Compare commits

..
Author SHA1 Message Date
regnat 4f84b2c4d6 Remove the visit machinery in RealisedPath
In addition to being some ugly template trickery, it was also totally
useless as it was used in only one place where I could replace it by
just a few extra characters
2021-02-04 15:15:22 +01:00
regnat 343687405d Move the GENERATE_CMP macro to its own file
Despite being an ugly hack, it can probably be useful in a couple extra
places
2021-02-04 15:11:05 +01:00
Shea Levy 8864c606ad Move command plugin interface to libnixcmd 2021-02-04 15:01:28 +01:00
regnat 9fe4bf8e59 Add a new Cmd type working on RealisedPaths
Where a `RealisedPath` is a store path with its history, meaning either
an opaque path for stuff that has been directly added to the store, or a
`Realisation` for stuff that has been built by a derivation

This is a low-level refactoring that doesn't bring anything by itself
(except a few dozen extra lines of code :/ ), but raising the
abstraction level a bit is important on a number of levels:

- Commands like `nix build` have to query for the realisations after the
  build is finished which is fragile (see
  27905f12e4a7207450abe37c9ed78e31603b67e1 for example). Having them
  oprate directly at the realisation level would avoid that
- Others like `nix copy` currently operate directly on (built) store
  paths, but need a bit more information as they will need to register
  the realisations on the remote side
2021-01-27 15:35:10 +01:00
37 changed files with 142 additions and 302 deletions

No files matched your search

-35
View File
@@ -1,35 +0,0 @@
# Stale bot information
- Thanks for your contribution!
- To remove the stale label, just leave a new comment.
- _How to find the right people to ping?_ → [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on the [#nixos IRC channel](https://webchat.freenode.net/#nixos).
## Suggestions for PRs
1. GitHub sometimes doesn't notify people who commented / reviewed a PR previously, when you (force) push commits. If you have addressed the reviews you can [officially ask for a review](https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from those who commented to you or anyone else.
2. If it is unfinished but you plan to finish it, please mark it as a draft.
3. If you don't expect to work on it any time soon, closing it with a short comment may encourage someone else to pick up your work.
4. To get things rolling again, rebase the PR against the target branch and address valid comments.
5. If you need a review to move forward, ask in [the Discourse thread for PRs that need help](https://discourse.nixos.org/t/prs-in-distress/3604).
6. If all you need is a merge, check the git history to find and [request reviews](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/requesting-a-pull-request-review) from people who usually merge related contributions.
## Suggestions for issues
1. If it is resolved (either for you personally, or in general), please consider closing it.
2. If this might still be an issue, but you are not interested in promoting its resolution, please consider closing it while encouraging others to take over and reopen an issue if they care enough.
3. If you still have interest in resolving it, try to ping somebody who you believe might have an interest in the topic. Consider discussing the problem in [our Discourse Forum](https://discourse.nixos.org/).
4. As with all open source projects, your best option is to submit a Pull Request that addresses this issue. We :heart: this attitude!
**Memorandum on closing issues**
Don't be afraid to close an issue that holds valuable information. Closed issues stay in the system for people to search, read, cross-reference, or even reopen--nothing is lost! Closing obsolete issues is an important way to help maintainers focus their time and effort.
## Useful GitHub search queries
- [Open PRs with any stale-bot interaction](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+)
- [Open PRs with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22)
- [Open PRs with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/pulls?q=is%3Apr+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
- [Open Issues with any stale-bot interaction](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+)
- [Open Issues with any stale-bot interaction and `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+label%3A%22stale%22+)
- [Open Issues with any stale-bot interaction and NOT `stale`](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+commenter%3Aapp%2Fstale+-label%3A%22stale%22+)
-10
View File
@@ -1,10 +0,0 @@
# Configuration for probot-stale - https://github.com/probot/stale
daysUntilStale: 180
daysUntilClose: 365
exemptLabels:
- "critical"
staleLabel: "stale"
markComment: |
I marked this as stale due to inactivity. → [More info](https://github.com/NixOS/nix/blob/master/.github/STALE-BOT.md)
closeComment: |
I closed this issue due to inactivity. → [More info](https://github.com/NixOS/nix/blob/master/.github/STALE-BOT.md)
+1 -4
View File
@@ -7,10 +7,7 @@ let
showCommand =
{ command, def, filename }:
''
**Warning**: This program is **experimental** and its interface is subject to change.
''
+ "# Name\n\n"
"# Name\n\n"
+ "`${command}` - ${def.description}\n\n"
+ "# Synopsis\n\n"
+ showSynopsis { inherit command; args = def.args; }
+7 -8
View File
@@ -232,23 +232,22 @@ terraform apply
> in a nix-shell shebang.
Finally, using the merging of multiple nix-shell shebangs the following
Haskell script uses a specific branch of Nixpkgs/NixOS (the 20.03 stable
Haskell script uses a specific branch of Nixpkgs/NixOS (the 18.03 stable
branch):
```haskell
#! /usr/bin/env nix-shell
#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])"
#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.03.tar.gz
#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.HTTP ps.tagsoup])"
#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-18.03.tar.gz
import Network.Curl.Download
import Network.HTTP
import Text.HTML.TagSoup
import Data.Either
import Data.ByteString.Char8 (unpack)
-- Fetch nixos.org and print all hrefs.
main = do
resp <- openURI "https://nixos.org/"
let tags = filter (isTagOpenName "a") $ parseTags $ unpack $ fromRight undefined resp
resp <- Network.HTTP.simpleHTTP (getRequest "http://nixos.org/")
body <- getResponseBody resp
let tags = filter (isTagOpenName "a") $ parseTags body
let tags' = map (fromAttrib "href") tags
mapM_ putStrLn $ filter (/= "") tags'
```
@@ -25,4 +25,5 @@ order of precedence (from strongest to weakest binding).
| Inequality | *e1* `!=` *e2* | none | Inequality. | 11 |
| Logical AND | *e1* `&&` *e2* | left | Logical AND. | 12 |
| Logical OR | *e1* `\|\|` *e2* | left | Logical OR. | 13 |
| Logical Implication | *e1* `->` *e2* | none | Logical implication (equivalent to `!e1 \|\| e2`). | 14 |
| Logical Implication | *e1* `->` *e2* | none | Logical implication (equivalent to `!e1 \|\|
e2`). | 14 |
+7 -6
View File
@@ -115,7 +115,7 @@
# 'nix.perl-bindings' packages.
overlay = final: prev: {
nix = with final; with commonDeps pkgs; stdenv.mkDerivation {
nix = with final; with commonDeps pkgs; (stdenv.mkDerivation {
name = "nix-${version}";
inherit version;
@@ -163,8 +163,9 @@
installCheckFlags = "sysconfdir=$(out)/etc";
separateDebugInfo = true;
}) // {
passthru.perl-bindings = with final; stdenv.mkDerivation {
perl-bindings = with final; stdenv.mkDerivation {
name = "nix-perl-${version}";
src = self;
@@ -198,12 +199,12 @@
};
lowdown = with final; stdenv.mkDerivation rec {
name = "lowdown-0.8.0";
lowdown = with final; stdenv.mkDerivation {
name = "lowdown-0.7.9";
src = fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/${name}.tar.gz";
hash = "sha512-U9WeGoInT9vrawwa57t6u9dEdRge4/P+0wLxmQyOL9nhzOEUU2FRz2Be9H0dCjYE7p2v3vCXIYk40M+jjULATw==";
url = https://kristaps.bsd.lv/lowdown/snapshots/lowdown-0.7.9.tar.gz;
hash = "sha512-7GQrKFICyTI5T4SinATfohiCq9TC0OgN8NmVfG3B3BZJM9J00DT8llAco8kNykLIKtl/AXuS4X8fETiCFEWEUQ==";
};
#src = lowdown-src;
+3 -1
View File
@@ -118,8 +118,10 @@ void StorePathsCommand::run(ref<Store> store, std::vector<RealisedPath> paths)
run(store, std::move(storePaths));
}
void StorePathCommand::run(ref<Store> store, std::vector<StorePath> storePaths)
void StorePathCommand::run(ref<Store> store)
{
auto storePaths = toStorePaths(store, Realise::Nothing, operateOn, installables);
if (storePaths.size() != 1)
throw UsageError("this command requires exactly one store path");
+3 -3
View File
@@ -177,13 +177,13 @@ struct StorePathsCommand : public RealisedPathsCommand
};
/* A command that operates on exactly one store path. */
struct StorePathCommand : public StorePathsCommand
struct StorePathCommand : public InstallablesCommand
{
using StorePathsCommand::run;
using StoreCommand::run;
virtual void run(ref<Store> store, const StorePath & storePath) = 0;
void run(ref<Store> store, std::vector<StorePath> storePaths) override;
void run(ref<Store> store) override;
};
/* A helper class for registering commands globally. */
+5 -24
View File
@@ -496,23 +496,6 @@ static std::string showAttrPaths(const std::vector<std::string> & paths)
return s;
}
InstallableFlake::InstallableFlake(
SourceExprCommand * cmd,
ref<EvalState> state,
FlakeRef && flakeRef,
Strings && attrPaths,
Strings && prefixes,
const flake::LockFlags & lockFlags)
: InstallableValue(state),
flakeRef(flakeRef),
attrPaths(attrPaths),
prefixes(prefixes),
lockFlags(lockFlags)
{
if (cmd && cmd->getAutoArgs(*state)->size())
throw UsageError("'--arg' and '--argstr' are incompatible with flakes");
}
std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableFlake::toDerivation()
{
auto lockedFlake = getLockedFlake();
@@ -645,12 +628,9 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
try {
auto [flakeRef, fragment] = parseFlakeRefWithFragment(s, absPath("."));
result.push_back(std::make_shared<InstallableFlake>(
this,
getEvalState(),
std::move(flakeRef),
getEvalState(), std::move(flakeRef),
fragment == "" ? getDefaultFlakeAttrPaths() : Strings{fragment},
getDefaultFlakeAttrPathPrefixes(),
lockFlags));
getDefaultFlakeAttrPathPrefixes(), lockFlags));
continue;
} catch (...) {
ex = std::current_exception();
@@ -744,13 +724,14 @@ std::set<RealisedPath> toRealisedPaths(
if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
if (!outputHashes.count(output.first))
throw Error(
"the derivation '%s' doesn't have an output named '%s'",
"The derivation %s doesn't have an output "
"named %s",
store->printStorePath(bfd.drvPath),
output.first);
auto outputId = DrvOutput{outputHashes.at(output.first), output.first};
auto realisation = store->queryRealisation(outputId);
if (!realisation)
throw Error("cannot operate on an output of unbuilt content-addresed derivation '%s'", outputId.to_string());
throw Error("Cannot operate on output of unbuilt CA drv %s", outputId.to_string());
res.insert(RealisedPath{*realisation});
}
else {
+5 -7
View File
@@ -104,13 +104,11 @@ struct InstallableFlake : InstallableValue
const flake::LockFlags & lockFlags;
mutable std::shared_ptr<flake::LockedFlake> _lockedFlake;
InstallableFlake(
SourceExprCommand * cmd,
ref<EvalState> state,
FlakeRef && flakeRef,
Strings && attrPaths,
Strings && prefixes,
const flake::LockFlags & lockFlags);
InstallableFlake(ref<EvalState> state, FlakeRef && flakeRef,
Strings && attrPaths, Strings && prefixes, const flake::LockFlags & lockFlags)
: InstallableValue(state), flakeRef(flakeRef), attrPaths(attrPaths),
prefixes(prefixes), lockFlags(lockFlags)
{ }
std::string what() override { return flakeRef.to_string() + "#" + *attrPaths.begin(); }
+3 -3
View File
@@ -3,7 +3,9 @@
#include "finally.hh"
#include <sys/queue.h>
extern "C" {
#include <lowdown.h>
}
namespace nix {
@@ -40,9 +42,7 @@ std::string renderMarkdownToTerminal(std::string_view markdown)
throw Error("cannot allocate Markdown output buffer");
Finally freeBuffer([&]() { lowdown_buf_free(buf); });
int rndr_res = lowdown_term_rndr(buf, nullptr, renderer, node);
if (!rndr_res)
throw Error("allocation error while rendering Markdown");
lowdown_term_rndr(buf, nullptr, renderer, node);
return std::string(buf->data, buf->size);
}
+4 -4
View File
@@ -1381,10 +1381,10 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
} else if (!i.def) {
throwMissingArgumentError(i.pos, R"(cannot evaluate a function that has an argument without a value ('%1%')
Nix attempted to evaluate a function as a top level expression; in
this case it must have its arguments supplied either by default
values, or passed explicitly with '--arg' or '--argstr'. See
https://nixos.org/manual/nix/stable/#ss-functions.)", i.name);
nix attempted to evaluate a function as a top level expression; in this case it must have its
arguments supplied either by default values, or passed explicitly with --arg or --argstr.
https://nixos.org/manual/nix/stable/#ss-functions)", i.name);
}
}
+1 -1
View File
@@ -17,7 +17,7 @@ MakeError(ThrownError, AssertionError);
MakeError(Abort, EvalError);
MakeError(TypeError, EvalError);
MakeError(UndefinedVarError, Error);
MakeError(MissingArgumentError, EvalError);
MakeError(MissingArgumentError, Error);
MakeError(RestrictedPathError, Error);
+4 -8
View File
@@ -696,14 +696,10 @@ static RegisterPrimOp primop_tryEval({
Try to shallowly evaluate *e*. Return a set containing the
attributes `success` (`true` if *e* evaluated successfully,
`false` if an error was thrown) and `value`, equalling *e* if
successful and `false` otherwise. `tryEval` will only prevent
errors created by `throw` or `assert` from being thrown.
Errors `tryEval` will not catch are for example those created
by `abort` and type errors generated by builtins. Also note that
this doesn't evaluate *e* deeply, so `let e = { x = throw ""; };
in (builtins.tryEval e).success` will be `true`. Using
`builtins.deepSeq` one can get the expected result:
`let e = { x = throw ""; }; in
successful and `false` otherwise. Note that this doesn't evaluate
*e* deeply, so ` let e = { x = throw ""; }; in (builtins.tryEval
e).success ` will be `true`. Using ` builtins.deepSeq ` one can
get the expected result: `let e = { x = throw ""; }; in
(builtins.tryEval (builtins.deepSeq e e)).success` will be
`false`.
)",
+14 -86
View File
@@ -123,8 +123,7 @@ DerivationGoal::DerivationGoal(const StorePath & drvPath, const BasicDerivation
, wantedOutputs(wantedOutputs)
, buildMode(buildMode)
{
this->drv = std::make_unique<Derivation>(drv);
this->drv = std::make_unique<BasicDerivation>(BasicDerivation(drv));
state = &DerivationGoal::haveDerivation;
name = fmt(
"building of '%s' from in-memory derivation",
@@ -259,10 +258,8 @@ void DerivationGoal::loadDerivation()
assert(worker.store.isValidPath(drvPath));
auto fullDrv = new Derivation(worker.store.derivationFromPath(drvPath));
/* Get the derivation. */
drv = std::unique_ptr<Derivation>(fullDrv);
drv = std::unique_ptr<BasicDerivation>(new Derivation(worker.store.derivationFromPath(drvPath)));
haveDerivation();
}
@@ -281,16 +278,6 @@ void DerivationGoal::haveDerivation()
if (i.second.second)
worker.store.addTempRoot(*i.second.second);
auto outputHashes = staticOutputHashes(worker.store, *drv);
for (auto &[outputName, outputHash] : outputHashes)
initialOutputs.insert({
outputName,
InitialOutput{
.wanted = true, // Will be refined later
.outputHash = outputHash
}
});
/* Check what outputs paths are not already valid. */
checkPathValidity();
bool allValid = true;
@@ -519,7 +506,6 @@ void DerivationGoal::inputsRealised()
Derivation drvResolved { *std::move(attempt) };
auto pathResolved = writeDerivation(worker.store, drvResolved);
resolvedDrv = drvResolved;
auto msg = fmt("Resolved derivation: '%s' -> '%s'",
worker.store.printStorePath(drvPath),
@@ -550,12 +536,12 @@ void DerivationGoal::inputsRealised()
if (!optRealizedInput)
throw Error(
"derivation '%s' requires output '%s' from input derivation '%s', which is supposedly realized already, yet we still don't know what path corresponds to that output",
worker.store.printStorePath(drvPath), j, worker.store.printStorePath(depDrvPath));
worker.store.printStorePath(drvPath), j, worker.store.printStorePath(drvPath));
worker.store.computeFSClosure(*optRealizedInput, inputPaths);
} else
throw Error(
"derivation '%s' requires non-existent output '%s' from input derivation '%s'",
worker.store.printStorePath(drvPath), j, worker.store.printStorePath(depDrvPath));
worker.store.printStorePath(drvPath), j, worker.store.printStorePath(drvPath));
}
}
}
@@ -1033,37 +1019,7 @@ void DerivationGoal::buildDone()
}
void DerivationGoal::resolvedFinished() {
assert(resolvedDrv);
auto resolvedHashes = staticOutputHashes(worker.store, *resolvedDrv);
// `wantedOutputs` might be empty, which means “all the outputs”
auto realWantedOutputs = wantedOutputs;
if (realWantedOutputs.empty())
realWantedOutputs = resolvedDrv->outputNames();
for (auto & wantedOutput : realWantedOutputs) {
assert(initialOutputs.count(wantedOutput) != 0);
assert(resolvedHashes.count(wantedOutput) != 0);
auto realisation = worker.store.queryRealisation(
DrvOutput{resolvedHashes.at(wantedOutput), wantedOutput}
);
// We've just built it, but maybe the build failed, in which case the
// realisation won't be there
if (realisation) {
auto newRealisation = *realisation;
newRealisation.id = DrvOutput{initialOutputs.at(wantedOutput).outputHash, wantedOutput};
worker.store.registerDrvOutput(newRealisation);
} else {
// If we don't have a realisation, then it must mean that something
// failed when building the resolved drv
assert(!result.success());
}
}
// This is potentially a bit fishy in terms of error reporting. Not sure
// how to do it in a cleaner way
amDone(nrFailed == 0 ? ecSuccess : ecFailed, ex);
done(BuildResult::Built);
}
HookReply DerivationGoal::tryBuildHook()
@@ -1088,14 +1044,7 @@ HookReply DerivationGoal::tryBuildHook()
whether the hook wishes to perform the build. */
string reply;
while (true) {
auto s = [&]() {
try {
return readLine(worker.hook->fromHook.readSide.get());
} catch (Error & e) {
e.addTrace({}, "while reading the response from the build hook");
throw e;
}
}();
string s = readLine(worker.hook->fromHook.readSide.get());
if (handleJSONLogMessage(s, worker.act, worker.hook->activities, true))
;
else if (string(s, 0, 2) == "# ") {
@@ -1135,12 +1084,7 @@ HookReply DerivationGoal::tryBuildHook()
hook = std::move(worker.hook);
try {
machineName = readLine(hook->fromHook.readSide.get());
} catch (Error & e) {
e.addTrace({}, "while reading the machine name from the build hook");
throw e;
}
machineName = readLine(hook->fromHook.readSide.get());
/* Tell the hook all the inputs that have to be copied to the
remote system. */
@@ -1828,17 +1772,8 @@ void DerivationGoal::startBuilder()
worker.childStarted(shared_from_this(), {builderOut.readSide.get()}, true, true);
/* Check if setting up the build environment failed. */
std::vector<std::string> msgs;
while (true) {
string msg = [&]() {
try {
return readLine(builderOut.readSide.get());
} catch (Error & e) {
e.addTrace({}, "while waiting for the build environment to initialize (previous messages: %s)",
concatStringsSep("|", msgs));
throw e;
}
}();
string msg = readLine(builderOut.readSide.get());
if (string(msg, 0, 1) == "\2") break;
if (string(msg, 0, 1) == "\1") {
FdSource source(builderOut.readSide.get());
@@ -1847,7 +1782,6 @@ void DerivationGoal::startBuilder()
throw ex;
}
debug("sandbox setup: " + msg);
msgs.push_back(std::move(msg));
}
}
@@ -3507,9 +3441,10 @@ void DerivationGoal::registerOutputs()
but it's fine to do in all cases. */
if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
auto outputHashes = staticOutputHashes(worker.store, *drv);
for (auto& [outputName, newInfo] : infos)
worker.store.registerDrvOutput(Realisation{
.id = DrvOutput{initialOutputs.at(outputName).outputHash, outputName},
.id = DrvOutput{outputHashes.at(outputName), outputName},
.outPath = newInfo.path});
}
}
@@ -3833,8 +3768,9 @@ void DerivationGoal::checkPathValidity()
{
bool checkHash = buildMode == bmRepair;
for (auto & i : queryPartialDerivationOutputMap()) {
InitialOutput & info = initialOutputs.at(i.first);
info.wanted = wantOutput(i.first, wantedOutputs);
InitialOutput info {
.wanted = wantOutput(i.first, wantedOutputs),
};
if (i.second) {
auto outputPath = *i.second;
info.known = {
@@ -3846,15 +3782,7 @@ void DerivationGoal::checkPathValidity()
: PathStatus::Corrupt,
};
}
if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
if (auto real = worker.store.queryRealisation(
DrvOutput{initialOutputs.at(i.first).outputHash, i.first})) {
info.known = {
.path = real->outPath,
.status = PathStatus::Valid,
};
}
}
initialOutputs.insert_or_assign(i.first, info);
}
}
+1 -5
View File
@@ -37,7 +37,6 @@ struct InitialOutputStatus {
struct InitialOutput {
bool wanted;
Hash outputHash;
std::optional<InitialOutputStatus> known;
};
@@ -49,9 +48,6 @@ struct DerivationGoal : public Goal
/* The path of the derivation. */
StorePath drvPath;
/* The path of the corresponding resolved derivation */
std::optional<BasicDerivation> resolvedDrv;
/* The specific outputs that we need to build. Empty means all of
them. */
StringSet wantedOutputs;
@@ -64,7 +60,7 @@ struct DerivationGoal : public Goal
bool retrySubstitution;
/* The derivation stored at drvPath. */
std::unique_ptr<Derivation> drv;
std::unique_ptr<BasicDerivation> drv;
std::unique_ptr<ParsedDerivation> parsedDrv;
+32 -7
View File
@@ -745,7 +745,7 @@ static void rewriteDerivation(Store & store, BasicDerivation & drv, const String
}
std::optional<BasicDerivation> Derivation::tryResolve(Store & store) {
std::optional<BasicDerivation> Derivation::tryResolveUncached(Store & store) {
BasicDerivation resolved { *this };
// Input paths that we'll want to rewrite in the derivation
@@ -756,13 +756,8 @@ std::optional<BasicDerivation> Derivation::tryResolve(Store & store) {
StringSet newOutputNames;
for (auto & outputName : input.second) {
auto actualPathOpt = inputDrvOutputs.at(outputName);
if (!actualPathOpt) {
warn("output %s of input %s missing, aborting the resolving",
outputName,
store.printStorePath(input.first)
);
if (!actualPathOpt)
return std::nullopt;
}
auto actualPath = *actualPathOpt;
inputRewrites.emplace(
downstreamPlaceholder(store, input.first, outputName),
@@ -776,4 +771,34 @@ std::optional<BasicDerivation> Derivation::tryResolve(Store & store) {
return resolved;
}
std::optional<BasicDerivation> Derivation::tryResolve(Store& store)
{
auto drvPath = writeDerivation(store, *this, NoRepair, false);
return Derivation::tryResolve(store, drvPath);
}
std::optional<BasicDerivation> Derivation::tryResolve(Store& store, const StorePath& drvPath)
{
// This is quite dirty and leaky, but will disappear once #4340 is merged
static Sync<std::map<StorePath, std::optional<Derivation>>> resolutionsCache;
{
auto resolutions = resolutionsCache.lock();
auto resolvedDrvIter = resolutions->find(drvPath);
if (resolvedDrvIter != resolutions->end()) {
auto & [_, resolvedDrv] = *resolvedDrvIter;
return *resolvedDrv;
}
}
/* Try resolve drv and use that path instead. */
auto drv = store.readDerivation(drvPath);
auto attempt = drv.tryResolveUncached(store);
if (!attempt)
return std::nullopt;
/* Store in memo table. */
resolutionsCache.lock()->insert_or_assign(drvPath, *attempt);
return *attempt;
}
}
+4
View File
@@ -138,10 +138,14 @@ struct Derivation : BasicDerivation
2. Input placeholders are replaced with realized input store paths. */
std::optional<BasicDerivation> tryResolve(Store & store);
static std::optional<BasicDerivation> tryResolve(Store & store, const StorePath & drvPath);
Derivation() = default;
Derivation(const BasicDerivation & bd) : BasicDerivation(bd) { }
Derivation(BasicDerivation && bd) : BasicDerivation(std::move(bd)) { }
private:
std::optional<BasicDerivation> tryResolveUncached(Store & store);
};
-3
View File
@@ -831,9 +831,6 @@ public:
command, and RegisterSetting to add new nix config settings. See the
constructors for those types for more details.
Warning! These APIs are inherently unstable and may change from
release to release.
Since these files are loaded into the same address space as Nix
itself, they must be DSOs compatible with the instance of Nix
running at the time (i.e. compiled against the same headers, not
+1 -1
View File
@@ -883,7 +883,7 @@ StorePathSet LocalStore::queryValidDerivers(const StorePath & path)
std::map<std::string, std::optional<StorePath>>
LocalStore::queryPartialDerivationOutputMap(const StorePath & path_)
LocalStore::queryDerivationOutputMapNoResolve(const StorePath& path_)
{
auto path = path_;
auto outputs = retrySQLite<std::map<std::string, std::optional<StorePath>>>([&]() {
+1 -1
View File
@@ -127,7 +127,7 @@ public:
StorePathSet queryValidDerivers(const StorePath & path) override;
std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path) override;
std::map<std::string, std::optional<StorePath>> queryDerivationOutputMapNoResolve(const StorePath & path) override;
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override;
+1 -1
View File
@@ -65,7 +65,7 @@ void RealisedPath::closure(
ret.insert(pathsClosure.begin(), pathsClosure.end());
}
void RealisedPath::closure(Store& store, RealisedPath::Set & ret) const
void RealisedPath::closure(Store& store, RealisedPath::Set& ret) const
{
RealisedPath::closure(store, {*this}, ret);
}
+15 -2
View File
@@ -366,7 +366,7 @@ bool Store::PathInfoCacheValue::isKnownNow()
return std::chrono::steady_clock::now() < time_point + ttl;
}
std::map<std::string, std::optional<StorePath>> Store::queryPartialDerivationOutputMap(const StorePath & path)
std::map<std::string, std::optional<StorePath>> Store::queryDerivationOutputMapNoResolve(const StorePath & path)
{
std::map<std::string, std::optional<StorePath>> outputs;
auto drv = readInvalidDerivation(path);
@@ -376,12 +376,25 @@ std::map<std::string, std::optional<StorePath>> Store::queryPartialDerivationOut
return outputs;
}
std::map<std::string, std::optional<StorePath>> Store::queryPartialDerivationOutputMap(const StorePath & path)
{
if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
auto resolvedDrv = Derivation::tryResolve(*this, path);
if (resolvedDrv) {
auto resolvedDrvPath = writeDerivation(*this, *resolvedDrv, NoRepair, true);
if (isValidPath(resolvedDrvPath))
return queryDerivationOutputMapNoResolve(resolvedDrvPath);
}
}
return queryDerivationOutputMapNoResolve(path);
}
OutputPathMap Store::queryDerivationOutputMap(const StorePath & path) {
auto resp = queryPartialDerivationOutputMap(path);
OutputPathMap result;
for (auto & [outName, optOutPath] : resp) {
if (!optOutPath)
throw Error("output '%s' of derivation '%s' has no store path mapped to it", outName, printStorePath(path));
throw Error("output '%s' has no store path mapped to it", outName);
result.insert_or_assign(outName, *optOutPath);
}
return result;
+6
View File
@@ -415,6 +415,12 @@ public:
`std::nullopt`. */
virtual std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path);
/*
* Similar to `queryPartialDerivationOutputMap`, but doesn't try to resolve
* the derivation
*/
virtual std::map<std::string, std::optional<StorePath>> queryDerivationOutputMapNoResolve(const StorePath & path);
/* Query the mapping outputName=>outputPath for the given derivation.
Assume every output has a mapping and throw an exception otherwise. */
OutputPathMap queryDerivationOutputMap(const StorePath & path);
-3
View File
@@ -14,8 +14,6 @@ void Args::addFlag(Flag && flag_)
assert(flag->handler.arity == flag->labels.size());
assert(flag->longName != "");
longFlags[flag->longName] = flag;
for (auto & alias : flag->aliases)
longFlags[alias] = flag;
if (flag->shortName) shortFlags[flag->shortName] = flag;
}
@@ -193,7 +191,6 @@ nlohmann::json Args::toJSON()
for (auto & [name, flag] : longFlags) {
auto j = nlohmann::json::object();
if (flag->aliases.count(name)) continue;
if (flag->shortName)
j["shortName"] = std::string(1, flag->shortName);
if (flag->description != "")
-1
View File
@@ -97,7 +97,6 @@ protected:
typedef std::shared_ptr<Flag> ptr;
std::string longName;
std::set<std::string> aliases;
char shortName = 0;
std::string description;
std::string category;
+2 -4
View File
@@ -518,11 +518,9 @@ static void main_nix_build(int argc, char * * argv)
if (counter)
drvPrefix += fmt("-%d", counter + 1);
auto builtOutputs = store->queryPartialDerivationOutputMap(drvPath);
auto builtOutputs = store->queryDerivationOutputMap(drvPath);
auto maybeOutputPath = builtOutputs.at(outputName);
assert(maybeOutputPath);
auto outputPath = *maybeOutputPath;
auto outputPath = builtOutputs.at(outputName);
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) {
std::string symlink = drvPrefix;
+3 -16
View File
@@ -12,16 +12,11 @@ App Installable::toApp(EvalState & state)
auto type = cursor->getAttr("type")->getString();
auto checkProgram = [&](const Path & program)
{
if (!state.store->isInStore(program))
throw Error("app program '%s' is not in the Nix store", program);
};
if (type == "app") {
auto [program, context] = cursor->getAttr("program")->getStringWithContext();
checkProgram(program);
if (!state.store->isInStore(program))
throw Error("app program '%s' is not in the Nix store", program);
std::vector<StorePathWithOutputs> context2;
for (auto & [path, name] : context)
@@ -38,17 +33,9 @@ App Installable::toApp(EvalState & state)
auto outPath = cursor->getAttr(state.sOutPath)->getString();
auto outputName = cursor->getAttr(state.sOutputName)->getString();
auto name = cursor->getAttr(state.sName)->getString();
auto aMeta = cursor->maybeGetAttr("meta");
auto aMainProgram = aMeta ? aMeta->maybeGetAttr("mainProgram") : nullptr;
auto mainProgram =
aMainProgram
? aMainProgram->getString()
: DrvName(name).name;
auto program = outPath + "/bin/" + mainProgram;
checkProgram(program);
return App {
.context = { { drvPath, {outputName} } },
.program = program,
.program = outPath + "/bin/" + DrvName(name).name,
};
}
+1 -1
View File
@@ -74,7 +74,7 @@ struct CmdBundle : InstallableCommand
auto [bundlerFlakeRef, bundlerName] = parseFlakeRefWithFragment(bundler, absPath("."));
const flake::LockFlags lockFlags{ .writeLockFile = false };
auto bundler = InstallableFlake(this,
auto bundler = InstallableFlake(
evalState, std::move(bundlerFlakeRef),
Strings{bundlerName == "" ? "defaultBundler" : bundlerName},
Strings({"bundlers."}), lockFlags);
+2 -3
View File
@@ -59,7 +59,7 @@ BuildEnvironment readEnvironment(const Path & path)
R"re((?:\$?"(?:[^"\\]|\\[$`"\\\n])*"))re";
static std::string squotedStringRegex =
R"re((?:\$?(?:'(?:[^'\\]|\\[abeEfnrtv\\'"?])*'|\\')+))re";
R"re((?:\$?'(?:[^'\\]|\\[abeEfnrtv\\'"?])*'))re";
static std::string indexedArrayRegex =
R"re((?:\(( *\[[0-9]+\]="(?:[^"\\]|\\.)*")*\)))re";
@@ -239,7 +239,7 @@ struct Common : InstallableCommand, MixProfile
out << buildEnvironment.bashFunctions << "\n";
out << "export NIX_BUILD_TOP=\"$(mktemp -d -t nix-shell.XXXXXX)\"\n";
out << "export NIX_BUILD_TOP=\"$(mktemp -d --tmpdir nix-shell.XXXXXX)\"\n";
for (auto & i : {"TMP", "TMPDIR", "TEMP", "TEMPDIR"})
out << fmt("export %s=\"$NIX_BUILD_TOP\"\n", i);
@@ -443,7 +443,6 @@ struct CmdDevelop : Common, MixEnvironment
auto state = getEvalState();
auto bashInstallable = std::make_shared<InstallableFlake>(
this,
state,
installable->nixpkgsFlakeRef(),
Strings{"bashInteractive"},
+3 -5
View File
@@ -595,7 +595,7 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
auto [templateFlakeRef, templateName] = parseFlakeRefWithFragment(templateUrl, absPath("."));
auto installable = InstallableFlake(nullptr,
auto installable = InstallableFlake(
evalState, std::move(templateFlakeRef),
Strings{templateName == "" ? "defaultTemplate" : templateName},
Strings(attrsPathPrefixes), lockFlags);
@@ -880,8 +880,7 @@ struct CmdFlakeShow : FlakeCommand
|| attrPath[0] == "nixosConfigurations"
|| attrPath[0] == "nixosModules"
|| attrPath[0] == "defaultApp"
|| attrPath[0] == "templates"
|| attrPath[0] == "overlays"))
|| attrPath[0] == "templates"))
|| ((attrPath.size() == 1 || attrPath.size() == 2)
&& (attrPath[0] == "checks"
|| attrPath[0] == "packages"
@@ -944,8 +943,7 @@ struct CmdFlakeShow : FlakeCommand
else {
logger->cout("%s: %s",
headerPrefix,
(attrPath.size() == 1 && attrPath[0] == "overlay")
|| (attrPath.size() == 2 && attrPath[0] == "overlays") ? "Nixpkgs overlay" :
attrPath.size() == 1 && attrPath[0] == "overlay" ? "Nixpkgs overlay" :
attrPath.size() == 2 && attrPath[0] == "nixosConfigurations" ? "NixOS configuration" :
attrPath.size() == 2 && attrPath[0] == "nixosModules" ? "NixOS module" :
ANSI_YELLOW "unknown" ANSI_NORMAL);
+2 -24
View File
@@ -17,10 +17,6 @@
#include <netdb.h>
#include <netinet/in.h>
#if __linux__
#include <sys/resource.h>
#endif
#include <nlohmann/json.hpp>
extern std::string chrootHelperName;
@@ -65,7 +61,6 @@ struct NixArgs : virtual MultiCommand, virtual MixCommonArgs
bool printBuildLogs = false;
bool useNet = true;
bool refresh = false;
bool showVersion = false;
NixArgs() : MultiCommand(RegisterCommand::getCommandsFor({})), MixCommonArgs("nix")
{
@@ -92,12 +87,11 @@ struct NixArgs : virtual MultiCommand, virtual MixCommonArgs
addFlag({
.longName = "version",
.description = "Show version information.",
.handler = {[&]() { showVersion = true; }},
.handler = {[&]() { if (!completions) printVersion(programName); }},
});
addFlag({
.longName = "offline",
.aliases = {"no-net"}, // FIXME: remove
.longName = "no-net",
.description = "Disable substituters and consider all previously downloaded files up-to-date.",
.handler = {[&]() { useNet = false; }},
});
@@ -285,11 +279,6 @@ void mainWrapped(int argc, char * * argv)
initPlugins();
if (args.showVersion) {
printVersion(programName);
return;
}
if (!args.command)
throw UsageError("no subcommand specified");
@@ -329,17 +318,6 @@ void mainWrapped(int argc, char * * argv)
int main(int argc, char * * argv)
{
// Increase the default stack size for the evaluator and for
// libstdc++'s std::regex.
#if __linux__
rlim_t stackSize = 64 * 1024 * 1024;
struct rlimit limit;
if (getrlimit(RLIMIT_STACK, &limit) == 0 && limit.rlim_cur < stackSize) {
limit.rlim_cur = stackSize;
setrlimit(RLIMIT_STACK, &limit);
}
#endif
return nix::handleExceptions(argv[0], [&]() {
nix::mainWrapped(argc, argv);
});
+2 -8
View File
@@ -249,7 +249,7 @@ struct CmdProfileInstall : InstallablesCommand, MixDefaultProfile
attrPath,
};
pathsToBuild.push_back({drv.drvPath, StringSet{drv.outputName}});
pathsToBuild.push_back({drv.drvPath, StringSet{"out"}}); // FIXME
manifest.elements.emplace_back(std::move(element));
} else {
@@ -399,13 +399,7 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
Activity act(*logger, lvlChatty, actUnknown,
fmt("checking '%s' for updates", element.source->attrPath));
InstallableFlake installable(
this,
getEvalState(),
FlakeRef(element.source->originalRef),
{element.source->attrPath},
{},
lockFlags);
InstallableFlake installable(getEvalState(), FlakeRef(element.source->originalRef), {element.source->attrPath}, {}, lockFlags);
auto [attrPath, resolvedRef, drv] = installable.toDerivation();
+3 -4
View File
@@ -43,10 +43,9 @@ program specified by the app definition.
If *installable* evaluates to a derivation, it will try to execute the
program `<out>/bin/<name>`, where *out* is the primary output store
path of the derivation and *name* is the `meta.mainProgram` attribute
of the derivation if it exists, and otherwise the name part of the
value of the `name` attribute of the derivation (e.g. if `name` is set
to `hello-1.10`, it will run `$out/bin/hello`).
path of the derivation and *name* is the name part of the value of the
`name` attribute of the derivation (e.g. if `name` is set to
`hello-1.10`, it will run `$out/bin/hello`).
# Flake output attributes
+4 -7
View File
@@ -81,9 +81,9 @@ struct CmdSearch : InstallableCommand, MixJSON
uint64_t results = 0;
std::function<void(eval_cache::AttrCursor & cursor, const std::vector<Symbol> & attrPath, bool initialRecurse)> visit;
std::function<void(eval_cache::AttrCursor & cursor, const std::vector<Symbol> & attrPath)> visit;
visit = [&](eval_cache::AttrCursor & cursor, const std::vector<Symbol> & attrPath, bool initialRecurse)
visit = [&](eval_cache::AttrCursor & cursor, const std::vector<Symbol> & attrPath)
{
Activity act(*logger, lvlInfo, actUnknown,
fmt("evaluating '%s'", concatStringsSep(".", attrPath)));
@@ -94,7 +94,7 @@ struct CmdSearch : InstallableCommand, MixJSON
auto cursor2 = cursor.getAttr(attr);
auto attrPath2(attrPath);
attrPath2.push_back(attr);
visit(*cursor2, attrPath2, false);
visit(*cursor2, attrPath2);
}
};
@@ -150,9 +150,6 @@ struct CmdSearch : InstallableCommand, MixJSON
|| (attrPath[0] == "packages" && attrPath.size() <= 2))
recurse();
else if (initialRecurse)
recurse();
else if (attrPath[0] == "legacyPackages" && attrPath.size() > 2) {
auto attr = cursor.maybeGetAttr(state->sRecurseForDerivations);
if (attr && attr->getBool())
@@ -166,7 +163,7 @@ struct CmdSearch : InstallableCommand, MixJSON
};
for (auto & [cursor, prefix] : installable->getCursors(*state))
visit(*cursor, parseAttrPath(*state, prefix), true);
visit(*cursor, parseAttrPath(*state, prefix));
if (!json && !results)
throw Error("no results for the given search term(s)!");
-4
View File
@@ -48,10 +48,6 @@ testCutoff () {
testGC () {
nix-instantiate --experimental-features ca-derivations ./content-addressed.nix -A rootCA --arg seed 5
nix-collect-garbage --experimental-features ca-derivations --option keep-derivations true
clearStore
buildAttr rootCA 1 --out-link $TEST_ROOT/rootCA
nix-collect-garbage --experimental-features ca-derivations
buildAttr rootCA 1 -j0
}
testNixCommand () {
-1
View File
@@ -21,7 +21,6 @@ experimental-features = nix-command flakes
gc-reserved-space = 0
substituters =
flake-registry = $TEST_ROOT/registry.json
show-trace = true
include nix.conf.extra
EOF