Compare commits

...
Author SHA1 Message Date
regnat ba1a256d08 Make DerivationGoal::drv a full Derivation
This field used to be a `BasicDerivation`, but this `BasicDerivation`
was downcasted to a `Derivation` when needed (implicitely or not), so we
might as well make it a full `Derivation` and upcast it when needed.

This also allows getting rid of a weird duplication in the way we
compute the static output hashes for the derivation. We had to
do it differently and in a different place depending on whether the
derivation was a full derivation or just a basic drv, but we can now do
it unconditionally on the full derivation.

Fix #4559
2021-02-23 14:15:45 +01:00
Shea Levy 35205e2e92 Warn about instability of plugin API 2021-02-22 17:10:55 -05:00
Eelco Dolstra 8043bc6c8d Merge pull request #4567 from NixOS/make-auto-call-error-eval-error
Make missing auto-call arguments throw an eval error
2021-02-22 16:35:52 +01:00
regnat e2f3b2eb42 Make missing auto-call arguments throw an eval error
The PR #4240 changed messag of the error that was thrown when an auto-called
function was missing an argument.
However this change also changed the type of the error, from `EvalError`
to a new `MissingArgumentError`. This broke hydra which was relying on
an `EvalError` being thrown.

Make `MissingArgumentError` a subclass of `EvalError` to un-break hydra.
2021-02-22 16:13:09 +01:00
Eelco Dolstra 574eb2be81 Tweak error message 2021-02-22 15:24:14 +01:00
Eelco Dolstra 548437c234 Merge pull request #4541 from obsidiansystems/simpler-store-path-command
Deeper `Command` hierarchy to remove redundancy
2021-02-19 16:18:53 +01:00
regnat f483b623e9 Remove the drv resolution caching mechanism
It isn't needed anymore now that don't need to eagerly resolve
everything like we used to do. So we can safely get rid of it
2021-02-19 15:48:31 +01:00
regnat 4bc28c44f2 Store the output hashes in the initialOutputs of the drv goal
That way we
1. Don't have to recompute them several times
2. Can compute them in a place where we know the type of the parent
  derivation, meaning that we don't need the casting dance we had before
2021-02-19 15:48:31 +01:00
Théophane HufschmittandEelco Dolstra 0bfbd04369 Don't expose the "bang" drvoutput syntax
It's not fixed nor useful atm, so better keep it hidden

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-19 15:48:31 +01:00
Théophane HufschmittandEelco Dolstra 93d9eb78a0 Syntactic fixes
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-02-19 15:48:31 +01:00
regnat 87c8d3d702 Register the realisations for unresolved drvs
Once a build is done, get back to the original derivation, and register
all the newly built outputs for this derivation.

This allows Nix to work properly with derivations that don't have all
their build inputs available − thus allowing garbage collection and
(once it's implemented) binary substitution
2021-02-19 15:48:31 +01:00
regnat be1b5c4e59 Test the garbage collection of CA derivations
Simple test to ensure that `nix-build && nix-collect-garbage &&
nix-build -j0` works as it should
2021-02-19 15:48:31 +01:00
regnat 263f6dbd1c Don't crash nix-build when not all outputs are realised
Change the `nix-build` logic for linking/printing the output paths to allow for
some outputs to be missing. This might happen when the toplevel
derivation didn't have to be built, either because all the required
outputs were already there, or because they have all been substituted.
2021-02-19 15:48:31 +01:00
Eelco Dolstra cd44c0af71 Increase default stack size on Linux
Workaround for #4550.
2021-02-18 19:22:37 +01:00
Eelco Dolstra 1b57825524 Document meta.mainProgram
Issue #4498.
2021-02-17 17:58:47 +01:00
Eelco Dolstra 7bd9898d5c nix run: Allow program name to be set in meta.mainProgram
This is useful when the program name doesn't match the package name
(e.g. ripgrep vs rg).

Fixes #4498.
2021-02-17 17:55:15 +01:00
Eelco Dolstra 13897afbe6 Throw an error if --arg / --argstr is used with a flake
Fixes #3949.
2021-02-17 17:32:10 +01:00
Eelco Dolstra f33878b656 Make 'nix --version -vv' work
Fixes #3743.
2021-02-17 17:11:21 +01:00
Eelco Dolstra cced73496b nix flake show: Handle 'overlays' output
Fixes #4542.
2021-02-17 16:53:39 +01:00
Eelco Dolstra 063de66909 nix develop: Fix quoted string handling
Fixes #4540.
2021-02-17 16:42:03 +01:00
Eelco Dolstra 6042febfce Restore warning about 'nix' being experimental
Fixes #4552.
2021-02-17 15:30:49 +01:00
Eelco Dolstra 54ab377288 Merge pull request #4553 from mausch/patch-1
Fix Haskell example
2021-02-17 10:18:42 +01:00
Mauricio SchefferandCole Helbling 5f4701e70d Update doc/manual/src/command-ref/nix-shell.md
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2021-02-16 23:27:04 +00:00
Mauricio Scheffer 35129884f9 Fix Haskell example
http://nixos.org redirects to https://nixos.org and apparently the HTTP library doesn't follow the redirect, so the output is empty.
When defining https in the request it crashes because the library doesn't seem to support https.
So this switches the example to a different http library.
2021-02-16 23:19:42 +00:00
John Ericson ad337c8697 Deeper Command hierarchy to remove redundancy
Simply put, we now have `StorePathCommand : public StorePathsCommand` so
`StorePathCommand` doesn't reimplement work.
2021-02-12 17:48:09 +00:00
26 changed files with 192 additions and 109 deletions

No files matched your search

+4 -1
View File
@@ -7,7 +7,10 @@ let
showCommand = showCommand =
{ command, def, filename }: { command, def, filename }:
"# Name\n\n" ''
**Warning**: This program is **experimental** and its interface is subject to change.
''
+ "# Name\n\n"
+ "`${command}` - ${def.description}\n\n" + "`${command}` - ${def.description}\n\n"
+ "# Synopsis\n\n" + "# Synopsis\n\n"
+ showSynopsis { inherit command; args = def.args; } + showSynopsis { inherit command; args = def.args; }
+8 -7
View File
@@ -232,22 +232,23 @@ terraform apply
> in a nix-shell shebang. > in a nix-shell shebang.
Finally, using the merging of multiple nix-shell shebangs the following Finally, using the merging of multiple nix-shell shebangs the following
Haskell script uses a specific branch of Nixpkgs/NixOS (the 18.03 stable Haskell script uses a specific branch of Nixpkgs/NixOS (the 20.03 stable
branch): branch):
```haskell ```haskell
#! /usr/bin/env nix-shell #! /usr/bin/env nix-shell
#! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.HTTP ps.tagsoup])" #! nix-shell -i runghc -p "haskellPackages.ghcWithPackages (ps: [ps.download-curl ps.tagsoup])"
#! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-18.03.tar.gz #! nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs/archive/nixos-20.03.tar.gz
import Network.HTTP import Network.Curl.Download
import Text.HTML.TagSoup import Text.HTML.TagSoup
import Data.Either
import Data.ByteString.Char8 (unpack)
-- Fetch nixos.org and print all hrefs. -- Fetch nixos.org and print all hrefs.
main = do main = do
resp <- Network.HTTP.simpleHTTP (getRequest "http://nixos.org/") resp <- openURI "https://nixos.org/"
body <- getResponseBody resp let tags = filter (isTagOpenName "a") $ parseTags $ unpack $ fromRight undefined resp
let tags = filter (isTagOpenName "a") $ parseTags body
let tags' = map (fromAttrib "href") tags let tags' = map (fromAttrib "href") tags
mapM_ putStrLn $ filter (/= "") tags' mapM_ putStrLn $ filter (/= "") tags'
``` ```
+1 -3
View File
@@ -118,10 +118,8 @@ void StorePathsCommand::run(ref<Store> store, std::vector<RealisedPath> paths)
run(store, std::move(storePaths)); run(store, std::move(storePaths));
} }
void StorePathCommand::run(ref<Store> store) void StorePathCommand::run(ref<Store> store, std::vector<StorePath> storePaths)
{ {
auto storePaths = toStorePaths(store, Realise::Nothing, operateOn, installables);
if (storePaths.size() != 1) if (storePaths.size() != 1)
throw UsageError("this command requires exactly one store path"); throw UsageError("this command requires exactly one store path");
+3 -3
View File
@@ -177,13 +177,13 @@ struct StorePathsCommand : public RealisedPathsCommand
}; };
/* A command that operates on exactly one store path. */ /* A command that operates on exactly one store path. */
struct StorePathCommand : public InstallablesCommand struct StorePathCommand : public StorePathsCommand
{ {
using StoreCommand::run; using StorePathsCommand::run;
virtual void run(ref<Store> store, const StorePath & storePath) = 0; virtual void run(ref<Store> store, const StorePath & storePath) = 0;
void run(ref<Store> store) override; void run(ref<Store> store, std::vector<StorePath> storePaths) override;
}; };
/* A helper class for registering commands globally. */ /* A helper class for registering commands globally. */
+22 -2
View File
@@ -496,6 +496,23 @@ static std::string showAttrPaths(const std::vector<std::string> & paths)
return s; return s;
} }
InstallableFlake::InstallableFlake(
SourceExprCommand * cmd,
ref<EvalState> state,
FlakeRef && flakeRef,
Strings && attrPaths,
Strings && prefixes,
const flake::LockFlags & lockFlags)
: InstallableValue(state),
flakeRef(flakeRef),
attrPaths(attrPaths),
prefixes(prefixes),
lockFlags(lockFlags)
{
if (cmd && cmd->getAutoArgs(*state)->size())
throw UsageError("'--arg' and '--argstr' are incompatible with flakes");
}
std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableFlake::toDerivation() std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableFlake::toDerivation()
{ {
auto lockedFlake = getLockedFlake(); auto lockedFlake = getLockedFlake();
@@ -628,9 +645,12 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
try { try {
auto [flakeRef, fragment] = parseFlakeRefWithFragment(s, absPath(".")); auto [flakeRef, fragment] = parseFlakeRefWithFragment(s, absPath("."));
result.push_back(std::make_shared<InstallableFlake>( result.push_back(std::make_shared<InstallableFlake>(
getEvalState(), std::move(flakeRef), this,
getEvalState(),
std::move(flakeRef),
fragment == "" ? getDefaultFlakeAttrPaths() : Strings{fragment}, fragment == "" ? getDefaultFlakeAttrPaths() : Strings{fragment},
getDefaultFlakeAttrPathPrefixes(), lockFlags)); getDefaultFlakeAttrPathPrefixes(),
lockFlags));
continue; continue;
} catch (...) { } catch (...) {
ex = std::current_exception(); ex = std::current_exception();
+7 -5
View File
@@ -104,11 +104,13 @@ struct InstallableFlake : InstallableValue
const flake::LockFlags & lockFlags; const flake::LockFlags & lockFlags;
mutable std::shared_ptr<flake::LockedFlake> _lockedFlake; mutable std::shared_ptr<flake::LockedFlake> _lockedFlake;
InstallableFlake(ref<EvalState> state, FlakeRef && flakeRef, InstallableFlake(
Strings && attrPaths, Strings && prefixes, const flake::LockFlags & lockFlags) SourceExprCommand * cmd,
: InstallableValue(state), flakeRef(flakeRef), attrPaths(attrPaths), ref<EvalState> state,
prefixes(prefixes), lockFlags(lockFlags) FlakeRef && flakeRef,
{ } Strings && attrPaths,
Strings && prefixes,
const flake::LockFlags & lockFlags);
std::string what() override { return flakeRef.to_string() + "#" + *attrPaths.begin(); } std::string what() override { return flakeRef.to_string() + "#" + *attrPaths.begin(); }
+4 -4
View File
@@ -1381,10 +1381,10 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
} else if (!i.def) { } else if (!i.def) {
throwMissingArgumentError(i.pos, R"(cannot evaluate a function that has an argument without a value ('%1%') throwMissingArgumentError(i.pos, R"(cannot evaluate a function that has an argument without a value ('%1%')
nix attempted to evaluate a function as a top level expression; in this case it must have its Nix attempted to evaluate a function as a top level expression; in
arguments supplied either by default values, or passed explicitly with --arg or --argstr. this case it must have its arguments supplied either by default
values, or passed explicitly with '--arg' or '--argstr'. See
https://nixos.org/manual/nix/stable/#ss-functions)", i.name); https://nixos.org/manual/nix/stable/#ss-functions.)", i.name);
} }
} }
+1 -1
View File
@@ -17,7 +17,7 @@ MakeError(ThrownError, AssertionError);
MakeError(Abort, EvalError); MakeError(Abort, EvalError);
MakeError(TypeError, EvalError); MakeError(TypeError, EvalError);
MakeError(UndefinedVarError, Error); MakeError(UndefinedVarError, Error);
MakeError(MissingArgumentError, Error); MakeError(MissingArgumentError, EvalError);
MakeError(RestrictedPathError, Error); MakeError(RestrictedPathError, Error);
+59 -9
View File
@@ -123,7 +123,8 @@ DerivationGoal::DerivationGoal(const StorePath & drvPath, const BasicDerivation
, wantedOutputs(wantedOutputs) , wantedOutputs(wantedOutputs)
, buildMode(buildMode) , buildMode(buildMode)
{ {
this->drv = std::make_unique<BasicDerivation>(BasicDerivation(drv)); this->drv = std::make_unique<Derivation>(drv);
state = &DerivationGoal::haveDerivation; state = &DerivationGoal::haveDerivation;
name = fmt( name = fmt(
"building of '%s' from in-memory derivation", "building of '%s' from in-memory derivation",
@@ -258,8 +259,10 @@ void DerivationGoal::loadDerivation()
assert(worker.store.isValidPath(drvPath)); assert(worker.store.isValidPath(drvPath));
auto fullDrv = new Derivation(worker.store.derivationFromPath(drvPath));
/* Get the derivation. */ /* Get the derivation. */
drv = std::unique_ptr<BasicDerivation>(new Derivation(worker.store.derivationFromPath(drvPath))); drv = std::unique_ptr<Derivation>(fullDrv);
haveDerivation(); haveDerivation();
} }
@@ -278,6 +281,16 @@ void DerivationGoal::haveDerivation()
if (i.second.second) if (i.second.second)
worker.store.addTempRoot(*i.second.second); worker.store.addTempRoot(*i.second.second);
auto outputHashes = staticOutputHashes(worker.store, *drv);
for (auto &[outputName, outputHash] : outputHashes)
initialOutputs.insert({
outputName,
InitialOutput{
.wanted = true, // Will be refined later
.outputHash = outputHash
}
});
/* Check what outputs paths are not already valid. */ /* Check what outputs paths are not already valid. */
checkPathValidity(); checkPathValidity();
bool allValid = true; bool allValid = true;
@@ -506,6 +519,7 @@ void DerivationGoal::inputsRealised()
Derivation drvResolved { *std::move(attempt) }; Derivation drvResolved { *std::move(attempt) };
auto pathResolved = writeDerivation(worker.store, drvResolved); auto pathResolved = writeDerivation(worker.store, drvResolved);
resolvedDrv = drvResolved;
auto msg = fmt("Resolved derivation: '%s' -> '%s'", auto msg = fmt("Resolved derivation: '%s' -> '%s'",
worker.store.printStorePath(drvPath), worker.store.printStorePath(drvPath),
@@ -1019,7 +1033,37 @@ void DerivationGoal::buildDone()
} }
void DerivationGoal::resolvedFinished() { void DerivationGoal::resolvedFinished() {
done(BuildResult::Built); assert(resolvedDrv);
auto resolvedHashes = staticOutputHashes(worker.store, *resolvedDrv);
// `wantedOutputs` might be empty, which means “all the outputs”
auto realWantedOutputs = wantedOutputs;
if (realWantedOutputs.empty())
realWantedOutputs = resolvedDrv->outputNames();
for (auto & wantedOutput : realWantedOutputs) {
assert(initialOutputs.count(wantedOutput) != 0);
assert(resolvedHashes.count(wantedOutput) != 0);
auto realisation = worker.store.queryRealisation(
DrvOutput{resolvedHashes.at(wantedOutput), wantedOutput}
);
// We've just built it, but maybe the build failed, in which case the
// realisation won't be there
if (realisation) {
auto newRealisation = *realisation;
newRealisation.id = DrvOutput{initialOutputs.at(wantedOutput).outputHash, wantedOutput};
worker.store.registerDrvOutput(newRealisation);
} else {
// If we don't have a realisation, then it must mean that something
// failed when building the resolved drv
assert(!result.success());
}
}
// This is potentially a bit fishy in terms of error reporting. Not sure
// how to do it in a cleaner way
amDone(nrFailed == 0 ? ecSuccess : ecFailed, ex);
} }
HookReply DerivationGoal::tryBuildHook() HookReply DerivationGoal::tryBuildHook()
@@ -3463,10 +3507,9 @@ void DerivationGoal::registerOutputs()
but it's fine to do in all cases. */ but it's fine to do in all cases. */
if (settings.isExperimentalFeatureEnabled("ca-derivations")) { if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
auto outputHashes = staticOutputHashes(worker.store, *drv);
for (auto& [outputName, newInfo] : infos) for (auto& [outputName, newInfo] : infos)
worker.store.registerDrvOutput(Realisation{ worker.store.registerDrvOutput(Realisation{
.id = DrvOutput{outputHashes.at(outputName), outputName}, .id = DrvOutput{initialOutputs.at(outputName).outputHash, outputName},
.outPath = newInfo.path}); .outPath = newInfo.path});
} }
} }
@@ -3790,9 +3833,8 @@ void DerivationGoal::checkPathValidity()
{ {
bool checkHash = buildMode == bmRepair; bool checkHash = buildMode == bmRepair;
for (auto & i : queryPartialDerivationOutputMap()) { for (auto & i : queryPartialDerivationOutputMap()) {
InitialOutput info { InitialOutput & info = initialOutputs.at(i.first);
.wanted = wantOutput(i.first, wantedOutputs), info.wanted = wantOutput(i.first, wantedOutputs);
};
if (i.second) { if (i.second) {
auto outputPath = *i.second; auto outputPath = *i.second;
info.known = { info.known = {
@@ -3804,7 +3846,15 @@ void DerivationGoal::checkPathValidity()
: PathStatus::Corrupt, : PathStatus::Corrupt,
}; };
} }
initialOutputs.insert_or_assign(i.first, info); if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
if (auto real = worker.store.queryRealisation(
DrvOutput{initialOutputs.at(i.first).outputHash, i.first})) {
info.known = {
.path = real->outPath,
.status = PathStatus::Valid,
};
}
}
} }
} }
+5 -1
View File
@@ -37,6 +37,7 @@ struct InitialOutputStatus {
struct InitialOutput { struct InitialOutput {
bool wanted; bool wanted;
Hash outputHash;
std::optional<InitialOutputStatus> known; std::optional<InitialOutputStatus> known;
}; };
@@ -48,6 +49,9 @@ struct DerivationGoal : public Goal
/* The path of the derivation. */ /* The path of the derivation. */
StorePath drvPath; StorePath drvPath;
/* The path of the corresponding resolved derivation */
std::optional<BasicDerivation> resolvedDrv;
/* The specific outputs that we need to build. Empty means all of /* The specific outputs that we need to build. Empty means all of
them. */ them. */
StringSet wantedOutputs; StringSet wantedOutputs;
@@ -60,7 +64,7 @@ struct DerivationGoal : public Goal
bool retrySubstitution; bool retrySubstitution;
/* The derivation stored at drvPath. */ /* The derivation stored at drvPath. */
std::unique_ptr<BasicDerivation> drv; std::unique_ptr<Derivation> drv;
std::unique_ptr<ParsedDerivation> parsedDrv; std::unique_ptr<ParsedDerivation> parsedDrv;
+7 -32
View File
@@ -745,7 +745,7 @@ static void rewriteDerivation(Store & store, BasicDerivation & drv, const String
} }
std::optional<BasicDerivation> Derivation::tryResolveUncached(Store & store) { std::optional<BasicDerivation> Derivation::tryResolve(Store & store) {
BasicDerivation resolved { *this }; BasicDerivation resolved { *this };
// Input paths that we'll want to rewrite in the derivation // Input paths that we'll want to rewrite in the derivation
@@ -756,8 +756,13 @@ std::optional<BasicDerivation> Derivation::tryResolveUncached(Store & store) {
StringSet newOutputNames; StringSet newOutputNames;
for (auto & outputName : input.second) { for (auto & outputName : input.second) {
auto actualPathOpt = inputDrvOutputs.at(outputName); auto actualPathOpt = inputDrvOutputs.at(outputName);
if (!actualPathOpt) if (!actualPathOpt) {
warn("output %s of input %s missing, aborting the resolving",
outputName,
store.printStorePath(input.first)
);
return std::nullopt; return std::nullopt;
}
auto actualPath = *actualPathOpt; auto actualPath = *actualPathOpt;
inputRewrites.emplace( inputRewrites.emplace(
downstreamPlaceholder(store, input.first, outputName), downstreamPlaceholder(store, input.first, outputName),
@@ -771,34 +776,4 @@ std::optional<BasicDerivation> Derivation::tryResolveUncached(Store & store) {
return resolved; return resolved;
} }
std::optional<BasicDerivation> Derivation::tryResolve(Store& store)
{
auto drvPath = writeDerivation(store, *this, NoRepair, false);
return Derivation::tryResolve(store, drvPath);
}
std::optional<BasicDerivation> Derivation::tryResolve(Store& store, const StorePath& drvPath)
{
// This is quite dirty and leaky, but will disappear once #4340 is merged
static Sync<std::map<StorePath, std::optional<Derivation>>> resolutionsCache;
{
auto resolutions = resolutionsCache.lock();
auto resolvedDrvIter = resolutions->find(drvPath);
if (resolvedDrvIter != resolutions->end()) {
auto & [_, resolvedDrv] = *resolvedDrvIter;
return *resolvedDrv;
}
}
/* Try resolve drv and use that path instead. */
auto drv = store.readDerivation(drvPath);
auto attempt = drv.tryResolveUncached(store);
if (!attempt)
return std::nullopt;
/* Store in memo table. */
resolutionsCache.lock()->insert_or_assign(drvPath, *attempt);
return *attempt;
}
} }
-4
View File
@@ -138,14 +138,10 @@ struct Derivation : BasicDerivation
2. Input placeholders are replaced with realized input store paths. */ 2. Input placeholders are replaced with realized input store paths. */
std::optional<BasicDerivation> tryResolve(Store & store); std::optional<BasicDerivation> tryResolve(Store & store);
static std::optional<BasicDerivation> tryResolve(Store & store, const StorePath & drvPath);
Derivation() = default; Derivation() = default;
Derivation(const BasicDerivation & bd) : BasicDerivation(bd) { } Derivation(const BasicDerivation & bd) : BasicDerivation(bd) { }
Derivation(BasicDerivation && bd) : BasicDerivation(std::move(bd)) { } Derivation(BasicDerivation && bd) : BasicDerivation(std::move(bd)) { }
private:
std::optional<BasicDerivation> tryResolveUncached(Store & store);
}; };
+3
View File
@@ -831,6 +831,9 @@ public:
command, and RegisterSetting to add new nix config settings. See the command, and RegisterSetting to add new nix config settings. See the
constructors for those types for more details. constructors for those types for more details.
Warning! These APIs are inherently unstable and may change from
release to release.
Since these files are loaded into the same address space as Nix Since these files are loaded into the same address space as Nix
itself, they must be DSOs compatible with the instance of Nix itself, they must be DSOs compatible with the instance of Nix
running at the time (i.e. compiled against the same headers, not running at the time (i.e. compiled against the same headers, not
+1 -1
View File
@@ -883,7 +883,7 @@ StorePathSet LocalStore::queryValidDerivers(const StorePath & path)
std::map<std::string, std::optional<StorePath>> std::map<std::string, std::optional<StorePath>>
LocalStore::queryDerivationOutputMapNoResolve(const StorePath& path_) LocalStore::queryPartialDerivationOutputMap(const StorePath & path_)
{ {
auto path = path_; auto path = path_;
auto outputs = retrySQLite<std::map<std::string, std::optional<StorePath>>>([&]() { auto outputs = retrySQLite<std::map<std::string, std::optional<StorePath>>>([&]() {
+1 -1
View File
@@ -127,7 +127,7 @@ public:
StorePathSet queryValidDerivers(const StorePath & path) override; StorePathSet queryValidDerivers(const StorePath & path) override;
std::map<std::string, std::optional<StorePath>> queryDerivationOutputMapNoResolve(const StorePath & path) override; std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path) override;
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override; std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override;
+1 -14
View File
@@ -366,7 +366,7 @@ bool Store::PathInfoCacheValue::isKnownNow()
return std::chrono::steady_clock::now() < time_point + ttl; return std::chrono::steady_clock::now() < time_point + ttl;
} }
std::map<std::string, std::optional<StorePath>> Store::queryDerivationOutputMapNoResolve(const StorePath & path) std::map<std::string, std::optional<StorePath>> Store::queryPartialDerivationOutputMap(const StorePath & path)
{ {
std::map<std::string, std::optional<StorePath>> outputs; std::map<std::string, std::optional<StorePath>> outputs;
auto drv = readInvalidDerivation(path); auto drv = readInvalidDerivation(path);
@@ -376,19 +376,6 @@ std::map<std::string, std::optional<StorePath>> Store::queryDerivationOutputMapN
return outputs; return outputs;
} }
std::map<std::string, std::optional<StorePath>> Store::queryPartialDerivationOutputMap(const StorePath & path)
{
if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
auto resolvedDrv = Derivation::tryResolve(*this, path);
if (resolvedDrv) {
auto resolvedDrvPath = writeDerivation(*this, *resolvedDrv, NoRepair, true);
if (isValidPath(resolvedDrvPath))
return queryDerivationOutputMapNoResolve(resolvedDrvPath);
}
}
return queryDerivationOutputMapNoResolve(path);
}
OutputPathMap Store::queryDerivationOutputMap(const StorePath & path) { OutputPathMap Store::queryDerivationOutputMap(const StorePath & path) {
auto resp = queryPartialDerivationOutputMap(path); auto resp = queryPartialDerivationOutputMap(path);
OutputPathMap result; OutputPathMap result;
-6
View File
@@ -415,12 +415,6 @@ public:
`std::nullopt`. */ `std::nullopt`. */
virtual std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path); virtual std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path);
/*
* Similar to `queryPartialDerivationOutputMap`, but doesn't try to resolve
* the derivation
*/
virtual std::map<std::string, std::optional<StorePath>> queryDerivationOutputMapNoResolve(const StorePath & path);
/* Query the mapping outputName=>outputPath for the given derivation. /* Query the mapping outputName=>outputPath for the given derivation.
Assume every output has a mapping and throw an exception otherwise. */ Assume every output has a mapping and throw an exception otherwise. */
OutputPathMap queryDerivationOutputMap(const StorePath & path); OutputPathMap queryDerivationOutputMap(const StorePath & path);
+4 -2
View File
@@ -518,9 +518,11 @@ static void main_nix_build(int argc, char * * argv)
if (counter) if (counter)
drvPrefix += fmt("-%d", counter + 1); drvPrefix += fmt("-%d", counter + 1);
auto builtOutputs = store->queryDerivationOutputMap(drvPath); auto builtOutputs = store->queryPartialDerivationOutputMap(drvPath);
auto outputPath = builtOutputs.at(outputName); auto maybeOutputPath = builtOutputs.at(outputName);
assert(maybeOutputPath);
auto outputPath = *maybeOutputPath;
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) { if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) {
std::string symlink = drvPrefix; std::string symlink = drvPrefix;
+16 -3
View File
@@ -12,11 +12,16 @@ App Installable::toApp(EvalState & state)
auto type = cursor->getAttr("type")->getString(); auto type = cursor->getAttr("type")->getString();
auto checkProgram = [&](const Path & program)
{
if (!state.store->isInStore(program))
throw Error("app program '%s' is not in the Nix store", program);
};
if (type == "app") { if (type == "app") {
auto [program, context] = cursor->getAttr("program")->getStringWithContext(); auto [program, context] = cursor->getAttr("program")->getStringWithContext();
if (!state.store->isInStore(program)) checkProgram(program);
throw Error("app program '%s' is not in the Nix store", program);
std::vector<StorePathWithOutputs> context2; std::vector<StorePathWithOutputs> context2;
for (auto & [path, name] : context) for (auto & [path, name] : context)
@@ -33,9 +38,17 @@ App Installable::toApp(EvalState & state)
auto outPath = cursor->getAttr(state.sOutPath)->getString(); auto outPath = cursor->getAttr(state.sOutPath)->getString();
auto outputName = cursor->getAttr(state.sOutputName)->getString(); auto outputName = cursor->getAttr(state.sOutputName)->getString();
auto name = cursor->getAttr(state.sName)->getString(); auto name = cursor->getAttr(state.sName)->getString();
auto aMeta = cursor->maybeGetAttr("meta");
auto aMainProgram = aMeta ? aMeta->maybeGetAttr("mainProgram") : nullptr;
auto mainProgram =
aMainProgram
? aMainProgram->getString()
: DrvName(name).name;
auto program = outPath + "/bin/" + mainProgram;
checkProgram(program);
return App { return App {
.context = { { drvPath, {outputName} } }, .context = { { drvPath, {outputName} } },
.program = outPath + "/bin/" + DrvName(name).name, .program = program,
}; };
} }
+1 -1
View File
@@ -74,7 +74,7 @@ struct CmdBundle : InstallableCommand
auto [bundlerFlakeRef, bundlerName] = parseFlakeRefWithFragment(bundler, absPath(".")); auto [bundlerFlakeRef, bundlerName] = parseFlakeRefWithFragment(bundler, absPath("."));
const flake::LockFlags lockFlags{ .writeLockFile = false }; const flake::LockFlags lockFlags{ .writeLockFile = false };
auto bundler = InstallableFlake( auto bundler = InstallableFlake(this,
evalState, std::move(bundlerFlakeRef), evalState, std::move(bundlerFlakeRef),
Strings{bundlerName == "" ? "defaultBundler" : bundlerName}, Strings{bundlerName == "" ? "defaultBundler" : bundlerName},
Strings({"bundlers."}), lockFlags); Strings({"bundlers."}), lockFlags);
+2 -1
View File
@@ -59,7 +59,7 @@ BuildEnvironment readEnvironment(const Path & path)
R"re((?:\$?"(?:[^"\\]|\\[$`"\\\n])*"))re"; R"re((?:\$?"(?:[^"\\]|\\[$`"\\\n])*"))re";
static std::string squotedStringRegex = static std::string squotedStringRegex =
R"re((?:\$?'(?:[^'\\]|\\[abeEfnrtv\\'"?])*'))re"; R"re((?:\$?(?:'(?:[^'\\]|\\[abeEfnrtv\\'"?])*'|\\')+))re";
static std::string indexedArrayRegex = static std::string indexedArrayRegex =
R"re((?:\(( *\[[0-9]+\]="(?:[^"\\]|\\.)*")*\)))re"; R"re((?:\(( *\[[0-9]+\]="(?:[^"\\]|\\.)*")*\)))re";
@@ -443,6 +443,7 @@ struct CmdDevelop : Common, MixEnvironment
auto state = getEvalState(); auto state = getEvalState();
auto bashInstallable = std::make_shared<InstallableFlake>( auto bashInstallable = std::make_shared<InstallableFlake>(
this,
state, state,
installable->nixpkgsFlakeRef(), installable->nixpkgsFlakeRef(),
Strings{"bashInteractive"}, Strings{"bashInteractive"},
+5 -3
View File
@@ -595,7 +595,7 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
auto [templateFlakeRef, templateName] = parseFlakeRefWithFragment(templateUrl, absPath(".")); auto [templateFlakeRef, templateName] = parseFlakeRefWithFragment(templateUrl, absPath("."));
auto installable = InstallableFlake( auto installable = InstallableFlake(nullptr,
evalState, std::move(templateFlakeRef), evalState, std::move(templateFlakeRef),
Strings{templateName == "" ? "defaultTemplate" : templateName}, Strings{templateName == "" ? "defaultTemplate" : templateName},
Strings(attrsPathPrefixes), lockFlags); Strings(attrsPathPrefixes), lockFlags);
@@ -880,7 +880,8 @@ struct CmdFlakeShow : FlakeCommand
|| attrPath[0] == "nixosConfigurations" || attrPath[0] == "nixosConfigurations"
|| attrPath[0] == "nixosModules" || attrPath[0] == "nixosModules"
|| attrPath[0] == "defaultApp" || attrPath[0] == "defaultApp"
|| attrPath[0] == "templates")) || attrPath[0] == "templates"
|| attrPath[0] == "overlays"))
|| ((attrPath.size() == 1 || attrPath.size() == 2) || ((attrPath.size() == 1 || attrPath.size() == 2)
&& (attrPath[0] == "checks" && (attrPath[0] == "checks"
|| attrPath[0] == "packages" || attrPath[0] == "packages"
@@ -943,7 +944,8 @@ struct CmdFlakeShow : FlakeCommand
else { else {
logger->cout("%s: %s", logger->cout("%s: %s",
headerPrefix, headerPrefix,
attrPath.size() == 1 && attrPath[0] == "overlay" ? "Nixpkgs overlay" : (attrPath.size() == 1 && attrPath[0] == "overlay")
|| (attrPath.size() == 2 && attrPath[0] == "overlays") ? "Nixpkgs overlay" :
attrPath.size() == 2 && attrPath[0] == "nixosConfigurations" ? "NixOS configuration" : attrPath.size() == 2 && attrPath[0] == "nixosConfigurations" ? "NixOS configuration" :
attrPath.size() == 2 && attrPath[0] == "nixosModules" ? "NixOS module" : attrPath.size() == 2 && attrPath[0] == "nixosModules" ? "NixOS module" :
ANSI_YELLOW "unknown" ANSI_NORMAL); ANSI_YELLOW "unknown" ANSI_NORMAL);
+22 -1
View File
@@ -17,6 +17,10 @@
#include <netdb.h> #include <netdb.h>
#include <netinet/in.h> #include <netinet/in.h>
#if __linux__
#include <sys/resource.h>
#endif
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
extern std::string chrootHelperName; extern std::string chrootHelperName;
@@ -61,6 +65,7 @@ struct NixArgs : virtual MultiCommand, virtual MixCommonArgs
bool printBuildLogs = false; bool printBuildLogs = false;
bool useNet = true; bool useNet = true;
bool refresh = false; bool refresh = false;
bool showVersion = false;
NixArgs() : MultiCommand(RegisterCommand::getCommandsFor({})), MixCommonArgs("nix") NixArgs() : MultiCommand(RegisterCommand::getCommandsFor({})), MixCommonArgs("nix")
{ {
@@ -87,7 +92,7 @@ struct NixArgs : virtual MultiCommand, virtual MixCommonArgs
addFlag({ addFlag({
.longName = "version", .longName = "version",
.description = "Show version information.", .description = "Show version information.",
.handler = {[&]() { if (!completions) printVersion(programName); }}, .handler = {[&]() { showVersion = true; }},
}); });
addFlag({ addFlag({
@@ -280,6 +285,11 @@ void mainWrapped(int argc, char * * argv)
initPlugins(); initPlugins();
if (args.showVersion) {
printVersion(programName);
return;
}
if (!args.command) if (!args.command)
throw UsageError("no subcommand specified"); throw UsageError("no subcommand specified");
@@ -319,6 +329,17 @@ void mainWrapped(int argc, char * * argv)
int main(int argc, char * * argv) int main(int argc, char * * argv)
{ {
// Increase the default stack size for the evaluator and for
// libstdc++'s std::regex.
#if __linux__
rlim_t stackSize = 64 * 1024 * 1024;
struct rlimit limit;
if (getrlimit(RLIMIT_STACK, &limit) == 0 && limit.rlim_cur < stackSize) {
limit.rlim_cur = stackSize;
setrlimit(RLIMIT_STACK, &limit);
}
#endif
return nix::handleExceptions(argv[0], [&]() { return nix::handleExceptions(argv[0], [&]() {
nix::mainWrapped(argc, argv); nix::mainWrapped(argc, argv);
}); });
+7 -1
View File
@@ -399,7 +399,13 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
Activity act(*logger, lvlChatty, actUnknown, Activity act(*logger, lvlChatty, actUnknown,
fmt("checking '%s' for updates", element.source->attrPath)); fmt("checking '%s' for updates", element.source->attrPath));
InstallableFlake installable(getEvalState(), FlakeRef(element.source->originalRef), {element.source->attrPath}, {}, lockFlags); InstallableFlake installable(
this,
getEvalState(),
FlakeRef(element.source->originalRef),
{element.source->attrPath},
{},
lockFlags);
auto [attrPath, resolvedRef, drv] = installable.toDerivation(); auto [attrPath, resolvedRef, drv] = installable.toDerivation();
+4 -3
View File
@@ -43,9 +43,10 @@ program specified by the app definition.
If *installable* evaluates to a derivation, it will try to execute the If *installable* evaluates to a derivation, it will try to execute the
program `<out>/bin/<name>`, where *out* is the primary output store program `<out>/bin/<name>`, where *out* is the primary output store
path of the derivation and *name* is the name part of the value of the path of the derivation and *name* is the `meta.mainProgram` attribute
`name` attribute of the derivation (e.g. if `name` is set to of the derivation if it exists, and otherwise the name part of the
`hello-1.10`, it will run `$out/bin/hello`). value of the `name` attribute of the derivation (e.g. if `name` is set
to `hello-1.10`, it will run `$out/bin/hello`).
# Flake output attributes # Flake output attributes
+4
View File
@@ -48,6 +48,10 @@ testCutoff () {
testGC () { testGC () {
nix-instantiate --experimental-features ca-derivations ./content-addressed.nix -A rootCA --arg seed 5 nix-instantiate --experimental-features ca-derivations ./content-addressed.nix -A rootCA --arg seed 5
nix-collect-garbage --experimental-features ca-derivations --option keep-derivations true nix-collect-garbage --experimental-features ca-derivations --option keep-derivations true
clearStore
buildAttr rootCA 1 --out-link $TEST_ROOT/rootCA
nix-collect-garbage --experimental-features ca-derivations
buildAttr rootCA 1 -j0
} }
testNixCommand () { testNixCommand () {