Compare commits

...
Author SHA1 Message Date
Eelco Dolstra 5ed3a9db6a rl-2.7.md: Fix title
(cherry picked from commit f98d76ff1a)
2022-04-05 14:14:28 +02:00
Eelco Dolstra c9afca59e8 Merge pull request #6297 from NixOS/backport-6296-to-2.7-maintenance
[Backport 2.7-maintenance] Don't hide repeated values while generating manifest.nix
2022-03-22 21:01:24 +01:00
Eelco Dolstra 0a26f9ae4a Don't hide repeated values while generating manifest.nix
Fixes #6243.

(cherry picked from commit a0259a21a4)
2022-03-22 13:09:47 +00:00
Eelco Dolstra fc553fb632 printValue(): <REPEAT> -> «repeated»
This ensures that it doesn't get parsed as a valid Nix expression.

(cherry picked from commit 732296ddc0)
2022-03-22 13:09:47 +00:00
Eelco Dolstra ffe155abd3 Mark official release 2022-03-07 20:11:22 +01:00
Eelco Dolstra ad7c99ef20 Move rl-next.md to rl-2.7.md 2022-03-07 20:10:18 +01:00
Eelco Dolstra 504e3b2a8f Style 2022-03-07 20:07:43 +01:00
Eelco Dolstra fa614fac7f Merge branch 'GuillaumeDesforges/issue6192' of https://github.com/GuillaumeDesforges/nix 2022-03-07 19:58:06 +01:00
Eelco Dolstra 0123b9aec8 Tweak release notes 2022-03-07 19:56:10 +01:00
Eelco Dolstra 30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
Guillaume Desforges 436c6e900f Explicit error in flake init/new when not in store 2022-03-07 17:57:52 +01:00
regnat dcf4f77fac Merge or-suggestions.hh into suggestions.hh
No real need for keeping a separate header for such a simple class.

This requires changing a bit `OrSuggestions<T>::operator*` to not throw
an `Error` to prevent a cyclic dependency. But since this error is only
thrown on programmer error, we can replace the whole method by a direct
call to `std::get` which will raise its own assertion if needs be.
2022-03-07 17:49:02 +01:00
Eelco Dolstra 60a68a4fee Merge pull request #6173 from hercules-ci/fix-mkString-for-empty-string-view
Fix `mkString` for empty `string_view`
2022-03-07 17:10:41 +01:00
Robert Hensing da260f579d dupStringWithLen -> makeImmutableString
Refactor the `size == 0` logic into a new helper function that
replaces dupStringWithLen.
The name had to change, because unlike a `dup`-function, it does
not always allocate a new string.
2022-03-07 16:09:12 +01:00
Robert Hensing bbf55383e7 Value::mkPath: Avoid potential crash from null string_view 2022-03-07 16:09:12 +01:00
Robert Hensing 1b978596b5 Value::mkString: Avoid crash from null string_view 2022-03-07 16:09:12 +01:00
Eelco Dolstra c28e2b1b29 Tweak release notes 2022-03-07 11:30:40 +01:00
regnat 313bbc07a8 Implement operator<< for Suggestions
That way there’s no need to explicitely convert it to a string when
printing it
2022-03-07 10:09:10 +01:00
regnat fd45d85b41 Move OrSuggestions to its own header
Prevents a recursive inclusion
2022-03-07 10:09:10 +01:00
regnat b44cebd1fd Add a release-notes entry for the cli suggestions 2022-03-07 10:09:10 +01:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat 98e361ad4c Also display suggestions for the commands using the eval cache
Make `nix build .#nix-armv8l-linux` work for example
2022-03-07 10:09:10 +01:00
regnat 2405bbbb5e Add some tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat c0792b1546 Implement a suggestions mechanism
Each `Error` class now includes a set of suggestions, and these are printed by
the top-level handler.
2022-03-07 10:09:09 +01:00
Théophane Hufschmitt a2ace54fe4 Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt 860016bcbf Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Théophane Hufschmitt 2051c0f409 Merge pull request #6208 from sielicki/fix-url-github-oauth
Point to new github oauth docs url
2022-03-07 08:54:50 +01:00
Nicholas Sielicki 314852a10e Point to new github oauth docs url
Previous URL was 404'ing.
2022-03-06 17:01:14 -06:00
Maximilian Bosch 697d1dac01 tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra b09baf690b Merge pull request #6188 from obsidiansystems/store-gc-subclass
Factor out a `GcStore` interface
2022-03-03 20:53:10 +01:00
John Ericson 6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
Eelco Dolstra 391f4fcabe Merge pull request #6201 from edolstra/print-value
printValue(): Don't show repeated values
2022-03-03 14:35:16 +01:00
Eelco Dolstra e9c04c3351 Be more aggressive in hiding repeated values
We now memoize on Bindings / list element vectors rather than Values,
so that e.g. two Values that point to the same Bindings will be
printed only once.
2022-03-03 13:33:34 +01:00
Eelco Dolstra ecff9d969a printValue(): Don't show repeated values
Fixes #6157.
2022-03-03 13:18:23 +01:00
Eelco Dolstra 6097790863 Fix segfault in headerCallback()
https://hydra.nixos.org/build/168594664
2022-03-03 11:11:16 +01:00
Eelco Dolstra 885d709393 Merge pull request #6198 from edolstra/coerce-store-path
Add EvalState::coerceToStorePath() helper
2022-03-03 10:02:51 +01:00
Eelco DolstraandJohn Ericson a7c835e9cb Use C++11-style initializer
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-03 10:02:11 +01:00
Eelco Dolstra b55d79728c Add EvalState::coerceToStorePath() helper
This is useful whenever we want to evaluate something to a store path
(e.g. in get-drvs.cc).

Extracted from the lazy-trees branch (where we can require that a
store path must come from a store source tree accessor).
2022-03-02 23:58:58 +01:00
Eelco Dolstra 3e3d0711d4 Merge pull request #6197 from edolstra/nix-profile-ca
nix profile: Support CA derivations
2022-03-02 23:48:52 +01:00
Eelco Dolstra d4538034b7 nix profile test: Restart daemon
Fixes

  nix-daemon: src/libstore/sqlite.cc:97: nix::SQLiteStmt::Use::Use(nix::SQLiteStmt&): Assertion `stmt.stmt' failed.

which happens because the daemon doesn't properly handle the case
where ca-derivations isn't enabled at daemon startup.
2022-03-02 23:08:49 +01:00
Eelco Dolstra 3a3821bcd7 Remove obsolete todo 2022-03-02 22:22:55 +01:00
Eelco Dolstra b0d65b3d11 Silence kill output 2022-03-02 22:22:23 +01:00
Eelco Dolstra d2586188fe tests/common.sh.in: Add enableFeatures helper 2022-03-02 21:48:25 +01:00
Eelco Dolstra 5d208cbe41 mk/run_test.sh: Add missing backslash 2022-03-02 21:36:46 +01:00
Eelco Dolstra f9375778ae nix profile: Add a test for non-flake packages 2022-03-02 20:56:40 +01:00
Eelco Dolstra 161f798aa1 nix profile: Support CA derivations 2022-03-02 20:38:51 +01:00
Eelco Dolstra 54888b92de Move installables-related operations 2022-03-02 19:19:51 +01:00
Eelco Dolstra b2da2a22c6 Merge pull request #6194 from edolstra/nix-profile
Add basic tests for 'nix profile'
2022-03-02 16:07:00 +01:00
Eelco Dolstra 5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra 1aca6b9f17 Merge pull request #4345 from bjornfor/installer-configurable-uid-gid
installer: allow overriding nix user GID and UIDs
2022-03-02 12:38:52 +01:00
Eelco Dolstra b39ef07414 Style 2022-03-02 11:46:15 +01:00
Eelco Dolstra c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
Eelco Dolstra 010ffc31f8 Remove stray debug line 2022-03-02 11:20:32 +01:00
Eelco Dolstra 03df331c3f Merge pull request #6189 from obsidiansystems/build-result-header
Move `BuildResult` defintion to its own header
2022-03-02 08:47:51 +01:00
Eelco Dolstra 3c66839b9d Merge pull request #6190 from andersk/bel
filterANSIEscapes: Ignore BEL character
2022-03-02 08:46:19 +01:00
Anders Kaseorg b5cd3e2d5c filterANSIEscapes: Ignore BEL character
GCC is not as good at music as it seems to think it is.  Fixes #4546.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-03-01 15:35:42 -08:00
John Ericson e862833ec6 Move BuildResult defintion to its own header
Just like we did for `ValidPathInfo` in
d92d4f85a5.
2022-03-01 19:43:07 +00:00
Bjørn Forsman f4d57aa490 installer: allow overriding nix user GID and UIDs
Needed to resolve conflict in case the default GID and UIDs are in use.
2022-03-01 19:17:41 +01:00
Eelco Dolstra 9087fe0760 Merge pull request #6187 from thufschmitt/allow-the-darwin-tests-to-be-flaky
Acknowledge that the macOS tests are flaky
2022-03-01 19:11:36 +01:00
regnat c511134a94 Acknowledge that the macOS tests are flaky
Restart the tests (at most once) on `unexpected EOF` errors.

This is truly ugly, but might prevent half of the CI runs to fail
because of https://github.com/NixOS/nix/issues/3605
2022-03-01 15:25:05 +01:00
Eelco Dolstra 0cb5af5000 Merge pull request #6185 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Use isValidPath, add comment
2022-03-01 13:15:42 +01:00
Robert Hensing b6deca7c0d fetchTree: Use isValidPath, add comment 2022-03-01 12:11:10 +01:00
Eelco Dolstra d974d2ad59 fetch{url,Tarball}: Remove 'narHash' attribute
This was introduced in #6174. However fetch{url,Tarball} are legacy
and we shouldn't have an undocumented attribute that does the same
thing as one that already exists ('sha256').
2022-03-01 11:30:26 +01:00
Eelco Dolstra 9ab81a9d38 Merge pull request #6183 from obsidiansystems/sort-config
Move some stuff from `Settings` to a new `FetchSettings`.
2022-03-01 10:52:55 +01:00
Théophane Hufschmitt 983c991652 Merge pull request #6174 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Do not re-fetch paths already present + refactor
2022-03-01 10:29:32 +01:00
John Ericson ea71da395f Move some stuff from Settings to a new FetchSettings.
Starting work on #5638

The exact boundary between `FetchSettings` and `EvalSettings` is not
clear to me, but that's fine. First lets clean out `libstore`, and then
worry about what, if anything, should be the separation between those
two.
2022-03-01 01:39:25 +00:00
Robert Hensing ee019d0afc Add EvalState::allowAndSetStorePathString helper
This switches addPath from `printStorePath` to `toRealPath`.
2022-02-28 21:37:49 +01:00
Robert Hensing 158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
Théophane Hufschmitt 1c985428c4 Merge pull request #6179 from NixOS/properly-start-the-daemon-in-tests
tests: Fix the start of the daemon
2022-02-28 19:23:50 +01:00
Maximilian Bosch 6a8f1b548f logging.hh: json.hpp -> json_fwd.hpp 2022-02-28 17:27:52 +01:00
Maximilian Bosch 102cb39086 libstore/build: add a few explanatory comments; simplify 2022-02-28 17:27:52 +01:00
Maximilian Bosch 287642f132 tests: implement test for nix log with ssh-ng:// builds
A few notes:

* The `echo hi` is needed to make sure that a file that can be read by
  `nix log` is properly created (i.e. some output is needed). This is
  known and to be fixed in #6051.
* We explicitly ignore the floating-CA case here: the `$out` of `input3`
  depends on `$out` of `input2`. This means that there are actually two
  derivations - I assume that this is because at eval time (i.e.
  `nix-instantiate -A`) the hash of `input2` isn't known yet and the
  other .drv is created as soon as `input2` was built. This is another
  issue on its own, so we ignore the case here explicitly.
2022-02-28 17:27:52 +01:00
Maximilian Bosch cd92ea5885 libstore/derivation-goal: avoid double-parsing of JSON messages
To avoid that JSON messages are parsed twice in case of
remote builds with `ssh-ng://`, I split up the original
`handleJSONLogMessage` into three parts:

* `parseJSONMessage(const std::string&)` checks if it's a message in the
  form of `@nix {...}` and tries to parse it (and prints an error if the
  parsing fails).
* `handleJSONLogMessage(nlohmann::json&, ...)` reads the fields from the
  message and passes them to the logger.
* `handleJSONLogMessage(const std::string&, ...)` behaves as before, but
  uses the two functions mentioned above as implementation.

In case of `ssh-ng://`-logs the first two methods are invoked manually.
2022-02-28 17:27:52 +01:00
Maximilian Bosch 7a04839ea5 ssh-ng: also store build logs to make them accessible by nix log
Right now when building a derivation remotely via

    $ nix build -j0 -f . hello -L --builders 'ssh://builder'

it's possible later to read through the entire build-log by running
`nix log -f . hello`. This isn't possible however when using `ssh-ng`
rather than `ssh`.

The reason for that is that there are two different ways to transfer
logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng`
respectively):

* `ssh://` receives its logs from the fd pointing to `builderOut`. This
  is directly passed to the "log-sink" (and to the logger on each `\n`),
  hence `nix log` works here.
* `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data
  in here}`) and passes it directly to the logger without doing anything
  with the `logSink`. However it's certainly possible to extract
  log-lines from this format as these have their own message-type in the
  JSON payload (i.e. `resBuildLogLine`).

  This is basically what I changed in this patch: if the code-path for
  `builderOut` is not reached and a `logSink` is initialized, the
  message was successfully processed by the JSON logger (i.e. it's in
  the expected format) and the line is of the expected type (i.e.
  `resBuildLogLine`), the line will be written to the log-sink as well.

Closes #5079
2022-02-28 17:27:52 +01:00
regnat b8f8aef9d3 tests: Fix the start of the daemon
- Make sure that it starts even without the `nix-command` xp feature
- Fail if it doesn’t manage to start

This fixes a 30s wait for every test in `init.sh` as the daemon couldn’t
start, but the code was just waiting 30s and continuing as if everything
was all right.
2022-02-28 17:00:31 +01:00
99 changed files with 1385 additions and 680 deletions

No files matched your search

+1 -1
View File
@@ -71,7 +71,7 @@
- [Hacking](contributing/hacking.md)
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
- [Release 2.7 (2022-03-07)](release-notes/rl-2.7.md)
- [Release 2.6 (2022-01-24)](release-notes/rl-2.6.md)
- [Release 2.5 (2021-12-13)](release-notes/rl-2.5.md)
- [Release 2.4 (2021-11-01)](release-notes/rl-2.4.md)
+33
View File
@@ -0,0 +1,33 @@
# Release 2.7 (2022-03-07)
* Nix will now make some helpful suggestions when you mistype
something on the command line. For instance, if you type `nix build
nixpkgs#thunderbrd`, it will suggest `thunderbird`.
* A number of "default" flake output attributes have been
renamed. These are:
* `defaultPackage.<system>` → `packages.<system>.default`
* `defaultApps.<system>` → `apps.<system>.default`
* `defaultTemplate` → `templates.default`
* `defaultBundler.<system>` → `bundlers.<system>.default`
* `overlay` → `overlays.default`
* `devShell.<system>` → `devShells.<system>.default`
The old flake output attributes still work, but `nix flake check`
will warn about them.
* Breaking API change: `nix bundle` now supports bundlers of the form
`bundler.<system>.<name>= derivation: another-derivation;`. This
supports additional functionality to inspect evaluation information
during bundling. A new
[repository](https://github.com/NixOS/bundlers) has various bundlers
implemented.
* `nix store ping` now reports the version of the remote Nix daemon.
* `nix flake {init,new}` now display information about which files have been
created.
* Templates can now define a `welcomeText` attribute, which is printed out by
`nix flake {init,new} --template <template>`.
-27
View File
@@ -1,28 +1 @@
# Release X.Y (202?-??-??)
* A number of "default" flake output attributes have been
renamed. These are:
* `defaultPackage.<system>` → `packages.<system>.default`
* `defaultApps.<system>` → `apps.<system>.default`
* `defaultTemplate` → `templates.default`
* `defaultBundler.<system>` → `bundlers.<system>.default`
* `overlay` → `overlays.default`
* `devShell.<system>` → `devShells.<system>.default`
The old flake output attributes still work, but `nix flake check`
will warn about them.
* `nix bundle` breaking API change now supports bundlers of the form
`bundler.<system>.<name>= derivation: another-derivation;`. This supports
additional functionality to inspect evaluation information during bundling. A
new [repository](https://github.com/NixOS/bundlers) has various bundlers
implemented.
* `nix store ping` now reports the version of the remote Nix daemon.
* `nix flake {init,new}` now display information about which files have been
created.
* Templates can now define a `welcomeText` attribute, which is printed out by
`nix flake {init,new} --template <template>`.
+1 -1
View File
@@ -15,7 +15,7 @@
then ""
else "pre${builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")}_${self.shortRev or "dirty"}";
officialRelease = false;
officialRelease = true;
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ];
linuxSystems = linux64BitSystems ++ [ "i686-linux" ];
+21 -3
View File
@@ -14,9 +14,27 @@ if [ -t 1 ]; then
yellow=""
normal=""
fi
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
run_test () {
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
}
run_test "$1"
# Hack: Retry the test if it fails with “unexpected EOF reading a line” as these
# appear randomly without anyone knowing why.
# See https://github.com/NixOS/nix/issues/3605 for more info
if [[ $status -ne 0 && $status -ne 99 && \
"$(uname)" == "Darwin" && \
"$log" =~ "unexpected EOF reading a line" \
]]; then
echo "$post_run_msg [${yellow}FAIL$normal] (possibly flaky, so will be retried)"
echo "$log" | sed 's/^/ /'
run_test "$1"
fi
if [ $status -eq 0 ]; then
echo "$post_run_msg [${green}PASS$normal]"
elif [ $status -eq 99 ]; then
+2 -2
View File
@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="30000"
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# darwin installer needs to override these
NIX_FIRST_BUILD_UID="30001"
NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
+1
View File
@@ -14,6 +14,7 @@
#include "pathlocks.hh"
#include "globals.hh"
#include "serialise.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "derivations.hh"
#include "local-store.hh"
+1 -1
View File
@@ -153,7 +153,7 @@ void BuiltPathsCommand::run(ref<Store> store)
for (auto & p : store->queryAllValidPaths())
paths.push_back(BuiltPath::Opaque{p});
} else {
paths = toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
paths = Installable::toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) {
// XXX: This only computes the store path closure, ignoring
// intermediate realisations
-54
View File
@@ -5,7 +5,6 @@
#include "common-eval-args.hh"
#include "path.hh"
#include "flake/lockfile.hh"
#include "store-api.hh"
#include <optional>
@@ -82,14 +81,6 @@ struct MixFlakeOptions : virtual Args, EvalCommand
{ return {}; }
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
};
struct SourceExprCommand : virtual Args, MixFlakeOptions
{
std::optional<Path> file;
@@ -113,19 +104,6 @@ struct SourceExprCommand : virtual Args, MixFlakeOptions
void completeInstallable(std::string_view prefix);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* A command that operates on a list of "installables", which can be
store paths, attribute paths, Nix expressions, etc. */
struct InstallablesCommand : virtual Args, SourceExprCommand
@@ -238,38 +216,6 @@ static RegisterCommand registerCommand2(std::vector<std::string> && name)
return RegisterCommand(std::move(name), [](){ return make_ref<T>(); });
}
BuiltPaths build(
ref<Store> evalStore,
ref<Store> store, Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
/* Helper function to generate args that invoke $EDITOR on
filename:lineno. */
Strings editorFor(const Pos & pos);
+34 -22
View File
@@ -272,9 +272,9 @@ void completeFlakeRefWithFragment(
auto attr = root->findAlongAttrPath(attrPath);
if (!attr) continue;
for (auto & attr2 : attr->getAttrs()) {
for (auto & attr2 : (*attr)->getAttrs()) {
if (hasPrefix(attr2, lastAttr)) {
auto attrPath2 = attr->getAttrPath(attr2);
auto attrPath2 = (*attr)->getAttrPath(attr2);
/* Strip the attrpath prefix. */
attrPath2.erase(attrPath2.begin(), attrPath2.begin() + attrPathPrefix.size());
completions->add(flakeRefS + "#" + concatStringsSep(".", attrPath2));
@@ -468,11 +468,10 @@ std::vector<InstallableValue::DerivationInfo> InstallableAttrPath::toDerivations
std::vector<DerivationInfo> res;
for (auto & drvInfo : drvInfos) {
res.push_back({
state->store->parseStorePath(drvInfo.queryDrvPath()),
state->store->maybeParseStorePath(drvInfo.queryOutPath()),
drvInfo.queryOutputName()
});
auto drvPath = drvInfo.queryDrvPath();
if (!drvPath)
throw Error("'%s' is not a derivation", what());
res.push_back({ *drvPath, drvInfo.queryOutputName() });
}
return res;
@@ -569,31 +568,37 @@ std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableF
auto cache = openEvalCache(*state, lockedFlake);
auto root = cache->getRoot();
Suggestions suggestions;
for (auto & attrPath : getActualAttrPaths()) {
debug("trying flake output attribute '%s'", attrPath);
auto attr = root->findAlongAttrPath(
auto attrOrSuggestions = root->findAlongAttrPath(
parseAttrPath(*state, attrPath),
true
);
if (!attr) continue;
if (!attrOrSuggestions) {
suggestions += attrOrSuggestions.getSuggestions();
continue;
}
auto attr = *attrOrSuggestions;
if (!attr->isDerivation())
throw Error("flake output attribute '%s' is not a derivation", attrPath);
auto drvPath = attr->forceDerivation();
auto drvInfo = DerivationInfo{
auto drvInfo = DerivationInfo {
std::move(drvPath),
state->store->maybeParseStorePath(attr->getAttr(state->sOutPath)->getString()),
attr->getAttr(state->sOutputName)->getString()
};
return {attrPath, lockedFlake->flake.lockedRef, std::move(drvInfo)};
}
throw Error("flake '%s' does not provide attribute %s",
throw Error(suggestions, "flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
}
@@ -612,17 +617,24 @@ std::pair<Value *, Pos> InstallableFlake::toValue(EvalState & state)
auto emptyArgs = state.allocBindings(0);
Suggestions suggestions;
for (auto & attrPath : getActualAttrPaths()) {
try {
auto [v, pos] = findAlongAttrPath(state, attrPath, *emptyArgs, *vOutputs);
state.forceValue(*v, pos);
return {v, pos};
} catch (AttrPathNotFound & e) {
suggestions += e.info().suggestions;
}
}
throw Error("flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
throw Error(
suggestions,
"flake '%s' does not provide attribute %s",
flakeRef,
showAttrPaths(getActualAttrPaths())
);
}
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
@@ -637,7 +649,7 @@ InstallableFlake::getCursors(EvalState & state)
for (auto & attrPath : getActualAttrPaths()) {
auto attr = root->findAlongAttrPath(parseAttrPath(state, attrPath));
if (attr) res.push_back({attr, attrPath});
if (attr) res.push_back({*attr, attrPath});
}
return res;
@@ -787,7 +799,7 @@ BuiltPaths getBuiltPaths(ref<Store> evalStore, ref<Store> store, const DerivedPa
return res;
}
BuiltPaths build(
BuiltPaths Installable::build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -812,7 +824,7 @@ BuiltPaths build(
return getBuiltPaths(evalStore, store, pathsToBuild);
}
BuiltPaths toBuiltPaths(
BuiltPaths Installable::toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -820,19 +832,19 @@ BuiltPaths toBuiltPaths(
const std::vector<std::shared_ptr<Installable>> & installables)
{
if (operateOn == OperateOn::Output)
return build(evalStore, store, mode, installables);
return Installable::build(evalStore, store, mode, installables);
else {
if (mode == Realise::Nothing)
settings.readOnlyMode = true;
BuiltPaths res;
for (auto & drvPath : toDerivations(store, installables, true))
for (auto & drvPath : Installable::toDerivations(store, installables, true))
res.push_back(BuiltPath::Opaque{drvPath});
return res;
}
}
StorePathSet toStorePaths(
StorePathSet Installable::toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -846,7 +858,7 @@ StorePathSet toStorePaths(
return outPaths;
}
StorePath toStorePath(
StorePath Installable::toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -860,7 +872,7 @@ StorePath toStorePath(
return *paths.begin();
}
StorePathSet toDerivations(
StorePathSet Installable::toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver)
+60 -1
View File
@@ -5,6 +5,7 @@
#include "path-with-outputs.hh"
#include "derived-path.hh"
#include "eval.hh"
#include "store-api.hh"
#include "flake/flake.hh"
#include <optional>
@@ -29,6 +30,27 @@ struct UnresolvedApp
App resolve(ref<Store> evalStore, ref<Store> store);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
};
struct Installable
{
virtual ~Installable() { }
@@ -68,6 +90,39 @@ struct Installable
{
return FlakeRef::fromAttrs({{"type","indirect"}, {"id", "nixpkgs"}});
}
static BuiltPaths build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
static std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
static StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
static std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
static BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
};
struct InstallableValue : Installable
@@ -79,7 +134,6 @@ struct InstallableValue : Installable
struct DerivationInfo
{
StorePath drvPath;
std::optional<StorePath> outPath;
std::string outputName;
};
@@ -131,4 +185,9 @@ ref<eval_cache::EvalCache> openEvalCache(
EvalState & state,
std::shared_ptr<flake::LockedFlake> lockedFlake);
BuiltPaths getBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
const DerivedPaths & hopefullyBuiltPaths);
}
+8 -2
View File
@@ -74,8 +74,14 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const std::string &
throw Error("empty attribute name in selection path '%1%'", attrPath);
Bindings::iterator a = v->attrs->find(state.symbols.create(attr));
if (a == v->attrs->end())
throw AttrPathNotFound("attribute '%1%' in selection path '%2%' not found", attr, attrPath);
if (a == v->attrs->end()) {
std::set<std::string> attrNames;
for (auto & attr : *v->attrs)
attrNames.insert(attr.name);
auto suggestions = Suggestions::bestMatches(attrNames, attr);
throw AttrPathNotFound(suggestions, "attribute '%1%' in selection path '%2%' not found", attr, attrPath);
}
v = &*a->value;
pos = *a->pos;
}
+27 -8
View File
@@ -406,6 +406,16 @@ Value & AttrCursor::forceValue()
return v;
}
Suggestions AttrCursor::getSuggestionsForAttr(Symbol name)
{
auto attrNames = getAttrs();
std::set<std::string> strAttrNames;
for (auto & name : attrNames)
strAttrNames.insert(std::string(name));
return Suggestions::bestMatches(strAttrNames, name);
}
std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErrors)
{
if (root->db) {
@@ -446,6 +456,11 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
return nullptr;
//throw TypeError("'%s' is not an attribute set", getAttrPathStr());
for (auto & attr : *v.attrs) {
if (root->db)
root->db->setPlaceholder({cachedValue->first, attr.name});
}
auto attr = v.attrs->get(name);
if (!attr) {
@@ -464,7 +479,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
cachedValue2 = {root->db->setPlaceholder({cachedValue->first, name}), placeholder_t()};
}
return std::make_shared<AttrCursor>(
return make_ref<AttrCursor>(
root, std::make_pair(shared_from_this(), name), attr->value, std::move(cachedValue2));
}
@@ -473,27 +488,31 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(std::string_view name)
return maybeGetAttr(root->state.symbols.create(name));
}
std::shared_ptr<AttrCursor> AttrCursor::getAttr(Symbol name, bool forceErrors)
ref<AttrCursor> AttrCursor::getAttr(Symbol name, bool forceErrors)
{
auto p = maybeGetAttr(name, forceErrors);
if (!p)
throw Error("attribute '%s' does not exist", getAttrPathStr(name));
return p;
return ref(p);
}
std::shared_ptr<AttrCursor> AttrCursor::getAttr(std::string_view name)
ref<AttrCursor> AttrCursor::getAttr(std::string_view name)
{
return getAttr(root->state.symbols.create(name));
}
std::shared_ptr<AttrCursor> AttrCursor::findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force)
OrSuggestions<ref<AttrCursor>> AttrCursor::findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force)
{
auto res = shared_from_this();
for (auto & attr : attrPath) {
res = res->maybeGetAttr(attr, force);
if (!res) return {};
auto child = res->maybeGetAttr(attr, force);
if (!child) {
auto suggestions = res->getSuggestionsForAttr(attr);
return OrSuggestions<ref<AttrCursor>>::failed(suggestions);
}
res = child;
}
return res;
return ref(res);
}
std::string AttrCursor::getString()
+5 -3
View File
@@ -94,15 +94,17 @@ public:
std::string getAttrPathStr(Symbol name) const;
Suggestions getSuggestionsForAttr(Symbol name);
std::shared_ptr<AttrCursor> maybeGetAttr(Symbol name, bool forceErrors = false);
std::shared_ptr<AttrCursor> maybeGetAttr(std::string_view name);
std::shared_ptr<AttrCursor> getAttr(Symbol name, bool forceErrors = false);
ref<AttrCursor> getAttr(Symbol name, bool forceErrors = false);
std::shared_ptr<AttrCursor> getAttr(std::string_view name);
ref<AttrCursor> getAttr(std::string_view name);
std::shared_ptr<AttrCursor> findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force = false);
OrSuggestions<ref<AttrCursor>> findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force = false);
std::string getString();
+67 -30
View File
@@ -63,9 +63,15 @@ static char * dupString(const char * s)
}
static char * dupStringWithLen(const char * s, size_t size)
// When there's no need to write to the string, we can optimize away empty
// string allocations.
// This function handles makeImmutableStringWithLen(null, 0) by returning the
// empty string.
static const char * makeImmutableStringWithLen(const char * s, size_t size)
{
char * t;
if (size == 0)
return "";
#if HAVE_BOEHMGC
t = GC_STRNDUP(s, size);
#else
@@ -75,6 +81,10 @@ static char * dupStringWithLen(const char * s, size_t size)
return t;
}
static inline const char * makeImmutableString(std::string_view s) {
return makeImmutableStringWithLen(s.data(), s.size());
}
RootValue allocRootValue(Value * v)
{
@@ -86,25 +96,20 @@ RootValue allocRootValue(Value * v)
}
void printValue(std::ostream & str, std::set<const Value *> & active, const Value & v)
void Value::print(std::ostream & str, std::set<const void *> * seen) const
{
checkInterrupt();
if (!active.insert(&v).second) {
str << "<CYCLE>";
return;
}
switch (v.internalType) {
switch (internalType) {
case tInt:
str << v.integer;
str << integer;
break;
case tBool:
str << (v.boolean ? "true" : "false");
str << (boolean ? "true" : "false");
break;
case tString:
str << "\"";
for (const char * i = v.string.s; *i; i++)
for (const char * i = string.s; *i; i++)
if (*i == '\"' || *i == '\\') str << "\\" << *i;
else if (*i == '\n') str << "\\n";
else if (*i == '\r') str << "\\r";
@@ -114,30 +119,38 @@ void printValue(std::ostream & str, std::set<const Value *> & active, const Valu
str << "\"";
break;
case tPath:
str << v.path; // !!! escaping?
str << path; // !!! escaping?
break;
case tNull:
str << "null";
break;
case tAttrs: {
str << "{ ";
for (auto & i : v.attrs->lexicographicOrder()) {
str << i->name << " = ";
printValue(str, active, *i->value);
str << "; ";
if (seen && !attrs->empty() && !seen->insert(attrs).second)
str << "«repeated»";
else {
str << "{ ";
for (auto & i : attrs->lexicographicOrder()) {
str << i->name << " = ";
i->value->print(str, seen);
str << "; ";
}
str << "}";
}
str << "}";
break;
}
case tList1:
case tList2:
case tListN:
str << "[ ";
for (auto v2 : v.listItems()) {
printValue(str, active, *v2);
str << " ";
if (seen && listSize() && !seen->insert(listElems()).second)
str << "«repeated»";
else {
str << "[ ";
for (auto v2 : listItems()) {
v2->print(str, seen);
str << " ";
}
str << "]";
}
str << "]";
break;
case tThunk:
case tApp:
@@ -153,23 +166,27 @@ void printValue(std::ostream & str, std::set<const Value *> & active, const Valu
str << "<PRIMOP-APP>";
break;
case tExternal:
str << *v.external;
str << *external;
break;
case tFloat:
str << v.fpoint;
str << fpoint;
break;
default:
abort();
}
}
active.erase(&v);
void Value::print(std::ostream & str, bool showRepeated) const
{
std::set<const void *> seen;
print(str, showRepeated ? nullptr : &seen);
}
std::ostream & operator << (std::ostream & str, const Value & v)
{
std::set<const Value *> active;
printValue(str, active, v);
v.print(str, false);
return str;
}
@@ -517,6 +534,14 @@ void EvalState::allowPath(const StorePath & storePath)
allowedPaths->insert(store->toRealPath(storePath));
}
void EvalState::allowAndSetStorePathString(const StorePath &storePath, Value & v)
{
allowPath(storePath);
auto path = store->printStorePath(storePath);
v.mkString(path, PathSet({path}));
}
Path EvalState::checkSourcePath(const Path & path_)
{
if (!allowedPaths) return path_;
@@ -796,7 +821,7 @@ LocalNoInline(void addErrorTrace(Error & e, const Pos & pos, const char * s, con
void Value::mkString(std::string_view s)
{
mkString(dupStringWithLen(s.data(), s.size()));
mkString(makeImmutableString(s));
}
@@ -827,7 +852,7 @@ void Value::mkStringMove(const char * s, const PathSet & context)
void Value::mkPath(std::string_view s)
{
mkPath(dupStringWithLen(s.data(), s.size()));
mkPath(makeImmutableString(s));
}
@@ -2050,6 +2075,18 @@ Path EvalState::coerceToPath(const Pos & pos, Value & v, PathSet & context)
}
StorePath EvalState::coerceToStorePath(const Pos & pos, Value & v, PathSet & context)
{
auto path = coerceToString(pos, v, context, false, false).toOwned();
if (auto storePath = store->maybeParseStorePath(path))
return *storePath;
throw EvalError({
.msg = hintfmt("path '%1%' is not in the Nix store", path),
.errPos = pos
});
}
bool EvalState::eqValues(Value & v1, Value & v2)
{
forceValue(v1, noPos);
+6
View File
@@ -161,6 +161,9 @@ public:
the real store path if `store` is a chroot store. */
void allowPath(const StorePath & storePath);
/* Allow access to a store path and return it as a string. */
void allowAndSetStorePathString(const StorePath & storePath, Value & v);
/* Check whether access to a path is allowed and throw an error if
not. Otherwise return the canonicalised path. */
Path checkSourcePath(const Path & path);
@@ -269,6 +272,9 @@ public:
path. Nothing is copied to the store. */
Path coerceToPath(const Pos & pos, Value & v, PathSet & context);
/* Like coerceToPath, but the result must be a store path. */
StorePath coerceToStorePath(const Pos & pos, Value & v, PathSet & context);
public:
/* The base environment, containing the builtin functions and
+2 -1
View File
@@ -1,5 +1,6 @@
#include "flake.hh"
#include "globals.hh"
#include "fetch-settings.hh"
#include <nlohmann/json.hpp>
@@ -53,7 +54,7 @@ void ConfigFile::apply()
auto trustedList = readTrustedList();
bool trusted = false;
if (nix::settings.acceptFlakeConfig){
if (nix::fetchSettings.acceptFlakeConfig){
trusted = true;
} else if (auto saved = get(get(trustedList, name).value_or(std::map<std::string, bool>()), valueS)) {
trusted = *saved;
+5 -4
View File
@@ -6,6 +6,7 @@
#include "store-api.hh"
#include "fetchers.hh"
#include "finally.hh"
#include "fetch-settings.hh"
namespace nix {
@@ -315,7 +316,7 @@ LockedFlake lockFlake(
FlakeCache flakeCache;
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto useRegistries = lockFlags.useRegistries.value_or(fetchSettings.useRegistries);
auto flake = getFlake(state, topRef, useRegistries, flakeCache);
@@ -591,7 +592,7 @@ LockedFlake lockFlake(
if (lockFlags.writeLockFile) {
if (auto sourcePath = topRef.input.getSourcePath()) {
if (!newLockFile.isImmutable()) {
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("will not write lock file of flake '%s' because it has a mutable input", topRef);
} else {
if (!lockFlags.updateLockFile)
@@ -618,7 +619,7 @@ LockedFlake lockFlake(
if (lockFlags.commitLockFile) {
std::string cm;
cm = settings.commitLockFileSummary.get();
cm = fetchSettings.commitLockFileSummary.get();
if (cm == "") {
cm = fmt("%s: %s", relPath, lockFileExists ? "Update" : "Add");
@@ -716,7 +717,7 @@ static void prim_getFlake(EvalState & state, const Pos & pos, Value * * args, Va
lockFlake(state, flakeRef,
LockFlags {
.updateLockFile = false,
.useRegistries = !evalSettings.pureEval && settings.useRegistries,
.useRegistries = !evalSettings.pureEval && fetchSettings.useRegistries,
.allowMutable = !evalSettings.pureEval,
}),
v);
+3 -3
View File
@@ -98,7 +98,7 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
if (std::regex_match(url, match, flakeRegex)) {
auto parsedURL = ParsedURL{
.url = url,
.base = "flake:" + std::string(match[1]),
.base = "flake:" + match.str(1),
.scheme = "flake",
.authority = "",
.path = match[1],
@@ -106,12 +106,12 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
return std::make_pair(
FlakeRef(Input::fromURL(parsedURL), ""),
percentDecode(std::string(match[6])));
percentDecode(match.str(6)));
}
else if (std::regex_match(url, match, pathUrlRegex)) {
std::string path = match[1];
std::string fragment = percentDecode(std::string(match[3]));
std::string fragment = percentDecode(match.str(3));
if (baseDir) {
/* Check if 'url' is a path (either absolute or relative
+21 -12
View File
@@ -22,7 +22,7 @@ DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPat
{
auto [drvPath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
this->drvPath = store->printStorePath(drvPath);
this->drvPath = drvPath;
auto drv = store->derivationFromPath(drvPath);
@@ -41,9 +41,7 @@ DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPat
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvPath), outputName);
auto & [outputName, output] = *i;
auto optStorePath = output.path(*store, drv.name, outputName);
if (optStorePath)
outPath = store->printStorePath(*optStorePath);
outPath = {output.path(*store, drv.name, outputName)};
}
@@ -68,24 +66,35 @@ std::string DrvInfo::querySystem() const
}
std::string DrvInfo::queryDrvPath() const
std::optional<StorePath> DrvInfo::queryDrvPath() const
{
if (drvPath == "" && attrs) {
if (!drvPath && attrs) {
Bindings::iterator i = attrs->find(state->sDrvPath);
PathSet context;
drvPath = i != attrs->end() ? state->coerceToPath(*i->pos, *i->value, context) : "";
if (i == attrs->end())
drvPath = {std::nullopt};
else
drvPath = {state->coerceToStorePath(*i->pos, *i->value, context)};
}
return drvPath;
return drvPath.value_or(std::nullopt);
}
std::string DrvInfo::queryOutPath() const
StorePath DrvInfo::requireDrvPath() const
{
if (auto drvPath = queryDrvPath())
return *drvPath;
throw Error("derivation does not contain a 'drvPath' attribute");
}
StorePath DrvInfo::queryOutPath() const
{
if (!outPath && attrs) {
Bindings::iterator i = attrs->find(state->sOutPath);
PathSet context;
if (i != attrs->end())
outPath = state->coerceToPath(*i->pos, *i->value, context);
outPath = state->coerceToStorePath(*i->pos, *i->value, context);
}
if (!outPath)
throw UnimplementedError("CA derivations are not yet supported");
@@ -113,10 +122,10 @@ DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
Bindings::iterator outPath = out->value->attrs->find(state->sOutPath);
if (outPath == out->value->attrs->end()) continue; // FIXME: throw error?
PathSet context;
outputs[name] = state->coerceToPath(*outPath->pos, *outPath->value, context);
outputs.emplace(name, state->coerceToStorePath(*outPath->pos, *outPath->value, context));
}
} else
outputs["out"] = queryOutPath();
outputs.emplace("out", queryOutPath());
}
if (!onlyOutputsToInstall || !attrs)
return outputs;
+9 -7
View File
@@ -1,6 +1,7 @@
#pragma once
#include "eval.hh"
#include "path.hh"
#include <string>
#include <map>
@@ -12,15 +13,15 @@ namespace nix {
struct DrvInfo
{
public:
typedef std::map<std::string, Path> Outputs;
typedef std::map<std::string, StorePath> Outputs;
private:
EvalState * state;
mutable std::string name;
mutable std::string system;
mutable std::string drvPath;
mutable std::optional<std::string> outPath;
mutable std::optional<std::optional<StorePath>> drvPath;
mutable std::optional<StorePath> outPath;
mutable std::string outputName;
Outputs outputs;
@@ -41,8 +42,9 @@ public:
std::string queryName() const;
std::string querySystem() const;
std::string queryDrvPath() const;
std::string queryOutPath() const;
std::optional<StorePath> queryDrvPath() const;
StorePath requireDrvPath() const;
StorePath queryOutPath() const;
std::string queryOutputName() const;
/** Return the list of outputs. The "outputs to install" are determined by `meta.outputsToInstall`. */
Outputs queryOutputs(bool onlyOutputsToInstall = false);
@@ -61,8 +63,8 @@ public:
*/
void setName(const std::string & s) { name = s; }
void setDrvPath(const std::string & s) { drvPath = s; }
void setOutPath(const std::string & s) { outPath = s; }
void setDrvPath(StorePath path) { drvPath = {{std::move(path)}}; }
void setOutPath(StorePath path) { outPath = {{std::move(path)}}; }
void setFailed() { failed = true; };
bool hasFailed() { return failed; };
+5 -10
View File
@@ -1919,20 +1919,15 @@ static void addPath(
if (expectedHash)
expectedStorePath = state.store->makeFixedOutputPath(method, *expectedHash, name);
Path dstPath;
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
dstPath = state.store->printStorePath(settings.readOnlyMode
StorePath dstPath = settings.readOnlyMode
? state.store->computeStorePathForPath(name, path, method, htSHA256, filter).first
: state.store->addToStore(name, path, method, htSHA256, filter, state.repair, refs));
if (expectedHash && expectedStorePath != state.store->parseStorePath(dstPath))
: state.store->addToStore(name, path, method, htSHA256, filter, state.repair, refs);
if (expectedHash && expectedStorePath != dstPath)
throw Error("store path mismatch in (possibly filtered) path added from '%s'", path);
state.allowAndSetStorePathString(dstPath, v);
} else
dstPath = state.store->printStorePath(*expectedStorePath);
v.mkString(dstPath, {dstPath});
state.allowPath(dstPath);
state.allowAndSetStorePathString(*expectedStorePath, v);
} catch (Error & e) {
e.addTrace(pos, "while adding path '%s'", path);
throw;
+16 -4
View File
@@ -230,6 +230,21 @@ static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
if (evalSettings.pureEval && !expectedHash)
throw Error("in pure evaluation mode, '%s' requires a 'sha256' argument", who);
// early exit if pinned and already in the store
if (expectedHash && expectedHash->type == htSHA256) {
auto expectedPath =
unpack
? state.store->makeFixedOutputPath(FileIngestionMethod::Recursive, *expectedHash, name, {})
: state.store->makeFixedOutputPath(FileIngestionMethod::Flat, *expectedHash, name, {});
if (state.store->isValidPath(expectedPath)) {
state.allowAndSetStorePathString(expectedPath, v);
return;
}
}
// TODO: fetching may fail, yet the path may be substitutable.
// https://github.com/NixOS/nix/issues/4313
auto storePath =
unpack
? fetchers::downloadTarball(state.store, *url, name, (bool) expectedHash).first.storePath
@@ -244,10 +259,7 @@ static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
*url, expectedHash->to_string(Base32, true), hash.to_string(Base32, true));
}
state.allowPath(storePath);
auto path = state.store->printStorePath(storePath);
v.mkString(path, PathSet({path}));
state.allowAndSetStorePathString(storePath, v);
}
static void prim_fetchurl(EvalState & state, const Pos & pos, Value * * args, Value & v)
+5 -2
View File
@@ -114,11 +114,14 @@ struct Value
private:
InternalType internalType;
friend std::string showType(const Value & v);
friend void printValue(std::ostream & str, std::set<const Value *> & active, const Value & v);
friend std::string showType(const Value & v);
void print(std::ostream & str, std::set<const void *> * seen) const;
public:
void print(std::ostream & str, bool showRepeated = false) const;
// Functions needed to distinguish the type
// These should be removed eventually, by putting the functionality that's
// needed by callers into methods of this type
+13
View File
@@ -0,0 +1,13 @@
#include "fetch-settings.hh"
namespace nix {
FetchSettings::FetchSettings()
{
}
FetchSettings fetchSettings;
static GlobalConfig::Register rFetchSettings(&fetchSettings);
}
+93
View File
@@ -0,0 +1,93 @@
#pragma once
#include "types.hh"
#include "config.hh"
#include "util.hh"
#include <map>
#include <limits>
#include <sys/types.h>
namespace nix {
struct FetchSettings : public Config
{
FetchSettings();
Setting<StringMap> accessTokens{this, {}, "access-tokens",
R"(
Access tokens used to access protected GitHub, GitLab, or
other locations requiring token-based authentication.
Access tokens are specified as a string made up of
space-separated `host=token` values. The specific token
used is selected by matching the `host` portion against the
"host" specification of the input. The actual use of the
`token` value is determined by the type of resource being
accessed:
* Github: the token value is the OAUTH-TOKEN string obtained
as the Personal Access Token from the Github server (see
https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps).
* Gitlab: the token value is either the OAuth2 token or the
Personal Access Token (these are different types tokens
for gitlab, see
https://docs.gitlab.com/12.10/ee/api/README.html#authentication).
The `token` value should be `type:tokenstring` where
`type` is either `OAuth2` or `PAT` to indicate which type
of token is being specified.
Example `~/.config/nix/nix.conf`:
```
access-tokens = github.com=23ac...b289 gitlab.mycompany.com=PAT:A123Bp_Cd..EfG gitlab.com=OAuth2:1jklw3jk
```
Example `~/code/flake.nix`:
```nix
input.foo = {
type = "gitlab";
host = "gitlab.mycompany.com";
owner = "mycompany";
repo = "pro";
};
```
This example specifies three tokens, one each for accessing
github.com, gitlab.mycompany.com, and sourceforge.net.
The `input.foo` uses the "gitlab" fetcher, which might
requires specifying the token type along with the token
value.
)"};
Setting<bool> allowDirty{this, true, "allow-dirty",
"Whether to allow dirty Git/Mercurial trees."};
Setting<bool> warnDirty{this, true, "warn-dirty",
"Whether to warn about dirty Git/Mercurial trees."};
Setting<std::string> flakeRegistry{this, "https://github.com/NixOS/flake-registry/raw/master/flake-registry.json", "flake-registry",
"Path or URI of the global flake registry."};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references."};
Setting<bool> acceptFlakeConfig{this, false, "accept-flake-config",
"Whether to accept nix configuration from a flake without prompting."};
Setting<std::string> commitLockFileSummary{
this, "", "commit-lockfile-summary",
R"(
The commit summary to use when committing changed flake lock files. If
empty, the summary is generated based on the action performed.
)"};
};
// FIXME: don't use a global variable.
extern FetchSettings fetchSettings;
}
+4 -2
View File
@@ -6,6 +6,8 @@
#include "url-parts.hh"
#include "pathlocks.hh"
#include "fetch-settings.hh"
#include <sys/time.h>
#include <sys/wait.h>
@@ -246,10 +248,10 @@ struct GitInputScheme : InputScheme
/* This is an unclean working tree. So copy all tracked files. */
if (!settings.allowDirty)
if (!fetchSettings.allowDirty)
throw Error("Git tree '%s' is dirty", actualUrl);
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("Git tree '%s' is dirty", actualUrl);
auto gitOpts = Strings({ "-C", actualUrl, "ls-files", "-z" });
+4 -2
View File
@@ -1,11 +1,13 @@
#include "filetransfer.hh"
#include "cache.hh"
#include "fetchers.hh"
#include "globals.hh"
#include "store-api.hh"
#include "types.hh"
#include "url-parts.hh"
#include "fetchers.hh"
#include "fetch-settings.hh"
#include <optional>
#include <nlohmann/json.hpp>
#include <fstream>
@@ -157,7 +159,7 @@ struct GitArchiveInputScheme : InputScheme
std::optional<std::string> getAccessToken(const std::string & host) const
{
auto tokens = settings.accessTokens.get();
auto tokens = fetchSettings.accessTokens.get();
if (auto token = get(tokens, host))
return *token;
return {};
+4 -2
View File
@@ -5,6 +5,8 @@
#include "store-api.hh"
#include "url-parts.hh"
#include "fetch-settings.hh"
#include <sys/time.h>
using namespace std::string_literals;
@@ -165,10 +167,10 @@ struct MercurialInputScheme : InputScheme
/* This is an unclean working tree. So copy all tracked
files. */
if (!settings.allowDirty)
if (!fetchSettings.allowDirty)
throw Error("Mercurial tree '%s' is unclean", actualUrl);
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("Mercurial tree '%s' is unclean", actualUrl);
input.attrs.insert_or_assign("ref", chomp(runHg({ "branch", "-R", actualUrl })));
+3 -1
View File
@@ -5,6 +5,8 @@
#include "store-api.hh"
#include "local-fs-store.hh"
#include "fetch-settings.hh"
#include <nlohmann/json.hpp>
namespace nix::fetchers {
@@ -150,7 +152,7 @@ void overrideRegistry(
static std::shared_ptr<Registry> getGlobalRegistry(ref<Store> store)
{
static auto reg = [&]() {
auto path = settings.flakeRegistry.get();
auto path = fetchSettings.flakeRegistry.get();
if (!hasPrefix(path, "/")) {
auto storePath = downloadFile(store, path, "flake-registry.json", false).storePath;
+1 -2
View File
@@ -1,6 +1,7 @@
#include "globals.hh"
#include "shared.hh"
#include "store-api.hh"
#include "gc-store.hh"
#include "util.hh"
#include "loggers.hh"
@@ -227,8 +228,6 @@ LegacyArgs::LegacyArgs(const std::string & programName,
std::function<bool(Strings::iterator & arg, const Strings::iterator & end)> parseArg)
: MixCommonArgs(programName), parseArg(parseArg)
{
printError("FOO %s", programName);
addFlag({
.longName = "no-build-output",
.shortName = 'Q',
+77
View File
@@ -0,0 +1,77 @@
#pragma once
#include "realisation.hh"
#include <string>
#include <chrono>
namespace nix {
struct BuildResult
{
/* Note: don't remove status codes, and only add new status codes
at the end of the list, to prevent client/server
incompatibilities in the nix-store --serve protocol. */
enum Status {
Built = 0,
Substituted,
AlreadyValid,
PermanentFailure,
InputRejected,
OutputRejected,
TransientFailure, // possibly transient
CachedFailure, // no longer used
TimedOut,
MiscFailure,
DependencyFailed,
LogLimitExceeded,
NotDeterministic,
ResolvesToAlreadyValid,
} status = MiscFailure;
std::string errorMsg;
std::string toString() const {
auto strStatus = [&]() {
switch (status) {
case Built: return "Built";
case Substituted: return "Substituted";
case AlreadyValid: return "AlreadyValid";
case PermanentFailure: return "PermanentFailure";
case InputRejected: return "InputRejected";
case OutputRejected: return "OutputRejected";
case TransientFailure: return "TransientFailure";
case CachedFailure: return "CachedFailure";
case TimedOut: return "TimedOut";
case MiscFailure: return "MiscFailure";
case DependencyFailed: return "DependencyFailed";
case LogLimitExceeded: return "LogLimitExceeded";
case NotDeterministic: return "NotDeterministic";
case ResolvesToAlreadyValid: return "ResolvesToAlreadyValid";
default: return "Unknown";
};
}();
return strStatus + ((errorMsg == "") ? "" : " : " + errorMsg);
}
/* How many times this build was performed. */
unsigned int timesBuilt = 0;
/* If timesBuilt > 1, whether some builds did not produce the same
result. (Note that 'isNonDeterministic = false' does not mean
the build is deterministic, just that we don't have evidence of
non-determinism.) */
bool isNonDeterministic = false;
DrvOutputs builtOutputs;
/* The start/stop times of the build (or one of the rounds, if it
was repeated). */
time_t startTime = 0, stopTime = 0;
bool success() {
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
}
};
}
+13 -3
View File
@@ -1175,10 +1175,11 @@ bool DerivationGoal::isReadDesc(int fd)
return fd == hook->builderOut.readSide.get();
}
void DerivationGoal::handleChildOutput(int fd, std::string_view data)
{
if (isReadDesc(fd))
// local & `ssh://`-builds are dealt with here.
auto isWrittenToLog = isReadDesc(fd);
if (isWrittenToLog)
{
logSize += data.size();
if (settings.maxLogSize && logSize > settings.maxLogSize) {
@@ -1207,7 +1208,16 @@ void DerivationGoal::handleChildOutput(int fd, std::string_view data)
if (hook && fd == hook->fromHook.readSide.get()) {
for (auto c : data)
if (c == '\n') {
handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
auto json = parseJSONMessage(currentHookLine);
if (json) {
auto s = handleJSONLogMessage(*json, worker.act, hook->activities, true);
// ensure that logs from a builder using `ssh-ng://` as protocol
// are also available to `nix log`.
if (s && !isWrittenToLog && logSink && (*json)["type"] == resBuildLogLine) {
auto f = (*json)["fields"];
(*logSink)((f.size() > 0 ? f.at(0).get<std::string>() : "") + "\n");
}
}
currentHookLine.clear();
} else
currentHookLine += c;
+1
View File
@@ -2,6 +2,7 @@
#include "parsed-derivations.hh"
#include "lock.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "pathlocks.hh"
#include "goal.hh"
+2 -1
View File
@@ -1,4 +1,5 @@
#include "local-derivation-goal.hh"
#include "gc-store.hh"
#include "hook-instance.hh"
#include "worker.hh"
#include "builtins.hh"
@@ -1127,7 +1128,7 @@ struct RestrictedStoreConfig : virtual LocalFSStoreConfig
/* A wrapper around LocalStore that only allows building/querying of
paths that are in the input closures of the build or were added via
recursive Nix calls. */
struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual LocalFSStore
struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual LocalFSStore, public virtual GcStore
{
ref<LocalStore> next;
+10 -3
View File
@@ -1,7 +1,9 @@
#include "daemon.hh"
#include "monitor-fd.hh"
#include "worker-protocol.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "gc-store.hh"
#include "path-with-outputs.hh"
#include "finally.hh"
#include "archive.hh"
@@ -622,9 +624,12 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
case wopAddIndirectRoot: {
Path path = absPath(readString(from));
logger->startWork();
store->addIndirectRoot(path);
auto & gcStore = requireGcStore(*store);
gcStore.addIndirectRoot(path);
logger->stopWork();
to << 1;
break;
}
@@ -639,7 +644,8 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
case wopFindRoots: {
logger->startWork();
Roots roots = store->findRoots(!trusted);
auto & gcStore = requireGcStore(*store);
Roots roots = gcStore.findRoots(!trusted);
logger->stopWork();
size_t size = 0;
@@ -670,7 +676,8 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
logger->startWork();
if (options.ignoreLiveness)
throw Error("you are not allowed to ignore liveness");
store->collectGarbage(options, results);
auto & gcStore = requireGcStore(*store);
gcStore.collectGarbage(options, results);
logger->stopWork();
to << results.paths << results.bytesFreed << 0 /* obsolete */;
+1 -1
View File
@@ -197,7 +197,7 @@ struct curlFileTransfer : public FileTransfer
result.etag = "";
result.data.clear();
result.bodySize = 0;
statusMsg = trim((std::string &) match[1]);
statusMsg = trim(match.str(1));
acceptRanges = false;
encoding = "";
} else {
+13
View File
@@ -0,0 +1,13 @@
#include "gc-store.hh"
namespace nix {
GcStore & requireGcStore(Store & store)
{
auto * gcStore = dynamic_cast<GcStore *>(&store);
if (!gcStore)
throw UsageError("Garbage collection not supported by this store");
return *gcStore;
}
}
+84
View File
@@ -0,0 +1,84 @@
#pragma once
#include "store-api.hh"
namespace nix {
typedef std::unordered_map<StorePath, std::unordered_set<std::string>> Roots;
struct GCOptions
{
/* Garbage collector operation:
- `gcReturnLive': return the set of paths reachable from
(i.e. in the closure of) the roots.
- `gcReturnDead': return the set of paths not reachable from
the roots.
- `gcDeleteDead': actually delete the latter set.
- `gcDeleteSpecific': delete the paths listed in
`pathsToDelete', insofar as they are not reachable.
*/
typedef enum {
gcReturnLive,
gcReturnDead,
gcDeleteDead,
gcDeleteSpecific,
} GCAction;
GCAction action{gcDeleteDead};
/* If `ignoreLiveness' is set, then reachability from the roots is
ignored (dangerous!). However, the paths must still be
unreferenced *within* the store (i.e., there can be no other
store paths that depend on them). */
bool ignoreLiveness{false};
/* For `gcDeleteSpecific', the paths to delete. */
StorePathSet pathsToDelete;
/* Stop after at least `maxFreed' bytes have been freed. */
uint64_t maxFreed{std::numeric_limits<uint64_t>::max()};
};
struct GCResults
{
/* Depending on the action, the GC roots, or the paths that would
be or have been deleted. */
PathSet paths;
/* For `gcReturnDead', `gcDeleteDead' and `gcDeleteSpecific', the
number of bytes that would be or was freed. */
uint64_t bytesFreed = 0;
};
struct GcStore : public virtual Store
{
/* Add an indirect root, which is merely a symlink to `path' from
/nix/var/nix/gcroots/auto/<hash of `path'>. `path' is supposed
to be a symlink to a store path. The garbage collector will
automatically remove the indirect root when it finds that
`path' has disappeared. */
virtual void addIndirectRoot(const Path & path) = 0;
/* Find the roots of the garbage collector. Each root is a pair
(link, storepath) where `link' is the path of the symlink
outside of the Nix store that point to `storePath'. If
'censor' is true, privacy-sensitive information about roots
found in /proc is censored. */
virtual Roots findRoots(bool censor) = 0;
/* Perform a garbage collection. */
virtual void collectGarbage(const GCOptions & options, GCResults & results) = 0;
};
GcStore & requireGcStore(Store & store);
}
-71
View File
@@ -880,55 +880,6 @@ public:
are loaded as plugins (non-recursively).
)"};
Setting<StringMap> accessTokens{this, {}, "access-tokens",
R"(
Access tokens used to access protected GitHub, GitLab, or
other locations requiring token-based authentication.
Access tokens are specified as a string made up of
space-separated `host=token` values. The specific token
used is selected by matching the `host` portion against the
"host" specification of the input. The actual use of the
`token` value is determined by the type of resource being
accessed:
* Github: the token value is the OAUTH-TOKEN string obtained
as the Personal Access Token from the Github server (see
https://docs.github.com/en/developers/apps/authorizing-oath-apps).
* Gitlab: the token value is either the OAuth2 token or the
Personal Access Token (these are different types tokens
for gitlab, see
https://docs.gitlab.com/12.10/ee/api/README.html#authentication).
The `token` value should be `type:tokenstring` where
`type` is either `OAuth2` or `PAT` to indicate which type
of token is being specified.
Example `~/.config/nix/nix.conf`:
```
access-tokens = github.com=23ac...b289 gitlab.mycompany.com=PAT:A123Bp_Cd..EfG gitlab.com=OAuth2:1jklw3jk
```
Example `~/code/flake.nix`:
```nix
input.foo = {
type = "gitlab";
host = "gitlab.mycompany.com";
owner = "mycompany";
repo = "pro";
};
```
This example specifies three tokens, one each for accessing
github.com, gitlab.mycompany.com, and sourceforge.net.
The `input.foo` uses the "gitlab" fetcher, which might
requires specifying the token type along with the token
value.
)"};
Setting<std::set<ExperimentalFeature>> experimentalFeatures{this, {}, "experimental-features",
"Experimental Nix features to enable."};
@@ -936,18 +887,9 @@ public:
void requireExperimentalFeature(const ExperimentalFeature &);
Setting<bool> allowDirty{this, true, "allow-dirty",
"Whether to allow dirty Git/Mercurial trees."};
Setting<bool> warnDirty{this, true, "warn-dirty",
"Whether to warn about dirty Git/Mercurial trees."};
Setting<size_t> narBufferSize{this, 32 * 1024 * 1024, "nar-buffer-size",
"Maximum size of NARs before spilling them to disk."};
Setting<std::string> flakeRegistry{this, "https://github.com/NixOS/flake-registry/raw/master/flake-registry.json", "flake-registry",
"Path or URI of the global flake registry."};
Setting<bool> allowSymlinkedStore{
this, false, "allow-symlinked-store",
R"(
@@ -960,19 +902,6 @@ public:
resolves to a different location from that of the build machine. You
can enable this setting if you are sure you're not going to do that.
)"};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references."};
Setting<bool> acceptFlakeConfig{this, false, "accept-flake-config",
"Whether to accept nix configuration from a flake without prompting."};
Setting<std::string> commitLockFileSummary{
this, "", "commit-lockfile-summary",
R"(
The commit summary to use when committing changed flake lock files. If
empty, the summary is generated based on the action performed.
)"};
};
+1
View File
@@ -2,6 +2,7 @@
#include "pool.hh"
#include "remote-store.hh"
#include "serve-protocol.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "path-with-outputs.hh"
#include "worker-protocol.hh"
+2 -1
View File
@@ -1,6 +1,7 @@
#pragma once
#include "store-api.hh"
#include "gc-store.hh"
namespace nix {
@@ -23,7 +24,7 @@ struct LocalFSStoreConfig : virtual StoreConfig
"physical path to the Nix store"};
};
class LocalFSStore : public virtual LocalFSStoreConfig, public virtual Store
class LocalFSStore : public virtual LocalFSStoreConfig, public virtual Store, virtual GcStore
{
public:
+2 -1
View File
@@ -5,6 +5,7 @@
#include "pathlocks.hh"
#include "store-api.hh"
#include "local-fs-store.hh"
#include "gc-store.hh"
#include "sync.hh"
#include "util.hh"
@@ -43,7 +44,7 @@ struct LocalStoreConfig : virtual LocalFSStoreConfig
};
class LocalStore : public virtual LocalStoreConfig, public virtual LocalFSStore
class LocalStore : public virtual LocalStoreConfig, public virtual LocalFSStore, public virtual GcStore
{
private:
+3 -3
View File
@@ -22,9 +22,9 @@ DerivedPath StorePathWithOutputs::toDerivedPath() const
std::vector<DerivedPath> toDerivedPaths(const std::vector<StorePathWithOutputs> ss)
{
std::vector<DerivedPath> reqs;
for (auto & s : ss) reqs.push_back(s.toDerivedPath());
return reqs;
std::vector<DerivedPath> reqs;
for (auto & s : ss) reqs.push_back(s.toDerivedPath());
return reqs;
}
+2
View File
@@ -1,7 +1,9 @@
#include "serialise.hh"
#include "util.hh"
#include "path-with-outputs.hh"
#include "gc-store.hh"
#include "remote-fs-accessor.hh"
#include "build-result.hh"
#include "remote-store.hh"
#include "worker-protocol.hh"
#include "archive.hh"
+2 -1
View File
@@ -4,6 +4,7 @@
#include <string>
#include "store-api.hh"
#include "gc-store.hh"
namespace nix {
@@ -29,7 +30,7 @@ struct RemoteStoreConfig : virtual StoreConfig
/* FIXME: RemoteStore is a misnomer - should be something like
DaemonStore. */
class RemoteStore : public virtual RemoteStoreConfig, public virtual Store
class RemoteStore : public virtual RemoteStoreConfig, public virtual Store, public virtual GcStore
{
public:
+1 -138
View File
@@ -76,127 +76,10 @@ enum AllowInvalidFlag : bool { DisallowInvalid = false, AllowInvalid = true };
const uint32_t exportMagic = 0x4558494e;
typedef std::unordered_map<StorePath, std::unordered_set<std::string>> Roots;
struct GCOptions
{
/* Garbage collector operation:
- `gcReturnLive': return the set of paths reachable from
(i.e. in the closure of) the roots.
- `gcReturnDead': return the set of paths not reachable from
the roots.
- `gcDeleteDead': actually delete the latter set.
- `gcDeleteSpecific': delete the paths listed in
`pathsToDelete', insofar as they are not reachable.
*/
typedef enum {
gcReturnLive,
gcReturnDead,
gcDeleteDead,
gcDeleteSpecific,
} GCAction;
GCAction action{gcDeleteDead};
/* If `ignoreLiveness' is set, then reachability from the roots is
ignored (dangerous!). However, the paths must still be
unreferenced *within* the store (i.e., there can be no other
store paths that depend on them). */
bool ignoreLiveness{false};
/* For `gcDeleteSpecific', the paths to delete. */
StorePathSet pathsToDelete;
/* Stop after at least `maxFreed' bytes have been freed. */
uint64_t maxFreed{std::numeric_limits<uint64_t>::max()};
};
struct GCResults
{
/* Depending on the action, the GC roots, or the paths that would
be or have been deleted. */
PathSet paths;
/* For `gcReturnDead', `gcDeleteDead' and `gcDeleteSpecific', the
number of bytes that would be or was freed. */
uint64_t bytesFreed = 0;
};
enum BuildMode { bmNormal, bmRepair, bmCheck };
struct BuildResult;
struct BuildResult
{
/* Note: don't remove status codes, and only add new status codes
at the end of the list, to prevent client/server
incompatibilities in the nix-store --serve protocol. */
enum Status {
Built = 0,
Substituted,
AlreadyValid,
PermanentFailure,
InputRejected,
OutputRejected,
TransientFailure, // possibly transient
CachedFailure, // no longer used
TimedOut,
MiscFailure,
DependencyFailed,
LogLimitExceeded,
NotDeterministic,
ResolvesToAlreadyValid,
} status = MiscFailure;
std::string errorMsg;
std::string toString() const {
auto strStatus = [&]() {
switch (status) {
case Built: return "Built";
case Substituted: return "Substituted";
case AlreadyValid: return "AlreadyValid";
case PermanentFailure: return "PermanentFailure";
case InputRejected: return "InputRejected";
case OutputRejected: return "OutputRejected";
case TransientFailure: return "TransientFailure";
case CachedFailure: return "CachedFailure";
case TimedOut: return "TimedOut";
case MiscFailure: return "MiscFailure";
case DependencyFailed: return "DependencyFailed";
case LogLimitExceeded: return "LogLimitExceeded";
case NotDeterministic: return "NotDeterministic";
case ResolvesToAlreadyValid: return "ResolvesToAlreadyValid";
default: return "Unknown";
};
}();
return strStatus + ((errorMsg == "") ? "" : " : " + errorMsg);
}
/* How many times this build was performed. */
unsigned int timesBuilt = 0;
/* If timesBuilt > 1, whether some builds did not produce the same
result. (Note that 'isNonDeterministic = false' does not mean
the build is deterministic, just that we don't have evidence of
non-determinism.) */
bool isNonDeterministic = false;
DrvOutputs builtOutputs;
/* The start/stop times of the build (or one of the rounds, if it
was repeated). */
time_t startTime = 0, stopTime = 0;
bool success() {
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
}
};
struct StoreConfig : public Config
{
@@ -595,26 +478,6 @@ public:
virtual void addTempRoot(const StorePath & path)
{ debug("not creating temporary root, store doesn't support GC"); }
/* Add an indirect root, which is merely a symlink to `path' from
/nix/var/nix/gcroots/auto/<hash of `path'>. `path' is supposed
to be a symlink to a store path. The garbage collector will
automatically remove the indirect root when it finds that
`path' has disappeared. */
virtual void addIndirectRoot(const Path & path)
{ unsupported("addIndirectRoot"); }
/* Find the roots of the garbage collector. Each root is a pair
(link, storepath) where `link' is the path of the symlink
outside of the Nix store that point to `storePath'. If
'censor' is true, privacy-sensitive information about roots
found in /proc is censored. */
virtual Roots findRoots(bool censor)
{ unsupported("findRoots"); }
/* Perform a garbage collection. */
virtual void collectGarbage(const GCOptions & options, GCResults & results)
{ unsupported("collectGarbage"); }
/* Return a string representing information about the path that
can be loaded into the database using `nix-store --load-db' or
`nix-store --register-validity'. */
+7 -2
View File
@@ -328,8 +328,13 @@ MultiCommand::MultiCommand(const Commands & commands_)
completions->add(name);
}
auto i = commands.find(s);
if (i == commands.end())
throw UsageError("'%s' is not a recognised command", s);
if (i == commands.end()) {
std::set<std::string> commandNames;
for (auto & [name, _] : commands)
commandNames.insert(name);
auto suggestions = Suggestions::bestMatches(commandNames, s);
throw UsageError(suggestions, "'%s' is not a recognised command", s);
}
command = {s, i->second()};
command->second->parent = this;
}}
+7
View File
@@ -282,6 +282,13 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
}
}
auto suggestions = einfo.suggestions.trim();
if (! suggestions.suggestions.empty()){
oss << "Did you mean " <<
suggestions.trim() <<
"?" << std::endl;
}
// traces
if (showTrace && !einfo.traces.empty()) {
for (auto iter = einfo.traces.rbegin(); iter != einfo.traces.rend(); ++iter) {
+8
View File
@@ -1,5 +1,6 @@
#pragma once
#include "suggestions.hh"
#include "ref.hh"
#include "types.hh"
#include "fmt.hh"
@@ -112,6 +113,8 @@ struct ErrorInfo {
std::optional<ErrPos> errPos;
std::list<Trace> traces;
Suggestions suggestions;
static std::optional<std::string> programName;
};
@@ -141,6 +144,11 @@ public:
: err { .level = lvlError, .msg = hintfmt(fs, args...) }
{ }
template<typename... Args>
BaseError(const Suggestions & sug, const Args & ... args)
: err { .level = lvlError, .msg = hintfmt(args...), .suggestions = sug }
{ }
BaseError(hintformat hint)
: err { .level = lvlError, .msg = hint }
{ }
+48 -36
View File
@@ -266,51 +266,63 @@ static Logger::Fields getFields(nlohmann::json & json)
return fields;
}
bool handleJSONLogMessage(const std::string & msg,
const Activity & act, std::map<ActivityId, Activity> & activities, bool trusted)
std::optional<nlohmann::json> parseJSONMessage(const std::string & msg)
{
if (!hasPrefix(msg, "@nix ")) return false;
if (!hasPrefix(msg, "@nix ")) return std::nullopt;
try {
auto json = nlohmann::json::parse(std::string(msg, 5));
std::string action = json["action"];
if (action == "start") {
auto type = (ActivityType) json["type"];
if (trusted || type == actFileTransfer)
activities.emplace(std::piecewise_construct,
std::forward_as_tuple(json["id"]),
std::forward_as_tuple(*logger, (Verbosity) json["level"], type,
json["text"], getFields(json["fields"]), act.id));
}
else if (action == "stop")
activities.erase((ActivityId) json["id"]);
else if (action == "result") {
auto i = activities.find((ActivityId) json["id"]);
if (i != activities.end())
i->second.result((ResultType) json["type"], getFields(json["fields"]));
}
else if (action == "setPhase") {
std::string phase = json["phase"];
act.result(resSetPhase, phase);
}
else if (action == "msg") {
std::string msg = json["msg"];
logger->log((Verbosity) json["level"], msg);
}
return nlohmann::json::parse(std::string(msg, 5));
} catch (std::exception & e) {
printError("bad JSON log message from builder: %s", e.what());
}
return std::nullopt;
}
bool handleJSONLogMessage(nlohmann::json & json,
const Activity & act, std::map<ActivityId, Activity> & activities,
bool trusted)
{
std::string action = json["action"];
if (action == "start") {
auto type = (ActivityType) json["type"];
if (trusted || type == actFileTransfer)
activities.emplace(std::piecewise_construct,
std::forward_as_tuple(json["id"]),
std::forward_as_tuple(*logger, (Verbosity) json["level"], type,
json["text"], getFields(json["fields"]), act.id));
}
else if (action == "stop")
activities.erase((ActivityId) json["id"]);
else if (action == "result") {
auto i = activities.find((ActivityId) json["id"]);
if (i != activities.end())
i->second.result((ResultType) json["type"], getFields(json["fields"]));
}
else if (action == "setPhase") {
std::string phase = json["phase"];
act.result(resSetPhase, phase);
}
else if (action == "msg") {
std::string msg = json["msg"];
logger->log((Verbosity) json["level"], msg);
}
return true;
}
bool handleJSONLogMessage(const std::string & msg,
const Activity & act, std::map<ActivityId, Activity> & activities, bool trusted)
{
auto json = parseJSONMessage(msg);
if (!json) return false;
return handleJSONLogMessage(*json, act, activities, trusted);
}
Activity::~Activity()
{
try {
+8
View File
@@ -4,6 +4,8 @@
#include "error.hh"
#include "config.hh"
#include <nlohmann/json_fwd.hpp>
namespace nix {
typedef enum {
@@ -166,6 +168,12 @@ Logger * makeSimpleLogger(bool printBuildLogs = true);
Logger * makeJSONLogger(Logger & prevLogger);
std::optional<nlohmann::json> parseJSONMessage(const std::string & msg);
bool handleJSONLogMessage(nlohmann::json & json,
const Activity & act, std::map<ActivityId, Activity> & activities,
bool trusted);
bool handleJSONLogMessage(const std::string & msg,
const Activity & act, std::map<ActivityId, Activity> & activities,
bool trusted);
+114
View File
@@ -0,0 +1,114 @@
#include "suggestions.hh"
#include "ansicolor.hh"
#include "util.hh"
#include <algorithm>
namespace nix {
int levenshteinDistance(std::string_view first, std::string_view second)
{
// Implementation borrowed from
// https://en.wikipedia.org/wiki/Levenshtein_distance#Iterative_with_two_matrix_rows
int m = first.size();
int n = second.size();
auto v0 = std::vector<int>(n+1);
auto v1 = std::vector<int>(n+1);
for (auto i = 0; i <= n; i++)
v0[i] = i;
for (auto i = 0; i < m; i++) {
v1[0] = i+1;
for (auto j = 0; j < n; j++) {
auto deletionCost = v0[j+1] + 1;
auto insertionCost = v1[j] + 1;
auto substitutionCost = first[i] == second[j] ? v0[j] : v0[j] + 1;
v1[j+1] = std::min({deletionCost, insertionCost, substitutionCost});
}
std::swap(v0, v1);
}
return v0[n];
}
Suggestions Suggestions::bestMatches (
std::set<std::string> allMatches,
std::string query)
{
std::set<Suggestion> res;
for (const auto & possibleMatch : allMatches) {
res.insert(Suggestion {
.distance = levenshteinDistance(query, possibleMatch),
.suggestion = possibleMatch,
});
}
return Suggestions { res };
}
Suggestions Suggestions::trim(int limit, int maxDistance) const
{
std::set<Suggestion> res;
int count = 0;
for (auto & elt : suggestions) {
if (count >= limit || elt.distance > maxDistance)
break;
count++;
res.insert(elt);
}
return Suggestions{res};
}
std::string Suggestion::to_string() const
{
return ANSI_WARNING + filterANSIEscapes(suggestion) + ANSI_NORMAL;
}
std::string Suggestions::to_string() const
{
switch (suggestions.size()) {
case 0:
return "";
case 1:
return suggestions.begin()->to_string();
default: {
std::string res = "one of ";
auto iter = suggestions.begin();
res += iter->to_string(); // Iter can’t be end() because the container isn’t null
iter++;
auto last = suggestions.end(); last--;
for ( ; iter != suggestions.end() ; iter++) {
res += (iter == last) ? " or " : ", ";
res += iter->to_string();
}
return res;
}
}
}
Suggestions & Suggestions::operator+=(const Suggestions & other)
{
suggestions.insert(
other.suggestions.begin(),
other.suggestions.end()
);
return *this;
}
std::ostream & operator<<(std::ostream & str, const Suggestion & suggestion)
{
return str << suggestion.to_string();
}
std::ostream & operator<<(std::ostream & str, const Suggestions & suggestions)
{
return str << suggestions.to_string();
}
}
+102
View File
@@ -0,0 +1,102 @@
#pragma once
#include "comparator.hh"
#include "types.hh"
#include <set>
namespace nix {
int levenshteinDistance(std::string_view first, std::string_view second);
/**
* A potential suggestion for the cli interface.
*/
class Suggestion {
public:
int distance; // The smaller the better
std::string suggestion;
std::string to_string() const;
GENERATE_CMP(Suggestion, me->distance, me->suggestion)
};
class Suggestions {
public:
std::set<Suggestion> suggestions;
std::string to_string() const;
Suggestions trim(
int limit = 5,
int maxDistance = 2
) const;
static Suggestions bestMatches (
std::set<std::string> allMatches,
std::string query
);
Suggestions& operator+=(const Suggestions & other);
};
std::ostream & operator<<(std::ostream & str, const Suggestion &);
std::ostream & operator<<(std::ostream & str, const Suggestions &);
// Either a value of type `T`, or some suggestions
template<typename T>
class OrSuggestions {
public:
using Raw = std::variant<T, Suggestions>;
Raw raw;
T* operator ->()
{
return &**this;
}
T& operator *()
{
return std::get<T>(raw);
}
operator bool() const noexcept
{
return std::holds_alternative<T>(raw);
}
OrSuggestions(T t)
: raw(t)
{
}
OrSuggestions()
: raw(Suggestions{})
{
}
static OrSuggestions<T> failed(const Suggestions & s)
{
auto res = OrSuggestions<T>();
res.raw = s;
return res;
}
static OrSuggestions<T> failed()
{
return OrSuggestions<T>::failed(Suggestions{});
}
const Suggestions & getSuggestions()
{
static Suggestions noSuggestions;
if (const auto & suggestions = std::get_if<Suggestions>(&raw))
return *suggestions;
else
return noSuggestions;
}
};
}
+1 -1
View File
@@ -359,7 +359,7 @@ namespace nix {
// constructing without access violation.
ErrPos ep(invalid);
// assignment without access violation.
ep = invalid;
+43
View File
@@ -0,0 +1,43 @@
#include "suggestions.hh"
#include <gtest/gtest.h>
namespace nix {
struct LevenshteinDistanceParam {
std::string s1, s2;
int distance;
};
class LevenshteinDistanceTest :
public testing::TestWithParam<LevenshteinDistanceParam> {
};
TEST_P(LevenshteinDistanceTest, CorrectlyComputed) {
auto params = GetParam();
ASSERT_EQ(levenshteinDistance(params.s1, params.s2), params.distance);
ASSERT_EQ(levenshteinDistance(params.s2, params.s1), params.distance);
}
INSTANTIATE_TEST_SUITE_P(LevenshteinDistance, LevenshteinDistanceTest,
testing::Values(
LevenshteinDistanceParam{"foo", "foo", 0},
LevenshteinDistanceParam{"foo", "", 3},
LevenshteinDistanceParam{"", "", 0},
LevenshteinDistanceParam{"foo", "fo", 1},
LevenshteinDistanceParam{"foo", "oo", 1},
LevenshteinDistanceParam{"foo", "fao", 1},
LevenshteinDistanceParam{"foo", "abc", 3}
)
);
TEST(Suggestions, Trim) {
auto suggestions = Suggestions::bestMatches({"foooo", "bar", "fo", "gao"}, "foo");
auto onlyOne = suggestions.trim(1);
ASSERT_EQ(onlyOne.suggestions.size(), 1);
ASSERT_TRUE(onlyOne.suggestions.begin()->suggestion == "fo");
auto closest = suggestions.trim(999, 2);
ASSERT_EQ(closest.suggestions.size(), 3);
}
}
+3 -3
View File
@@ -1263,9 +1263,9 @@ std::string chomp(std::string_view s)
std::string trim(std::string_view s, std::string_view whitespace)
{
auto i = s.find_first_not_of(whitespace);
if (i == std::string_view::npos) return "";
if (i == s.npos) return "";
auto j = s.find_last_not_of(whitespace);
return std::string(s, i, j == std::string::npos ? j : j - i + 1);
return std::string(s, i, j == s.npos ? j : j - i + 1);
}
@@ -1412,7 +1412,7 @@ std::string filterANSIEscapes(const std::string & s, bool filterAll, unsigned in
}
}
else if (*i == '\r')
else if (*i == '\r' || *i == '\a')
// do nothing for now
i++;
+4 -7
View File
@@ -346,7 +346,7 @@ static void main_nix_build(int argc, char * * argv)
throw UsageError("nix-shell requires a single derivation");
auto & drvInfo = drvs.front();
auto drv = evalStore->derivationFromPath(evalStore->parseStorePath(drvInfo.queryDrvPath()));
auto drv = evalStore->derivationFromPath(drvInfo.requireDrvPath());
std::vector<StorePathWithOutputs> pathsToBuild;
RealisedPath::Set pathsToCopy;
@@ -369,7 +369,7 @@ static void main_nix_build(int argc, char * * argv)
if (!drv)
throw Error("the 'bashInteractive' attribute in <nixpkgs> did not evaluate to a derivation");
auto bashDrv = store->parseStorePath(drv->queryDrvPath());
auto bashDrv = drv->requireDrvPath();
pathsToBuild.push_back({bashDrv});
pathsToCopy.insert(bashDrv);
shellDrv = bashDrv;
@@ -458,10 +458,7 @@ static void main_nix_build(int argc, char * * argv)
}
}
ParsedDerivation parsedDrv(
StorePath(store->parseStorePath(drvInfo.queryDrvPath())),
drv
);
ParsedDerivation parsedDrv(drvInfo.requireDrvPath(), drv);
if (auto structAttrs = parsedDrv.prepareStructuredAttrs(*store, inputs)) {
auto json = structAttrs.value();
@@ -553,7 +550,7 @@ static void main_nix_build(int argc, char * * argv)
std::map<StorePath, std::pair<size_t, StringSet>> drvMap;
for (auto & drvInfo : drvs) {
auto drvPath = store->parseStorePath(drvInfo.queryDrvPath());
auto drvPath = drvInfo.requireDrvPath();
auto outputName = drvInfo.queryOutputName();
if (outputName == "")
+1 -1
View File
@@ -96,7 +96,7 @@ static void update(const StringSet & channelNames)
std::smatch match;
auto urlBase = std::string(baseNameOf(url));
if (std::regex_search(urlBase, match, std::regex("(-\\d.*)$")))
cname = cname + (std::string) match[1];
cname = cname + match.str(1);
std::string extraAttrs;
@@ -1,4 +1,5 @@
#include "store-api.hh"
#include "gc-store.hh"
#include "profiles.hh"
#include "shared.hh"
#include "globals.hh"
@@ -80,10 +81,11 @@ static int main_nix_collect_garbage(int argc, char * * argv)
// Run the actual garbage collector.
if (!dryRun) {
auto store = openStore();
auto & gcStore = requireGcStore(*store);
options.action = GCOptions::gcDeleteDead;
GCResults results;
PrintFreed freed(true, results);
store->collectGarbage(options, results);
gcStore.collectGarbage(options, results);
}
return 0;
+36 -39
View File
@@ -211,7 +211,7 @@ static long comparePriorities(EvalState & state, DrvInfo & drv1, DrvInfo & drv2)
// at a time.
static bool isPrebuilt(EvalState & state, DrvInfo & elem)
{
auto path = state.store->parseStorePath(elem.queryOutPath());
auto path = elem.queryOutPath();
if (state.store->isValidPath(path)) return true;
return state.store->querySubstitutablePaths({path}).count(path);
}
@@ -429,14 +429,15 @@ static void queryInstSources(EvalState & state,
elem.setName(name);
if (path.isDerivation()) {
elem.setDrvPath(state.store->printStorePath(path));
elem.setDrvPath(path);
auto outputs = state.store->queryDerivationOutputMap(path);
elem.setOutPath(state.store->printStorePath(outputs.at("out")));
elem.setOutPath(outputs.at("out"));
if (name.size() >= drvExtension.size() &&
std::string(name, name.size() - drvExtension.size()) == drvExtension)
name = name.substr(0, name.size() - drvExtension.size());
}
else elem.setOutPath(state.store->printStorePath(path));
else
elem.setOutPath(path);
elems.push_back(elem);
}
@@ -470,13 +471,11 @@ static void queryInstSources(EvalState & state,
static void printMissing(EvalState & state, DrvInfos & elems)
{
std::vector<DerivedPath> targets;
for (auto & i : elems) {
Path drvPath = i.queryDrvPath();
if (drvPath != "")
targets.push_back(DerivedPath::Built{state.store->parseStorePath(drvPath)});
for (auto & i : elems)
if (auto drvPath = i.queryDrvPath())
targets.push_back(DerivedPath::Built{*drvPath});
else
targets.push_back(DerivedPath::Opaque{state.store->parseStorePath(i.queryOutPath())});
}
targets.push_back(DerivedPath::Opaque{i.queryOutPath()});
printMissing(state.store, targets);
}
@@ -744,14 +743,11 @@ static void opSet(Globals & globals, Strings opFlags, Strings opArgs)
if (globals.forceName != "")
drv.setName(globals.forceName);
auto drvPath = drv.queryDrvPath();
std::vector<DerivedPath> paths {
(drv.queryDrvPath() != "")
? (DerivedPath) (DerivedPath::Built {
globals.state->store->parseStorePath(drv.queryDrvPath())
})
: (DerivedPath) (DerivedPath::Opaque {
globals.state->store->parseStorePath(drv.queryOutPath())
}),
drvPath
? (DerivedPath) (DerivedPath::Built { *drvPath })
: (DerivedPath) (DerivedPath::Opaque { drv.queryOutPath() }),
};
printMissing(globals.state->store, paths);
if (globals.dryRun) return;
@@ -759,8 +755,9 @@ static void opSet(Globals & globals, Strings opFlags, Strings opArgs)
debug(format("switching to new user environment"));
Path generation = createGeneration(
ref<LocalFSStore>(store2), globals.profile,
store2->parseStorePath(drv.queryOutPath()));
ref<LocalFSStore>(store2),
globals.profile,
drv.queryOutPath());
switchLink(globals.profile, generation);
}
@@ -780,7 +777,7 @@ static void uninstallDerivations(Globals & globals, Strings & selectors,
split = std::partition(
workingElems.begin(), workingElems.end(),
[&selectorStorePath, globals](auto &elem) {
return selectorStorePath != globals.state->store->parseStorePath(elem.queryOutPath());
return selectorStorePath != elem.queryOutPath();
}
);
} else {
@@ -925,9 +922,8 @@ static void queryJSON(Globals & globals, std::vector<DrvInfo> & elems, bool prin
if (printOutPath) {
DrvInfo::Outputs outputs = i.queryOutputs();
JSONObject outputObj = pkgObj.object("outputs");
for (auto & j : outputs) {
outputObj.attr(j.first, j.second);
}
for (auto & j : outputs)
outputObj.attr(j.first, globals.state->store->printStorePath(j.second));
}
if (printMeta) {
@@ -957,6 +953,8 @@ static void queryJSON(Globals & globals, std::vector<DrvInfo> & elems, bool prin
static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
{
auto & store { *globals.state->store };
Strings remaining;
std::string attrPath;
@@ -1027,12 +1025,11 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
/* We only need to know the installed paths when we are querying
the status of the derivation. */
PathSet installed; /* installed paths */
StorePathSet installed; /* installed paths */
if (printStatus) {
if (printStatus)
for (auto & i : installedElems)
installed.insert(i.queryOutPath());
}
/* Query which paths have substitutes. */
@@ -1042,13 +1039,13 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
StorePathSet paths;
for (auto & i : elems)
try {
paths.insert(globals.state->store->parseStorePath(i.queryOutPath()));
paths.insert(i.queryOutPath());
} catch (AssertionError & e) {
printMsg(lvlTalkative, "skipping derivation named '%s' which gives an assertion failure", i.queryName());
i.setFailed();
}
validPaths = globals.state->store->queryValidPaths(paths);
substitutablePaths = globals.state->store->querySubstitutablePaths(paths);
validPaths = store.queryValidPaths(paths);
substitutablePaths = store.querySubstitutablePaths(paths);
}
@@ -1073,8 +1070,8 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
//Activity act(*logger, lvlDebug, format("outputting query result '%1%'") % i.attrPath);
if (globals.prebuiltOnly &&
!validPaths.count(globals.state->store->parseStorePath(i.queryOutPath())) &&
!substitutablePaths.count(globals.state->store->parseStorePath(i.queryOutPath())))
!validPaths.count(i.queryOutPath()) &&
!substitutablePaths.count(i.queryOutPath()))
continue;
/* For table output. */
@@ -1084,10 +1081,10 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
XMLAttrs attrs;
if (printStatus) {
Path outPath = i.queryOutPath();
bool hasSubs = substitutablePaths.count(globals.state->store->parseStorePath(outPath));
bool isInstalled = installed.find(outPath) != installed.end();
bool isValid = validPaths.count(globals.state->store->parseStorePath(outPath));
auto outPath = i.queryOutPath();
bool hasSubs = substitutablePaths.count(outPath);
bool isInstalled = installed.count(outPath);
bool isValid = validPaths.count(outPath);
if (xmlOutput) {
attrs["installed"] = isInstalled ? "1" : "0";
attrs["valid"] = isValid ? "1" : "0";
@@ -1152,9 +1149,9 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
if (printDrvPath) {
auto drvPath = i.queryDrvPath();
if (xmlOutput) {
if (drvPath != "") attrs["drvPath"] = drvPath;
if (drvPath) attrs["drvPath"] = store.printStorePath(*drvPath);
} else
columns.push_back(drvPath == "" ? "-" : drvPath);
columns.push_back(drvPath ? store.printStorePath(*drvPath) : "-");
}
if (printOutPath && !xmlOutput) {
@@ -1163,7 +1160,7 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
for (auto & j : outputs) {
if (!s.empty()) s += ';';
if (j.first != "out") { s += j.first; s += "="; }
s += j.second;
s += store.printStorePath(j.second);
}
columns.push_back(s);
}
@@ -1184,7 +1181,7 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
for (auto & j : outputs) {
XMLAttrs attrs2;
attrs2["name"] = j.first;
attrs2["path"] = j.second;
attrs2["path"] = store.printStorePath(j.second);
xml.writeEmptyElement("output", attrs2);
}
}
+17 -16
View File
@@ -38,8 +38,8 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
exist already. */
std::vector<StorePathWithOutputs> drvsToBuild;
for (auto & i : elems)
if (i.queryDrvPath() != "")
drvsToBuild.push_back({state.store->parseStorePath(i.queryDrvPath())});
if (auto drvPath = i.queryDrvPath())
drvsToBuild.push_back({*drvPath});
debug(format("building user environment dependencies"));
state.store->buildPaths(
@@ -55,7 +55,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
/* Create a pseudo-derivation containing the name, system,
output paths, and optionally the derivation path, as well
as the meta attributes. */
Path drvPath = keepDerivations ? i.queryDrvPath() : "";
std::optional<StorePath> drvPath = keepDerivations ? i.queryDrvPath() : std::nullopt;
DrvInfo::Outputs outputs = i.queryOutputs(true);
StringSet metaNames = i.queryMetaNames();
@@ -66,9 +66,9 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
auto system = i.querySystem();
if (!system.empty())
attrs.alloc(state.sSystem).mkString(system);
attrs.alloc(state.sOutPath).mkString(i.queryOutPath());
if (drvPath != "")
attrs.alloc(state.sDrvPath).mkString(i.queryDrvPath());
attrs.alloc(state.sOutPath).mkString(state.store->printStorePath(i.queryOutPath()));
if (drvPath)
attrs.alloc(state.sDrvPath).mkString(state.store->printStorePath(*drvPath));
// Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.sOutputs);
@@ -76,15 +76,15 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
for (const auto & [m, j] : enumerate(outputs)) {
(vOutputs.listElems()[m] = state.allocValue())->mkString(j.first);
auto outputAttrs = state.buildBindings(2);
outputAttrs.alloc(state.sOutPath).mkString(j.second);
outputAttrs.alloc(state.sOutPath).mkString(state.store->printStorePath(j.second));
attrs.alloc(j.first).mkAttrs(outputAttrs);
/* This is only necessary when installing store paths, e.g.,
`nix-env -i /nix/store/abcd...-foo'. */
state.store->addTempRoot(state.store->parseStorePath(j.second));
state.store->ensurePath(state.store->parseStorePath(j.second));
state.store->addTempRoot(j.second);
state.store->ensurePath(j.second);
references.insert(state.store->parseStorePath(j.second));
references.insert(j.second);
}
// Copy the meta attributes.
@@ -99,14 +99,16 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
(manifest.listElems()[n++] = state.allocValue())->mkAttrs(attrs);
if (drvPath != "") references.insert(state.store->parseStorePath(drvPath));
if (drvPath) references.insert(*drvPath);
}
/* Also write a copy of the list of user environment elements to
the store; we need it for future modifications of the
environment. */
std::ostringstream str;
manifest.print(str, true);
auto manifestFile = state.store->addTextToStore("env-manifest.nix",
fmt("%s", manifest), references);
str.str(), references);
/* Get the environment builder expression. */
Value envBuilder;
@@ -132,9 +134,9 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
state.forceValue(topLevel, [&]() { return topLevel.determinePos(noPos); });
PathSet context;
Attr & aDrvPath(*topLevel.attrs->find(state.sDrvPath));
auto topLevelDrv = state.store->parseStorePath(state.coerceToPath(*aDrvPath.pos, *aDrvPath.value, context));
auto topLevelDrv = state.coerceToStorePath(*aDrvPath.pos, *aDrvPath.value, context);
Attr & aOutPath(*topLevel.attrs->find(state.sOutPath));
Path topLevelOut = state.coerceToPath(*aOutPath.pos, *aOutPath.value, context);
auto topLevelOut = state.coerceToStorePath(*aOutPath.pos, *aOutPath.value, context);
/* Realise the resulting store expression. */
debug("building user environment");
@@ -158,8 +160,7 @@ bool createUserEnv(EvalState & state, DrvInfos & elems,
}
debug(format("switching to new user environment"));
Path generation = createGeneration(ref<LocalFSStore>(store2), profile,
store2->parseStorePath(topLevelOut));
Path generation = createGeneration(ref<LocalFSStore>(store2), profile, topLevelOut);
switchLink(profile, generation);
}
+5 -4
View File
@@ -61,12 +61,13 @@ void processExpr(EvalState & state, const Strings & attrPaths,
DrvInfos drvs;
getDerivations(state, v, "", autoArgs, drvs, false);
for (auto & i : drvs) {
Path drvPath = i.queryDrvPath();
auto drvPath = i.requireDrvPath();
auto drvPathS = state.store->printStorePath(drvPath);
/* What output do we want? */
std::string outputName = i.queryOutputName();
if (outputName == "")
throw Error("derivation '%1%' lacks an 'outputName' attribute ", drvPath);
throw Error("derivation '%1%' lacks an 'outputName' attribute", drvPathS);
if (gcRoot == "")
printGCWarning();
@@ -75,9 +76,9 @@ void processExpr(EvalState & state, const Strings & attrPaths,
if (++rootNr > 1) rootName += "-" + std::to_string(rootNr);
auto store2 = state.store.dynamic_pointer_cast<LocalFSStore>();
if (store2)
drvPath = store2->addPermRoot(store2->parseStorePath(drvPath), rootName);
drvPathS = store2->addPermRoot(drvPath, rootName);
}
std::cout << fmt("%s%s\n", drvPath, (outputName != "out" ? "!" + outputName : ""));
std::cout << fmt("%s%s\n", drvPathS, (outputName != "out" ? "!" + outputName : ""));
}
}
}
+13 -6
View File
@@ -2,6 +2,8 @@
#include "derivations.hh"
#include "dotgraph.hh"
#include "globals.hh"
#include "build-result.hh"
#include "gc-store.hh"
#include "local-store.hh"
#include "monitor-fd.hh"
#include "serve-protocol.hh"
@@ -427,11 +429,12 @@ static void opQuery(Strings opFlags, Strings opArgs)
store->computeFSClosure(
args, referrers, true, settings.gcKeepOutputs, settings.gcKeepDerivations);
Roots roots = store->findRoots(false);
auto & gcStore = requireGcStore(*store);
Roots roots = gcStore.findRoots(false);
for (auto & [target, links] : roots)
if (referrers.find(target) != referrers.end())
for (auto & link : links)
cout << fmt("%1% -> %2%\n", link, store->printStorePath(target));
cout << fmt("%1% -> %2%\n", link, gcStore.printStorePath(target));
break;
}
@@ -587,20 +590,22 @@ static void opGC(Strings opFlags, Strings opArgs)
if (!opArgs.empty()) throw UsageError("no arguments expected");
auto & gcStore = requireGcStore(*store);
if (printRoots) {
Roots roots = store->findRoots(false);
Roots roots = gcStore.findRoots(false);
std::set<std::pair<Path, StorePath>> roots2;
// Transpose and sort the roots.
for (auto & [target, links] : roots)
for (auto & link : links)
roots2.emplace(link, target);
for (auto & [link, target] : roots2)
std::cout << link << " -> " << store->printStorePath(target) << "\n";
std::cout << link << " -> " << gcStore.printStorePath(target) << "\n";
}
else {
PrintFreed freed(options.action == GCOptions::gcDeleteDead, results);
store->collectGarbage(options, results);
gcStore.collectGarbage(options, results);
if (options.action != GCOptions::gcDeleteDead)
for (auto & i : results.paths)
@@ -624,9 +629,11 @@ static void opDelete(Strings opFlags, Strings opArgs)
for (auto & i : opArgs)
options.pathsToDelete.insert(store->followLinksToStorePath(i));
auto & gcStore = requireGcStore(*store);
GCResults results;
PrintFreed freed(true, results);
store->collectGarbage(options, results);
gcStore.collectGarbage(options, results);
}
+1 -1
View File
@@ -114,7 +114,7 @@ App UnresolvedApp::resolve(ref<Store> evalStore, ref<Store> store)
installableContext.push_back(
std::make_shared<InstallableDerivedPath>(store, ctxElt.toDerivedPath()));
auto builtContext = build(evalStore, store, Realise::Outputs, installableContext);
auto builtContext = Installable::build(evalStore, store, Realise::Outputs, installableContext);
res.program = resolveString(*store, unresolved.program, builtContext);
if (!store->isInStore(res.program))
throw Error("app program '%s' is not in the Nix store", res.program);
+1 -1
View File
@@ -52,7 +52,7 @@ struct CmdBuild : InstallablesCommand, MixDryRun, MixJSON, MixProfile
void run(ref<Store> store) override
{
auto buildables = build(
auto buildables = Installable::build(
getEvalStore(), store,
dryRun ? Realise::Derivation : Realise::Outputs,
installables, buildMode);
+2 -2
View File
@@ -97,13 +97,13 @@ struct CmdBundle : InstallableCommand
throw Error("the bundler '%s' does not produce a derivation", bundler.what());
PathSet context2;
StorePath drvPath = store->parseStorePath(evalState->coerceToPath(*attr1->pos, *attr1->value, context2));
auto drvPath = evalState->coerceToStorePath(*attr1->pos, *attr1->value, context2);
auto attr2 = vRes->attrs->get(evalState->sOutPath);
if (!attr2)
throw Error("the bundler '%s' does not produce a derivation", bundler.what());
StorePath outPath = store->parseStorePath(evalState->coerceToPath(*attr2->pos, *attr2->value, context2));
auto outPath = evalState->coerceToStorePath(*attr2->pos, *attr2->value, context2);
store->buildPaths({ DerivedPath::Built { drvPath } });
+4 -3
View File
@@ -307,7 +307,7 @@ struct Common : InstallableCommand, MixProfile
for (auto & [installable_, dir_] : redirects) {
auto dir = absPath(dir_);
auto installable = parseInstallable(store, installable_);
auto builtPaths = toStorePaths(
auto builtPaths = Installable::toStorePaths(
getEvalStore(), store, Realise::Nothing, OperateOn::Output, {installable});
for (auto & path: builtPaths) {
auto from = store->printStorePath(path);
@@ -347,7 +347,7 @@ struct Common : InstallableCommand, MixProfile
if (path && hasSuffix(path->to_string(), "-env"))
return *path;
else {
auto drvs = toDerivations(store, {installable});
auto drvs = Installable::toDerivations(store, {installable});
if (drvs.size() != 1)
throw Error("'%s' needs to evaluate to a single derivation, but it evaluated to %d derivations",
@@ -511,7 +511,8 @@ struct CmdDevelop : Common, MixEnvironment
nixpkgsLockFlags);
shell = store->printStorePath(
toStorePath(getEvalStore(), store, Realise::Outputs, OperateOn::Output, bashInstallable)) + "/bin/bash";
Installable::toStorePath(getEvalStore(), store, Realise::Outputs, OperateOn::Output, bashInstallable))
+ "/bin/bash";
} catch (Error &) {
ignoreException();
}
+2 -2
View File
@@ -131,9 +131,9 @@ struct CmdDiffClosures : SourceExprCommand
void run(ref<Store> store) override
{
auto before = parseInstallable(store, _before);
auto beforePath = toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, before);
auto beforePath = Installable::toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, before);
auto after = parseInstallable(store, _after);
auto afterPath = toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, after);
auto afterPath = Installable::toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, after);
printClosureDiff(store, beforePath, afterPath, "");
}
};
+8 -3
View File
@@ -327,7 +327,7 @@ struct CmdFlakeCheck : FlakeCommand
if (!drvInfo)
throw Error("flake attribute '%s' is not a derivation", attrPath);
// FIXME: check meta attributes
return std::make_optional(store->parseStorePath(drvInfo->queryDrvPath()));
return drvInfo->queryDrvPath();
} catch (Error & e) {
e.addTrace(pos, hintfmt("while checking the derivation '%s'", attrPath));
reportError(e);
@@ -706,9 +706,14 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
auto [cursor, attrPath] = installable.getCursor(*evalState);
auto templateDir = cursor->getAttr("path")->getString();
auto templateDirAttr = cursor->getAttr("path");
auto templateDir = templateDirAttr->getString();
assert(store->isInStore(templateDir));
if (!store->isInStore(templateDir))
throw TypeError(
"'%s' was not found in the Nix store\n"
"If you've set '%s' to a string, try using a path instead.",
templateDir, templateDirAttr->getAttrPathStr());
std::vector<Path> files;
+60 -63
View File
@@ -61,6 +61,27 @@ struct ProfileElement
{
return std::tuple(describe(), storePaths) < std::tuple(other.describe(), other.storePaths);
}
void updateStorePaths(ref<Store> evalStore, ref<Store> store, Installable & installable)
{
// FIXME: respect meta.outputsToInstall
storePaths.clear();
for (auto & buildable : getBuiltPaths(evalStore, store, installable.toDerivedPaths())) {
std::visit(overloaded {
[&](const BuiltPath::Opaque & bo) {
storePaths.insert(bo.path);
},
[&](const BuiltPath::Built & bfd) {
// TODO: Why are we querying if we know the output
// names already? Is it just to figure out what the
// default one is?
for (auto & output : store->queryDerivationOutputMap(bfd.drvPath)) {
storePaths.insert(output.second);
}
},
}, buildable.raw());
}
}
};
struct ProfileManifest
@@ -105,7 +126,7 @@ struct ProfileManifest
for (auto & drvInfo : drvInfos) {
ProfileElement element;
element.storePaths = {state.store->parseStorePath(drvInfo.queryOutPath())};
element.storePaths = {drvInfo.queryOutPath()};
elements.emplace_back(std::move(element));
}
}
@@ -232,53 +253,25 @@ struct CmdProfileInstall : InstallablesCommand, MixDefaultProfile
{
ProfileManifest manifest(*getEvalState(), *profile);
std::vector<DerivedPath> pathsToBuild;
auto builtPaths = Installable::build(getEvalStore(), store, Realise::Outputs, installables, bmNormal);
for (auto & installable : installables) {
if (auto installable2 = std::dynamic_pointer_cast<InstallableFlake>(installable)) {
auto [attrPath, resolvedRef, drv] = installable2->toDerivation();
ProfileElement element;
ProfileElement element;
if (!drv.outPath)
throw UnimplementedError("CA derivations are not yet supported by 'nix profile'");
element.storePaths = {*drv.outPath}; // FIXME
if (auto installable2 = std::dynamic_pointer_cast<InstallableFlake>(installable)) {
// FIXME: make build() return this?
auto [attrPath, resolvedRef, drv] = installable2->toDerivation();
element.source = ProfileElementSource{
installable2->flakeRef,
resolvedRef,
attrPath,
};
pathsToBuild.push_back(DerivedPath::Built{drv.drvPath, StringSet{drv.outputName}});
manifest.elements.emplace_back(std::move(element));
} else {
auto buildables = build(getEvalStore(), store, Realise::Outputs, {installable}, bmNormal);
for (auto & buildable : buildables) {
ProfileElement element;
std::visit(overloaded {
[&](const BuiltPath::Opaque & bo) {
pathsToBuild.push_back(bo);
element.storePaths.insert(bo.path);
},
[&](const BuiltPath::Built & bfd) {
// TODO: Why are we querying if we know the output
// names already? Is it just to figure out what the
// default one is?
for (auto & output : store->queryDerivationOutputMap(bfd.drvPath)) {
pathsToBuild.push_back(DerivedPath::Built{bfd.drvPath, {output.first}});
element.storePaths.insert(output.second);
}
},
}, buildable.raw());
manifest.elements.emplace_back(std::move(element));
}
}
}
store->buildPaths(pathsToBuild);
element.updateStorePaths(getEvalStore(), store, *installable);
manifest.elements.push_back(std::move(element));
}
updateProfile(manifest.build(store));
}
@@ -373,15 +366,15 @@ struct CmdProfileRemove : virtual EvalCommand, MixDefaultProfile, MixProfileElem
if (removedCount == 0) {
for (auto matcher: matchers) {
if (const size_t* index = std::get_if<size_t>(&matcher)){
warn("'%d' is not a valid index in profile", *index);
} else if (const Path* path = std::get_if<Path>(&matcher)){
warn("'%s' does not match any paths in profile", *path);
} else if (const RegexPattern* regex = std::get_if<RegexPattern>(&matcher)){
warn("'%s' does not match any packages in profile", regex->pattern);
if (const size_t * index = std::get_if<size_t>(&matcher)){
warn("'%d' is not a valid index", *index);
} else if (const Path * path = std::get_if<Path>(&matcher)){
warn("'%s' does not match any paths", *path);
} else if (const RegexPattern * regex = std::get_if<RegexPattern>(&matcher)){
warn("'%s' does not match any packages", regex->pattern);
}
}
warn ("Try `nix profile list` to see the current profile.");
warn ("Use 'nix profile list' to see the current profile.");
}
updateProfile(newManifest.build(store));
}
@@ -407,8 +400,8 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
auto matchers = getMatchers(store);
// FIXME: code duplication
std::vector<DerivedPath> pathsToBuild;
std::vector<std::shared_ptr<Installable>> installables;
std::vector<size_t> indices;
auto upgradedCount = 0;
@@ -423,49 +416,53 @@ struct CmdProfileUpgrade : virtual SourceExprCommand, MixDefaultProfile, MixProf
Activity act(*logger, lvlChatty, actUnknown,
fmt("checking '%s' for updates", element.source->attrPath));
InstallableFlake installable(
auto installable = std::make_shared<InstallableFlake>(
this,
getEvalState(),
FlakeRef(element.source->originalRef),
"",
{element.source->attrPath},
{},
Strings{element.source->attrPath},
Strings{},
lockFlags);
auto [attrPath, resolvedRef, drv] = installable.toDerivation();
auto [attrPath, resolvedRef, drv] = installable->toDerivation();
if (element.source->resolvedRef == resolvedRef) continue;
printInfo("upgrading '%s' from flake '%s' to '%s'",
element.source->attrPath, element.source->resolvedRef, resolvedRef);
if (!drv.outPath)
throw UnimplementedError("CA derivations are not yet supported by 'nix profile'");
element.storePaths = {*drv.outPath}; // FIXME
element.source = ProfileElementSource{
installable.flakeRef,
installable->flakeRef,
resolvedRef,
attrPath,
};
pathsToBuild.push_back(DerivedPath::Built{drv.drvPath, {drv.outputName}});
installables.push_back(installable);
indices.push_back(i);
}
}
if (upgradedCount == 0) {
for (auto & matcher : matchers) {
if (const size_t* index = std::get_if<size_t>(&matcher)){
warn("'%d' is not a valid index in profile", *index);
} else if (const Path* path = std::get_if<Path>(&matcher)){
warn("'%s' does not match any paths in profile", *path);
} else if (const RegexPattern* regex = std::get_if<RegexPattern>(&matcher)){
warn("'%s' does not match any packages in profile", regex->pattern);
if (const size_t * index = std::get_if<size_t>(&matcher)){
warn("'%d' is not a valid index", *index);
} else if (const Path * path = std::get_if<Path>(&matcher)){
warn("'%s' does not match any paths", *path);
} else if (const RegexPattern * regex = std::get_if<RegexPattern>(&matcher)){
warn("'%s' does not match any packages", regex->pattern);
}
}
warn ("Use 'nix profile list' to see the current profile.");
}
store->buildPaths(pathsToBuild);
auto builtPaths = Installable::build(getEvalStore(), store, Realise::Outputs, installables, bmNormal);
for (size_t i = 0; i < installables.size(); ++i) {
auto & installable = installables.at(i);
auto & element = manifest.elements[indices.at(i)];
element.updateStorePaths(getEvalStore(), store, *installable);
}
updateProfile(manifest.build(store));
}
+12 -10
View File
@@ -384,13 +384,12 @@ StorePath NixRepl::getDerivationPath(Value & v) {
auto drvInfo = getDerivation(*state, v, false);
if (!drvInfo)
throw Error("expression does not evaluate to a derivation, so I can't build it");
Path drvPathRaw = drvInfo->queryDrvPath();
if (drvPathRaw == "")
throw Error("expression did not evaluate to a valid derivation (no drv path)");
StorePath drvPath = state->store->parseStorePath(drvPathRaw);
if (!state->store->isValidPath(drvPath))
throw Error("expression did not evaluate to a valid derivation (invalid drv path)");
return drvPath;
auto drvPath = drvInfo->queryDrvPath();
if (!drvPath)
throw Error("expression did not evaluate to a valid derivation (no 'drvPath' attribute)");
if (!state->store->isValidPath(*drvPath))
throw Error("expression evaluated to invalid derivation '%s'", state->store->printStorePath(*drvPath));
return *drvPath;
}
@@ -780,8 +779,11 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
str << "«derivation ";
Bindings::iterator i = v.attrs->find(state->sDrvPath);
PathSet context;
Path drvPath = i != v.attrs->end() ? state->coerceToPath(*i->pos, *i->value, context) : "???";
str << drvPath << "»";
if (i != v.attrs->end())
str << state->store->printStorePath(state->coerceToStorePath(*i->pos, *i->value, context));
else
str << "???";
str << "»";
}
else if (maxDepth > 0) {
@@ -798,7 +800,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
else
printStringValue(str, i.first.c_str());
str << " = ";
if (seen.find(i.second) != seen.end())
if (seen.count(i.second))
str << "«repeated»";
else
try {
+1 -1
View File
@@ -91,7 +91,7 @@ struct CmdShell : InstallablesCommand, MixEnvironment
void run(ref<Store> store) override
{
auto outPaths = toStorePaths(getEvalStore(), store, Realise::Outputs, OperateOn::Output, installables);
auto outPaths = Installable::toStorePaths(getEvalStore(), store, Realise::Outputs, OperateOn::Output, installables);
auto accessor = store->getFSAccessor();
+1 -1
View File
@@ -40,7 +40,7 @@ struct CmdShowDerivation : InstallablesCommand
void run(ref<Store> store) override
{
auto drvPaths = toDerivations(store, installables, true);
auto drvPaths = Installable::toDerivations(store, installables, true);
if (recursive) {
StorePathSet closure;
+4 -1
View File
@@ -2,6 +2,7 @@
#include "common-args.hh"
#include "shared.hh"
#include "store-api.hh"
#include "gc-store.hh"
using namespace nix;
@@ -32,12 +33,14 @@ struct CmdStoreDelete : StorePathsCommand
void run(ref<Store> store, std::vector<StorePath> && storePaths) override
{
auto & gcStore = requireGcStore(*store);
for (auto & path : storePaths)
options.pathsToDelete.insert(path);
GCResults results;
PrintFreed freed(true, results);
store->collectGarbage(options, results);
gcStore.collectGarbage(options, results);
}
};
+4 -1
View File
@@ -2,6 +2,7 @@
#include "common-args.hh"
#include "shared.hh"
#include "store-api.hh"
#include "gc-store.hh"
using namespace nix;
@@ -33,10 +34,12 @@ struct CmdStoreGC : StoreCommand, MixDryRun
void run(ref<Store> store) override
{
auto & gcStore = requireGcStore(*store);
options.action = dryRun ? GCOptions::gcReturnDead : GCOptions::gcDeleteDead;
GCResults results;
PrintFreed freed(options.action == GCOptions::gcDeleteDead, results);
store->collectGarbage(options, results);
gcStore.collectGarbage(options, results);
}
};
+2 -2
View File
@@ -82,9 +82,9 @@ struct CmdWhyDepends : SourceExprCommand
void run(ref<Store> store) override
{
auto package = parseInstallable(store, _package);
auto packagePath = toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, package);
auto packagePath = Installable::toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, package);
auto dependency = parseInstallable(store, _dependency);
auto dependencyPath = toStorePath(getEvalStore(), store, Realise::Derivation, operateOn, dependency);
auto dependencyPath = Installable::toStorePath(getEvalStore(), store, Realise::Derivation, operateOn, dependency);
auto dependencyPathHash = dependencyPath.hashPart();
StorePathSet closure;
+9
View File
@@ -0,0 +1,9 @@
source common.sh
sed -e "s|@localstatedir@|$TEST_ROOT/profile-var|g" -e "s|@coreutils@|$coreutils|g" < ../scripts/nix-profile.sh.in > $TEST_ROOT/nix-profile.sh
user=$(whoami)
rm -rf $TEST_HOME $TEST_ROOT/profile-var
mkdir -p $TEST_HOME
USER=$user $SHELL -e -c ". $TEST_ROOT/nix-profile.sh; set"
USER=$user $SHELL -e -c ". $TEST_ROOT/nix-profile.sh" # test idempotency
+6 -4
View File
@@ -11,13 +11,13 @@ let
args = ["sh" "-e" args.builder or (builtins.toFile "builder-${args.name}.sh" "if [ -e .attrs.sh ]; then source .attrs.sh; fi; eval \"$buildCommand\"")];
outputHashMode = "recursive";
outputHashAlgo = "sha256";
} // removeAttrs args ["builder" "meta"])
// { meta = args.meta or {}; };
} // removeAttrs args ["builder" "meta" "passthru"])
// { meta = args.meta or {}; passthru = args.passthru or {}; };
input1 = mkDerivation {
shell = busybox;
name = "build-remote-input-1";
buildCommand = "echo FOO > $out";
buildCommand = "echo hi-input1; echo FOO > $out";
requiredSystemFeatures = ["foo"];
outputHash = "sha256-FePFYIlMuycIXPZbWi7LGEiMmZSX9FMbaQenWBzm1Sc=";
};
@@ -25,7 +25,7 @@ let
input2 = mkDerivation {
shell = busybox;
name = "build-remote-input-2";
buildCommand = "echo BAR > $out";
buildCommand = "echo hi; echo BAR > $out";
requiredSystemFeatures = ["bar"];
outputHash = "sha256-XArauVH91AVwP9hBBQNlkX9ccuPpSYx9o0zeIHb6e+Q=";
};
@@ -34,6 +34,7 @@ let
shell = busybox;
name = "build-remote-input-3";
buildCommand = ''
echo hi-input3
read x < ${input2}
echo $x BAZ > $out
'';
@@ -46,6 +47,7 @@ in
mkDerivation {
shell = busybox;
name = "build-remote";
passthru = { inherit input1 input2 input3; };
buildCommand =
''
read x < ${input1}
+6 -4
View File
@@ -9,20 +9,20 @@ let
inherit system;
builder = busybox;
args = ["sh" "-e" args.builder or (builtins.toFile "builder-${args.name}.sh" "if [ -e .attrs.sh ]; then source .attrs.sh; fi; eval \"$buildCommand\"")];
} // removeAttrs args ["builder" "meta"])
// { meta = args.meta or {}; };
} // removeAttrs args ["builder" "meta" "passthru"])
// { meta = args.meta or {}; passthru = args.passthru or {}; };
input1 = mkDerivation {
shell = busybox;
name = "build-remote-input-1";
buildCommand = "echo FOO > $out";
buildCommand = "echo hi-input1; echo FOO > $out";
requiredSystemFeatures = ["foo"];
};
input2 = mkDerivation {
shell = busybox;
name = "build-remote-input-2";
buildCommand = "echo BAR > $out";
buildCommand = "echo hi; echo BAR > $out";
requiredSystemFeatures = ["bar"];
};
@@ -30,6 +30,7 @@ let
shell = busybox;
name = "build-remote-input-3";
buildCommand = ''
echo hi-input3
read x < ${input2}
echo $x BAZ > $out
'';
@@ -41,6 +42,7 @@ in
mkDerivation {
shell = busybox;
name = "build-remote";
passthru = { inherit input1 input2 input3; };
buildCommand =
''
read x < ${input1}
@@ -2,7 +2,7 @@ source common.sh
file=build-hook-ca-floating.nix
sed -i 's/experimental-features .*/& ca-derivations/' "$NIX_CONF_DIR"/nix.conf
enableFeatures "ca-derivations ca-references"
CONTENT_ADDRESSED=true
+8
View File
@@ -54,6 +54,14 @@ nix path-info --store $TEST_ROOT/machine3 --all \
| grep -v builder-build-remote-input-2.sh \
| grep builder-build-remote-input-3.sh
# Temporarily disabled because of https://github.com/NixOS/nix/issues/6209
if [[ -z "$CONTENT_ADDRESSED" ]]; then
for i in input1 input3; do
nix log --store $TEST_ROOT/machine0 --file "$file" --arg busybox $busybox passthru."$i" | grep hi-$i
done
fi
# Behavior of keep-failed
out="$(nix-build 2>&1 failing.nix \
--builders "$(join_by '; ' "${builders[@]}")" \
+1 -1
View File
@@ -37,7 +37,7 @@ testCutoffFor () {
}
testCutoff () {
# Don't directly build depenentCA, that way we'll make sure we dodn't rely on
# Don't directly build dependentCA, that way we'll make sure we don't rely on
# dependent derivations always being already built.
#testDerivation dependentCA
testCutoffFor transitivelyDependentCA
+1 -1
View File
@@ -1,5 +1,5 @@
source ../common.sh
sed -i 's/experimental-features .*/& ca-derivations ca-references/' "$NIX_CONF_DIR"/nix.conf
enableFeatures "ca-derivations ca-references"
restartDaemon
@@ -2,8 +2,6 @@ source ./common.sh
requireDaemonNewerThan "2.4pre20210625"
sed -i 's/experimental-features .*/& ca-derivations ca-references/' "$NIX_CONF_DIR"/nix.conf
export REMOTE_STORE_DIR="$TEST_ROOT/remote_store"
export REMOTE_STORE="file://$REMOTE_STORE_DIR"
-3
View File
@@ -2,9 +2,6 @@
source common.sh
# Globally enable the ca derivations experimental flag
sed -i 's/experimental-features = .*/& ca-derivations ca-references/' "$NIX_CONF_DIR/nix.conf"
export REMOTE_STORE_DIR="$TEST_ROOT/remote_store"
export REMOTE_STORE="file://$REMOTE_STORE_DIR"
-2
View File
@@ -2,8 +2,6 @@
source common.sh
sed -i 's/experimental-features .*/& ca-derivations ca-references nix-command flakes/' "$NIX_CONF_DIR"/nix.conf
FLAKE_PATH=path:$PWD
nix run --no-write-lock-file $FLAKE_PATH#runnable
-2
View File
@@ -2,8 +2,6 @@
source common.sh
sed -i 's/experimental-features .*/& ca-derivations ca-references nix-command flakes/' "$NIX_CONF_DIR"/nix.conf
CONTENT_ADDRESSED=true
cd ..
source ./nix-shell.sh
-2
View File
@@ -4,8 +4,6 @@ source common.sh
requireDaemonNewerThan "2.4pre20210626"
sed -i 's/experimental-features .*/& ca-derivations ca-references nix-command flakes/' "$NIX_CONF_DIR"/nix.conf
export NIX_TESTS_CA_BY_DEFAULT=1
cd ..
source ./post-hook.sh
-2
View File
@@ -4,8 +4,6 @@ source common.sh
requireDaemonNewerThan "2.4pre20210623"
sed -i 's/experimental-features .*/& ca-derivations ca-references nix-command flakes/' "$NIX_CONF_DIR"/nix.conf
export NIX_TESTS_CA_BY_DEFAULT=1
cd ..
source ./recursive.sh
-3
View File
@@ -1,8 +1,5 @@
source common.sh
# Globally enable the ca derivations experimental flag
sed -i 's/experimental-features = .*/& ca-derivations ca-references/' "$NIX_CONF_DIR/nix.conf"
clearStore
clearCache
+18 -8
View File
@@ -87,15 +87,20 @@ startDaemon() {
if [[ "$NIX_REMOTE" == daemon ]]; then
return
fi
# Start the daemon, wait for the socket to appear. !!!
# ‘nix-daemon’ should have an option to fork into the background.
# Start the daemon, wait for the socket to appear.
rm -f $NIX_DAEMON_SOCKET_PATH
PATH=$DAEMON_PATH nix daemon &
PATH=$DAEMON_PATH nix-daemon&
pidDaemon=$!
for ((i = 0; i < 300; i++)); do
if [[ -S $NIX_DAEMON_SOCKET_PATH ]]; then break; fi
if [[ -S $NIX_DAEMON_SOCKET_PATH ]]; then
DAEMON_STARTED=1
break;
fi
sleep 0.1
done
pidDaemon=$!
if [[ -z ${DAEMON_STARTED+x} ]]; then
fail "Didn’t manage to start the daemon"
fi
trap "killDaemon" EXIT
export NIX_REMOTE=daemon
}
@@ -103,10 +108,10 @@ startDaemon() {
killDaemon() {
kill $pidDaemon
for i in {0..100}; do
kill -0 $pidDaemon || break
kill -0 $pidDaemon 2> /dev/null || break
sleep 0.1
done
kill -9 $pidDaemon || true
kill -9 $pidDaemon 2> /dev/null || true
wait $pidDaemon || true
trap "" EXIT
}
@@ -167,10 +172,15 @@ needLocalStore() {
}
# Just to make it easy to find which tests should be fixed
buggyNeedLocalStore () {
buggyNeedLocalStore() {
needLocalStore
}
enableFeatures() {
local features="$1"
sed -i 's/experimental-features .*/& '"$features"'/' "$NIX_CONF_DIR"/nix.conf
}
set -x
if [[ -n "${NIX_DAEMON_PACKAGE:-}" ]]; then
+4
View File
@@ -9,6 +9,10 @@ outPath=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:
cmp $outPath fetchurl.sh
# Do not re-fetch paths already present.
outPath2=$(nix-build -vvvvv --expr 'import <nix/fetchurl.nix>' --argstr url file:///does-not-exist/must-remain-unused/fetchurl.sh --argstr sha256 $hash --no-out-link)
test "$outPath" == "$outPath2"
# Now using a base-64 hash.
clearStore
+3 -1
View File
@@ -89,9 +89,11 @@ nix_tests = \
build.sh \
ca/nix-run.sh \
db-migration.sh \
nix-profile.sh \
bash-profile.sh \
pass-as-file.sh \
describe-stores.sh \
nix-profile.sh \
suggestions.sh \
store-ping.sh
ifeq ($(HAVE_LIBCPUID), 1)
+94 -6
View File
@@ -1,9 +1,97 @@
source common.sh
sed -e "s|@localstatedir@|$TEST_ROOT/profile-var|g" -e "s|@coreutils@|$coreutils|g" < ../scripts/nix-profile.sh.in > $TEST_ROOT/nix-profile.sh
clearStore
clearProfiles
user=$(whoami)
rm -rf $TEST_HOME $TEST_ROOT/profile-var
mkdir -p $TEST_HOME
USER=$user $SHELL -e -c ". $TEST_ROOT/nix-profile.sh; set"
USER=$user $SHELL -e -c ". $TEST_ROOT/nix-profile.sh" # test idempotency
enableFeatures "ca-derivations ca-references"
restartDaemon
# Make a flake.
flake1Dir=$TEST_ROOT/flake1
mkdir -p $flake1Dir
cat > $flake1Dir/flake.nix <<EOF
{
description = "Bla bla";
outputs = { self }: with import ./config.nix; rec {
packages.$system.default = mkDerivation {
name = "profile-test-\${builtins.readFile ./version}";
builder = builtins.toFile "builder.sh"
''
mkdir -p \$out/bin
cat > \$out/bin/hello <<EOF
#! ${shell}
echo Hello \${builtins.readFile ./who}
EOF
chmod +x \$out/bin/hello
echo DONE
'';
__contentAddressed = import ./ca.nix;
outputHashMode = "recursive";
outputHashAlgo = "sha256";
};
};
}
EOF
printf World > $flake1Dir/who
printf 1.0 > $flake1Dir/version
printf false > $flake1Dir/ca.nix
cp ./config.nix $flake1Dir/
# Test upgrading from nix-env.
nix-env -f ./user-envs.nix -i foo-1.0
nix profile list | grep '0 - - .*-foo-1.0'
nix profile install $flake1Dir -L
[[ $($TEST_HOME/.nix-profile/bin/hello) = "Hello World" ]]
nix profile history
nix profile history | grep "packages.$system.default: ∅ -> 1.0"
nix profile diff-closures | grep 'env-manifest.nix: ε → ∅'
# Test upgrading a package.
printf NixOS > $flake1Dir/who
printf 2.0 > $flake1Dir/version
nix profile upgrade 1
[[ $($TEST_HOME/.nix-profile/bin/hello) = "Hello NixOS" ]]
nix profile history | grep "packages.$system.default: 1.0 -> 2.0"
# Test 'history', 'diff-closures'.
nix profile diff-closures
# Test rollback.
nix profile rollback
[[ $($TEST_HOME/.nix-profile/bin/hello) = "Hello World" ]]
# Test uninstall.
[ -e $TEST_HOME/.nix-profile/bin/foo ]
nix profile remove 0
(! [ -e $TEST_HOME/.nix-profile/bin/foo ])
nix profile history | grep 'foo: 1.0 -> ∅'
nix profile diff-closures | grep 'Version 3 -> 4'
# Test installing a non-flake package.
nix profile install --file ./simple.nix ''
[[ $(cat $TEST_HOME/.nix-profile/hello) = "Hello World!" ]]
nix profile remove 1
nix profile install $(nix-build --no-out-link ./simple.nix)
[[ $(cat $TEST_HOME/.nix-profile/hello) = "Hello World!" ]]
# Test wipe-history.
nix profile wipe-history
[[ $(nix profile history | grep Version | wc -l) -eq 1 ]]
# Test upgrade to CA package.
printf true > $flake1Dir/ca.nix
printf 3.0 > $flake1Dir/version
nix profile upgrade 0
nix profile history | grep "packages.$system.default: 1.0 -> 3.0"
# Test new install of CA package.
nix profile remove 0
printf 4.0 > $flake1Dir/version
printf Utrecht > $flake1Dir/who
nix profile install $flake1Dir
[[ $($TEST_HOME/.nix-profile/bin/hello) = "Hello Utrecht" ]]
[[ $(nix path-info --json $(realpath $TEST_HOME/.nix-profile/bin/hello) | jq -r .[].ca) =~ fixed:r:sha256: ]]
+36
View File
@@ -0,0 +1,36 @@
source common.sh
clearStore
cd "$TEST_HOME"
cat <<EOF > flake.nix
{
outputs = a: {
packages.$system = {
foo = 1;
fo1 = 1;
fo2 = 1;
fooo = 1;
foooo = 1;
fooooo = 1;
fooooo1 = 1;
fooooo2 = 1;
fooooo3 = 1;
fooooo4 = 1;
fooooo5 = 1;
fooooo6 = 1;
};
};
}
EOF
# Probable typo in the requested attribute path. Suggest some close possibilities
NIX_BUILD_STDERR_WITH_SUGGESTIONS=$(! nix build .\#fob 2>&1 1>/dev/null)
[[ "$NIX_BUILD_STDERR_WITH_SUGGESTIONS" =~ "Did you mean one of fo1, fo2, foo or fooo?" ]] || \
fail "The nix build stderr should suggest the three closest possiblities"
# None of the possible attributes is close to `bar`, so shouldn’t suggest anything
NIX_BUILD_STDERR_WITH_NO_CLOSE_SUGGESTION=$(! nix build .\#bar 2>&1 1>/dev/null)
[[ ! "$NIX_BUILD_STDERR_WITH_NO_CLOSE_SUGGESTION" =~ "Did you mean" ]] || \
fail "The nix build stderr shouldn’t suggest anything if there’s nothing relevant to suggest"
+4
View File
@@ -26,10 +26,14 @@ test_tarball() {
nix-build -o $TEST_ROOT/result '<foo>' -I foo=file://$tarball
nix-build -o $TEST_ROOT/result -E "import (fetchTarball file://$tarball)"
# Do not re-fetch paths already present
nix-build -o $TEST_ROOT/result -E "import (fetchTarball { url = file:///does-not-exist/must-remain-unused/$tarball; sha256 = \"$hash\"; })"
nix-build -o $TEST_ROOT/result -E "import (fetchTree file://$tarball)"
nix-build -o $TEST_ROOT/result -E "import (fetchTree { type = \"tarball\"; url = file://$tarball; })"
nix-build -o $TEST_ROOT/result -E "import (fetchTree { type = \"tarball\"; url = file://$tarball; narHash = \"$hash\"; })"
# Do not re-fetch paths already present
nix-build -o $TEST_ROOT/result -E "import (fetchTree { type = \"tarball\"; url = file:///does-not-exist/must-remain-unused/$tarball; narHash = \"$hash\"; })"
nix-build -o $TEST_ROOT/result -E "import (fetchTree { type = \"tarball\"; url = file://$tarball; narHash = \"sha256-xdKv2pq/IiwLSnBBJXW8hNowI4MrdZfW+SYqDQs7Tzc=\"; })" 2>&1 | grep 'NAR hash mismatch in input'
nix-instantiate --strict --eval -E "!((import (fetchTree { type = \"tarball\"; url = file://$tarball; narHash = \"$hash\"; })) ? submodules)" >&2
+3
View File
@@ -8,6 +8,8 @@ assert foo == "foo";
let
platforms = let x = "foobar"; in [ x x ];
makeDrv = name: progName: (mkDerivation {
name = assert progName != "fail"; name;
inherit progName system;
@@ -15,6 +17,7 @@ let
} // {
meta = {
description = "A silly test package with some \${escaped anti-quotation} in it";
inherit platforms;
};
});
-1
View File
@@ -9,7 +9,6 @@ clearProfiles
# Query installed: should be empty.
test "$(nix-env -p $profiles/test -q '*' | wc -l)" -eq 0
mkdir -p $TEST_HOME
nix-env --switch-profile $profiles/test
# Query available: should contain several.