Compare commits

..
235 Commits
Author SHA1 Message Date
Eelco Dolstra 5ed3a9db6a rl-2.7.md: Fix title
(cherry picked from commit f98d76ff1a)
2022-04-05 14:14:28 +02:00
Eelco Dolstra c9afca59e8 Merge pull request #6297 from NixOS/backport-6296-to-2.7-maintenance
[Backport 2.7-maintenance] Don't hide repeated values while generating manifest.nix
2022-03-22 21:01:24 +01:00
Eelco Dolstra 0a26f9ae4a Don't hide repeated values while generating manifest.nix
Fixes #6243.

(cherry picked from commit a0259a21a4)
2022-03-22 13:09:47 +00:00
Eelco Dolstra fc553fb632 printValue(): <REPEAT> -> «repeated»
This ensures that it doesn't get parsed as a valid Nix expression.

(cherry picked from commit 732296ddc0)
2022-03-22 13:09:47 +00:00
Eelco Dolstra ffe155abd3 Mark official release 2022-03-07 20:11:22 +01:00
Eelco Dolstra ad7c99ef20 Move rl-next.md to rl-2.7.md 2022-03-07 20:10:18 +01:00
Eelco Dolstra 504e3b2a8f Style 2022-03-07 20:07:43 +01:00
Eelco Dolstra fa614fac7f Merge branch 'GuillaumeDesforges/issue6192' of https://github.com/GuillaumeDesforges/nix 2022-03-07 19:58:06 +01:00
Eelco Dolstra 0123b9aec8 Tweak release notes 2022-03-07 19:56:10 +01:00
Eelco Dolstra 30ddd37873 Merge branch 'cli-suggestions' of https://github.com/thufschmitt/nix 2022-03-07 19:47:45 +01:00
Guillaume Desforges 436c6e900f Explicit error in flake init/new when not in store 2022-03-07 17:57:52 +01:00
regnat dcf4f77fac Merge or-suggestions.hh into suggestions.hh
No real need for keeping a separate header for such a simple class.

This requires changing a bit `OrSuggestions<T>::operator*` to not throw
an `Error` to prevent a cyclic dependency. But since this error is only
thrown on programmer error, we can replace the whole method by a direct
call to `std::get` which will raise its own assertion if needs be.
2022-03-07 17:49:02 +01:00
Eelco Dolstra 60a68a4fee Merge pull request #6173 from hercules-ci/fix-mkString-for-empty-string-view
Fix `mkString` for empty `string_view`
2022-03-07 17:10:41 +01:00
Robert Hensing da260f579d dupStringWithLen -> makeImmutableString
Refactor the `size == 0` logic into a new helper function that
replaces dupStringWithLen.
The name had to change, because unlike a `dup`-function, it does
not always allocate a new string.
2022-03-07 16:09:12 +01:00
Robert Hensing bbf55383e7 Value::mkPath: Avoid potential crash from null string_view 2022-03-07 16:09:12 +01:00
Robert Hensing 1b978596b5 Value::mkString: Avoid crash from null string_view 2022-03-07 16:09:12 +01:00
Eelco Dolstra c28e2b1b29 Tweak release notes 2022-03-07 11:30:40 +01:00
regnat 313bbc07a8 Implement operator<< for Suggestions
That way there’s no need to explicitely convert it to a string when
printing it
2022-03-07 10:09:10 +01:00
regnat fd45d85b41 Move OrSuggestions to its own header
Prevents a recursive inclusion
2022-03-07 10:09:10 +01:00
regnat b44cebd1fd Add a release-notes entry for the cli suggestions 2022-03-07 10:09:10 +01:00
regnat 91635206c0 Add some end-to-end tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat 98e361ad4c Also display suggestions for the commands using the eval cache
Make `nix build .#nix-armv8l-linux` work for example
2022-03-07 10:09:10 +01:00
regnat 2405bbbb5e Add some tests for the suggestions 2022-03-07 10:09:10 +01:00
regnat c0792b1546 Implement a suggestions mechanism
Each `Error` class now includes a set of suggestions, and these are printed by
the top-level handler.
2022-03-07 10:09:09 +01:00
Théophane Hufschmitt a2ace54fe4 Merge pull request #6029 from Ma27/nix-log-ssh-ng
ssh-ng: also store build logs to make them accessible by `nix log`
2022-03-07 09:51:40 +01:00
Théophane Hufschmitt 860016bcbf Explain why the log tests are disabled for CA derivations 2022-03-07 09:15:34 +01:00
Théophane Hufschmitt 2051c0f409 Merge pull request #6208 from sielicki/fix-url-github-oauth
Point to new github oauth docs url
2022-03-07 08:54:50 +01:00
Nicholas Sielicki 314852a10e Point to new github oauth docs url
Previous URL was 404'ing.
2022-03-06 17:01:14 -06:00
Maximilian Bosch 697d1dac01 tests: grep for string in nix log for remote-builds 2022-03-04 16:58:27 +01:00
Eelco Dolstra b09baf690b Merge pull request #6188 from obsidiansystems/store-gc-subclass
Factor out a `GcStore` interface
2022-03-03 20:53:10 +01:00
John Ericson 6636202356 Factor out a GcStore interface
Starts progress on #5729.

The idea is that we should not have these default methods throwing
"unimplemented". This is a small step in that direction.

I kept `addTempRoot` because it is a no-op, rather than failure. Also,
as a practical matter, it is called all over the place, while doing
other tasks, so the downcasting would be annoying.

Maybe in the future I could move the "real" `addTempRoot` to `GcStore`,
and the existing usecases use a `tryAddTempRoot` wrapper to downcast or
do nothing, but I wasn't sure whether that was a good idea so with a
bias to less churn I didn't do it yet.
2022-03-03 19:01:25 +00:00
Eelco Dolstra 391f4fcabe Merge pull request #6201 from edolstra/print-value
printValue(): Don't show repeated values
2022-03-03 14:35:16 +01:00
Eelco Dolstra e9c04c3351 Be more aggressive in hiding repeated values
We now memoize on Bindings / list element vectors rather than Values,
so that e.g. two Values that point to the same Bindings will be
printed only once.
2022-03-03 13:33:34 +01:00
Eelco Dolstra ecff9d969a printValue(): Don't show repeated values
Fixes #6157.
2022-03-03 13:18:23 +01:00
Eelco Dolstra 6097790863 Fix segfault in headerCallback()
https://hydra.nixos.org/build/168594664
2022-03-03 11:11:16 +01:00
Eelco Dolstra 885d709393 Merge pull request #6198 from edolstra/coerce-store-path
Add EvalState::coerceToStorePath() helper
2022-03-03 10:02:51 +01:00
Eelco DolstraandJohn Ericson a7c835e9cb Use C++11-style initializer
Co-authored-by: John Ericson <git@JohnEricson.me>
2022-03-03 10:02:11 +01:00
Eelco Dolstra b55d79728c Add EvalState::coerceToStorePath() helper
This is useful whenever we want to evaluate something to a store path
(e.g. in get-drvs.cc).

Extracted from the lazy-trees branch (where we can require that a
store path must come from a store source tree accessor).
2022-03-02 23:58:58 +01:00
Eelco Dolstra 3e3d0711d4 Merge pull request #6197 from edolstra/nix-profile-ca
nix profile: Support CA derivations
2022-03-02 23:48:52 +01:00
Eelco Dolstra d4538034b7 nix profile test: Restart daemon
Fixes

  nix-daemon: src/libstore/sqlite.cc:97: nix::SQLiteStmt::Use::Use(nix::SQLiteStmt&): Assertion `stmt.stmt' failed.

which happens because the daemon doesn't properly handle the case
where ca-derivations isn't enabled at daemon startup.
2022-03-02 23:08:49 +01:00
Eelco Dolstra 3a3821bcd7 Remove obsolete todo 2022-03-02 22:22:55 +01:00
Eelco Dolstra b0d65b3d11 Silence kill output 2022-03-02 22:22:23 +01:00
Eelco Dolstra d2586188fe tests/common.sh.in: Add enableFeatures helper 2022-03-02 21:48:25 +01:00
Eelco Dolstra 5d208cbe41 mk/run_test.sh: Add missing backslash 2022-03-02 21:36:46 +01:00
Eelco Dolstra f9375778ae nix profile: Add a test for non-flake packages 2022-03-02 20:56:40 +01:00
Eelco Dolstra 161f798aa1 nix profile: Support CA derivations 2022-03-02 20:38:51 +01:00
Eelco Dolstra 54888b92de Move installables-related operations 2022-03-02 19:19:51 +01:00
Eelco Dolstra b2da2a22c6 Merge pull request #6194 from edolstra/nix-profile
Add basic tests for 'nix profile'
2022-03-02 16:07:00 +01:00
Eelco Dolstra 5850fd17b4 Add basic tests for 'nix profile'
Fixes #6193.
2022-03-02 14:40:46 +01:00
Eelco Dolstra 1aca6b9f17 Merge pull request #4345 from bjornfor/installer-configurable-uid-gid
installer: allow overriding nix user GID and UIDs
2022-03-02 12:38:52 +01:00
Eelco Dolstra b39ef07414 Style 2022-03-02 11:46:15 +01:00
Eelco Dolstra c10865a46e tests: Rename nix-profile.sh -> bash-profile.sh 2022-03-02 11:21:00 +01:00
Eelco Dolstra 010ffc31f8 Remove stray debug line 2022-03-02 11:20:32 +01:00
Eelco Dolstra 03df331c3f Merge pull request #6189 from obsidiansystems/build-result-header
Move `BuildResult` defintion to its own header
2022-03-02 08:47:51 +01:00
Eelco Dolstra 3c66839b9d Merge pull request #6190 from andersk/bel
filterANSIEscapes: Ignore BEL character
2022-03-02 08:46:19 +01:00
Anders Kaseorg b5cd3e2d5c filterANSIEscapes: Ignore BEL character
GCC is not as good at music as it seems to think it is.  Fixes #4546.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-03-01 15:35:42 -08:00
John Ericson e862833ec6 Move BuildResult defintion to its own header
Just like we did for `ValidPathInfo` in
d92d4f85a5.
2022-03-01 19:43:07 +00:00
Bjørn Forsman f4d57aa490 installer: allow overriding nix user GID and UIDs
Needed to resolve conflict in case the default GID and UIDs are in use.
2022-03-01 19:17:41 +01:00
Eelco Dolstra 9087fe0760 Merge pull request #6187 from thufschmitt/allow-the-darwin-tests-to-be-flaky
Acknowledge that the macOS tests are flaky
2022-03-01 19:11:36 +01:00
regnat c511134a94 Acknowledge that the macOS tests are flaky
Restart the tests (at most once) on `unexpected EOF` errors.

This is truly ugly, but might prevent half of the CI runs to fail
because of https://github.com/NixOS/nix/issues/3605
2022-03-01 15:25:05 +01:00
Eelco Dolstra 0cb5af5000 Merge pull request #6185 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Use isValidPath, add comment
2022-03-01 13:15:42 +01:00
Robert Hensing b6deca7c0d fetchTree: Use isValidPath, add comment 2022-03-01 12:11:10 +01:00
Eelco Dolstra d974d2ad59 fetch{url,Tarball}: Remove 'narHash' attribute
This was introduced in #6174. However fetch{url,Tarball} are legacy
and we shouldn't have an undocumented attribute that does the same
thing as one that already exists ('sha256').
2022-03-01 11:30:26 +01:00
Eelco Dolstra 9ab81a9d38 Merge pull request #6183 from obsidiansystems/sort-config
Move some stuff from `Settings` to a new `FetchSettings`.
2022-03-01 10:52:55 +01:00
Théophane Hufschmitt 983c991652 Merge pull request #6174 from hercules-ci/fetchTree-reuse-local-paths
fetchTree: Do not re-fetch paths already present + refactor
2022-03-01 10:29:32 +01:00
John Ericson ea71da395f Move some stuff from Settings to a new FetchSettings.
Starting work on #5638

The exact boundary between `FetchSettings` and `EvalSettings` is not
clear to me, but that's fine. First lets clean out `libstore`, and then
worry about what, if anything, should be the separation between those
two.
2022-03-01 01:39:25 +00:00
Robert Hensing ee019d0afc Add EvalState::allowAndSetStorePathString helper
This switches addPath from `printStorePath` to `toRealPath`.
2022-02-28 21:37:49 +01:00
Robert Hensing 158280d8e9 fetchTree: Do not re-fetch paths already present 2022-02-28 21:37:49 +01:00
Théophane Hufschmitt 1c985428c4 Merge pull request #6179 from NixOS/properly-start-the-daemon-in-tests
tests: Fix the start of the daemon
2022-02-28 19:23:50 +01:00
Maximilian Bosch 6a8f1b548f logging.hh: json.hpp -> json_fwd.hpp 2022-02-28 17:27:52 +01:00
Maximilian Bosch 102cb39086 libstore/build: add a few explanatory comments; simplify 2022-02-28 17:27:52 +01:00
Maximilian Bosch 287642f132 tests: implement test for nix log with ssh-ng:// builds
A few notes:

* The `echo hi` is needed to make sure that a file that can be read by
  `nix log` is properly created (i.e. some output is needed). This is
  known and to be fixed in #6051.
* We explicitly ignore the floating-CA case here: the `$out` of `input3`
  depends on `$out` of `input2`. This means that there are actually two
  derivations - I assume that this is because at eval time (i.e.
  `nix-instantiate -A`) the hash of `input2` isn't known yet and the
  other .drv is created as soon as `input2` was built. This is another
  issue on its own, so we ignore the case here explicitly.
2022-02-28 17:27:52 +01:00
Maximilian Bosch cd92ea5885 libstore/derivation-goal: avoid double-parsing of JSON messages
To avoid that JSON messages are parsed twice in case of
remote builds with `ssh-ng://`, I split up the original
`handleJSONLogMessage` into three parts:

* `parseJSONMessage(const std::string&)` checks if it's a message in the
  form of `@nix {...}` and tries to parse it (and prints an error if the
  parsing fails).
* `handleJSONLogMessage(nlohmann::json&, ...)` reads the fields from the
  message and passes them to the logger.
* `handleJSONLogMessage(const std::string&, ...)` behaves as before, but
  uses the two functions mentioned above as implementation.

In case of `ssh-ng://`-logs the first two methods are invoked manually.
2022-02-28 17:27:52 +01:00
Maximilian Bosch 7a04839ea5 ssh-ng: also store build logs to make them accessible by nix log
Right now when building a derivation remotely via

    $ nix build -j0 -f . hello -L --builders 'ssh://builder'

it's possible later to read through the entire build-log by running
`nix log -f . hello`. This isn't possible however when using `ssh-ng`
rather than `ssh`.

The reason for that is that there are two different ways to transfer
logs in Nix through e.g. an SSH tunnel (that are used by `ssh`/`ssh-ng`
respectively):

* `ssh://` receives its logs from the fd pointing to `builderOut`. This
  is directly passed to the "log-sink" (and to the logger on each `\n`),
  hence `nix log` works here.
* `ssh-ng://` however expects JSON-like messages (i.e. `@nix {log data
  in here}`) and passes it directly to the logger without doing anything
  with the `logSink`. However it's certainly possible to extract
  log-lines from this format as these have their own message-type in the
  JSON payload (i.e. `resBuildLogLine`).

  This is basically what I changed in this patch: if the code-path for
  `builderOut` is not reached and a `logSink` is initialized, the
  message was successfully processed by the JSON logger (i.e. it's in
  the expected format) and the line is of the expected type (i.e.
  `resBuildLogLine`), the line will be written to the log-sink as well.

Closes #5079
2022-02-28 17:27:52 +01:00
regnat b8f8aef9d3 tests: Fix the start of the daemon
- Make sure that it starts even without the `nix-command` xp feature
- Fail if it doesn’t manage to start

This fixes a 30s wait for every test in `init.sh` as the daemon couldn’t
start, but the code was just waiting 30s and continuing as if everything
was all right.
2022-02-28 17:00:31 +01:00
Eelco Dolstra b592359c56 Merge pull request #6178 from edolstra/fix-darwin
Fix Darwin build
2022-02-28 16:13:00 +01:00
Eelco Dolstra b91500a14e Fix clang warning 2022-02-28 15:48:26 +01:00
Eelco Dolstra a949673a5b Fix Darwin build
Fixes #6169
2022-02-28 15:48:26 +01:00
Théophane Hufschmitt c2720797d5 Merge pull request #6072 from Ma27/fix-nix-why-depends-non-precise
nix/why-depends: fix output when not using `--precise`
2022-02-28 11:46:50 +01:00
Domen Kožar e85d565b5a Merge pull request #6123 from abathur/fix_6122
install-darwin: fix mount permission edge-case
2022-02-28 08:45:22 +00:00
Eelco Dolstra df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra 14b38d0887 xml-writer: Remove std aliases 2022-02-25 16:13:02 +01:00
Théophane Hufschmitt fd4b693ca2 Merge pull request #6164 from kamadorueda/issue-5859
refactor: remove verbose-build from docs
2022-02-25 15:29:59 +01:00
Kevin Amado b312d4d096 refactor: remove verbose-build from docs
- From what I see it is an implementation detail
  but is no longer configurable from the settings
2022-02-25 09:16:42 -05:00
Eelco Dolstra 9d4697453f ExprAttrs::show(): Show attrs in sorted order
Issue #6160.
2022-02-25 09:21:04 +01:00
Eelco Dolstra 59683733f5 Merge pull request #6089 from edolstra/dot-default
Replace defaultBla.$system with bla.$system.default
2022-02-24 21:16:58 +01:00
Eelco Dolstra cf7f98483a Merge pull request #6161 from edolstra/locked
libfetchers: Rename immutable -> locked
2022-02-24 19:01:33 +01:00
Eelco Dolstra 8518cebfce libfetchers: Rename immutable -> locked
This is more consistent with flake terminology.
2022-02-24 18:09:00 +01:00
Eelco Dolstra 3317866060 Merge pull request #6150 from GuillaumeDesforges/doc/builtins-function-anchor
Add anchor to builtin functions in HTML documentation
2022-02-24 17:54:48 +01:00
Eelco Dolstra 7ec244aec2 Merge pull request #6159 from NixOS/more-eager-daemon-polling-in-tests
testS: poll more eagerly for the daemon start/stop
2022-02-24 17:54:09 +01:00
Eelco Dolstra 5f88411af0 Merge pull request #6158 from NixOS/reorder-tests
Sort the tests by wall time
2022-02-24 17:53:48 +01:00
regnat 9c470cb969 testS: poll more eagerly for the daemon start/stop
Polling every 1 second means that even the simplest test takes at least
2 seconds. We can reasonably poll 1/10 of that to make things much
  quicker (esp. given that most of the time 0.1s is enough for the
  daemon to be started or stopped)
2022-02-24 15:00:23 +01:00
regnat 42766f8924 Sort the tests by wall time
The tests are scheduled in the order they appear, so running the long
ones first slightly improves the scheduling.

On my machine, this decreases the time of `make install` from 40s to 36s
2022-02-24 14:32:31 +01:00
Guillaume Desforges cfbf9ee2ce Anchor with prefix 2022-02-24 13:50:50 +01:00
Théophane Hufschmitt 6524eb4b77 Merge pull request #5342 from Misterio77/add-sourcehut
Add support for sourcehut input scheme
2022-02-24 09:09:21 +01:00
Gabriel Fontes 770f3af31d add sourcehut integration test 2022-02-23 11:58:09 -03:00
Guillaume Desforges 6462ee61c7 Anchor link for builtin functions in HTML doc 2022-02-23 15:07:01 +01:00
Guillaume Desforges 2cc645a91a Add html make target 2022-02-23 15:06:13 +01:00
Eelco Dolstra 3144b373a4 Merge pull request #6147 from NixOS/include-outputs-doc
Precise the doc for `--include-outputs`
2022-02-23 10:22:20 +01:00
Théophane Hufschmitt 54f07b66c8 Precise the doc for --include-outputs
Make it explicit that it only includes the existing outputs and not the ones that haven’t been realised
2022-02-23 09:17:08 +01:00
Travis A. Everett ba9a8c4b3d install-darwin: track mount permission edge-case fix
Same as 1fd127a068, but applied to a
code path (volume_pass_works -> verify_volume_pass) that the reporting
user didn't hit and wasn't able to trigger manually. I am not certain
but I suspect it will be easier to add prophylactically than to debug
if its absence causes trouble some day.
2022-02-22 12:44:15 -06:00
Eelco Dolstra 1a6548ca75 Update docs 2022-02-22 14:32:56 +01:00
Eelco Dolstra 38eea2c503 Update release notes 2022-02-22 14:23:39 +01:00
Eelco Dolstra d4428d00ae nix flake check: Warn about deprecated flake outputs 2022-02-22 14:19:39 +01:00
Eelco Dolstra 162fbe31ff Replace defaultBla.$system with bla.$system.default
This also simplifies some InstallableFlake logic and fixes 'nix
bundle' parsing its installable twice.

Fixes #5532.
2022-02-22 11:47:41 +01:00
Eelco Dolstra caf5172945 Merge pull request #6136 from matthewbauer/profile-upgrade-verbosity
Add verbosity to nix profile upgrade
2022-02-21 22:30:47 +01:00
Eelco Dolstra 3848a8edb8 Merge pull request #6139 from edolstra/no-std-aliases
Remove std aliases
2022-02-21 18:18:42 +01:00
Eelco Dolstra 8ffb09a08a Fix macOS build 2022-02-21 17:14:22 +01:00
Matthew BauerandEelco Dolstra 3cd958849b Apply suggestions from code review
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-02-21 10:04:04 -06:00
Eelco Dolstra b98ce19544 Merge pull request #6138 from nmattia/nm-clarify-manpath
Document usage of MANPATH in nix-profile.sh
2022-02-21 16:54:01 +01:00
Eelco Dolstra 36c7b12f33 Remove std::string alias 2022-02-21 16:37:25 +01:00
Eelco Dolstra 1ac2664472 Remove std::vector alias 2022-02-21 16:32:34 +01:00
Eelco Dolstra fe9afb65bb Remove std::set alias 2022-02-21 16:28:23 +01:00
Eelco Dolstra afcdc7606c Remove std::list alias 2022-02-21 16:25:12 +01:00
Eelco Dolstra e2422c4582 Merge pull request #6052 from hercules-ci/issue-3294-fix-interruptCallback-deadlock
Fix deadlocked nix-daemon zombies on darwin #3294
2022-02-21 16:21:45 +01:00
Robert Hensing ddb6740e7d triggerInterrupt: Refactor to use break 2022-02-21 15:43:43 +01:00
Nicolas Mattia 44b7d104b4 Document usage of MANPATH in nix-profile.sh
While trying to figure out how `nix-env`/`nix profile` work I had a hard
time understand how man pages were being installed.

Took me quite some time to figure this out, thought it might be useful
to others too!
2022-02-21 13:35:55 +01:00
Eelco Dolstra f22b9e72f5 Merge pull request #6120 from mayflower/print-full-names
path-info: use full store paths when we have them
2022-02-21 11:03:20 +01:00
Matthew Bauer 61295b910f Add verbosity to nix profile upgrade
Followup of https://github.com/NixOS/nix/pull/6086. This time adding a
warning if no packages are upgraded.
2022-02-20 23:46:11 -06:00
Travis A. Everett 1fd127a068 install-darwin: fix mount permission edge-case
Fixes #6122, which reports a problem with trying to run the installer
under another user (probably: user is not the disk "owner" and thus
can't mount the volume).
2022-02-18 23:15:37 -06:00
Linus Heckemann dbdc63bc41 path-info: use full store paths when we have them
Fixes #5645
2022-02-19 00:32:05 +01:00
Eelco Dolstra 9bc03adbba Integrate push-docker.sh into the release script
This also makes sure that we get the Docker images from the same Hydra
eval, rather than the latest build from job/nix/.../dockerImage, which
may not be the same.
2022-02-18 13:58:01 +01:00
Eelco Dolstra 50e3840f14 Merge remote-tracking branch 'origin/script-to-make-docker-release' 2022-02-18 10:55:07 +01:00
Eelco Dolstra 26e99c817c Merge pull request #6114 from Radvendii/welcomeText
add release notes for welcomeText
2022-02-18 10:52:03 +01:00
Rok Garbas f0de5fb8e7 remove the manifest before creating them 2022-02-18 00:32:38 +01:00
Rok Garbas bf435664d7 Merge remote-tracking branch 'origin/master' into script-to-make-docker-release 2022-02-18 00:15:23 +01:00
Taeer Bar-Yam 219fa2e43d add release notes for welcomeText 2022-02-17 15:17:20 -05:00
Eelco Dolstra a768e85e2f Merge pull request #6103 from Radvendii/welcomeText
add descriptive output when creating templates
2022-02-17 21:06:10 +01:00
Taeer Bar-Yam f56dd3a36b make flake template welcomeText markdown 2022-02-17 13:59:32 -05:00
Eelco Dolstra b24d541c34 Merge pull request #6110 from layus/patch-2
Create daemon-socket folder during install
2022-02-17 13:00:20 +01:00
Guillaume Maudoux 1bec333788 Create to daemon-socket folder during install 2022-02-17 09:32:15 +01:00
Eelco Dolstra aa5b83d93c InputScheme::fetch(): Return a StorePath instead of a Tree 2022-02-16 11:14:01 +01:00
Eelco Dolstra 2d6d9a28eb addToStoreFromDump(): Take std::string_view 2022-02-16 11:02:35 +01:00
Eelco Dolstra f450edc78b Merge pull request #6104 from rrbutani/master
Fix formatting for options with "machine-specific" defaults
2022-02-16 10:47:51 +01:00
Rahul Butani d82cf4a016 manual: fix formatting for options with "machine-specific" defaults 2022-02-15 15:44:05 -06:00
Taeer Bar-Yam f3a2940e70 add descriptive output when creating templates
this includes a `welcomeText` attribute which can be set in the
template, as well as outputing which files were created.
2022-02-15 11:50:14 -05:00
Gabriel Fontes 72e8f94081 add sourcehut input scheme 2022-02-14 23:53:01 -03:00
Eelco Dolstra 94992a9196 Merge pull request #6100 from edolstra/installables-cleanup
InstallableFlake: Default attr paths cleanup
2022-02-14 22:06:11 +01:00
Eelco Dolstra 023e459777 InstallableFlake: Default attr paths cleanup
This removes some duplicated logic, and fixes "nix bundle" parsing its
installable twice.
2022-02-14 21:06:11 +01:00
Eelco Dolstra 744a101a36 Merge pull request #6092 from Kha/sandbox-wo-userns
Fix using sandbox without user namespaces
2022-02-14 11:40:08 +01:00
Sebastian Ullrich c437e1326d Fix using sandbox without user namespaces 2022-02-12 16:28:36 +01:00
Eelco Dolstra 4d67ecbbb2 Merge pull request #6085 from edolstra/fix-flake-defaults
parseInstallables(): Don't try the flake attr path prefixes when no fragment is specified
2022-02-11 18:00:00 +01:00
Eelco Dolstra 36845dc9a3 Merge pull request #6086 from tomberek/profile_remove
profile: add verbosity
2022-02-11 16:47:11 +01:00
Tom Bereknyei 270fb5f192 profile: add verbosity
warn if there are no matches and give notice of removing packages as
they are found
2022-02-11 10:44:33 -05:00
Eelco Dolstra cdc90c2776 parseInstallables(): Don't try the flake attr path prefixes when no fragment is specified
Fixes #5880.
2022-02-11 15:50:12 +01:00
Eelco Dolstra d2f9a081b8 flake.nix: Fix indent 2022-02-11 14:45:46 +01:00
Eelco Dolstra 5b809f9e0e check-hydra-status.sh: Ignore unfinished builds 2022-02-10 21:15:07 +01:00
Maximilian Bosch 5b586575ac nix/why-depends: fix output when not using --precise
On Nix 2.6 the output of `nix why-depends --all` seems to be somewhat
off:

    $ nix why-depends /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv /nix/store/srn5jbs1q30jpybdmxqrwskyny659qgc-nix-2.6.drv --derivation  --extra-experimental-features nix-command  --all
    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
        └───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
        │   └───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
        | [...]
            └───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
                └───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
                    └───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
                        └───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv

Please note that `[...]-system-units.drv` is supposed to be a direct
child of `[...]-etc.drv`.

The reason for that is that each new level printed by `printNode` is
four spaces off in comparison to `nix why-depends --precise` because the
recursive `printNode()` only prints the path and not the `tree*`-chars in
the case of `--precise` and in this format the path is four spaces further
indented, i.e. on a newline, but on the same level as the path's children, i.e.

    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
    └───/: …1-p8.drv",["out"]),("/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv",["out"]),("/nix/store/…
        → /nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv

As you can see `[...]-etc.drv` is a direct child of the root, but four
spaces indented. This logic was directly applied to the code-path with
`precise=false` which resulted in `tree*` being printed four spaces too
deep.

In case of no `--precise`, `hits[hash]` is empty and the path itself
should be printed rather than hits using the same logic as for `hits[hash]`.

With this fix, the output looks correct now:

    /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
    └───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
        ├───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
        | [...]
        └───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
            └───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
                └───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
                    └───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv
2022-02-10 14:26:00 +01:00
Eelco Dolstra b8d57e2883 check-hydra-status.sh: Improve error behaviour 2022-02-10 11:10:58 +01:00
Eelco Dolstra 52f52319ad Merge pull request #6067 from trofi/revert-6060-prefer-inplace-libs
Revert "mk: prefert inplace library paths to system ones (take 2)"
2022-02-09 14:24:54 +01:00
Sergei Trofimovich 28b9bd784c Revert "mk: prefert inplace library paths to system ones (take 2)" 2022-02-09 13:00:53 +00:00
Eelco Dolstra 0b3d8e1a29 Merge pull request #6060 from trofi/prefer-inplace-libs
mk: prefert inplace library paths to system ones (take 2)
2022-02-08 19:49:21 +01:00
Sergei Trofimovich 579dcbabd5 mk: prefert inplace library paths to system ones (take 2)
It's a second attempt to merge the change. Previous attempt
was reverted in b976b34a5b.
Since then underlying failure exposed by original change was
fixed by https://github.com/NixOS/nix/pull/5354.

Below goes description of original change:

The link failure happens on a system with stable nix-2.3.15
installed in /usr/lib64 (it's libutil.so API differs from master):

```
LANG=C make V=1
g++ -o /home/slyfox/dev/git/nix/src/libstore/libnixstore.so \
    -shared -L/usr/lib64 -Wl,--no-copy-dt-needed-entries \
    src/libstore/binary-cache-store.o ... src/libstore/uds-remote-store.o \
    -lsqlite3 -lcurl -lsodium -pthread -ldl -lseccomp -Wl,-z,defs -Wl,-soname=libnixstore.so
      -Wl,-rpath,/home/slyfox/dev/git/nix/src/libutil -Lsrc/libutil -lnixutil
ld: src/libstore/binary-cache-store.o: in function `nix::BinaryCacheStore::BinaryCacheStore(
    std::map<std::__cxx11::basic_string<char, std::char_traits<char>, ...
nix/src/libstore/binary-cache-store.cc:30: undefined reference to `nix::readFile(
    std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)' ...
...
```

This happens due to `-L/usr/lib64 -Lsrc/libutil` search path ordering.
The change turns it into `-Lsrc/libutil -L/usr/lib64`.

Closes: https://github.com/NixOS/nix/issues/3087
2022-02-07 23:39:33 +00:00
Eelco Dolstra 7c64a9dfd4 Merge pull request #6054 from lincolnauster/lf-reporterr
repl/load-flake: throw error if path isn't specified
2022-02-07 20:44:10 +01:00
Eelco Dolstra 725817c223 Merge pull request #6057 from trofi/fix-daemon-crashes
Make sure no exceptions leave ignoreException()
2022-02-07 20:42:19 +01:00
Sergei Trofimovich 3ec02deb20 Make sure no exceptions leave ignoreException()
I noticed that occasional Ctrl-C leaves *.lock files around.
`nix-daemon`'s journal logs contained crashes like:

    nix-daemon[30416]: terminate called after throwing an instance of 'nix::SysError'
    nix-daemon[30416]:   what():  error: writing to file: Broken pipe

And core dump backtraces pointed at `teriminate()` call from
destructors:

    ...
    _Unwind_Resume ()
    nix::ignoreException() ()
    nix::LocalDerivationGoal::~LocalDerivationGoal()
    ...

    void ignoreException()
    {
        try {
            throw;
        } catch (std::exception & e) {
            printError("error (ignored): %1%", e.what());
        }
    }

The crashes happen when client side closes early and printError() throws
an IO error.

The change wraps `ignoreException()` into blanket `try { ... } catch (...) {}`.

Closes: https://github.com/NixOS/nix/issues/6046
2022-02-07 16:20:56 +00:00
lincoln austerandEelco Dolstra b1abfcd0c2 fix markup
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2022-02-07 08:35:50 -07:00
Eelco Dolstra 97e02c23bd Fix 'basic_string::_M_construct null not valid' in interrupted download
Fixes #5985.
2022-02-07 10:53:47 +01:00
lincoln auster [they/them] c23501a3b2 repl/load-flake: throw error if path isn't specified 2022-02-06 16:29:42 -07:00
Robert Hensing c3b942e0fc Don't hold interruptCallbacks lock during interrupt handling
This changes the representation of the interrupt callback list to
be safe to use during interrupt handling.

Holding a lock while executing arbitrary functions is something to
avoid in general, because of the risk of deadlock.

Such a deadlock occurs in https://github.com/NixOS/nix/issues/3294
where ~CurlDownloader tries to deregister its interrupt callback.

This happens during what seems to be a triggerInterrupt() by the
daemon connection's MonitorFdHup thread. This bit I can not confirm
based on the stack trace though; it's based on reading the code,
so no absolute certainty, but a smoking gun nonetheless.
2022-02-06 13:53:28 +01:00
Eelco Dolstra 4369771870 Merge pull request #6040 from matthewbauer/allow-missing-override-input
Allow missing flake.nix for --override-input target
2022-02-06 13:22:41 +01:00
Eelco Dolstra 3b64569601 Merge pull request #6047 from SuperSandro2000/nix-shell-BASH
nix-shell: set BASH variable to correct shell
2022-02-06 13:19:52 +01:00
Eelco Dolstra 93293fc66b Merge pull request #6042 from pennae/fix-repl-a
fix nix repl not overriding existing bindings in :a
2022-02-06 13:17:58 +01:00
Eelco Dolstra 9148be6bfc Merge pull request #6043 from Ma27/fix-comp
completions: pipe stderr to /dev/null
2022-02-06 13:17:05 +01:00
Sandro Jäckel 45eeb2fd6a nix-shell: set BASH variable to correct shell 2022-02-05 17:56:51 +01:00
Maximilian Bosch 9d840758a8 completions: pipe stderr to /dev/null
This fixes weird issues where e.g.

    nix build -L .#<tab>

deletes the current line from the prompt.
2022-02-04 22:43:16 +01:00
pennae 1daf1babf9 fix nix repl not overriding existing bindings in :a
previously :a would override old bindings of a name with new values if the added
set contained names that were already bound. in nix 2.6 this doesn't happen any
more, which is potentially confusing.

fixes #6041
2022-02-04 15:27:59 +01:00
Matthew Bauer f222fba4dc Allow missing flake.nix for --override-input target
At this point, we don’t know if the input is a flake or not. So, we
should allow the user to override the input with a directory without a
flake.nix.

Ideally, we could figure whether the input was originally a flake or
not, but that would require instantiating the whole flake. So just
allow it to be missing here, and rely on checks later on to verify the
input for us.
2022-02-03 20:51:47 -06:00
Eelco Dolstra bd383d1b6f Make most calls to determinePos() lazy 2022-02-04 00:33:21 +01:00
Eelco Dolstra 4c755c3b3f Merge branch 'issue-3505' of https://github.com/kamadorueda/nix 2022-02-04 00:33:13 +01:00
Eelco Dolstra 1aa5994e6d Merge pull request #5456 from tomberek/bundler_drv
bundle: pass drv attrset instead of path
2022-02-03 23:40:20 +01:00
tomberek 12ff354d01 Merge branch 'master' into bundler_drv 2022-02-03 02:39:18 -05:00
Eelco Dolstra fcb33440b6 Merge pull request #6036 from tweag/balsoft/and-yet-another-follows-bugfix
Flake follows: resolve all follows to absolute
2022-02-02 22:52:08 +01:00
Alexander Bantyev 169ea0b83f Flake follows: resolve all follows to absolute
It's not possible in general to know in computeLocks, relative to
which path the follows was intended to be. So, we always resolve
follows to their absolute states when we encounter them (which can
either be in parseFlakeInput or computeLocks' fake input population).

Fixes https://github.com/NixOS/nix/issues/6013
Fixes https://github.com/NixOS/nix/issues/5609
Fixes https://github.com/NixOS/nix/issues/5697 (again)
2022-02-02 21:41:45 +03:00
Eelco Dolstra 17e3f353df Merge branch 'parser-improvements' of https://github.com/pennae/nix 2022-02-02 12:45:44 +01:00
Eelco Dolstra cd35bbbeef Merge branch 'more-stringviews' of https://github.com/pennae/nix 2022-02-02 12:38:37 +01:00
Eelco Dolstra 73d5f38a47 Require lowdown 0.9.0
Fixes #6021.
2022-02-01 10:44:19 +01:00
Eelco Dolstra 59b6afec07 Merge pull request #6022 from thkoch2001/fix-lowdown_libs
use LOWDOWN_LIBS variable
2022-01-31 13:40:54 +01:00
Eelco Dolstra cc730bd46b Merge pull request #5990 from lincolnauster/flakes-nixconfig-docs
flakes: document nixConfig option
2022-01-31 10:21:15 +01:00
Thomas Koch 43509cc69d use LOWDOWN_LIBS variable
fixes: #5931
2022-01-30 20:59:58 +02:00
Eelco Dolstra 08fc3d6552 Merge pull request #6018 from dtzWill/fix/issue-6017
canonPath: fix missing slash when resolving links
2022-01-30 12:32:02 +01:00
Eelco Dolstra 8915f16aab Merge pull request #6019 from thkoch2001/fix_spelling_mistakes
fix spelling mistakes reported by Debian's lintian tool
2022-01-30 12:31:16 +01:00
Thomas Koch 85b1427662 fix spelling mistakes reported by Debian's lintian tool 2022-01-30 10:51:39 +02:00
Will Dietz a0357abda7 canonPath: fix missing slash when resolving links
Fixes #6017
2022-01-29 16:32:27 -06:00
Tom Bereknyei 6e5e64fc74 bundler: suggested doc fixes 2022-01-28 10:25:05 -05:00
tomberek 2bf96bd9f2 Merge branch 'master' into bundler_drv 2022-01-28 10:18:29 -05:00
Tom Bereknyei 73e82ae954 bundler: tests various combinations of referring to installables 2022-01-28 10:17:51 -05:00
Tom Bereknyei 4ebc50d92e bundler: revert default behavior to use defaultApp
Bundlers are now responsible for correctly handling their inputs which
are no longer constrained to be (Drv->Drv)->Drv->Drv, but can be of
type (attrset->Drv)->attrset->Drv.
2022-01-28 09:56:58 -05:00
Eelco Dolstra 4bf6af7b55 Remove a repeated std::move in a for loop 2022-01-28 15:10:43 +01:00
pennae d439dceb3b optionally return string_view from coerceToString
we'll retain the old coerceToString interface that returns a string, but callers
that don't need the returned value to outlive the Value it came from can save
copies by using the new interface instead. for values that weren't stringy we'll
pass a new buffer argument that'll be used for storage and shouldn't be
inspected.
2022-01-27 22:15:30 +01:00
Domen Kožar f05fefcd03 Merge pull request #5951 from abathur/install_add_getconf_fallback
install-darwin: dodge bash 3.2 command bug
2022-01-27 20:18:11 +01:00
pennae 41d70a2fc8 return string_views from forceString*
once a string has been forced we already have dynamic storage allocated for it,
so we can easily reuse that storage instead of copying.
2022-01-27 17:15:43 +01:00
pennae 0d7fae6a57 convert a for more utilities to string_view 2022-01-27 17:15:43 +01:00
Eelco Dolstra 558c4ee3e3 Merge pull request #6001 from NixOS/fix-nix-path
Don’t require `NIX_PATH` entries to be valid paths
2022-01-27 17:04:02 +01:00
Eelco Dolstra 27b4056154 Merge pull request #6000 from NixOS/use-flakes-in-ci
Use the `nix` command (and flakes) in the CI
2022-01-27 17:01:45 +01:00
regnat fcdc60ed22 Don’t require NIX_PATH entries to be valid paths
It’s totally valid to have entries in `NIX_PATH` that aren’t valid paths
(they can even be arbitrary urls or `channel:<channel-name>`).

Fix #5998 and #5980
2022-01-27 16:26:39 +01:00
regnat 7bd85a3bf6 Use the nix command (and flakes) in the CI
Apart from a slight simplification and a bit of dogfooding, this also
make the cache behavior more predictable.
For example `nix build .` and `nix build nix/$(git rev-parse HEAD)` will
yield the exact same path, while their “intuitive” non-flake equivalents
(`nix-build` and
`nix-build https://github.com/nixos/nix/archives/$(git rev-parse HEAD).tar.gz`)
don’t.

This was a pain for example in https://github.com/NixOS/nix/pull/5059

Also, the `bar-with-logs` log format is imho nicer (even in an
non-interactive context) because prefixing each log line with the name
of the derivation that produced it makes it much easier to follow what’s
going on.
2022-01-26 16:41:37 +01:00
Eelco Dolstra 1fe3bfdeaf Merge pull request #5997 from NixOS/test-nix-store-ping
Fix the `store ping` test
2022-01-26 15:22:15 +01:00
Eelco Dolstra b0de24cc89 Merge pull request #5996 from edolstra/remove-nlohmann
Stop vendoring nlohmann_json
2022-01-26 14:49:34 +01:00
regnat 3dc1418216 Fix the store ping test 2022-01-26 14:15:03 +01:00
Eelco Dolstra 9691f86ff7 Stop vendoring nlohmann_json 2022-01-26 11:50:53 +01:00
Eelco Dolstra e9d2ac6d7f Merge pull request #5995 from NixOS/test-nix-store-ping
Add some tests for `nix store ping`
2022-01-26 11:50:04 +01:00
regnat d139474f48 Add some tests for nix store ping
Always good to have :)
2022-01-26 11:01:25 +01:00
Rok Garbas 50a9c48db4 fail early 2022-01-26 10:05:33 +01:00
Rok Garbas 4fc3c4da7b typo 2022-01-26 10:01:58 +01:00
Eelco Dolstra 4f24a33d34 Merge pull request #5991 from edolstra/remote-nix-version
nix store ping: Report Nix daemon version
2022-01-26 09:59:59 +01:00
Rok Garbas 1eac5a6bd0 Script to push docker image for releases 2022-01-26 09:22:51 +01:00
lincoln austerandCole Helbling c746a429db fix typo
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2022-01-25 14:55:49 -07:00
Eelco Dolstra 35dbdbedd4 nix store ping: Report Nix daemon version
Fixes #5952.
2022-01-25 21:15:58 +01:00
lincoln auster [they/them] 203ef26974 flakes: document nixConfig option
Fixes #5988.
2022-01-25 12:23:52 -07:00
Eelco Dolstra 5fa624f59a Merge pull request #5987 from edolstra/rust-cleanup
Remove unused Rust stuff
2022-01-25 13:57:22 +01:00
Eelco Dolstra a04a66c196 Merge pull request #5922 from fzakaria/fzakaria/json-ignore-assertion
Add try/catch to queryJSON for assertion and errors
2022-01-25 12:44:20 +01:00
Eelco Dolstra a5bdffaae9 Merge pull request #5966 from SuperSandro2000/patch-2
Remove url literal
2022-01-25 11:59:33 +01:00
Eelco Dolstra fcf3528ad1 Remove unused Rust stuff
In particular we were still compiling rust-ffi.cc even though we're
not using it.
2022-01-25 11:58:00 +01:00
Eelco Dolstra a9a90b3c1f Merge pull request #5984 from NixOS/5982-correctly-parse-__curPosFoo
Fix parsing of variable names that are a suffix of '__curPos'
2022-01-25 11:54:04 +01:00
regnat f113ea6c73 Fix parsing of variable names that are a suffix of '__curPos'
Follow-up from #5969
Fix #5982
2022-01-25 10:49:27 +01:00
Tom Bereknyei dc85e20684 bundler: notes and doc update to include bundlers repo 2022-01-25 03:48:44 -05:00
Tom Bereknyei 93299efc7c bundler: add tests and change defaults to use a derivation 2022-01-25 03:39:18 -05:00
Farid Zakaria 8ba7a2d3a8 Do not suppress errors in nix-env from feedback by Eelco 2022-01-24 19:12:13 -08:00
Tom Bereknyei c94db0535c Refactor bundler API
Bundlers now expect to be located at bundlers.<system>.<name> and are a
function from derivations to derivations.
2022-01-24 21:43:04 -05:00
Tom Bereknyei 3be810f5db bundler: pass drv attrset instead of path 2022-01-24 21:43:04 -05:00
Eelco Dolstra 0a70b37b56 flake.nix: Fix indentation 2022-01-25 01:28:44 +01:00
Eelco Dolstra 2e4d5f220e Bump version 2022-01-25 00:14:59 +01:00
Sandro 72aeae54e5 Remove url literals 2022-01-24 13:28:21 +01:00
Kevin Amado 50efc5499a determinePos: remove from critical path 2022-01-21 16:32:43 -05:00
Kevin Amado 3d2ad2b70b forceList: make pos mandatory 2022-01-21 16:32:43 -05:00
Kevin Amado c3896e19d0 forceAttrs: make pos mandatory 2022-01-21 16:32:43 -05:00
Kevin Amado 1472e045a7 avoid unnecesary calls 2022-01-21 16:32:43 -05:00
Kevin Amado 49b0bb0206 forceValue: make pos mandatory
- Make passing the position to `forceValue` mandatory,
  this way we remember people that the position is
  important for better error messages
- Add pos to all `forceValue` calls
2022-01-21 16:32:43 -05:00
Travis A. Everett bdb5e03821 install-darwin: dodge bash 3.2 command bug
The script is trying to find chown in a cross-platform-like
way, but there's some sort of deficiency in `command -p` in
the default macOS bash 3.2. It looks like it will just use
whatever PATH is already set, instead of the "default" path.

This attempts to hard-set a PATH via `getconf PATH`. It will
just set an empty PATH if that fails for some reason. A
properly-functioning `command -p` should not care what we
set the PATH to here one way or the other.

Hopefully fixes #5768.
2022-01-21 10:47:06 -06:00
pennae 7d4cc5515c defer formals duplicate check for incresed efficiency all round
if we defer the duplicate argument check for lambda formals we can use more
efficient data structures for the formals set, and we can get rid of the
duplication of formals names to boot. instead of a list of formals we've seen
and a set of names we'll keep a vector instead and run a sort+dupcheck step
before moving the parsed formals into a newly created lambda. this improves
performance on search and rebuild by ~1%, pure parsing gains more (about 4%).

this does reorder lambda arguments in the xml output, but the output is still
stable. this shouldn't be a problem since argument order is not semantically
important anyway.

 before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.550 s ±  0.060 s    [User: 6.470 s, System: 1.664 s]
    Range (min … max):    8.435 s …  8.666 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     346.7 ms ±   2.1 ms    [User: 312.4 ms, System: 34.2 ms]
    Range (min … max):   343.8 ms … 353.4 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.720 s ±  0.031 s    [User: 2.415 s, System: 0.231 s]
    Range (min … max):    2.662 s …  2.780 s    20 runs

 after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.462 s ±  0.063 s    [User: 6.398 s, System: 1.661 s]
    Range (min … max):    8.339 s …  8.542 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     329.1 ms ±   1.4 ms    [User: 296.8 ms, System: 32.3 ms]
    Range (min … max):   326.1 ms … 330.8 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.687 s ±  0.035 s    [User: 2.392 s, System: 0.228 s]
    Range (min … max):    2.626 s …  2.754 s    20 runs
2022-01-19 17:07:29 +01:00
pennae 9ac836d1d6 don't use Symbols for strings
string expressions by and large do not need the benefits a Symbol gives us,
instead they pollute the symbol table and cause unnecessary overhead for almost
all strings. the one place we can think of that benefits from them (attrpaths
with expressions) extracts the benefit in the parser, which we'll have to touch
anyway when changing ExprString to hold strings.

this gives a sizeable improvement on of 3-5% on all benchmarks we've run.

 before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.844 s ±  0.045 s    [User: 6.750 s, System: 1.663 s]
    Range (min … max):    8.758 s …  8.922 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     367.4 ms ±   3.3 ms    [User: 332.3 ms, System: 35.2 ms]
    Range (min … max):   364.0 ms … 375.2 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.810 s ±  0.030 s    [User: 2.517 s, System: 0.225 s]
    Range (min … max):    2.742 s …  2.854 s    20 runs

 after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.533 s ±  0.068 s    [User: 6.485 s, System: 1.642 s]
    Range (min … max):    8.404 s …  8.657 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     347.6 ms ±   3.1 ms    [User: 313.1 ms, System: 34.5 ms]
    Range (min … max):   343.3 ms … 354.6 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.709 s ±  0.032 s    [User: 2.414 s, System: 0.232 s]
    Range (min … max):    2.655 s …  2.788 s    20 runs
2022-01-19 14:48:00 +01:00
pennae 0a7746603e remove ExprIndStr
it can be replaced with StringToken if we add another bit if information to
StringToken, namely whether this string should take part in indentation scanning
or not. since all escaping terminates indentation scanning we need to set this
bit only for the non-escaped IND_STRING rule.

this improves performance by about 1%.

 before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.880 s ±  0.048 s    [User: 6.809 s, System: 1.643 s]
    Range (min … max):    8.781 s …  8.993 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     375.0 ms ±   2.2 ms    [User: 339.8 ms, System: 35.2 ms]
    Range (min … max):   371.5 ms … 379.3 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.831 s ±  0.040 s    [User: 2.536 s, System: 0.225 s]
    Range (min … max):    2.769 s …  2.912 s    20 runs

 after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      8.832 s ±  0.048 s    [User: 6.757 s, System: 1.657 s]
    Range (min … max):    8.743 s …  8.921 s    20 runs

  nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
    Time (mean ± σ):     367.4 ms ±   3.2 ms    [User: 332.7 ms, System: 34.7 ms]
    Range (min … max):   364.6 ms … 374.6 ms    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.810 s ±  0.030 s    [User: 2.517 s, System: 0.225 s]
    Range (min … max):    2.742 s …  2.854 s    20 runs
2022-01-19 13:39:42 +01:00
Farid Zakaria 61f02f7f20 Make queryJSON not bail immediately on an assertion or error 2022-01-15 19:36:07 -08:00
Farid Zakaria 6ff2ce8caf Added result and .vscode to gitignore 2022-01-15 19:17:40 -08:00
251 changed files with 3947 additions and 29409 deletions

No files matched your search

+2 -2
View File
@@ -25,7 +25,7 @@ jobs:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix-build -A checks.$(nix-instantiate --eval -E '(builtins.currentSystem)')
- run: nix --experimental-features 'nix-command flakes' flake check -L
check_cachix:
name: Cachix secret present for installer tests
@@ -95,7 +95,7 @@ jobs:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix-build -A checks.$(nix-instantiate --eval -E 'builtins.currentSystem' --json).dockerImage
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION nixos/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION nixos/nix:master
+4
View File
@@ -120,3 +120,7 @@ GTAGS
compile_commands.json
nix-rust/target
result
.vscode/
+1 -1
View File
@@ -1 +1 @@
2.6.1
2.7.0
-1
View File
@@ -10,7 +10,6 @@ makefiles = \
src/libexpr/local.mk \
src/libcmd/local.mk \
src/nix/local.mk \
src/nlohmann/local.mk \
src/resolve-system-dependencies/local.mk \
scripts/local.mk \
misc/bash/local.mk \
+1
View File
@@ -16,6 +16,7 @@ LDFLAGS = @LDFLAGS@
LIBARCHIVE_LIBS = @LIBARCHIVE_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBCURL_LIBS = @LIBCURL_LIBS@
LOWDOWN_LIBS = @LOWDOWN_LIBS@
OPENSSL_LIBS = @OPENSSL_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
PACKAGE_NAME = @PACKAGE_NAME@
+5 -1
View File
@@ -262,13 +262,17 @@ fi
PKG_CHECK_MODULES([GTEST], [gtest_main])
# Look for nlohmann/json.
PKG_CHECK_MODULES([NLOHMANN_JSON], [nlohmann_json >= 3.9])
# documentation generation switch
AC_ARG_ENABLE(doc-gen, AS_HELP_STRING([--disable-doc-gen],[disable documentation generation]),
doc_generate=$enableval, doc_generate=yes)
AC_SUBST(doc_generate)
# Look for lowdown library.
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.8.0], [CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"])
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.9.0], [CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"])
# Setuid installations.
AC_CHECK_FUNCS([setresuid setreuid lchown])
+1 -1
View File
@@ -1,3 +1,3 @@
(import (fetchTarball https://github.com/edolstra/flake-compat/archive/master.tar.gz) {
(import (fetchTarball "https://github.com/edolstra/flake-compat/archive/master.tar.gz") {
src = ./.;
}).defaultNix
+2 -2
View File
@@ -6,9 +6,9 @@ builtins:
concatStrings (map
(name:
let builtin = builtins.${name}; in
"<dt><code>${name} "
"<dt id=\"builtins-${name}\"><a href=\"#builtins-${name}\"><code>${name} "
+ concatStringsSep " " (map (s: "<var>${s}</var>") builtin.args)
+ "</code></dt>"
+ "</code></a></dt>"
+ "<dd>\n\n"
+ builtin.doc
+ "\n\n</dd>"
+1 -1
View File
@@ -20,7 +20,7 @@ concatStrings (map
# JSON, but that converts to "{ }" here.
(if isAttrs option.value then "`\"\"`"
else "`" + toString option.value + "`")) + "\n\n"
else " **Default:** *machine-specific*")
else " **Default:** *machine-specific*\n")
+ (if option.aliases != []
then " **Deprecated alias:** " + (concatStringsSep ", " (map (s: "`${s}`") option.aliases)) + "\n\n"
else "")
+1
View File
@@ -72,6 +72,7 @@ $(d)/builtins.json: $(bindir)/nix
@mv $@.tmp $@
# Generate the HTML manual.
html: $(docdir)/manual/index.html
install: $(docdir)/manual/index.html
# Generate 'nix' manpages.
+1
View File
@@ -71,6 +71,7 @@
- [Hacking](contributing/hacking.md)
- [CLI guideline](contributing/cli-guideline.md)
- [Release Notes](release-notes/release-notes.md)
- [Release 2.7 (2022-03-07)](release-notes/rl-2.7.md)
- [Release 2.6 (2022-01-24)](release-notes/rl-2.6.md)
- [Release 2.5 (2021-12-13)](release-notes/rl-2.5.md)
- [Release 2.4 (2021-11-01)](release-notes/rl-2.4.md)
+2 -2
View File
@@ -321,8 +321,8 @@ symlink.
This query has one option:
- `--include-outputs`
Also include the output path of store derivations, and their
closures.
Also include the existing output paths of store derivations,
and their closures.
This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
+33
View File
@@ -0,0 +1,33 @@
# Release 2.7 (2022-03-07)
* Nix will now make some helpful suggestions when you mistype
something on the command line. For instance, if you type `nix build
nixpkgs#thunderbrd`, it will suggest `thunderbird`.
* A number of "default" flake output attributes have been
renamed. These are:
* `defaultPackage.<system>` → `packages.<system>.default`
* `defaultApps.<system>` → `apps.<system>.default`
* `defaultTemplate` → `templates.default`
* `defaultBundler.<system>` → `bundlers.<system>.default`
* `overlay` → `overlays.default`
* `devShell.<system>` → `devShells.<system>.default`
The old flake output attributes still work, but `nix flake check`
will warn about them.
* Breaking API change: `nix bundle` now supports bundlers of the form
`bundler.<system>.<name>= derivation: another-derivation;`. This
supports additional functionality to inspect evaluation information
during bundling. A new
[repository](https://github.com/NixOS/bundlers) has various bundlers
implemented.
* `nix store ping` now reports the version of the remote Nix daemon.
* `nix flake {init,new}` now display information about which files have been
created.
* Templates can now define a `welcomeText` attribute, which is printed out by
`nix flake {init,new} --template <template>`.
-1
View File
@@ -1,2 +1 @@
# Release X.Y (202?-??-??)
+205 -196
View File
@@ -133,6 +133,7 @@
./boehmgc-coroutine-sp-fallback.diff
];
}))
nlohmann_json
];
perlDeps =
@@ -141,8 +142,8 @@
];
};
installScriptFor = systems:
with nixpkgsFor.x86_64-linux;
installScriptFor = systems:
with nixpkgsFor.x86_64-linux;
runCommand "installer-script"
{ buildInputs = [ nix ];
}
@@ -206,203 +207,204 @@
installCheckPhase = "make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES";
};
binaryTarball = buildPackages: nix: pkgs: let
inherit (pkgs) cacert;
installerClosureInfo = buildPackages.closureInfo { rootPaths = [ nix cacert ]; };
in
binaryTarball = buildPackages: nix: pkgs:
let
inherit (pkgs) cacert;
installerClosureInfo = buildPackages.closureInfo { rootPaths = [ nix cacert ]; };
in
buildPackages.runCommand "nix-binary-tarball-${version}"
{ #nativeBuildInputs = lib.optional (system != "aarch64-linux") shellcheck;
meta.description = "Distribution-independent Nix bootstrap binaries for ${pkgs.system}";
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
cp ${./scripts/create-darwin-volume.sh} $TMPDIR/create-darwin-volume.sh
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
buildPackages.runCommand "nix-binary-tarball-${version}"
{ #nativeBuildInputs = lib.optional (system != "aarch64-linux") shellcheck;
meta.description = "Distribution-independent Nix bootstrap binaries for ${pkgs.system}";
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
cp ${./scripts/create-darwin-volume.sh} $TMPDIR/create-darwin-volume.sh
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
# SC1090: Don't worry about not being able to find
# $nix/etc/profile.d/nix.sh
shellcheck --exclude SC1090 $TMPDIR/install
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
if type -p shellcheck; then
# SC1090: Don't worry about not being able to find
# $nix/etc/profile.d/nix.sh
shellcheck --exclude SC1090 $TMPDIR/install
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
# SC1091: Don't panic about not being able to source
# /etc/profile
# SC2002: Ignore "useless cat" "error", when loading
# .reginfo, as the cat is a much cleaner
# implementation, even though it is "useless"
# SC2116: Allow ROOT_HOME=$(echo ~root) for resolving
# root's home directory
shellcheck --external-sources \
--exclude SC1091,SC2002,SC2116 $TMPDIR/install-multi-user
fi
# SC1091: Don't panic about not being able to source
# /etc/profile
# SC2002: Ignore "useless cat" "error", when loading
# .reginfo, as the cat is a much cleaner
# implementation, even though it is "useless"
# SC2116: Allow ROOT_HOME=$(echo ~root) for resolving
# root's home directory
shellcheck --external-sources \
--exclude SC1091,SC2002,SC2116 $TMPDIR/install-multi-user
fi
chmod +x $TMPDIR/install
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${pkgs.system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install \
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
'';
overlayFor = getStdenv: final: prev:
let currentStdenv = getStdenv final; in
{
nixStable = prev.nix;
# Forward from the previous stage as we don’t want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
nix = with final; with commonDeps pkgs; currentStdenv.mkDerivation {
name = "nix-${version}";
inherit version;
src = self;
VERSION_SUFFIX = versionSuffix;
outputs = [ "out" "dev" "doc" ];
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ awsDeps;
propagatedBuildInputs = propagatedDeps;
disallowedReferences = [ boost ];
preConfigure =
''
# Copy libboost_context so we don't get all of Boost in our closure.
# https://github.com/NixOS/nixpkgs/issues/45462
mkdir -p $out/lib
cp -pd ${boost}/lib/{libboost_context*,libboost_thread*,libboost_system*} $out/lib
rm -f $out/lib/*.a
${lib.optionalString currentStdenv.isLinux ''
chmod u+w $out/lib/*.so.*
patchelf --set-rpath $out/lib:${currentStdenv.cc.cc.lib}/lib $out/lib/libboost_thread.so.*
''}
${lib.optionalString currentStdenv.isDarwin ''
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
done
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''}
'';
configureFlags = configureFlags ++
[ "--sysconfdir=/etc" ];
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
doCheck = true;
installFlags = "sysconfdir=$(out)/etc";
postInstall = ''
mkdir -p $doc/nix-support
echo "doc manual $doc/share/doc/nix/manual" >> $doc/nix-support/hydra-build-products
${lib.optionalString currentStdenv.isDarwin ''
install_name_tool \
-change ${boost}/lib/libboost_context.dylib \
$out/lib/libboost_context.dylib \
$out/lib/libnixutil.dylib
''}
chmod +x $TMPDIR/install
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${pkgs.system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install \
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
'';
doInstallCheck = true;
installCheckFlags = "sysconfdir=$(out)/etc";
overlayFor = getStdenv: final: prev:
let currentStdenv = getStdenv final; in
{
nixStable = prev.nix;
separateDebugInfo = true;
# Forward from the previous stage as we don’t want it to pick the lowdown override
nixUnstable = prev.nixUnstable;
strictDeps = true;
passthru.perl-bindings = with final; currentStdenv.mkDerivation {
name = "nix-perl-${version}";
nix = with final; with commonDeps pkgs; currentStdenv.mkDerivation {
name = "nix-${version}";
inherit version;
src = self;
nativeBuildInputs =
[ buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkg-config
];
VERSION_SUFFIX = versionSuffix;
buildInputs =
[ nix
curl
bzip2
xz
pkgs.perl
boost
]
++ lib.optional (currentStdenv.isLinux || currentStdenv.isDarwin) libsodium
++ lib.optional currentStdenv.isDarwin darwin.apple_sdk.frameworks.Security;
outputs = [ "out" "dev" "doc" ];
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}
'';
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ awsDeps;
propagatedBuildInputs = propagatedDeps;
disallowedReferences = [ boost ];
preConfigure =
''
# Copy libboost_context so we don't get all of Boost in our closure.
# https://github.com/NixOS/nixpkgs/issues/45462
mkdir -p $out/lib
cp -pd ${boost}/lib/{libboost_context*,libboost_thread*,libboost_system*} $out/lib
rm -f $out/lib/*.a
${lib.optionalString currentStdenv.isLinux ''
chmod u+w $out/lib/*.so.*
patchelf --set-rpath $out/lib:${currentStdenv.cc.cc.lib}/lib $out/lib/libboost_thread.so.*
''}
${lib.optionalString currentStdenv.isDarwin ''
for LIB in $out/lib/*.dylib; do
chmod u+w $LIB
install_name_tool -id $LIB $LIB
done
install_name_tool -change ${boost}/lib/libboost_system.dylib $out/lib/libboost_system.dylib $out/lib/libboost_thread.dylib
''}
'';
configureFlags = configureFlags ++
[ "--sysconfdir=/etc" ];
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
doCheck = true;
installFlags = "sysconfdir=$(out)/etc";
postInstall = ''
mkdir -p $doc/nix-support
echo "doc manual $doc/share/doc/nix/manual" >> $doc/nix-support/hydra-build-products
${lib.optionalString currentStdenv.isDarwin ''
install_name_tool \
-change ${boost}/lib/libboost_context.dylib \
$out/lib/libboost_context.dylib \
$out/lib/libnixutil.dylib
''}
'';
doInstallCheck = true;
installCheckFlags = "sysconfdir=$(out)/etc";
separateDebugInfo = true;
strictDeps = true;
passthru.perl-bindings = with final; currentStdenv.mkDerivation {
name = "nix-perl-${version}";
src = self;
nativeBuildInputs =
[ buildPackages.autoconf-archive
buildPackages.autoreconfHook
buildPackages.pkg-config
];
buildInputs =
[ nix
curl
bzip2
xz
pkgs.perl
boost
]
++ lib.optional (currentStdenv.isLinux || currentStdenv.isDarwin) libsodium
++ lib.optional currentStdenv.isDarwin darwin.apple_sdk.frameworks.Security;
configureFlags = ''
--with-dbi=${perlPackages.DBI}/${pkgs.perl.libPrefix}
--with-dbd-sqlite=${perlPackages.DBDSQLite}/${pkgs.perl.libPrefix}
'';
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
};
};
lowdown-nix = with final; currentStdenv.mkDerivation rec {
name = "lowdown-0.9.0";
src = lowdown-src;
outputs = [ "out" "bin" "dev" ];
nativeBuildInputs = [ buildPackages.which ];
configurePhase = ''
${if (currentStdenv.isDarwin && currentStdenv.isAarch64) then "echo \"HAVE_SANDBOX_INIT=false\" > configure.local" else ""}
./configure \
PREFIX=${placeholder "dev"} \
BINDIR=${placeholder "bin"}/bin
'';
};
};
lowdown-nix = with final; currentStdenv.mkDerivation rec {
name = "lowdown-0.9.0";
src = lowdown-src;
outputs = [ "out" "bin" "dev" ];
nativeBuildInputs = [ buildPackages.which ];
configurePhase = ''
${if (currentStdenv.isDarwin && currentStdenv.isAarch64) then "echo \"HAVE_SANDBOX_INIT=false\" > configure.local" else ""}
./configure \
PREFIX=${placeholder "dev"} \
BINDIR=${placeholder "bin"}/bin
'';
};
};
in {
# A Nixpkgs overlay that overrides the 'nix' and
@@ -445,19 +447,7 @@
installerScriptForGHA = installScriptFor [ "x86_64-linux" "x86_64-darwin" "armv6l-linux" "armv7l-linux"];
# docker image with Nix inside
dockerImage = nixpkgs.lib.genAttrs linux64BitSystems (system:
let
pkgs = nixpkgsFor.${system};
image = import ./docker.nix { inherit pkgs; tag = version; };
in pkgs.runCommand "docker-image-tarball-${version}"
{ meta.description = "Docker image with Nix for ${system}";
}
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'');
dockerImage = nixpkgs.lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# Line coverage analysis.
coverage =
@@ -511,6 +501,12 @@
inherit (self) overlay;
});
tests.sourcehutFlakes = (import ./tests/sourcehut-flakes.nix rec {
system = "x86_64-linux";
inherit nixpkgs;
inherit (self) overlay;
});
tests.setuid = nixpkgs.lib.genAttrs
["i686-linux" "x86_64-linux"]
(system:
@@ -604,6 +600,20 @@
hardeningDisable = [ "pie" ];
};
dockerImage =
let
pkgs = nixpkgsFor.${system};
image = import ./docker.nix { inherit pkgs; tag = version; };
in
pkgs.runCommand
"docker-image-tarball-${version}"
{ meta.description = "Docker image with Nix for ${system}"; }
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
} // builtins.listToAttrs (map (crossSystem: {
name = "nix-${crossSystem}";
value = let
@@ -644,11 +654,10 @@
installCheckFlags = "sysconfdir=$(out)/etc";
};
}) crossSystems)) // (builtins.listToAttrs (map (stdenvName:
nixpkgsFor.${system}.lib.nameValuePair
"nix-${stdenvName}"
nixpkgsFor.${system}."${stdenvName}Packages".nix
) stdenvs))
);
nixpkgsFor.${system}.lib.nameValuePair
"nix-${stdenvName}"
nixpkgsFor.${system}."${stdenvName}Packages".nix
) stdenvs)));
defaultPackage = forAllSystems (system: self.packages.${system}.nix);
+72 -4
View File
@@ -55,6 +55,11 @@ my $releaseDir = "nix/$releaseName";
my $tmpDir = "$TMPDIR/nix-release/$releaseName";
File::Path::make_path($tmpDir);
my $narCache = "$TMPDIR/nar-cache";
File::Path::make_path($narCache);
my $binaryCache = "https://cache.nixos.org/?local-nar-cache=$narCache";
# S3 setup.
my $aws_access_key_id = $ENV{'AWS_ACCESS_KEY_ID'} or die "No AWS_ACCESS_KEY_ID given.";
my $aws_secret_access_key = $ENV{'AWS_SECRET_ACCESS_KEY'} or die "No AWS_SECRET_ACCESS_KEY given.";
@@ -80,6 +85,7 @@ sub downloadFile {
my ($jobName, $productNr, $dstName) = @_;
my $buildInfo = decode_json(fetch("$evalUrl/job/$jobName", 'application/json'));
#print STDERR "$jobName: ", Dumper($buildInfo), "\n";
my $srcFile = $buildInfo->{buildproducts}->{$productNr}->{path} or die "job '$jobName' lacks product $productNr\n";
$dstName //= basename($srcFile);
@@ -87,19 +93,27 @@ sub downloadFile {
if (!-e $tmpFile) {
print STDERR "downloading $srcFile to $tmpFile...\n";
system("NIX_REMOTE=https://cache.nixos.org/ nix store cat '$srcFile' > '$tmpFile'") == 0
my $fileInfo = decode_json(`NIX_REMOTE=$binaryCache nix store ls --json '$srcFile'`);
$srcFile = $fileInfo->{target} if $fileInfo->{type} eq 'symlink';
#print STDERR $srcFile, " ", Dumper($fileInfo), "\n";
system("NIX_REMOTE=$binaryCache nix store cat '$srcFile' > '$tmpFile'.tmp") == 0
or die "unable to fetch $srcFile\n";
rename("$tmpFile.tmp", $tmpFile) or die;
}
my $sha256_expected = $buildInfo->{buildproducts}->{$productNr}->{sha256hash} or die;
my $sha256_expected = $buildInfo->{buildproducts}->{$productNr}->{sha256hash};
my $sha256_actual = `nix hash file --base16 --type sha256 '$tmpFile'`;
chomp $sha256_actual;
if ($sha256_expected ne $sha256_actual) {
if (defined($sha256_expected) && $sha256_expected ne $sha256_actual) {
print STDERR "file $tmpFile is corrupt, got $sha256_actual, expected $sha256_expected\n";
exit 1;
}
write_file("$tmpFile.sha256", $sha256_expected);
write_file("$tmpFile.sha256", $sha256_actual);
if (! -e "$tmpFile.asc") {
system("gpg2 --detach-sign --armor $tmpFile") == 0 or die "unable to sign $tmpFile\n";
@@ -117,6 +131,60 @@ downloadFile("binaryTarballCross.x86_64-linux.armv6l-linux", "1");
downloadFile("binaryTarballCross.x86_64-linux.armv7l-linux", "1");
downloadFile("installerScript", "1");
# Upload docker images to dockerhub.
my $dockerManifest = "";
my $dockerManifestLatest = "";
for my $platforms (["x86_64-linux", "amd64"], ["aarch64-linux", "arm64"]) {
my $system = $platforms->[0];
my $dockerPlatform = $platforms->[1];
my $fn = "nix-$version-docker-image-$dockerPlatform.tar.gz";
downloadFile("dockerImage.$system", "1", $fn);
print STDERR "loading docker image for $dockerPlatform...\n";
system("docker load -i $tmpDir/$fn") == 0 or die;
my $tag = "nixos/nix:$version-$dockerPlatform";
my $latestTag = "nixos/nix:latest-$dockerPlatform";
print STDERR "tagging $version docker image for $dockerPlatform...\n";
system("docker tag nix:$version $tag") == 0 or die;
if ($isLatest) {
print STDERR "tagging latest docker image for $dockerPlatform...\n";
system("docker tag nix:$version $latestTag") == 0 or die;
}
print STDERR "pushing $version docker image for $dockerPlatform...\n";
system("docker push -q $tag") == 0 or die;
if ($isLatest) {
print STDERR "pushing latest docker image for $dockerPlatform...\n";
system("docker push -q $latestTag") == 0 or die;
}
$dockerManifest .= " --amend $tag";
$dockerManifestLatest .= " --amend $latestTag"
}
print STDERR "creating multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:$version");
system("docker manifest create nixos/nix:$version $dockerManifest") == 0 or die;
if ($isLatest) {
print STDERR "creating latest multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:latest");
system("docker manifest create nixos/nix:latest $dockerManifestLatest") == 0 or die;
}
print STDERR "pushing multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:$version") == 0 or die;
if ($isLatest) {
print STDERR "pushing latest multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:latest") == 0 or die;
}
# Upload release files to S3.
for my $fn (glob "$tmpDir/*") {
my $name = basename($fn);
my $dstKey = "$releaseDir/" . $name;
+1 -1
View File
@@ -15,7 +15,7 @@ function _complete_nix {
else
COMPREPLY+=("$completion")
fi
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]/#\~/$HOME}")
done < <(NIX_GET_COMPLETIONS=$cword "${words[@]/#\~/$HOME}" 2>/dev/null)
__ltrim_colon_completions "$cur"
}
+1 -1
View File
@@ -4,7 +4,7 @@ function _nix() {
local ifs_bk="$IFS"
local input=("${(Q)words[@]}")
IFS=$'\n'
local res=($(NIX_GET_COMPLETIONS=$((CURRENT - 1)) "$input[@]"))
local res=($(NIX_GET_COMPLETIONS=$((CURRENT - 1)) "$input[@]" 2>/dev/null))
IFS="$ifs_bk"
local tpe="${${res[1]}%%> *}"
local -a suggestions
+21 -3
View File
@@ -14,9 +14,27 @@ if [ -t 1 ]; then
yellow=""
normal=""
fi
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
run_test () {
(cd tests && env ${TESTS_ENVIRONMENT} init.sh 2>/dev/null > /dev/null)
log="$(cd $(dirname $1) && env ${TESTS_ENVIRONMENT} $(basename $1) 2>&1)"
status=$?
}
run_test "$1"
# Hack: Retry the test if it fails with “unexpected EOF reading a line” as these
# appear randomly without anyone knowing why.
# See https://github.com/NixOS/nix/issues/3605 for more info
if [[ $status -ne 0 && $status -ne 99 && \
"$(uname)" == "Darwin" && \
"$log" =~ "unexpected EOF reading a line" \
]]; then
echo "$post_run_msg [${yellow}FAIL$normal] (possibly flaky, so will be retried)"
echo "$log" | sed 's/^/ /'
run_test "$1"
fi
if [ $status -eq 0 ]; then
echo "$post_run_msg [${green}PASS$normal]"
elif [ $status -eq 99 ]; then
-399
View File
@@ -1,399 +0,0 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "assert_matches"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "autocfg"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "bit-set"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"bit-vec 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "bit-vec"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "bitflags"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "byteorder"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "c2-chacha"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "cfg-if"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "cloudabi"
version = "0.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "fnv"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "fuchsia-cprng"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "getrandom"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "hex"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "lazy_static"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "libc"
version = "0.2.66"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "nix-rust"
version = "0.1.0"
dependencies = [
"assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)",
"hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"proptest 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "num-traits"
version = "0.2.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "ppv-lite86"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "proptest"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"bit-set 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
"bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
"byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)",
"lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
"num-traits 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)",
"quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
"rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"regex-syntax 0.6.12 (registry+https://github.com/rust-lang/crates.io-index)",
"rusty-fork 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
"tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "quick-error"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "rand"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_jitter 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_pcg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_chacha"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_chacha"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_core"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_core"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "rand_core"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_hc"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_hc"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_isaac"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_jitter"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_os"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)",
"fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
"rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_pcg"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)",
"rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rand_xorshift"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rdrand"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "redox_syscall"
version = "0.1.56"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "regex-syntax"
version = "0.6.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "remove_dir_all"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "rusty-fork"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)",
"quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)",
"tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)",
"wait-timeout 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "tempfile"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)",
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
"rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)",
"redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)",
"remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "wait-timeout"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "wasi"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "winapi"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
dependencies = [
"winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
"winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[metadata]
"checksum assert_matches 1.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7deb0a829ca7bcfaf5da70b073a8d128619259a7be8216a355e23f00763059e5"
"checksum autocfg 0.1.7 (registry+https://github.com/rust-lang/crates.io-index)" = "1d49d90015b3c36167a20fe2810c5cd875ad504b39cff3d4eae7977e6b7c1cb2"
"checksum bit-set 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "e84c238982c4b1e1ee668d136c510c67a13465279c0cb367ea6baf6310620a80"
"checksum bit-vec 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f59bbe95d4e52a6398ec21238d31577f2b28a9d86807f06ca59d191d8440d0bb"
"checksum bitflags 1.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cf1de2fe8c75bc145a2f577add951f8134889b4795d47466a54a5c846d691693"
"checksum byteorder 1.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "a7c3dd8985a7111efc5c80b44e23ecdd8c007de8ade3b96595387e812b957cf5"
"checksum c2-chacha 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb"
"checksum cfg-if 0.1.10 (registry+https://github.com/rust-lang/crates.io-index)" = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
"checksum cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ddfc5b9aa5d4507acaf872de71051dfd0e309860e88966e1051e462a077aac4f"
"checksum fnv 1.0.6 (registry+https://github.com/rust-lang/crates.io-index)" = "2fad85553e09a6f881f739c29f0b00b0f01357c743266d478b68951ce23285f3"
"checksum fuchsia-cprng 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "a06f77d526c1a601b7c4cdd98f54b5eaabffc14d5f2f0296febdc7f357c6d3ba"
"checksum getrandom 0.1.13 (registry+https://github.com/rust-lang/crates.io-index)" = "e7db7ca94ed4cd01190ceee0d8a8052f08a247aa1b469a7f68c6a3b71afcf407"
"checksum hex 0.3.2 (registry+https://github.com/rust-lang/crates.io-index)" = "805026a5d0141ffc30abb3be3173848ad46a1b1664fe632428479619a3644d77"
"checksum lazy_static 1.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
"checksum libc 0.2.66 (registry+https://github.com/rust-lang/crates.io-index)" = "d515b1f41455adea1313a4a2ac8a8a477634fbae63cc6100e3aebb207ce61558"
"checksum num-traits 0.2.10 (registry+https://github.com/rust-lang/crates.io-index)" = "d4c81ffc11c212fa327657cb19dd85eb7419e163b5b076bede2bdb5c974c07e4"
"checksum ppv-lite86 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b"
"checksum proptest 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)" = "cf147e022eacf0c8a054ab864914a7602618adba841d800a9a9868a5237a529f"
"checksum quick-error 1.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "9274b940887ce9addde99c4eee6b5c44cc494b182b97e73dc8ffdcb3397fd3f0"
"checksum rand 0.6.5 (registry+https://github.com/rust-lang/crates.io-index)" = "6d71dacdc3c88c1fde3885a3be3fbab9f35724e6ce99467f7d9c5026132184ca"
"checksum rand 0.7.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3ae1b169243eaf61759b8475a998f0a385e42042370f3a7dbaf35246eacc8412"
"checksum rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "556d3a1ca6600bfcbab7c7c91ccb085ac7fbbcd70e008a98742e7847f4f7bcef"
"checksum rand_chacha 0.2.1 (registry+https://github.com/rust-lang/crates.io-index)" = "03a2a90da8c7523f554344f921aa97283eadf6ac484a6d2a7d0212fa7f8d6853"
"checksum rand_core 0.3.1 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6fdeb83b075e8266dcc8762c22776f6877a63111121f5f8c7411e5be7eed4b"
"checksum rand_core 0.4.2 (registry+https://github.com/rust-lang/crates.io-index)" = "9c33a3c44ca05fa6f1807d8e6743f3824e8509beca625669633be0acbdf509dc"
"checksum rand_core 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
"checksum rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7b40677c7be09ae76218dc623efbf7b18e34bced3f38883af07bb75630a21bc4"
"checksum rand_hc 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
"checksum rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ded997c9d5f13925be2a6fd7e66bf1872597f759fd9dd93513dd7e92e5a5ee08"
"checksum rand_jitter 0.1.4 (registry+https://github.com/rust-lang/crates.io-index)" = "1166d5c91dc97b88d1decc3285bb0a99ed84b05cfd0bc2341bdf2d43fc41e39b"
"checksum rand_os 0.1.3 (registry+https://github.com/rust-lang/crates.io-index)" = "7b75f676a1e053fc562eafbb47838d67c84801e38fc1ba459e8f180deabd5071"
"checksum rand_pcg 0.1.2 (registry+https://github.com/rust-lang/crates.io-index)" = "abf9b09b01790cfe0364f52bf32995ea3c39f4d2dd011eac241d2914146d0b44"
"checksum rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cbf7e9e623549b0e21f6e97cf8ecf247c1a8fd2e8a992ae265314300b2455d5c"
"checksum rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "678054eb77286b51581ba43620cc911abf02758c91f93f479767aed0f90458b2"
"checksum redox_syscall 0.1.56 (registry+https://github.com/rust-lang/crates.io-index)" = "2439c63f3f6139d1b57529d16bc3b8bb855230c8efcc5d3a896c8bea7c3b1e84"
"checksum regex-syntax 0.6.12 (registry+https://github.com/rust-lang/crates.io-index)" = "11a7e20d1cce64ef2fed88b66d347f88bd9babb82845b2b858f3edbf59a4f716"
"checksum remove_dir_all 0.5.2 (registry+https://github.com/rust-lang/crates.io-index)" = "4a83fa3702a688b9359eccba92d153ac33fd2e8462f9e0e3fdf155239ea7792e"
"checksum rusty-fork 0.2.2 (registry+https://github.com/rust-lang/crates.io-index)" = "3dd93264e10c577503e926bd1430193eeb5d21b059148910082245309b424fae"
"checksum tempfile 3.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7a6e24d9338a0a5be79593e2fa15a648add6138caa803e2d5bc782c371732ca9"
"checksum wait-timeout 0.2.0 (registry+https://github.com/rust-lang/crates.io-index)" = "9f200f5b12eb75f8c1ed65abd4b2db8a6e1b138a20de009dacee265a2498f3f6"
"checksum wasi 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "b89c3ce4ce14bdc6fb6beaf9ec7928ca331de5df7e5ea278375642a2f478570d"
"checksum winapi 0.3.8 (registry+https://github.com/rust-lang/crates.io-index)" = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6"
"checksum winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
"checksum winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
-23
View File
@@ -1,23 +0,0 @@
[package]
name = "nix-rust"
version = "0.1.0"
authors = ["Eelco Dolstra <edolstra@gmail.com>"]
edition = "2018"
[lib]
name = "nixrust"
crate-type = ["cdylib"]
[dependencies]
libc = "0.2"
#futures-preview = { version = "=0.3.0-alpha.19" }
#hyper = "0.13.0-alpha.4"
#http = "0.1"
#tokio = { version = "0.2.0-alpha.6", default-features = false, features = ["rt-full"] }
lazy_static = "1.4"
#byteorder = "1.3"
[dev-dependencies]
hex = "0.3"
assert_matches = "1.3"
proptest = "0.9"
-48
View File
@@ -1,48 +0,0 @@
ifeq ($(OPTIMIZE), 1)
RUST_MODE = --release
RUST_DIR = release
else
RUST_MODE =
RUST_DIR = debug
endif
libnixrust_PATH := $(d)/target/$(RUST_DIR)/libnixrust.$(SO_EXT)
libnixrust_INSTALL_PATH := $(libdir)/libnixrust.$(SO_EXT)
libnixrust_LDFLAGS_USE := -L$(d)/target/$(RUST_DIR) -lnixrust
libnixrust_LDFLAGS_USE_INSTALLED := -L$(libdir) -lnixrust
ifdef HOST_LINUX
libnixrust_LDFLAGS_USE += -ldl
libnixrust_LDFLAGS_USE_INSTALLED += -ldl
endif
ifdef HOST_DARWIN
libnixrust_BUILD_FLAGS = NIX_LDFLAGS="-undefined dynamic_lookup"
else
libnixrust_LDFLAGS_USE += -Wl,-rpath,$(abspath $(d)/target/$(RUST_DIR))
libnixrust_LDFLAGS_USE_INSTALLED += -Wl,-rpath,$(libdir)
endif
$(libnixrust_PATH): $(call rwildcard, $(d)/src, *.rs) $(d)/Cargo.toml
$(trace-gen) cd nix-rust && CARGO_HOME=$$(if [[ -d vendor ]]; then echo vendor; fi) \
$(libnixrust_BUILD_FLAGS) \
cargo build $(RUST_MODE) $$(if [[ -d vendor ]]; then echo --offline; fi) \
&& touch target/$(RUST_DIR)/libnixrust.$(SO_EXT)
$(libnixrust_INSTALL_PATH): $(libnixrust_PATH)
$(target-gen) cp $^ $@
ifdef HOST_DARWIN
install_name_tool -id $@ $@
endif
clean: clean-rust
clean-rust:
$(suppress) rm -rfv nix-rust/target
ifndef HOST_DARWIN
check: rust-tests
rust-tests:
$(trace-test) cd nix-rust && CARGO_HOME=$$(if [[ -d vendor ]]; then echo vendor; fi) cargo test --release $$(if [[ -d vendor ]]; then echo --offline; fi)
endif
-77
View File
@@ -1,77 +0,0 @@
use super::{error, store::path, store::StorePath, util};
#[no_mangle]
pub unsafe extern "C" fn ffi_String_new(s: &str, out: *mut String) {
// FIXME: check whether 's' is valid UTF-8?
out.write(s.to_string())
}
#[no_mangle]
pub unsafe extern "C" fn ffi_String_drop(self_: *mut String) {
std::ptr::drop_in_place(self_);
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_new(
path: &str,
store_dir: &str,
) -> Result<StorePath, error::CppException> {
StorePath::new(std::path::Path::new(path), std::path::Path::new(store_dir))
.map_err(|err| err.into())
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_new2(
hash: &[u8; crate::store::path::STORE_PATH_HASH_BYTES],
name: &str,
) -> Result<StorePath, error::CppException> {
StorePath::from_parts(*hash, name).map_err(|err| err.into())
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_fromBaseName(
base_name: &str,
) -> Result<StorePath, error::CppException> {
StorePath::new_from_base_name(base_name).map_err(|err| err.into())
}
#[no_mangle]
pub unsafe extern "C" fn ffi_StorePath_drop(self_: *mut StorePath) {
std::ptr::drop_in_place(self_);
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_to_string(self_: &StorePath) -> Vec<u8> {
let mut buf = vec![0; path::STORE_PATH_HASH_CHARS + 1 + self_.name.name().len()];
util::base32::encode_into(self_.hash.hash(), &mut buf[0..path::STORE_PATH_HASH_CHARS]);
buf[path::STORE_PATH_HASH_CHARS] = b'-';
buf[path::STORE_PATH_HASH_CHARS + 1..].clone_from_slice(self_.name.name().as_bytes());
buf
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_less_than(a: &StorePath, b: &StorePath) -> bool {
a < b
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_eq(a: &StorePath, b: &StorePath) -> bool {
a == b
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_clone(self_: &StorePath) -> StorePath {
self_.clone()
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_name(self_: &StorePath) -> &str {
self_.name.name()
}
#[no_mangle]
pub extern "C" fn ffi_StorePath_hash_data(
self_: &StorePath,
) -> &[u8; crate::store::path::STORE_PATH_HASH_BYTES] {
self_.hash.hash()
}
-118
View File
@@ -1,118 +0,0 @@
use std::fmt;
#[derive(Debug)]
pub enum Error {
InvalidPath(crate::store::StorePath),
BadStorePath(std::path::PathBuf),
NotInStore(std::path::PathBuf),
BadNarInfo,
BadBase32,
StorePathNameEmpty,
StorePathNameTooLong,
BadStorePathName,
NarSizeFieldTooBig,
BadNarString,
BadNarPadding,
BadNarVersionMagic,
MissingNarOpenTag,
MissingNarCloseTag,
MissingNarField,
BadNarField(String),
BadExecutableField,
IOError(std::io::Error),
#[cfg(unused)]
HttpError(hyper::error::Error),
Misc(String),
#[cfg(not(test))]
Foreign(CppException),
BadTarFileMemberName(String),
}
impl From<std::io::Error> for Error {
fn from(err: std::io::Error) -> Self {
Error::IOError(err)
}
}
#[cfg(unused)]
impl From<hyper::error::Error> for Error {
fn from(err: hyper::error::Error) -> Self {
Error::HttpError(err)
}
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Error::InvalidPath(_) => write!(f, "invalid path"),
Error::BadNarInfo => write!(f, ".narinfo file is corrupt"),
Error::BadStorePath(path) => write!(f, "path '{}' is not a store path", path.display()),
Error::NotInStore(path) => {
write!(f, "path '{}' is not in the Nix store", path.display())
}
Error::BadBase32 => write!(f, "invalid base32 string"),
Error::StorePathNameEmpty => write!(f, "store path name is empty"),
Error::StorePathNameTooLong => {
write!(f, "store path name is longer than 211 characters")
}
Error::BadStorePathName => write!(f, "store path name contains forbidden character"),
Error::NarSizeFieldTooBig => write!(f, "size field in NAR is too big"),
Error::BadNarString => write!(f, "NAR string is not valid UTF-8"),
Error::BadNarPadding => write!(f, "NAR padding is not zero"),
Error::BadNarVersionMagic => write!(f, "unsupported NAR version"),
Error::MissingNarOpenTag => write!(f, "NAR open tag is missing"),
Error::MissingNarCloseTag => write!(f, "NAR close tag is missing"),
Error::MissingNarField => write!(f, "expected NAR field is missing"),
Error::BadNarField(s) => write!(f, "unrecognized NAR field '{}'", s),
Error::BadExecutableField => write!(f, "bad 'executable' field in NAR"),
Error::IOError(err) => write!(f, "I/O error: {}", err),
#[cfg(unused)]
Error::HttpError(err) => write!(f, "HTTP error: {}", err),
#[cfg(not(test))]
Error::Foreign(_) => write!(f, "<C++ exception>"), // FIXME
Error::Misc(s) => write!(f, "{}", s),
Error::BadTarFileMemberName(s) => {
write!(f, "tar archive contains illegal file name '{}'", s)
}
}
}
}
#[cfg(not(test))]
impl From<Error> for CppException {
fn from(err: Error) -> Self {
match err {
Error::Foreign(ex) => ex,
_ => CppException::new(&err.to_string()),
}
}
}
#[cfg(not(test))]
#[repr(C)]
#[derive(Debug)]
pub struct CppException(*const libc::c_void); // == std::exception_ptr*
#[cfg(not(test))]
impl CppException {
fn new(s: &str) -> Self {
Self(unsafe { make_error(s) })
}
}
#[cfg(not(test))]
impl Drop for CppException {
fn drop(&mut self) {
unsafe {
destroy_error(self.0);
}
}
}
#[cfg(not(test))]
extern "C" {
#[allow(improper_ctypes)] // YOLO
fn make_error(s: &str) -> *const libc::c_void;
fn destroy_error(exc: *const libc::c_void);
}
-10
View File
@@ -1,10 +0,0 @@
#[allow(improper_ctypes_definitions)]
#[cfg(not(test))]
mod c;
mod error;
#[cfg(unused)]
mod nar;
mod store;
mod util;
pub use error::Error;
-126
View File
@@ -1,126 +0,0 @@
use crate::Error;
use byteorder::{LittleEndian, ReadBytesExt};
use std::convert::TryFrom;
use std::io::Read;
pub fn parse<R: Read>(input: &mut R) -> Result<(), Error> {
if String::read(input)? != NAR_VERSION_MAGIC {
return Err(Error::BadNarVersionMagic);
}
parse_file(input)
}
const NAR_VERSION_MAGIC: &str = "nix-archive-1";
fn parse_file<R: Read>(input: &mut R) -> Result<(), Error> {
if String::read(input)? != "(" {
return Err(Error::MissingNarOpenTag);
}
if String::read(input)? != "type" {
return Err(Error::MissingNarField);
}
match String::read(input)?.as_ref() {
"regular" => {
let mut _executable = false;
let mut tag = String::read(input)?;
if tag == "executable" {
_executable = true;
if String::read(input)? != "" {
return Err(Error::BadExecutableField);
}
tag = String::read(input)?;
}
if tag != "contents" {
return Err(Error::MissingNarField);
}
let _contents = Vec::<u8>::read(input)?;
if String::read(input)? != ")" {
return Err(Error::MissingNarCloseTag);
}
}
"directory" => loop {
match String::read(input)?.as_ref() {
"entry" => {
if String::read(input)? != "(" {
return Err(Error::MissingNarOpenTag);
}
if String::read(input)? != "name" {
return Err(Error::MissingNarField);
}
let _name = String::read(input)?;
if String::read(input)? != "node" {
return Err(Error::MissingNarField);
}
parse_file(input)?;
let tag = String::read(input)?;
if tag != ")" {
return Err(Error::MissingNarCloseTag);
}
}
")" => break,
s => return Err(Error::BadNarField(s.into())),
}
},
"symlink" => {
if String::read(input)? != "target" {
return Err(Error::MissingNarField);
}
let _target = String::read(input)?;
if String::read(input)? != ")" {
return Err(Error::MissingNarCloseTag);
}
}
s => return Err(Error::BadNarField(s.into())),
}
Ok(())
}
trait Deserialize: Sized {
fn read<R: Read>(input: &mut R) -> Result<Self, Error>;
}
impl Deserialize for String {
fn read<R: Read>(input: &mut R) -> Result<Self, Error> {
let buf = Deserialize::read(input)?;
Ok(String::from_utf8(buf).map_err(|_| Error::BadNarString)?)
}
}
impl Deserialize for Vec<u8> {
fn read<R: Read>(input: &mut R) -> Result<Self, Error> {
let n: usize = Deserialize::read(input)?;
let mut buf = vec![0; n];
input.read_exact(&mut buf)?;
skip_padding(input, n)?;
Ok(buf)
}
}
fn skip_padding<R: Read>(input: &mut R, len: usize) -> Result<(), Error> {
if len % 8 != 0 {
let mut buf = [0; 8];
let buf = &mut buf[0..8 - (len % 8)];
input.read_exact(buf)?;
if !buf.iter().all(|b| *b == 0) {
return Err(Error::BadNarPadding);
}
}
Ok(())
}
impl Deserialize for u64 {
fn read<R: Read>(input: &mut R) -> Result<Self, Error> {
Ok(input.read_u64::<LittleEndian>()?)
}
}
impl Deserialize for usize {
fn read<R: Read>(input: &mut R) -> Result<Self, Error> {
let n: u64 = Deserialize::read(input)?;
Ok(usize::try_from(n).map_err(|_| Error::NarSizeFieldTooBig)?)
}
}
-48
View File
@@ -1,48 +0,0 @@
use super::{PathInfo, Store, StorePath};
use crate::Error;
use hyper::client::Client;
pub struct BinaryCacheStore {
base_uri: String,
client: Client<hyper::client::HttpConnector, hyper::Body>,
}
impl BinaryCacheStore {
pub fn new(base_uri: String) -> Self {
Self {
base_uri,
client: Client::new(),
}
}
}
impl Store for BinaryCacheStore {
fn query_path_info(
&self,
path: &StorePath,
) -> std::pin::Pin<Box<dyn std::future::Future<Output = Result<PathInfo, Error>> + Send>> {
let uri = format!("{}/{}.narinfo", self.base_uri.clone(), path.hash);
let path = path.clone();
let client = self.client.clone();
let store_dir = self.store_dir().to_string();
Box::pin(async move {
let response = client.get(uri.parse::<hyper::Uri>().unwrap()).await?;
if response.status() == hyper::StatusCode::NOT_FOUND
|| response.status() == hyper::StatusCode::FORBIDDEN
{
return Err(Error::InvalidPath(path));
}
let mut body = response.into_body();
let mut bytes = Vec::new();
while let Some(next) = body.next().await {
bytes.extend(next?);
}
PathInfo::parse_nar_info(std::str::from_utf8(&bytes).unwrap(), &store_dir)
})
}
}
-17
View File
@@ -1,17 +0,0 @@
pub mod path;
#[cfg(unused)]
mod binary_cache_store;
#[cfg(unused)]
mod path_info;
#[cfg(unused)]
mod store;
pub use path::{StorePath, StorePathHash, StorePathName};
#[cfg(unused)]
pub use binary_cache_store::BinaryCacheStore;
#[cfg(unused)]
pub use path_info::PathInfo;
#[cfg(unused)]
pub use store::Store;
-224
View File
@@ -1,224 +0,0 @@
use crate::error::Error;
use crate::util::base32;
use std::fmt;
use std::path::Path;
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Debug)]
pub struct StorePath {
pub hash: StorePathHash,
pub name: StorePathName,
}
pub const STORE_PATH_HASH_BYTES: usize = 20;
pub const STORE_PATH_HASH_CHARS: usize = 32;
impl StorePath {
pub fn new(path: &Path, store_dir: &Path) -> Result<Self, Error> {
if path.parent() != Some(store_dir) {
return Err(Error::NotInStore(path.into()));
}
Self::new_from_base_name(
path.file_name()
.ok_or_else(|| Error::BadStorePath(path.into()))?
.to_str()
.ok_or_else(|| Error::BadStorePath(path.into()))?,
)
}
pub fn from_parts(hash: [u8; STORE_PATH_HASH_BYTES], name: &str) -> Result<Self, Error> {
Ok(StorePath {
hash: StorePathHash(hash),
name: StorePathName::new(name)?,
})
}
pub fn new_from_base_name(base_name: &str) -> Result<Self, Error> {
if base_name.len() < STORE_PATH_HASH_CHARS + 1
|| base_name.as_bytes()[STORE_PATH_HASH_CHARS] != b'-'
{
return Err(Error::BadStorePath(base_name.into()));
}
Ok(StorePath {
hash: StorePathHash::new(&base_name[0..STORE_PATH_HASH_CHARS])?,
name: StorePathName::new(&base_name[STORE_PATH_HASH_CHARS + 1..])?,
})
}
}
impl fmt::Display for StorePath {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}-{}", self.hash, self.name)
}
}
#[derive(Clone, PartialEq, Eq, Debug)]
pub struct StorePathHash([u8; STORE_PATH_HASH_BYTES]);
impl StorePathHash {
pub fn new(s: &str) -> Result<Self, Error> {
assert_eq!(s.len(), STORE_PATH_HASH_CHARS);
let v = base32::decode(s)?;
assert_eq!(v.len(), STORE_PATH_HASH_BYTES);
let mut bytes: [u8; 20] = Default::default();
bytes.copy_from_slice(&v[0..STORE_PATH_HASH_BYTES]);
Ok(Self(bytes))
}
pub fn hash(&self) -> &[u8; STORE_PATH_HASH_BYTES] {
&self.0
}
}
impl fmt::Display for StorePathHash {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let mut buf = vec![0; STORE_PATH_HASH_CHARS];
base32::encode_into(&self.0, &mut buf);
f.write_str(std::str::from_utf8(&buf).unwrap())
}
}
impl Ord for StorePathHash {
fn cmp(&self, other: &Self) -> std::cmp::Ordering {
// Historically we've sorted store paths by their base32
// serialization, but our base32 encodes bytes in reverse
// order. So compare them in reverse order as well.
self.0.iter().rev().cmp(other.0.iter().rev())
}
}
impl PartialOrd for StorePathHash {
fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
Some(self.cmp(other))
}
}
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Debug)]
pub struct StorePathName(String);
impl StorePathName {
pub fn new(s: &str) -> Result<Self, Error> {
if s.is_empty() {
return Err(Error::StorePathNameEmpty);
}
if s.len() > 211 {
return Err(Error::StorePathNameTooLong);
}
let is_good_path_name = s.chars().all(|c| {
c.is_ascii_alphabetic()
|| c.is_ascii_digit()
|| c == '+'
|| c == '-'
|| c == '.'
|| c == '_'
|| c == '?'
|| c == '='
});
if s.starts_with('.') || !is_good_path_name {
return Err(Error::BadStorePathName);
}
Ok(Self(s.to_string()))
}
pub fn name(&self) -> &str {
&self.0
}
}
impl fmt::Display for StorePathName {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(&self.0)
}
}
#[cfg(test)]
mod tests {
use super::*;
use assert_matches::assert_matches;
#[test]
fn test_parse() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-konsole-18.12.3";
let p = StorePath::new_from_base_name(&s).unwrap();
assert_eq!(p.name.0, "konsole-18.12.3");
assert_eq!(
p.hash.0,
[
0x9f, 0x76, 0x49, 0x20, 0xf6, 0x5d, 0xe9, 0x71, 0xc4, 0xca, 0x46, 0x21, 0xab, 0xff,
0x9b, 0x44, 0xef, 0x87, 0x0f, 0x3c
]
);
}
#[test]
fn test_no_name() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::StorePathNameEmpty)
);
}
#[test]
fn test_no_dash() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::BadStorePath(_))
);
}
#[test]
fn test_short_hash() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxl-konsole-18.12.3";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::BadStorePath(_))
);
}
#[test]
fn test_invalid_hash() {
let s = "7h7qgvs4kgzsn8e6rb273saxyqh4jxlz-konsole-18.12.3";
assert_matches!(StorePath::new_from_base_name(&s), Err(Error::BadBase32));
}
#[test]
fn test_long_name() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
assert_matches!(StorePath::new_from_base_name(&s), Ok(_));
}
#[test]
fn test_too_long_name() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::StorePathNameTooLong)
);
}
#[test]
fn test_bad_name() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-foo bar";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::BadStorePathName)
);
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-kónsole";
assert_matches!(
StorePath::new_from_base_name(&s),
Err(Error::BadStorePathName)
);
}
#[test]
fn test_roundtrip() {
let s = "7h7qgvs4kgzsn8a6rb273saxyqh4jxlz-konsole-18.12.3";
assert_eq!(StorePath::new_from_base_name(&s).unwrap().to_string(), s);
}
}
-70
View File
@@ -1,70 +0,0 @@
use crate::store::StorePath;
use crate::Error;
use std::collections::BTreeSet;
#[derive(Clone, Debug)]
pub struct PathInfo {
pub path: StorePath,
pub references: BTreeSet<StorePath>,
pub nar_size: u64,
pub deriver: Option<StorePath>,
// Additional binary cache info.
pub url: Option<String>,
pub compression: Option<String>,
pub file_size: Option<u64>,
}
impl PathInfo {
pub fn parse_nar_info(nar_info: &str, store_dir: &str) -> Result<Self, Error> {
let mut path = None;
let mut references = BTreeSet::new();
let mut nar_size = None;
let mut deriver = None;
let mut url = None;
let mut compression = None;
let mut file_size = None;
for line in nar_info.lines() {
let colon = line.find(':').ok_or(Error::BadNarInfo)?;
let (name, value) = line.split_at(colon);
if !value.starts_with(": ") {
return Err(Error::BadNarInfo);
}
let value = &value[2..];
if name == "StorePath" {
path = Some(StorePath::new(std::path::Path::new(value), store_dir)?);
} else if name == "NarSize" {
nar_size = Some(u64::from_str_radix(value, 10).map_err(|_| Error::BadNarInfo)?);
} else if name == "References" {
if !value.is_empty() {
for r in value.split(' ') {
references.insert(StorePath::new_from_base_name(r)?);
}
}
} else if name == "Deriver" {
deriver = Some(StorePath::new_from_base_name(value)?);
} else if name == "URL" {
url = Some(value.into());
} else if name == "Compression" {
compression = Some(value.into());
} else if name == "FileSize" {
file_size = Some(u64::from_str_radix(value, 10).map_err(|_| Error::BadNarInfo)?);
}
}
Ok(PathInfo {
path: path.ok_or(Error::BadNarInfo)?,
references,
nar_size: nar_size.ok_or(Error::BadNarInfo)?,
deriver,
url: Some(url.ok_or(Error::BadNarInfo)?),
compression,
file_size,
})
}
}
-53
View File
@@ -1,53 +0,0 @@
use super::{PathInfo, StorePath};
use crate::Error;
use std::collections::{BTreeMap, BTreeSet};
use std::path::Path;
pub trait Store: Send + Sync {
fn store_dir(&self) -> &str {
"/nix/store"
}
fn query_path_info(
&self,
store_path: &StorePath,
) -> std::pin::Pin<Box<dyn std::future::Future<Output = Result<PathInfo, Error>> + Send>>;
}
impl dyn Store {
pub fn parse_store_path(&self, path: &Path) -> Result<StorePath, Error> {
StorePath::new(path, self.store_dir())
}
pub async fn compute_path_closure(
&self,
roots: BTreeSet<StorePath>,
) -> Result<BTreeMap<StorePath, PathInfo>, Error> {
let mut done = BTreeSet::new();
let mut result = BTreeMap::new();
let mut pending = vec![];
for root in roots {
pending.push(self.query_path_info(&root));
done.insert(root);
}
while !pending.is_empty() {
let (info, _, remaining) = futures::future::select_all(pending).await;
pending = remaining;
let info = info?;
for path in &info.references {
if !done.contains(path) {
pending.push(self.query_path_info(&path));
done.insert(path.clone());
}
}
result.insert(info.path.clone(), info);
}
Ok(result)
}
}
-160
View File
@@ -1,160 +0,0 @@
use crate::error::Error;
use lazy_static::lazy_static;
pub fn encoded_len(input_len: usize) -> usize {
if input_len == 0 {
0
} else {
(input_len * 8 - 1) / 5 + 1
}
}
pub fn decoded_len(input_len: usize) -> usize {
input_len * 5 / 8
}
static BASE32_CHARS: &[u8; 32] = &b"0123456789abcdfghijklmnpqrsvwxyz";
lazy_static! {
static ref BASE32_CHARS_REVERSE: Box<[u8; 256]> = {
let mut xs = [0xffu8; 256];
for (n, c) in BASE32_CHARS.iter().enumerate() {
xs[*c as usize] = n as u8;
}
Box::new(xs)
};
}
pub fn encode(input: &[u8]) -> String {
let mut buf = vec![0; encoded_len(input.len())];
encode_into(input, &mut buf);
std::str::from_utf8(&buf).unwrap().to_string()
}
pub fn encode_into(input: &[u8], output: &mut [u8]) {
let len = encoded_len(input.len());
assert_eq!(len, output.len());
let mut nr_bits_left: usize = 0;
let mut bits_left: u16 = 0;
let mut pos = len;
for b in input {
bits_left |= (*b as u16) << nr_bits_left;
nr_bits_left += 8;
while nr_bits_left > 5 {
output[pos - 1] = BASE32_CHARS[(bits_left & 0x1f) as usize];
pos -= 1;
bits_left >>= 5;
nr_bits_left -= 5;
}
}
if nr_bits_left > 0 {
output[pos - 1] = BASE32_CHARS[(bits_left & 0x1f) as usize];
pos -= 1;
}
assert_eq!(pos, 0);
}
pub fn decode(input: &str) -> Result<Vec<u8>, crate::Error> {
let mut res = Vec::with_capacity(decoded_len(input.len()));
let mut nr_bits_left: usize = 0;
let mut bits_left: u16 = 0;
for c in input.chars().rev() {
let b = BASE32_CHARS_REVERSE[c as usize];
if b == 0xff {
return Err(Error::BadBase32);
}
bits_left |= (b as u16) << nr_bits_left;
nr_bits_left += 5;
if nr_bits_left >= 8 {
res.push((bits_left & 0xff) as u8);
bits_left >>= 8;
nr_bits_left -= 8;
}
}
if nr_bits_left > 0 && bits_left != 0 {
return Err(Error::BadBase32);
}
Ok(res)
}
#[cfg(test)]
mod tests {
use super::*;
use assert_matches::assert_matches;
use hex;
use proptest::proptest;
#[test]
fn test_encode() {
assert_eq!(encode(&[]), "");
assert_eq!(
encode(&hex::decode("0839703786356bca59b0f4a32987eb2e6de43ae8").unwrap()),
"x0xf8v9fxf3jk8zln1cwlsrmhqvp0f88"
);
assert_eq!(
encode(
&hex::decode("ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad")
.unwrap()
),
"1b8m03r63zqhnjf7l5wnldhh7c134ap5vpj0850ymkq1iyzicy5s"
);
assert_eq!(
encode(
&hex::decode("ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f")
.unwrap()
),
"2gs8k559z4rlahfx0y688s49m2vvszylcikrfinm30ly9rak69236nkam5ydvly1ai7xac99vxfc4ii84hawjbk876blyk1jfhkbbyx"
);
}
#[test]
fn test_decode() {
assert_eq!(hex::encode(decode("").unwrap()), "");
assert_eq!(
hex::encode(decode("x0xf8v9fxf3jk8zln1cwlsrmhqvp0f88").unwrap()),
"0839703786356bca59b0f4a32987eb2e6de43ae8"
);
assert_eq!(
hex::encode(decode("1b8m03r63zqhnjf7l5wnldhh7c134ap5vpj0850ymkq1iyzicy5s").unwrap()),
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
);
assert_eq!(
hex::encode(decode("2gs8k559z4rlahfx0y688s49m2vvszylcikrfinm30ly9rak69236nkam5ydvly1ai7xac99vxfc4ii84hawjbk876blyk1jfhkbbyx").unwrap()),
"ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f"
);
assert_matches!(
decode("xoxf8v9fxf3jk8zln1cwlsrmhqvp0f88"),
Err(Error::BadBase32)
);
assert_matches!(
decode("2b8m03r63zqhnjf7l5wnldhh7c134ap5vpj0850ymkq1iyzicy5s"),
Err(Error::BadBase32)
);
assert_matches!(decode("2"), Err(Error::BadBase32));
assert_matches!(decode("2gs"), Err(Error::BadBase32));
assert_matches!(decode("2gs8"), Err(Error::BadBase32));
}
proptest! {
#[test]
fn roundtrip(s: Vec<u8>) {
assert_eq!(s, decode(&encode(&s)).unwrap());
}
}
}
-1
View File
@@ -1 +0,0 @@
pub mod base32;
+1 -1
View File
@@ -240,7 +240,7 @@ SV * convertHash(char * algo, char * s, int toBase32)
PPCODE:
try {
auto h = Hash::parseAny(s, parseHashType(algo));
string s = h.to_string(toBase32 ? Base32 : Base16, false);
auto s = h.to_string(toBase32 ? Base32 : Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
+9 -4
View File
@@ -16,12 +16,17 @@ someBuildFailed=0
for buildId in $BUILDS_FOR_LATEST_EVAL; do
buildInfo=$(curl -sS -H 'Accept: application/json' "https://hydra.nixos.org/build/$buildId")
buildStatus=$(echo "$buildInfo" | \
jq -r '.buildstatus')
finished=$(echo "$buildInfo" | jq -r '.finished')
if [[ "$buildStatus" -ne 0 ]]; then
if [[ $finished = 0 ]]; then
continue
fi
buildStatus=$(echo "$buildInfo" | jq -r '.buildstatus')
if [[ $buildStatus != 0 ]]; then
someBuildFailed=1
echo "Job “$(echo "$buildInfo" | jq -r '.job')” failed on hydra"
echo "Job “$(echo "$buildInfo" | jq -r '.job')” failed on hydra: $buildInfo"
fi
done
+10 -4
View File
@@ -246,7 +246,8 @@ get_volume_pass() {
verify_volume_pass() {
local volume_special="$1" # (i.e., disk1s7)
local volume_uuid="$2"
/usr/sbin/diskutil apfs unlockVolume "$volume_special" -verify -stdinpassphrase -user "$volume_uuid"
_sudo "to confirm the password actually unlocks the volume" \
/usr/sbin/diskutil apfs unlockVolume "$volume_special" -verify -stdinpassphrase -user "$volume_uuid"
}
volume_pass_works() {
@@ -685,22 +686,27 @@ encrypt_volume() {
local volume_uuid="$1"
local volume_label="$2"
local password
task "Encrypt the Nix volume" >&2
# Note: mount/unmount are late additions to support the right order
# of operations for creating the volume and then baking its uuid into
# other artifacts; not as well-trod wrt to potential errors, race
# conditions, etc.
/usr/sbin/diskutil mount "$volume_label"
_sudo "to mount your Nix volume for encrypting" \
/usr/sbin/diskutil mount "$volume_label"
password="$(/usr/bin/xxd -l 32 -p -c 256 /dev/random)"
_sudo "to add your Nix volume's password to Keychain" \
/usr/bin/security -i <<EOF
add-generic-password -a "$volume_label" -s "$volume_uuid" -l "$volume_label encryption password" -D "Encrypted volume password" -j "Added automatically by the Nix installer for use by $NIX_VOLUME_MOUNTD_DEST" -w "$password" -T /System/Library/CoreServices/APFSUserAgent -T /System/Library/CoreServices/CSUserAgent -T /usr/bin/security "/Library/Keychains/System.keychain"
EOF
builtin printf "%s" "$password" | _sudo "to encrypt your Nix volume" \
builtin printf "%s" "$password" | _sudo "to actually encrypt your Nix volume" \
/usr/sbin/diskutil apfs encryptVolume "$volume_label" -user disk -stdinpassphrase
/usr/sbin/diskutil unmount force "$volume_label"
_sudo "to unmount the encrypted volume" \
/usr/sbin/diskutil unmount force "$volume_label"
}
create_volume() {
+24 -5
View File
@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="30000"
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# darwin installer needs to override these
NIX_FIRST_BUILD_UID="30001"
NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
@@ -576,18 +576,37 @@ create_directories() {
# since this bit is cross-platform:
# - first try with `command -vp` to try and find
# chown in the usual places
# * to work around some sort of deficiency in
# `command -p` in macOS bash 3.2, we also add
# PATH="$(getconf PATH 2>/dev/null)". As long as
# getconf is found, this should set a sane PATH
# which `command -p` in bash 3.2 appears to use.
# A bash with a properly-working `command -p`
# should ignore this hard-set PATH in favor of
# whatever it obtains internally. See
# github.com/NixOS/nix/issues/5768
# - fall back on `command -v` which would find
# any chown on path
# if we don't find one, the command is already
# hiding behind || true, and the general state
# should be one the user can repair once they
# figure out where chown is...
local get_chr_own="$(command -vp chown)"
local get_chr_own="$(PATH="$(getconf PATH 2>/dev/null)" command -vp chown)"
if [[ -z "$get_chr_own" ]]; then
get_chr_own="$(command -v chown)"
fi
_sudo "to take root ownership of existing Nix store files" \
"$get_chr_own" -R "root:$NIX_BUILD_GROUP_NAME" "$NIX_ROOT" || true
if [[ -z "$get_chr_own" ]]; then
reminder <<EOF
I wanted to take root ownership of existing Nix store files,
but I couldn't locate 'chown'. (You may need to fix your PATH.)
To manually change file ownership, you can run:
sudo chown -R 'root:$NIX_BUILD_GROUP_NAME' '$NIX_ROOT'
EOF
else
_sudo "to take root ownership of existing Nix store files" \
"$get_chr_own" -R "root:$NIX_BUILD_GROUP_NAME" "$NIX_ROOT" || true
fi
fi
_sudo "to make the basic directory structure of Nix (part 1)" \
install -dv -m 0755 /nix /nix/var /nix/var/log /nix/var/log/nix /nix/var/log/nix/drvs /nix/var/nix{,/db,/gcroots,/profiles,/temproots,/userpool,/daemon-socket} /nix/var/nix/{gcroots,profiles}/per-user
+3
View File
@@ -24,6 +24,9 @@ if [ -n "$HOME" ] && [ -n "$USER" ]; then
export NIX_SSL_CERT_FILE="$NIX_LINK/etc/ca-bundle.crt"
fi
# Only use MANPATH if it is already set. In general `man` will just simply
# pick up `.nix-profile/share/man` because is it close to `.nix-profile/bin`
# which is in the $PATH. For more info, run `manpath -d`.
if [ -n "${MANPATH-}" ]; then
export MANPATH="$NIX_LINK/share/man:$MANPATH"
fi
+1 -1
View File
@@ -1,3 +1,3 @@
(import (fetchTarball https://github.com/edolstra/flake-compat/archive/master.tar.gz) {
(import (fetchTarball "https://github.com/edolstra/flake-compat/archive/master.tar.gz") {
src = ./.;
}).shellNix
+6 -5
View File
@@ -14,6 +14,7 @@
#include "pathlocks.hh"
#include "globals.hh"
#include "serialise.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "derivations.hh"
#include "local-store.hh"
@@ -32,7 +33,7 @@ std::string escapeUri(std::string uri)
return uri;
}
static string currentLoad;
static std::string currentLoad;
static AutoCloseFD openSlotLock(const Machine & m, uint64_t slot)
{
@@ -97,7 +98,7 @@ static int main_build_remote(int argc, char * * argv)
}
std::optional<StorePath> drvPath;
string storeUri;
std::string storeUri;
while (true) {
@@ -183,7 +184,7 @@ static int main_build_remote(int argc, char * * argv)
else
{
// build the hint template.
string errorText =
std::string errorText =
"Failed to find a machine for remote build!\n"
"derivation: %s\nrequired (system, features): (%s, %s)";
errorText += "\n%s available machines:";
@@ -193,7 +194,7 @@ static int main_build_remote(int argc, char * * argv)
errorText += "\n(%s, %s, %s, %s)";
// add the template values.
string drvstr;
std::string drvstr;
if (drvPath.has_value())
drvstr = drvPath->to_string();
else
@@ -208,7 +209,7 @@ static int main_build_remote(int argc, char * * argv)
for (auto & m : machines)
error
% concatStringsSep<vector<string>>(", ", m.systemTypes)
% concatStringsSep<std::vector<std::string>>(", ", m.systemTypes)
% m.maxJobs
% concatStringsSep<StringSet>(", ", m.supportedFeatures)
% concatStringsSep<StringSet>(", ", m.mandatoryFeatures);
+1 -1
View File
@@ -153,7 +153,7 @@ void BuiltPathsCommand::run(ref<Store> store)
for (auto & p : store->queryAllValidPaths())
paths.push_back(BuiltPath::Opaque{p});
} else {
paths = toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
paths = Installable::toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) {
// XXX: This only computes the store path closure, ignoring
// intermediate realisations
-54
View File
@@ -5,7 +5,6 @@
#include "common-eval-args.hh"
#include "path.hh"
#include "flake/lockfile.hh"
#include "store-api.hh"
#include <optional>
@@ -82,14 +81,6 @@ struct MixFlakeOptions : virtual Args, EvalCommand
{ return {}; }
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
};
struct SourceExprCommand : virtual Args, MixFlakeOptions
{
std::optional<Path> file;
@@ -113,19 +104,6 @@ struct SourceExprCommand : virtual Args, MixFlakeOptions
void completeInstallable(std::string_view prefix);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* A command that operates on a list of "installables", which can be
store paths, attribute paths, Nix expressions, etc. */
struct InstallablesCommand : virtual Args, SourceExprCommand
@@ -238,38 +216,6 @@ static RegisterCommand registerCommand2(std::vector<std::string> && name)
return RegisterCommand(std::move(name), [](){ return make_ref<T>(); });
}
BuiltPaths build(
ref<Store> evalStore,
ref<Store> store, Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
/* Helper function to generate args that invoke $EDITOR on
filename:lineno. */
Strings editorFor(const Pos & pos);
+55 -35
View File
@@ -97,7 +97,7 @@ MixFlakeOptions::MixFlakeOptions()
lockFlags.writeLockFile = false;
lockFlags.inputOverrides.insert_or_assign(
flake::parseInputPath(inputPath),
parseFlakeRef(flakeRef, absPath(".")));
parseFlakeRef(flakeRef, absPath("."), true));
}}
});
@@ -158,7 +158,10 @@ SourceExprCommand::SourceExprCommand()
Strings SourceExprCommand::getDefaultFlakeAttrPaths()
{
return {"defaultPackage." + settings.thisSystem.get()};
return {
"packages." + settings.thisSystem.get() + ".default",
"defaultPackage." + settings.thisSystem.get()
};
}
Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
@@ -198,8 +201,9 @@ void SourceExprCommand::completeInstallable(std::string_view prefix)
prefix_ = "";
}
Value &v1(*findAlongAttrPath(*state, prefix_, *autoArgs, root).first);
state->forceValue(v1);
auto [v, pos] = findAlongAttrPath(*state, prefix_, *autoArgs, root);
Value &v1(*v);
state->forceValue(v1, pos);
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
@@ -268,9 +272,9 @@ void completeFlakeRefWithFragment(
auto attr = root->findAlongAttrPath(attrPath);
if (!attr) continue;
for (auto & attr2 : attr->getAttrs()) {
for (auto & attr2 : (*attr)->getAttrs()) {
if (hasPrefix(attr2, lastAttr)) {
auto attrPath2 = attr->getAttrPath(attr2);
auto attrPath2 = (*attr)->getAttrPath(attr2);
/* Strip the attrpath prefix. */
attrPath2.erase(attrPath2.begin(), attrPath2.begin() + attrPathPrefix.size());
completions->add(flakeRefS + "#" + concatStringsSep(".", attrPath2));
@@ -446,7 +450,7 @@ struct InstallableAttrPath : InstallableValue
std::pair<Value *, Pos> toValue(EvalState & state) override
{
auto [vRes, pos] = findAlongAttrPath(state, attrPath, *cmd.getAutoArgs(state), **v);
state.forceValue(*vRes);
state.forceValue(*vRes, pos);
return {vRes, pos};
}
@@ -464,11 +468,10 @@ std::vector<InstallableValue::DerivationInfo> InstallableAttrPath::toDerivations
std::vector<DerivationInfo> res;
for (auto & drvInfo : drvInfos) {
res.push_back({
state->store->parseStorePath(drvInfo.queryDrvPath()),
state->store->maybeParseStorePath(drvInfo.queryOutPath()),
drvInfo.queryOutputName()
});
auto drvPath = drvInfo.queryDrvPath();
if (!drvPath)
throw Error("'%s' is not a derivation", what());
res.push_back({ *drvPath, drvInfo.queryOutputName() });
}
return res;
@@ -496,7 +499,7 @@ Value * InstallableFlake::getFlakeOutputs(EvalState & state, const flake::Locked
auto aOutputs = vFlake->attrs->get(state.symbols.create("outputs"));
assert(aOutputs);
state.forceValue(*aOutputs->value);
state.forceValue(*aOutputs->value, [&]() { return aOutputs->value->determinePos(noPos); });
return aOutputs->value;
}
@@ -521,7 +524,7 @@ ref<eval_cache::EvalCache> openEvalCache(
auto vFlake = state.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake);
state.forceAttrs(*vFlake);
state.forceAttrs(*vFlake, noPos);
auto aOutputs = vFlake->attrs->get(state.symbols.create("outputs"));
assert(aOutputs);
@@ -544,13 +547,14 @@ InstallableFlake::InstallableFlake(
SourceExprCommand * cmd,
ref<EvalState> state,
FlakeRef && flakeRef,
Strings && attrPaths,
Strings && prefixes,
std::string_view fragment,
Strings attrPaths,
Strings prefixes,
const flake::LockFlags & lockFlags)
: InstallableValue(state),
flakeRef(flakeRef),
attrPaths(attrPaths),
prefixes(prefixes),
attrPaths(fragment == "" ? attrPaths : Strings{(std::string) fragment}),
prefixes(fragment == "" ? Strings{} : prefixes),
lockFlags(lockFlags)
{
if (cmd && cmd->getAutoArgs(*state)->size())
@@ -564,29 +568,37 @@ std::tuple<std::string, FlakeRef, InstallableValue::DerivationInfo> InstallableF
auto cache = openEvalCache(*state, lockedFlake);
auto root = cache->getRoot();
Suggestions suggestions;
for (auto & attrPath : getActualAttrPaths()) {
auto attr = root->findAlongAttrPath(
debug("trying flake output attribute '%s'", attrPath);
auto attrOrSuggestions = root->findAlongAttrPath(
parseAttrPath(*state, attrPath),
true
);
if (!attr) continue;
if (!attrOrSuggestions) {
suggestions += attrOrSuggestions.getSuggestions();
continue;
}
auto attr = *attrOrSuggestions;
if (!attr->isDerivation())
throw Error("flake output attribute '%s' is not a derivation", attrPath);
auto drvPath = attr->forceDerivation();
auto drvInfo = DerivationInfo{
auto drvInfo = DerivationInfo {
std::move(drvPath),
state->store->maybeParseStorePath(attr->getAttr(state->sOutPath)->getString()),
attr->getAttr(state->sOutputName)->getString()
};
return {attrPath, lockedFlake->flake.lockedRef, std::move(drvInfo)};
}
throw Error("flake '%s' does not provide attribute %s",
throw Error(suggestions, "flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
}
@@ -605,17 +617,24 @@ std::pair<Value *, Pos> InstallableFlake::toValue(EvalState & state)
auto emptyArgs = state.allocBindings(0);
Suggestions suggestions;
for (auto & attrPath : getActualAttrPaths()) {
try {
auto [v, pos] = findAlongAttrPath(state, attrPath, *emptyArgs, *vOutputs);
state.forceValue(*v);
state.forceValue(*v, pos);
return {v, pos};
} catch (AttrPathNotFound & e) {
suggestions += e.info().suggestions;
}
}
throw Error("flake '%s' does not provide attribute %s",
flakeRef, showAttrPaths(getActualAttrPaths()));
throw Error(
suggestions,
"flake '%s' does not provide attribute %s",
flakeRef,
showAttrPaths(getActualAttrPaths())
);
}
std::vector<std::pair<std::shared_ptr<eval_cache::AttrCursor>, std::string>>
@@ -630,7 +649,7 @@ InstallableFlake::getCursors(EvalState & state)
for (auto & attrPath : getActualAttrPaths()) {
auto attr = root->findAlongAttrPath(parseAttrPath(state, attrPath));
if (attr) res.push_back({attr, attrPath});
if (attr) res.push_back({*attr, attrPath});
}
return res;
@@ -707,7 +726,8 @@ std::vector<std::shared_ptr<Installable>> SourceExprCommand::parseInstallables(
this,
getEvalState(),
std::move(flakeRef),
fragment == "" ? getDefaultFlakeAttrPaths() : Strings{fragment},
fragment,
getDefaultFlakeAttrPaths(),
getDefaultFlakeAttrPathPrefixes(),
lockFlags));
continue;
@@ -779,7 +799,7 @@ BuiltPaths getBuiltPaths(ref<Store> evalStore, ref<Store> store, const DerivedPa
return res;
}
BuiltPaths build(
BuiltPaths Installable::build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -804,7 +824,7 @@ BuiltPaths build(
return getBuiltPaths(evalStore, store, pathsToBuild);
}
BuiltPaths toBuiltPaths(
BuiltPaths Installable::toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
@@ -812,19 +832,19 @@ BuiltPaths toBuiltPaths(
const std::vector<std::shared_ptr<Installable>> & installables)
{
if (operateOn == OperateOn::Output)
return build(evalStore, store, mode, installables);
return Installable::build(evalStore, store, mode, installables);
else {
if (mode == Realise::Nothing)
settings.readOnlyMode = true;
BuiltPaths res;
for (auto & drvPath : toDerivations(store, installables, true))
for (auto & drvPath : Installable::toDerivations(store, installables, true))
res.push_back(BuiltPath::Opaque{drvPath});
return res;
}
}
StorePathSet toStorePaths(
StorePathSet Installable::toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -838,7 +858,7 @@ StorePathSet toStorePaths(
return outPaths;
}
StorePath toStorePath(
StorePath Installable::toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn,
@@ -852,7 +872,7 @@ StorePath toStorePath(
return *paths.begin();
}
StorePathSet toDerivations(
StorePathSet Installable::toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver)
+63 -3
View File
@@ -5,6 +5,7 @@
#include "path-with-outputs.hh"
#include "derived-path.hh"
#include "eval.hh"
#include "store-api.hh"
#include "flake/flake.hh"
#include <optional>
@@ -29,6 +30,27 @@ struct UnresolvedApp
App resolve(ref<Store> evalStore, ref<Store> store);
};
enum class Realise {
/* Build the derivation. Postcondition: the
derivation outputs exist. */
Outputs,
/* Don't build the derivation. Postcondition: the store derivation
exists. */
Derivation,
/* Evaluate in dry-run mode. Postcondition: nothing. */
// FIXME: currently unused, but could be revived if we can
// evaluate derivations in-memory.
Nothing
};
/* How to handle derivations in commands that operate on store paths. */
enum class OperateOn {
/* Operate on the output path. */
Output,
/* Operate on the .drv path. */
Derivation
};
struct Installable
{
virtual ~Installable() { }
@@ -68,6 +90,39 @@ struct Installable
{
return FlakeRef::fromAttrs({{"type","indirect"}, {"id", "nixpkgs"}});
}
static BuiltPaths build(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
const std::vector<std::shared_ptr<Installable>> & installables,
BuildMode bMode = bmNormal);
static std::set<StorePath> toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
static StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable);
static std::set<StorePath> toDerivations(
ref<Store> store,
const std::vector<std::shared_ptr<Installable>> & installables,
bool useDeriver = false);
static BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
const std::vector<std::shared_ptr<Installable>> & installables);
};
struct InstallableValue : Installable
@@ -79,7 +134,6 @@ struct InstallableValue : Installable
struct DerivationInfo
{
StorePath drvPath;
std::optional<StorePath> outPath;
std::string outputName;
};
@@ -102,8 +156,9 @@ struct InstallableFlake : InstallableValue
SourceExprCommand * cmd,
ref<EvalState> state,
FlakeRef && flakeRef,
Strings && attrPaths,
Strings && prefixes,
std::string_view fragment,
Strings attrPaths,
Strings prefixes,
const flake::LockFlags & lockFlags);
std::string what() const override { return flakeRef.to_string() + "#" + *attrPaths.begin(); }
@@ -130,4 +185,9 @@ ref<eval_cache::EvalCache> openEvalCache(
EvalState & state,
std::shared_ptr<flake::LockedFlake> lockedFlake);
BuiltPaths getBuiltPaths(
ref<Store> evalStore,
ref<Store> store,
const DerivedPaths & hopefullyBuiltPaths);
}
+1 -1
View File
@@ -8,7 +8,7 @@ libcmd_SOURCES := $(wildcard $(d)/*.cc)
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers
libcmd_LDFLAGS += -llowdown -pthread
libcmd_LDFLAGS += $(LOWDOWN_LIBS) -pthread
libcmd_LIBS = libstore libutil libexpr libmain libfetchers
+12 -6
View File
@@ -9,7 +9,7 @@ namespace nix {
static Strings parseAttrPath(std::string_view s)
{
Strings res;
string cur;
std::string cur;
auto i = s.begin();
while (i != s.end()) {
if (*i == '.') {
@@ -41,7 +41,7 @@ std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s)
}
std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attrPath,
std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const std::string & attrPath,
Bindings & autoArgs, Value & vIn)
{
Strings tokens = parseAttrPath(attrPath);
@@ -58,7 +58,7 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
Value * vNew = state.allocValue();
state.autoCallFunction(autoArgs, *v, *vNew);
v = vNew;
state.forceValue(*v);
state.forceValue(*v, noPos);
/* It should evaluate to either a set or an expression,
according to what is specified in the attrPath. */
@@ -74,8 +74,14 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
throw Error("empty attribute name in selection path '%1%'", attrPath);
Bindings::iterator a = v->attrs->find(state.symbols.create(attr));
if (a == v->attrs->end())
throw AttrPathNotFound("attribute '%1%' in selection path '%2%' not found", attr, attrPath);
if (a == v->attrs->end()) {
std::set<std::string> attrNames;
for (auto & attr : *v->attrs)
attrNames.insert(attr.name);
auto suggestions = Suggestions::bestMatches(attrNames, attr);
throw AttrPathNotFound(suggestions, "attribute '%1%' in selection path '%2%' not found", attr, attrPath);
}
v = &*a->value;
pos = *a->pos;
}
@@ -121,7 +127,7 @@ Pos findPackageFilename(EvalState & state, Value & v, std::string what)
std::string filename(pos, 0, colon);
unsigned int lineno;
try {
lineno = std::stoi(std::string(pos, colon + 1));
lineno = std::stoi(std::string(pos, colon + 1, std::string::npos));
} catch (std::invalid_argument & e) {
throw ParseError("cannot parse line number '%s'", pos);
}
+5 -2
View File
@@ -10,8 +10,11 @@ namespace nix {
MakeError(AttrPathNotFound, Error);
MakeError(NoPositionInfo, Error);
std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attrPath,
Bindings & autoArgs, Value & vIn);
std::pair<Value *, Pos> findAlongAttrPath(
EvalState & state,
const std::string & attrPath,
Bindings & autoArgs,
Value & vIn);
/* Heuristic to find the filename and lineno or a nix value. */
Pos findPackageFilename(EvalState & state, Value & v, std::string what);
+1 -1
View File
@@ -105,7 +105,7 @@ public:
for (size_t n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
std::sort(res.begin(), res.end(), [](const Attr * a, const Attr * b) {
return (const string &) a->name < (const string &) b->name;
return (const std::string &) a->name < (const std::string &) b->name;
});
return res;
}
+4 -4
View File
@@ -77,7 +77,7 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
for (auto & i : autoArgs) {
auto v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(string(i.second, 1), absPath(".")));
state.mkThunk_(*v, state.parseExprFromString(i.second.substr(1), absPath(".")));
else
v->mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
@@ -85,17 +85,17 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
return res.finish();
}
Path lookupFileArg(EvalState & state, string s)
Path lookupFileArg(EvalState & state, std::string_view s)
{
if (isUri(s)) {
return state.store->toRealPath(
fetchers::downloadTarball(
state.store, resolveUri(s), "source", false).first.storePath);
} else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
Path p = s.substr(1, s.size() - 2);
Path p(s.substr(1, s.size() - 2));
return state.findFile(p);
} else
return absPath(s);
return absPath(std::string(s));
}
}
+1 -1
View File
@@ -22,6 +22,6 @@ private:
std::map<std::string, std::string> autoArgs;
};
Path lookupFileArg(EvalState & state, string s);
Path lookupFileArg(EvalState & state, std::string_view s);
}
+30 -11
View File
@@ -336,7 +336,7 @@ Value & AttrCursor::getValue()
if (!_value) {
if (parent) {
auto & vParent = parent->first->getValue();
root->state.forceAttrs(vParent);
root->state.forceAttrs(vParent, noPos);
auto attr = vParent.attrs->get(parent->second);
if (!attr)
throw Error("attribute '%s' is unexpectedly missing", getAttrPathStr());
@@ -381,7 +381,7 @@ Value & AttrCursor::forceValue()
auto & v = getValue();
try {
root->state.forceValue(v);
root->state.forceValue(v, noPos);
} catch (EvalError &) {
debug("setting '%s' to failed", getAttrPathStr());
if (root->db)
@@ -406,6 +406,16 @@ Value & AttrCursor::forceValue()
return v;
}
Suggestions AttrCursor::getSuggestionsForAttr(Symbol name)
{
auto attrNames = getAttrs();
std::set<std::string> strAttrNames;
for (auto & name : attrNames)
strAttrNames.insert(std::string(name));
return Suggestions::bestMatches(strAttrNames, name);
}
std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErrors)
{
if (root->db) {
@@ -446,6 +456,11 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
return nullptr;
//throw TypeError("'%s' is not an attribute set", getAttrPathStr());
for (auto & attr : *v.attrs) {
if (root->db)
root->db->setPlaceholder({cachedValue->first, attr.name});
}
auto attr = v.attrs->get(name);
if (!attr) {
@@ -464,7 +479,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
cachedValue2 = {root->db->setPlaceholder({cachedValue->first, name}), placeholder_t()};
}
return std::make_shared<AttrCursor>(
return make_ref<AttrCursor>(
root, std::make_pair(shared_from_this(), name), attr->value, std::move(cachedValue2));
}
@@ -473,27 +488,31 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(std::string_view name)
return maybeGetAttr(root->state.symbols.create(name));
}
std::shared_ptr<AttrCursor> AttrCursor::getAttr(Symbol name, bool forceErrors)
ref<AttrCursor> AttrCursor::getAttr(Symbol name, bool forceErrors)
{
auto p = maybeGetAttr(name, forceErrors);
if (!p)
throw Error("attribute '%s' does not exist", getAttrPathStr(name));
return p;
return ref(p);
}
std::shared_ptr<AttrCursor> AttrCursor::getAttr(std::string_view name)
ref<AttrCursor> AttrCursor::getAttr(std::string_view name)
{
return getAttr(root->state.symbols.create(name));
}
std::shared_ptr<AttrCursor> AttrCursor::findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force)
OrSuggestions<ref<AttrCursor>> AttrCursor::findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force)
{
auto res = shared_from_this();
for (auto & attr : attrPath) {
res = res->maybeGetAttr(attr, force);
if (!res) return {};
auto child = res->maybeGetAttr(attr, force);
if (!child) {
auto suggestions = res->getSuggestionsForAttr(attr);
return OrSuggestions<ref<AttrCursor>>::failed(suggestions);
}
res = child;
}
return res;
return ref(res);
}
std::string AttrCursor::getString()
@@ -596,7 +615,7 @@ std::vector<Symbol> AttrCursor::getAttrs()
for (auto & attr : *getValue().attrs)
attrs.push_back(attr.name);
std::sort(attrs.begin(), attrs.end(), [](const Symbol & a, const Symbol & b) {
return (const string &) a < (const string &) b;
return (const std::string &) a < (const std::string &) b;
});
if (root->db)
+5 -3
View File
@@ -94,15 +94,17 @@ public:
std::string getAttrPathStr(Symbol name) const;
Suggestions getSuggestionsForAttr(Symbol name);
std::shared_ptr<AttrCursor> maybeGetAttr(Symbol name, bool forceErrors = false);
std::shared_ptr<AttrCursor> maybeGetAttr(std::string_view name);
std::shared_ptr<AttrCursor> getAttr(Symbol name, bool forceErrors = false);
ref<AttrCursor> getAttr(Symbol name, bool forceErrors = false);
std::shared_ptr<AttrCursor> getAttr(std::string_view name);
ref<AttrCursor> getAttr(std::string_view name);
std::shared_ptr<AttrCursor> findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force = false);
OrSuggestions<ref<AttrCursor>> findAlongAttrPath(const std::vector<Symbol> & attrPath, bool force = false);
std::string getString();
+14 -22
View File
@@ -15,12 +15,6 @@ LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s))
});
}
LocalNoInlineNoReturn(void throwTypeError(const char * s, const Value & v))
{
throw TypeError(s, showType(v));
}
LocalNoInlineNoReturn(void throwTypeError(const Pos & pos, const char * s, const Value & v))
{
throw TypeError({
@@ -31,6 +25,13 @@ LocalNoInlineNoReturn(void throwTypeError(const Pos & pos, const char * s, const
void EvalState::forceValue(Value & v, const Pos & pos)
{
forceValue(v, [&]() { return pos; });
}
template<typename Callable>
void EvalState::forceValue(Value & v, Callable getPos)
{
if (v.isThunk()) {
Env * env = v.thunk.env;
@@ -47,31 +48,22 @@ void EvalState::forceValue(Value & v, const Pos & pos)
else if (v.isApp())
callFunction(*v.app.left, *v.app.right, v, noPos);
else if (v.isBlackhole())
throwEvalError(pos, "infinite recursion encountered");
}
inline void EvalState::forceAttrs(Value & v)
{
forceValue(v);
if (v.type() != nAttrs)
throwTypeError("value is %1% while a set was expected", v);
throwEvalError(getPos(), "infinite recursion encountered");
}
inline void EvalState::forceAttrs(Value & v, const Pos & pos)
{
forceValue(v, pos);
if (v.type() != nAttrs)
throwTypeError(pos, "value is %1% while a set was expected", v);
forceAttrs(v, [&]() { return pos; });
}
inline void EvalState::forceList(Value & v)
template <typename Callable>
inline void EvalState::forceAttrs(Value & v, Callable getPos)
{
forceValue(v);
if (!v.isList())
throwTypeError("value is %1% while a list was expected", v);
forceValue(v, getPos);
if (v.type() != nAttrs)
throwTypeError(getPos(), "value is %1% while a set was expected", v);
}
+149 -101
View File
@@ -1,5 +1,6 @@
#include "eval.hh"
#include "hash.hh"
#include "types.hh"
#include "util.hh"
#include "store-api.hh"
#include "derivations.hh"
@@ -62,9 +63,15 @@ static char * dupString(const char * s)
}
static char * dupStringWithLen(const char * s, size_t size)
// When there's no need to write to the string, we can optimize away empty
// string allocations.
// This function handles makeImmutableStringWithLen(null, 0) by returning the
// empty string.
static const char * makeImmutableStringWithLen(const char * s, size_t size)
{
char * t;
if (size == 0)
return "";
#if HAVE_BOEHMGC
t = GC_STRNDUP(s, size);
#else
@@ -74,6 +81,10 @@ static char * dupStringWithLen(const char * s, size_t size)
return t;
}
static inline const char * makeImmutableString(std::string_view s) {
return makeImmutableStringWithLen(s.data(), s.size());
}
RootValue allocRootValue(Value * v)
{
@@ -85,25 +96,20 @@ RootValue allocRootValue(Value * v)
}
void printValue(std::ostream & str, std::set<const Value *> & active, const Value & v)
void Value::print(std::ostream & str, std::set<const void *> * seen) const
{
checkInterrupt();
if (!active.insert(&v).second) {
str << "<CYCLE>";
return;
}
switch (v.internalType) {
switch (internalType) {
case tInt:
str << v.integer;
str << integer;
break;
case tBool:
str << (v.boolean ? "true" : "false");
str << (boolean ? "true" : "false");
break;
case tString:
str << "\"";
for (const char * i = v.string.s; *i; i++)
for (const char * i = string.s; *i; i++)
if (*i == '\"' || *i == '\\') str << "\\" << *i;
else if (*i == '\n') str << "\\n";
else if (*i == '\r') str << "\\r";
@@ -113,30 +119,38 @@ void printValue(std::ostream & str, std::set<const Value *> & active, const Valu
str << "\"";
break;
case tPath:
str << v.path; // !!! escaping?
str << path; // !!! escaping?
break;
case tNull:
str << "null";
break;
case tAttrs: {
str << "{ ";
for (auto & i : v.attrs->lexicographicOrder()) {
str << i->name << " = ";
printValue(str, active, *i->value);
str << "; ";
if (seen && !attrs->empty() && !seen->insert(attrs).second)
str << "«repeated»";
else {
str << "{ ";
for (auto & i : attrs->lexicographicOrder()) {
str << i->name << " = ";
i->value->print(str, seen);
str << "; ";
}
str << "}";
}
str << "}";
break;
}
case tList1:
case tList2:
case tListN:
str << "[ ";
for (auto v2 : v.listItems()) {
printValue(str, active, *v2);
str << " ";
if (seen && listSize() && !seen->insert(listElems()).second)
str << "«repeated»";
else {
str << "[ ";
for (auto v2 : listItems()) {
v2->print(str, seen);
str << " ";
}
str << "]";
}
str << "]";
break;
case tThunk:
case tApp:
@@ -152,28 +166,32 @@ void printValue(std::ostream & str, std::set<const Value *> & active, const Valu
str << "<PRIMOP-APP>";
break;
case tExternal:
str << *v.external;
str << *external;
break;
case tFloat:
str << v.fpoint;
str << fpoint;
break;
default:
abort();
}
}
active.erase(&v);
void Value::print(std::ostream & str, bool showRepeated) const
{
std::set<const void *> seen;
print(str, showRepeated ? nullptr : &seen);
}
std::ostream & operator << (std::ostream & str, const Value & v)
{
std::set<const Value *> active;
printValue(str, active, v);
v.print(str, false);
return str;
}
const Value *getPrimOp(const Value &v) {
const Value * getPrimOp(const Value &v) {
const Value * primOp = &v;
while (primOp->isPrimOpApp()) {
primOp = primOp->primOpApp.left;
@@ -182,7 +200,7 @@ const Value *getPrimOp(const Value &v) {
return primOp;
}
string showType(ValueType type)
std::string_view showType(ValueType type)
{
switch (type) {
case nInt: return "an integer";
@@ -201,24 +219,24 @@ string showType(ValueType type)
}
string showType(const Value & v)
std::string showType(const Value & v)
{
switch (v.internalType) {
case tString: return v.string.context ? "a string with context" : "a string";
case tPrimOp:
return fmt("the built-in function '%s'", string(v.primOp->name));
return fmt("the built-in function '%s'", std::string(v.primOp->name));
case tPrimOpApp:
return fmt("the partially applied built-in function '%s'", string(getPrimOp(v)->primOp->name));
return fmt("the partially applied built-in function '%s'", std::string(getPrimOp(v)->primOp->name));
case tExternal: return v.external->showType();
case tThunk: return "a thunk";
case tApp: return "a function application";
case tBlackhole: return "a black hole";
default:
return showType(v.type());
return std::string(showType(v.type()));
}
}
Pos Value::determinePos(const Pos &pos) const
Pos Value::determinePos(const Pos & pos) const
{
switch (internalType) {
case tAttrs: return *attrs->pos;
@@ -355,7 +373,7 @@ void initGC()
/* Very hacky way to parse $NIX_PATH, which is colon-separated, but
can contain URLs (e.g. "nixpkgs=https://bla...:foo=https://"). */
static Strings parseNixPath(const string & s)
static Strings parseNixPath(const std::string & s)
{
Strings res;
@@ -516,6 +534,14 @@ void EvalState::allowPath(const StorePath & storePath)
allowedPaths->insert(store->toRealPath(storePath));
}
void EvalState::allowAndSetStorePathString(const StorePath &storePath, Value & v)
{
allowPath(storePath);
auto path = store->printStorePath(storePath);
v.mkString(path, PathSet({path}));
}
Path EvalState::checkSourcePath(const Path & path_)
{
if (!allowedPaths) return path_;
@@ -605,7 +631,7 @@ Path EvalState::toRealPath(const Path & path, const PathSet & context)
}
Value * EvalState::addConstant(const string & name, Value & v)
Value * EvalState::addConstant(const std::string & name, Value & v)
{
Value * v2 = allocValue();
*v2 = v;
@@ -614,19 +640,19 @@ Value * EvalState::addConstant(const string & name, Value & v)
}
void EvalState::addConstant(const string & name, Value * v)
void EvalState::addConstant(const std::string & name, Value * v)
{
staticBaseEnv.vars.emplace_back(symbols.create(name), baseEnvDispl);
baseEnv.values[baseEnvDispl++] = v;
string name2 = string(name, 0, 2) == "__" ? string(name, 2) : name;
auto name2 = name.substr(0, 2) == "__" ? name.substr(2) : name;
baseEnv.values[0]->attrs->push_back(Attr(symbols.create(name2), v));
}
Value * EvalState::addPrimOp(const string & name,
Value * EvalState::addPrimOp(const std::string & name,
size_t arity, PrimOpFun primOp)
{
auto name2 = string(name, 0, 2) == "__" ? string(name, 2) : name;
auto name2 = name.substr(0, 2) == "__" ? name.substr(2) : name;
Symbol sym = symbols.create(name2);
/* Hack to make constants lazy: turn them into a application of
@@ -674,7 +700,7 @@ Value * EvalState::addPrimOp(PrimOp && primOp)
}
Value & EvalState::getBuiltin(const string & name)
Value & EvalState::getBuiltin(const std::string & name)
{
return *baseEnv.values[0]->attrs->find(symbols.create(name))->value;
}
@@ -702,12 +728,12 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
evaluator. So here are some helper functions for throwing
exceptions. */
LocalNoInlineNoReturn(void throwEvalError(const char * s, const string & s2))
LocalNoInlineNoReturn(void throwEvalError(const char * s, const std::string & s2))
{
throw EvalError(s, s2);
}
LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s, const string & s2))
LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s, const std::string & s2))
{
throw EvalError({
.msg = hintfmt(s, s2),
@@ -715,12 +741,12 @@ LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s, const
});
}
LocalNoInlineNoReturn(void throwEvalError(const char * s, const string & s2, const string & s3))
LocalNoInlineNoReturn(void throwEvalError(const char * s, const std::string & s2, const std::string & s3))
{
throw EvalError(s, s2, s3);
}
LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s, const string & s2, const string & s3))
LocalNoInlineNoReturn(void throwEvalError(const Pos & pos, const char * s, const std::string & s2, const std::string & s3))
{
throw EvalError({
.msg = hintfmt(s, s2, s3),
@@ -753,7 +779,12 @@ LocalNoInlineNoReturn(void throwTypeError(const Pos & pos, const char * s, const
});
}
LocalNoInlineNoReturn(void throwAssertionError(const Pos & pos, const char * s, const string & s1))
LocalNoInlineNoReturn(void throwTypeError(const char * s, const Value & v))
{
throw TypeError(s, showType(v));
}
LocalNoInlineNoReturn(void throwAssertionError(const Pos & pos, const char * s, const std::string & s1))
{
throw AssertionError({
.msg = hintfmt(s, s1),
@@ -761,7 +792,7 @@ LocalNoInlineNoReturn(void throwAssertionError(const Pos & pos, const char * s,
});
}
LocalNoInlineNoReturn(void throwUndefinedVarError(const Pos & pos, const char * s, const string & s1))
LocalNoInlineNoReturn(void throwUndefinedVarError(const Pos & pos, const char * s, const std::string & s1))
{
throw UndefinedVarError({
.msg = hintfmt(s, s1),
@@ -769,7 +800,7 @@ LocalNoInlineNoReturn(void throwUndefinedVarError(const Pos & pos, const char *
});
}
LocalNoInlineNoReturn(void throwMissingArgumentError(const Pos & pos, const char * s, const string & s1))
LocalNoInlineNoReturn(void throwMissingArgumentError(const Pos & pos, const char * s, const std::string & s1))
{
throw MissingArgumentError({
.msg = hintfmt(s, s1),
@@ -777,12 +808,12 @@ LocalNoInlineNoReturn(void throwMissingArgumentError(const Pos & pos, const char
});
}
LocalNoInline(void addErrorTrace(Error & e, const char * s, const string & s2))
LocalNoInline(void addErrorTrace(Error & e, const char * s, const std::string & s2))
{
e.addTrace(std::nullopt, s, s2);
}
LocalNoInline(void addErrorTrace(Error & e, const Pos & pos, const char * s, const string & s2))
LocalNoInline(void addErrorTrace(Error & e, const Pos & pos, const char * s, const std::string & s2))
{
e.addTrace(pos, s, s2);
}
@@ -790,7 +821,7 @@ LocalNoInline(void addErrorTrace(Error & e, const Pos & pos, const char * s, con
void Value::mkString(std::string_view s)
{
mkString(dupStringWithLen(s.data(), s.size()));
mkString(makeImmutableString(s));
}
@@ -821,7 +852,7 @@ void Value::mkStringMove(const char * s, const PathSet & context)
void Value::mkPath(std::string_view s)
{
mkPath(dupStringWithLen(s.data(), s.size()));
mkPath(makeImmutableString(s));
}
@@ -1137,7 +1168,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Hence we need __overrides.) */
if (hasOverrides) {
Value * vOverrides = (*v.attrs)[overrides->second.displ].value;
state.forceAttrs(*vOverrides);
state.forceAttrs(*vOverrides, [&]() { return vOverrides->determinePos(noPos); });
Bindings * newBnds = state.allocBindings(v.attrs->capacity() + vOverrides->attrs->size());
for (auto & i : *v.attrs)
newBnds->push_back(i);
@@ -1215,7 +1246,7 @@ void ExprVar::eval(EvalState & state, Env & env, Value & v)
}
static string showAttrPath(EvalState & state, Env & env, const AttrPath & attrPath)
static std::string showAttrPath(EvalState & state, Env & env, const AttrPath & attrPath)
{
std::ostringstream out;
bool first = true;
@@ -1285,7 +1316,7 @@ void ExprOpHasAttr::eval(EvalState & state, Env & env, Value & v)
e->eval(state, env, vTmp);
for (auto & i : attrPath) {
state.forceValue(*vAttrs);
state.forceValue(*vAttrs, noPos);
Bindings::iterator j;
Symbol name = getName(i, state, env);
if (vAttrs->type() != nAttrs ||
@@ -1373,7 +1404,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
/* Nope, so show the first unexpected argument to the
user. */
for (auto & i : *args[0]->attrs)
if (!lambda.formals->argNames.count(i.name))
if (!lambda.formals->has(i.name))
throwTypeError(pos, "%1% called with unexpected argument '%2%'", lambda, i.name);
abort(); // can't happen
}
@@ -1389,7 +1420,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
if (loggerSettings.showTrace.get()) {
addErrorTrace(e, lambda.pos, "while evaluating %s",
(lambda.name.set()
? "'" + (string) lambda.name + "'"
? "'" + (const std::string &) lambda.name + "'"
: "anonymous lambda"));
addErrorTrace(e, pos, "from call site%s", "");
}
@@ -1499,14 +1530,16 @@ void EvalState::incrFunctionCall(ExprLambda * fun)
void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
{
forceValue(fun);
auto pos = fun.determinePos(noPos);
forceValue(fun, pos);
if (fun.type() == nAttrs) {
auto found = fun.attrs->find(sFunctor);
if (found != fun.attrs->end()) {
Value * v = allocValue();
callFunction(*found->value, fun, *v, noPos);
forceValue(*v);
callFunction(*found->value, fun, *v, pos);
forceValue(*v, pos);
return autoCallFunction(args, *v, res);
}
}
@@ -1694,7 +1727,7 @@ void EvalState::concatLists(Value & v, size_t nrLists, Value * * lists, const Po
void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
{
PathSet context;
std::vector<std::string> s;
std::vector<BackedStringView> s;
size_t sSize = 0;
NixInt n = 0;
NixFloat nf = 0;
@@ -1705,7 +1738,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
const auto str = [&] {
std::string result;
result.reserve(sSize);
for (const auto & part : s) result += part;
for (const auto & part : s) result += *part;
return result;
};
/* c_str() is not str().c_str() because we want to create a string
@@ -1715,15 +1748,18 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
char * result = allocString(sSize + 1);
char * tmp = result;
for (const auto & part : s) {
memcpy(tmp, part.c_str(), part.size());
tmp += part.size();
memcpy(tmp, part->data(), part->size());
tmp += part->size();
}
*tmp = 0;
return result;
};
Value values[es->size()];
Value * vTmpP = values;
for (auto & [i_pos, i] : *es) {
Value vTmp;
Value & vTmp = *vTmpP++;
i->eval(state, env, vTmp);
/* If the first element is a path, then the result will also
@@ -1756,9 +1792,9 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
/* skip canonization of first path, which would only be not
canonized in the first place if it's coming from a ./${foo} type
path */
s.emplace_back(
state.coerceToString(i_pos, vTmp, context, false, firstType == nString, !first));
sSize += s.back().size();
auto part = state.coerceToString(i_pos, vTmp, context, false, firstType == nString, !first);
sSize += part->size();
s.emplace_back(std::move(part));
}
first = false;
@@ -1792,7 +1828,7 @@ void EvalState::forceValueDeep(Value & v)
recurse = [&](Value & v) {
if (!seen.insert(&v).second) return;
forceValue(v);
forceValue(v, [&]() { return v.determinePos(noPos); });
if (v.type() == nAttrs) {
for (auto & i : *v.attrs)
@@ -1857,7 +1893,7 @@ void EvalState::forceFunction(Value & v, const Pos & pos)
}
string EvalState::forceString(Value & v, const Pos & pos)
std::string_view EvalState::forceString(Value & v, const Pos & pos)
{
forceValue(v, pos);
if (v.type() != nString) {
@@ -1866,13 +1902,13 @@ string EvalState::forceString(Value & v, const Pos & pos)
else
throwTypeError("value is %1% while a string was expected", v);
}
return string(v.string.s);
return v.string.s;
}
/* Decode a context string ‘!<name>!<path>’ into a pair <path,
name>. */
std::pair<string, string> decodeContext(std::string_view s)
std::pair<std::string, std::string> decodeContext(std::string_view s)
{
if (s.at(0) == '!') {
size_t index = s.find("!", 1);
@@ -1901,17 +1937,17 @@ std::vector<std::pair<Path, std::string>> Value::getContext()
}
string EvalState::forceString(Value & v, PathSet & context, const Pos & pos)
std::string_view EvalState::forceString(Value & v, PathSet & context, const Pos & pos)
{
string s = forceString(v, pos);
auto s = forceString(v, pos);
copyContext(v, context);
return s;
}
string EvalState::forceStringNoCtx(Value & v, const Pos & pos)
std::string_view EvalState::forceStringNoCtx(Value & v, const Pos & pos)
{
string s = forceString(v, pos);
auto s = forceString(v, pos);
if (v.string.context) {
if (pos)
throwEvalError(pos, "the string '%1%' is not allowed to refer to a store path (such as '%2%')",
@@ -1929,47 +1965,48 @@ bool EvalState::isDerivation(Value & v)
if (v.type() != nAttrs) return false;
Bindings::iterator i = v.attrs->find(sType);
if (i == v.attrs->end()) return false;
forceValue(*i->value);
forceValue(*i->value, *i->pos);
if (i->value->type() != nString) return false;
return strcmp(i->value->string.s, "derivation") == 0;
}
std::optional<string> EvalState::tryAttrsToString(const Pos & pos, Value & v,
std::optional<std::string> EvalState::tryAttrsToString(const Pos & pos, Value & v,
PathSet & context, bool coerceMore, bool copyToStore)
{
auto i = v.attrs->find(sToString);
if (i != v.attrs->end()) {
Value v1;
callFunction(*i->value, v, v1, pos);
return coerceToString(pos, v1, context, coerceMore, copyToStore);
return coerceToString(pos, v1, context, coerceMore, copyToStore).toOwned();
}
return {};
}
string EvalState::coerceToString(const Pos & pos, Value & v, PathSet & context,
BackedStringView EvalState::coerceToString(const Pos & pos, Value & v, PathSet & context,
bool coerceMore, bool copyToStore, bool canonicalizePath)
{
forceValue(v, pos);
string s;
if (v.type() == nString) {
copyContext(v, context);
return v.string.s;
return std::string_view(v.string.s);
}
if (v.type() == nPath) {
Path path(canonicalizePath ? canonPath(v.path) : v.path);
return copyToStore ? copyPathToStore(context, path) : path;
BackedStringView path(PathView(v.path));
if (canonicalizePath)
path = canonPath(*path);
if (copyToStore)
path = copyPathToStore(context, std::move(path).toOwned());
return path;
}
if (v.type() == nAttrs) {
auto maybeString = tryAttrsToString(pos, v, context, coerceMore, copyToStore);
if (maybeString) {
return *maybeString;
}
if (maybeString)
return std::move(*maybeString);
auto i = v.attrs->find(sOutPath);
if (i == v.attrs->end()) throwTypeError(pos, "cannot coerce a set to a string");
return coerceToString(pos, *i->value, context, coerceMore, copyToStore);
@@ -1989,16 +2026,15 @@ string EvalState::coerceToString(const Pos & pos, Value & v, PathSet & context,
if (v.type() == nNull) return "";
if (v.isList()) {
string result;
std::string result;
for (auto [n, v2] : enumerate(v.listItems())) {
result += coerceToString(pos, *v2,
context, coerceMore, copyToStore);
result += *coerceToString(pos, *v2, context, coerceMore, copyToStore);
if (n < v.listSize() - 1
/* !!! not quite correct */
&& (!v2->isList() || v2->listSize() != 0))
result += " ";
}
return result;
return std::move(result);
}
}
@@ -2006,7 +2042,7 @@ string EvalState::coerceToString(const Pos & pos, Value & v, PathSet & context,
}
string EvalState::copyPathToStore(PathSet & context, const Path & path)
std::string EvalState::copyPathToStore(PathSet & context, const Path & path)
{
if (nix::isDerivation(path))
throwEvalError("file names are not allowed to end in '%1%'", drvExtension);
@@ -2032,17 +2068,29 @@ string EvalState::copyPathToStore(PathSet & context, const Path & path)
Path EvalState::coerceToPath(const Pos & pos, Value & v, PathSet & context)
{
string path = coerceToString(pos, v, context, false, false);
auto path = coerceToString(pos, v, context, false, false).toOwned();
if (path == "" || path[0] != '/')
throwEvalError(pos, "string '%1%' doesn't represent an absolute path", path);
return path;
}
StorePath EvalState::coerceToStorePath(const Pos & pos, Value & v, PathSet & context)
{
auto path = coerceToString(pos, v, context, false, false).toOwned();
if (auto storePath = store->maybeParseStorePath(path))
return *storePath;
throw EvalError({
.msg = hintfmt("path '%1%' is not in the Nix store", path),
.errPos = pos
});
}
bool EvalState::eqValues(Value & v1, Value & v2)
{
forceValue(v1);
forceValue(v2);
forceValue(v1, noPos);
forceValue(v2, noPos);
/* !!! Hack to support some old broken code that relies on pointer
equality tests between sets. (Specifically, builderDefs calls
@@ -2202,11 +2250,11 @@ void EvalState::printStats()
for (auto & i : functionCalls) {
auto obj = list.object();
if (i.first->name.set())
obj.attr("name", (const string &) i.first->name);
obj.attr("name", (const std::string &) i.first->name);
else
obj.attr("name", nullptr);
if (i.first->pos) {
obj.attr("file", (const string &) i.first->pos.file);
obj.attr("file", (const std::string &) i.first->pos.file);
obj.attr("line", i.first->pos.line);
obj.attr("column", i.first->pos.column);
}
@@ -2218,7 +2266,7 @@ void EvalState::printStats()
for (auto & i : attrSelects) {
auto obj = list.object();
if (i.first) {
obj.attr("file", (const string &) i.first.file);
obj.attr("file", (const std::string &) i.first.file);
obj.attr("line", i.first.line);
obj.attr("column", i.first.column);
}
@@ -2235,7 +2283,7 @@ void EvalState::printStats()
}
string ExternalValueBase::coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const
std::string ExternalValueBase::coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const
{
throw TypeError({
.msg = hintfmt("cannot coerce %1% to a string", showType()),
+35 -22
View File
@@ -1,6 +1,7 @@
#pragma once
#include "attr-set.hh"
#include "types.hh"
#include "value.hh"
#include "nixexpr.hh"
#include "symbol-table.hh"
@@ -149,7 +150,7 @@ public:
const Pos & pos
);
void addToSearchPath(const string & s);
void addToSearchPath(const std::string & s);
SearchPath getSearchPath() { return searchPath; }
@@ -160,6 +161,9 @@ public:
the real store path if `store` is a chroot store. */
void allowPath(const StorePath & storePath);
/* Allow access to a store path and return it as a string. */
void allowAndSetStorePathString(const StorePath & storePath, Value & v);
/* Check whether access to a path is allowed and throw an error if
not. Otherwise return the canonicalised path. */
Path checkSourcePath(const Path & path);
@@ -201,8 +205,8 @@ public:
void resetFileCache();
/* Look up a file in the search path. */
Path findFile(const string & path);
Path findFile(SearchPath & searchPath, const string & path, const Pos & pos = noPos);
Path findFile(const std::string_view path);
Path findFile(SearchPath & searchPath, const std::string_view path, const Pos & pos = noPos);
/* If the specified search path element is a URI, download it. */
std::pair<bool, std::string> resolveSearchPathElem(const SearchPathElem & elem);
@@ -221,7 +225,10 @@ public:
of the evaluation of the thunk. If `v' is a delayed function
application, call the function and overwrite `v' with the
result. Otherwise, this is a no-op. */
inline void forceValue(Value & v, const Pos & pos = noPos);
inline void forceValue(Value & v, const Pos & pos);
template <typename Callable>
inline void forceValue(Value & v, Callable getPos);
/* Force a value, then recursively force list elements and
attributes. */
@@ -231,37 +238,43 @@ public:
NixInt forceInt(Value & v, const Pos & pos);
NixFloat forceFloat(Value & v, const Pos & pos);
bool forceBool(Value & v, const Pos & pos);
inline void forceAttrs(Value & v);
inline void forceAttrs(Value & v, const Pos & pos);
inline void forceList(Value & v);
void forceAttrs(Value & v, const Pos & pos);
template <typename Callable>
inline void forceAttrs(Value & v, Callable getPos);
inline void forceList(Value & v, const Pos & pos);
void forceFunction(Value & v, const Pos & pos); // either lambda or primop
string forceString(Value & v, const Pos & pos = noPos);
string forceString(Value & v, PathSet & context, const Pos & pos = noPos);
string forceStringNoCtx(Value & v, const Pos & pos = noPos);
std::string_view forceString(Value & v, const Pos & pos = noPos);
std::string_view forceString(Value & v, PathSet & context, const Pos & pos = noPos);
std::string_view forceStringNoCtx(Value & v, const Pos & pos = noPos);
/* Return true iff the value `v' denotes a derivation (i.e. a
set with attribute `type = "derivation"'). */
bool isDerivation(Value & v);
std::optional<string> tryAttrsToString(const Pos & pos, Value & v,
std::optional<std::string> tryAttrsToString(const Pos & pos, Value & v,
PathSet & context, bool coerceMore = false, bool copyToStore = true);
/* String coercion. Converts strings, paths and derivations to a
string. If `coerceMore' is set, also converts nulls, integers,
booleans and lists to a string. If `copyToStore' is set,
referenced paths are copied to the Nix store as a side effect. */
string coerceToString(const Pos & pos, Value & v, PathSet & context,
BackedStringView coerceToString(const Pos & pos, Value & v, PathSet & context,
bool coerceMore = false, bool copyToStore = true,
bool canonicalizePath = true);
string copyPathToStore(PathSet & context, const Path & path);
std::string copyPathToStore(PathSet & context, const Path & path);
/* Path coercion. Converts strings, paths and derivations to a
path. The result is guaranteed to be a canonicalised, absolute
path. Nothing is copied to the store. */
Path coerceToPath(const Pos & pos, Value & v, PathSet & context);
/* Like coerceToPath, but the result must be a store path. */
StorePath coerceToStorePath(const Pos & pos, Value & v, PathSet & context);
public:
/* The base environment, containing the builtin functions and
@@ -277,18 +290,18 @@ private:
void createBaseEnv();
Value * addConstant(const string & name, Value & v);
Value * addConstant(const std::string & name, Value & v);
void addConstant(const string & name, Value * v);
void addConstant(const std::string & name, Value * v);
Value * addPrimOp(const string & name,
Value * addPrimOp(const std::string & name,
size_t arity, PrimOpFun primOp);
Value * addPrimOp(PrimOp && primOp);
public:
Value & getBuiltin(const string & name);
Value & getBuiltin(const std::string & name);
struct Doc
{
@@ -309,8 +322,8 @@ private:
friend struct ExprAttrs;
friend struct ExprLet;
Expr * parse(char * text, size_t length, FileOrigin origin, const Path & path,
const Path & basePath, StaticEnv & staticEnv);
Expr * parse(char * text, size_t length, FileOrigin origin, const PathView path,
const PathView basePath, StaticEnv & staticEnv);
public:
@@ -407,12 +420,12 @@ private:
/* Return a string representing the type of the value `v'. */
string showType(ValueType type);
string showType(const Value & v);
std::string_view showType(ValueType type);
std::string showType(const Value & v);
/* Decode a context string ‘!<name>!<path>’ into a pair <path,
name>. */
std::pair<string, string> decodeContext(std::string_view s);
std::pair<std::string, std::string> decodeContext(std::string_view s);
/* If `path' refers to a directory, then append "/default.nix". */
Path resolveExprPath(Path path);
+2 -1
View File
@@ -1,5 +1,6 @@
#include "flake.hh"
#include "globals.hh"
#include "fetch-settings.hh"
#include <nlohmann/json.hpp>
@@ -53,7 +54,7 @@ void ConfigFile::apply()
auto trustedList = readTrustedList();
bool trusted = false;
if (nix::settings.acceptFlakeConfig){
if (nix::fetchSettings.acceptFlakeConfig){
trusted = true;
} else if (auto saved = get(get(trustedList, name).value_or(std::map<std::string, bool>()), valueS)) {
trusted = *saved;
+48 -66
View File
@@ -6,6 +6,7 @@
#include "store-api.hh"
#include "fetchers.hh"
#include "finally.hh"
#include "fetch-settings.hh"
namespace nix {
@@ -89,11 +90,11 @@ static void expectType(EvalState & state, ValueType type,
static std::map<FlakeId, FlakeInput> parseFlakeInputs(
EvalState & state, Value * value, const Pos & pos,
const std::optional<Path> & baseDir);
const std::optional<Path> & baseDir, InputPath lockRootPath);
static FlakeInput parseFlakeInput(EvalState & state,
const std::string & inputName, Value * value, const Pos & pos,
const std::optional<Path> & baseDir)
const std::optional<Path> & baseDir, InputPath lockRootPath)
{
expectType(state, nAttrs, *value, pos);
@@ -117,10 +118,12 @@ static FlakeInput parseFlakeInput(EvalState & state,
expectType(state, nBool, *attr.value, *attr.pos);
input.isFlake = attr.value->boolean;
} else if (attr.name == sInputs) {
input.overrides = parseFlakeInputs(state, attr.value, *attr.pos, baseDir);
input.overrides = parseFlakeInputs(state, attr.value, *attr.pos, baseDir, lockRootPath);
} else if (attr.name == sFollows) {
expectType(state, nString, *attr.value, *attr.pos);
input.follows = parseInputPath(attr.value->string.s);
auto follows(parseInputPath(attr.value->string.s));
follows.insert(follows.begin(), lockRootPath.begin(), lockRootPath.end());
input.follows = follows;
} else {
switch (attr.value->type()) {
case nString:
@@ -166,7 +169,7 @@ static FlakeInput parseFlakeInput(EvalState & state,
static std::map<FlakeId, FlakeInput> parseFlakeInputs(
EvalState & state, Value * value, const Pos & pos,
const std::optional<Path> & baseDir)
const std::optional<Path> & baseDir, InputPath lockRootPath)
{
std::map<FlakeId, FlakeInput> inputs;
@@ -178,7 +181,8 @@ static std::map<FlakeId, FlakeInput> parseFlakeInputs(
inputAttr.name,
inputAttr.value,
*inputAttr.pos,
baseDir));
baseDir,
lockRootPath));
}
return inputs;
@@ -188,7 +192,8 @@ static Flake getFlake(
EvalState & state,
const FlakeRef & originalRef,
bool allowLookup,
FlakeCache & flakeCache)
FlakeCache & flakeCache,
InputPath lockRootPath)
{
auto [sourceInfo, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, originalRef, allowLookup, flakeCache);
@@ -223,7 +228,7 @@ static Flake getFlake(
auto sInputs = state.symbols.create("inputs");
if (auto inputs = vInfo.attrs->get(sInputs))
flake.inputs = parseFlakeInputs(state, inputs->value, *inputs->pos, flakeDir);
flake.inputs = parseFlakeInputs(state, inputs->value, *inputs->pos, flakeDir, lockRootPath);
auto sOutputs = state.symbols.create("outputs");
@@ -250,10 +255,12 @@ static Flake getFlake(
for (auto & setting : *nixConfig->value->attrs) {
forceTrivialValue(state, *setting.value, *setting.pos);
if (setting.value->type() == nString)
flake.config.settings.insert({setting.name, state.forceStringNoCtx(*setting.value, *setting.pos)});
flake.config.settings.insert({setting.name, std::string(state.forceStringNoCtx(*setting.value, *setting.pos))});
else if (setting.value->type() == nPath) {
PathSet emptyContext = {};
flake.config.settings.insert({setting.name, state.coerceToString(*setting.pos, *setting.value, emptyContext, false, true, true)});
flake.config.settings.emplace(
setting.name,
state.coerceToString(*setting.pos, *setting.value, emptyContext, false, true, true) .toOwned());
}
else if (setting.value->type() == nInt)
flake.config.settings.insert({setting.name, state.forceInt(*setting.value, *setting.pos)});
@@ -265,7 +272,7 @@ static Flake getFlake(
if (elem->type() != nString)
throw TypeError("list element in flake configuration setting '%s' is %s while a string is expected",
setting.name, showType(*setting.value));
ss.push_back(state.forceStringNoCtx(*elem, *setting.pos));
ss.emplace_back(state.forceStringNoCtx(*elem, *setting.pos));
}
flake.config.settings.insert({setting.name, ss});
}
@@ -287,6 +294,11 @@ static Flake getFlake(
return flake;
}
Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup, FlakeCache & flakeCache)
{
return getFlake(state, originalRef, allowLookup, flakeCache, {});
}
Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup)
{
FlakeCache flakeCache;
@@ -304,7 +316,7 @@ LockedFlake lockFlake(
FlakeCache flakeCache;
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto useRegistries = lockFlags.useRegistries.value_or(fetchSettings.useRegistries);
auto flake = getFlake(state, topRef, useRegistries, flakeCache);
@@ -332,22 +344,12 @@ LockedFlake lockFlake(
std::vector<FlakeRef> parents;
struct LockParent {
/* The path to this parent. */
InputPath path;
/* Whether we are currently inside a top-level lockfile
(inputs absolute) or subordinate lockfile (inputs
relative). */
bool absolute;
};
std::function<void(
const FlakeInputs & flakeInputs,
std::shared_ptr<Node> node,
const InputPath & inputPathPrefix,
std::shared_ptr<const Node> oldNode,
const LockParent & parent,
const InputPath & lockRootPath,
const Path & parentPath,
bool trustLock)>
computeLocks;
@@ -357,7 +359,7 @@ LockedFlake lockFlake(
std::shared_ptr<Node> node,
const InputPath & inputPathPrefix,
std::shared_ptr<const Node> oldNode,
const LockParent & parent,
const InputPath & lockRootPath,
const Path & parentPath,
bool trustLock)
{
@@ -402,17 +404,7 @@ LockedFlake lockFlake(
if (input.follows) {
InputPath target;
if (parent.absolute && !hasOverride) {
target = *input.follows;
} else {
if (hasOverride) {
target = inputPathPrefix;
target.pop_back();
} else
target = parent.path;
for (auto & i : *input.follows) target.push_back(i);
}
target.insert(target.end(), input.follows->begin(), input.follows->end());
debug("input '%s' follows '%s'", inputPathS, printInputPath(target));
node->inputs.insert_or_assign(id, target);
@@ -485,30 +477,32 @@ LockedFlake lockFlake(
break;
}
}
auto absoluteFollows(lockRootPath);
absoluteFollows.insert(absoluteFollows.end(), follows->begin(), follows->end());
fakeInputs.emplace(i.first, FlakeInput {
.follows = *follows,
.follows = absoluteFollows,
});
}
}
}
LockParent newParent {
.path = inputPath,
.absolute = true
};
auto localPath(parentPath);
// If this input is a path, recurse it down.
// This allows us to resolve path inputs relative to the current flake.
if ((*input.ref).input.getType() == "path")
localPath = absPath(*input.ref->input.getSourcePath(), parentPath);
computeLocks(
mustRefetch
? getFlake(state, oldLock->lockedRef, false, flakeCache).inputs
? getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath).inputs
: fakeInputs,
childNode, inputPath, oldLock, newParent, parentPath, !mustRefetch);
childNode, inputPath, oldLock, lockRootPath, parentPath, !mustRefetch);
} else {
/* We need to create a new lock file entry. So fetch
this input. */
debug("creating new input '%s'", inputPathS);
if (!lockFlags.allowMutable && !input.ref->input.isImmutable())
if (!lockFlags.allowMutable && !input.ref->input.isLocked())
throw Error("cannot update flake input '%s' in pure mode", inputPathS);
if (input.isFlake) {
@@ -520,7 +514,7 @@ LockedFlake lockFlake(
if (localRef.input.getType() == "path")
localPath = absPath(*input.ref->input.getSourcePath(), parentPath);
auto inputFlake = getFlake(state, localRef, useRegistries, flakeCache);
auto inputFlake = getFlake(state, localRef, useRegistries, flakeCache, inputPath);
/* Note: in case of an --override-input, we use
the *original* ref (input2.ref) for the
@@ -541,13 +535,6 @@ LockedFlake lockFlake(
parents.push_back(*input.ref);
Finally cleanup([&]() { parents.pop_back(); });
// Follows paths from existing inputs in the top-level lockfile are absolute,
// whereas paths in subordinate lockfiles are relative to those lockfiles.
LockParent newParent {
.path = inputPath,
.absolute = oldLock ? true : false
};
/* Recursively process the inputs of this
flake. Also, unless we already have this flake
in the top-level lock file, use this flake's
@@ -558,7 +545,7 @@ LockedFlake lockFlake(
? std::dynamic_pointer_cast<const Node>(oldLock)
: LockFile::read(
inputFlake.sourceInfo->actualPath + "/" + inputFlake.lockedRef.subdir + "/flake.lock").root,
newParent, localPath, false);
oldLock ? lockRootPath : inputPath, localPath, false);
}
else {
@@ -576,17 +563,12 @@ LockedFlake lockFlake(
}
};
LockParent parent {
.path = {},
.absolute = true
};
// Bring in the current ref for relative path resolution if we have it
auto parentPath = canonPath(flake.sourceInfo->actualPath + "/" + flake.lockedRef.subdir, true);
computeLocks(
flake.inputs, newLockFile.root, {},
lockFlags.recreateLockFile ? nullptr : oldLockFile.root, parent, parentPath, false);
lockFlags.recreateLockFile ? nullptr : oldLockFile.root, {}, parentPath, false);
for (auto & i : lockFlags.inputOverrides)
if (!overridesUsed.count(i.first))
@@ -610,7 +592,7 @@ LockedFlake lockFlake(
if (lockFlags.writeLockFile) {
if (auto sourcePath = topRef.input.getSourcePath()) {
if (!newLockFile.isImmutable()) {
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("will not write lock file of flake '%s' because it has a mutable input", topRef);
} else {
if (!lockFlags.updateLockFile)
@@ -637,7 +619,7 @@ LockedFlake lockFlake(
if (lockFlags.commitLockFile) {
std::string cm;
cm = settings.commitLockFileSummary.get();
cm = fetchSettings.commitLockFileSummary.get();
if (cm == "") {
cm = fmt("%s: %s", relPath, lockFileExists ? "Update" : "Add");
@@ -669,7 +651,7 @@ LockedFlake lockFlake(
now. Corner case: we could have reverted from a
dirty to a clean tree! */
if (flake.lockedRef.input == prevLockedRef.input
&& !flake.lockedRef.input.isImmutable())
&& !flake.lockedRef.input.isLocked())
throw Error("'%s' did not change after I updated its 'flake.lock' file; is 'flake.lock' under version control?", flake.originalRef);
}
} else
@@ -726,16 +708,16 @@ static void prim_getFlake(EvalState & state, const Pos & pos, Value * * args, Va
{
state.requireExperimentalFeatureOnEvaluation(Xp::Flakes, "builtins.getFlake", pos);
auto flakeRefS = state.forceStringNoCtx(*args[0], pos);
std::string flakeRefS(state.forceStringNoCtx(*args[0], pos));
auto flakeRef = parseFlakeRef(flakeRefS, {}, true);
if (evalSettings.pureEval && !flakeRef.input.isImmutable())
throw Error("cannot call 'getFlake' on mutable flake reference '%s', at %s (use --impure to override)", flakeRefS, pos);
if (evalSettings.pureEval && !flakeRef.input.isLocked())
throw Error("cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)", flakeRefS, pos);
callFlake(state,
lockFlake(state, flakeRef,
LockFlags {
.updateLockFile = false,
.useRegistries = !evalSettings.pureEval && settings.useRegistries,
.useRegistries = !evalSettings.pureEval && fetchSettings.useRegistries,
.allowMutable = !evalSettings.pureEval,
}),
v);
+3 -3
View File
@@ -98,7 +98,7 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
if (std::regex_match(url, match, flakeRegex)) {
auto parsedURL = ParsedURL{
.url = url,
.base = "flake:" + std::string(match[1]),
.base = "flake:" + match.str(1),
.scheme = "flake",
.authority = "",
.path = match[1],
@@ -106,12 +106,12 @@ std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
return std::make_pair(
FlakeRef(Input::fromURL(parsedURL), ""),
percentDecode(std::string(match[6])));
percentDecode(match.str(6)));
}
else if (std::regex_match(url, match, pathUrlRegex)) {
std::string path = match[1];
std::string fragment = percentDecode(std::string(match[3]));
std::string fragment = percentDecode(match.str(3));
if (baseDir) {
/* Check if 'url' is a path (either absolute or relative
+2 -2
View File
@@ -35,7 +35,7 @@ LockedNode::LockedNode(const nlohmann::json & json)
, originalRef(getFlakeRef(json, "original", nullptr))
, isFlake(json.find("flake") != json.end() ? (bool) json["flake"] : true)
{
if (!lockedRef.input.isImmutable())
if (!lockedRef.input.isLocked())
throw Error("lockfile contains mutable lock '%s'",
fetchers::attrsToJSON(lockedRef.input.toAttrs()));
}
@@ -220,7 +220,7 @@ bool LockFile::isImmutable() const
for (auto & i : nodes) {
if (i == root) continue;
auto lockedNode = std::dynamic_pointer_cast<const LockedNode>(i);
if (lockedNode && !lockedNode->lockedRef.input.isImmutable()) return false;
if (lockedNode && !lockedNode->lockedRef.input.isLocked()) return false;
}
return true;
+43 -34
View File
@@ -11,8 +11,8 @@
namespace nix {
DrvInfo::DrvInfo(EvalState & state, const string & attrPath, Bindings * attrs)
: state(&state), attrs(attrs), attrPath(attrPath)
DrvInfo::DrvInfo(EvalState & state, std::string attrPath, Bindings * attrs)
: state(&state), attrs(attrs), attrPath(std::move(attrPath))
{
}
@@ -22,7 +22,7 @@ DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPat
{
auto [drvPath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
this->drvPath = store->printStorePath(drvPath);
this->drvPath = drvPath;
auto drv = store->derivationFromPath(drvPath);
@@ -41,13 +41,11 @@ DrvInfo::DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPat
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvPath), outputName);
auto & [outputName, output] = *i;
auto optStorePath = output.path(*store, drv.name, outputName);
if (optStorePath)
outPath = store->printStorePath(*optStorePath);
outPath = {output.path(*store, drv.name, outputName)};
}
string DrvInfo::queryName() const
std::string DrvInfo::queryName() const
{
if (name == "" && attrs) {
auto i = attrs->find(state->sName);
@@ -58,7 +56,7 @@ string DrvInfo::queryName() const
}
string DrvInfo::querySystem() const
std::string DrvInfo::querySystem() const
{
if (system == "" && attrs) {
auto i = attrs->find(state->sSystem);
@@ -68,24 +66,35 @@ string DrvInfo::querySystem() const
}
string DrvInfo::queryDrvPath() const
std::optional<StorePath> DrvInfo::queryDrvPath() const
{
if (drvPath == "" && attrs) {
if (!drvPath && attrs) {
Bindings::iterator i = attrs->find(state->sDrvPath);
PathSet context;
drvPath = i != attrs->end() ? state->coerceToPath(*i->pos, *i->value, context) : "";
if (i == attrs->end())
drvPath = {std::nullopt};
else
drvPath = {state->coerceToStorePath(*i->pos, *i->value, context)};
}
return drvPath;
return drvPath.value_or(std::nullopt);
}
string DrvInfo::queryOutPath() const
StorePath DrvInfo::requireDrvPath() const
{
if (auto drvPath = queryDrvPath())
return *drvPath;
throw Error("derivation does not contain a 'drvPath' attribute");
}
StorePath DrvInfo::queryOutPath() const
{
if (!outPath && attrs) {
Bindings::iterator i = attrs->find(state->sOutPath);
PathSet context;
if (i != attrs->end())
outPath = state->coerceToPath(*i->pos, *i->value, context);
outPath = state->coerceToStorePath(*i->pos, *i->value, context);
}
if (!outPath)
throw UnimplementedError("CA derivations are not yet supported");
@@ -104,19 +113,19 @@ DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
/* For each output... */
for (auto elem : i->value->listItems()) {
/* Evaluate the corresponding set. */
string name = state->forceStringNoCtx(*elem, *i->pos);
std::string name(state->forceStringNoCtx(*elem, *i->pos));
Bindings::iterator out = attrs->find(state->symbols.create(name));
if (out == attrs->end()) continue; // FIXME: throw error?
state->forceAttrs(*out->value);
state->forceAttrs(*out->value, *i->pos);
/* And evaluate its ‘outPath’ attribute. */
Bindings::iterator outPath = out->value->attrs->find(state->sOutPath);
if (outPath == out->value->attrs->end()) continue; // FIXME: throw error?
PathSet context;
outputs[name] = state->coerceToPath(*outPath->pos, *outPath->value, context);
outputs.emplace(name, state->coerceToStorePath(*outPath->pos, *outPath->value, context));
}
} else
outputs["out"] = queryOutPath();
outputs.emplace("out", queryOutPath());
}
if (!onlyOutputsToInstall || !attrs)
return outputs;
@@ -138,7 +147,7 @@ DrvInfo::Outputs DrvInfo::queryOutputs(bool onlyOutputsToInstall)
}
string DrvInfo::queryOutputName() const
std::string DrvInfo::queryOutputName() const
{
if (outputName == "" && attrs) {
Bindings::iterator i = attrs->find(state->sOutputName);
@@ -172,7 +181,7 @@ StringSet DrvInfo::queryMetaNames()
bool DrvInfo::checkMeta(Value & v)
{
state->forceValue(v);
state->forceValue(v, [&]() { return v.determinePos(noPos); });
if (v.type() == nList) {
for (auto elem : v.listItems())
if (!checkMeta(*elem)) return false;
@@ -190,7 +199,7 @@ bool DrvInfo::checkMeta(Value & v)
}
Value * DrvInfo::queryMeta(const string & name)
Value * DrvInfo::queryMeta(const std::string & name)
{
if (!getMeta()) return 0;
Bindings::iterator a = meta->find(state->symbols.create(name));
@@ -199,7 +208,7 @@ Value * DrvInfo::queryMeta(const string & name)
}
string DrvInfo::queryMetaString(const string & name)
std::string DrvInfo::queryMetaString(const std::string & name)
{
Value * v = queryMeta(name);
if (!v || v->type() != nString) return "";
@@ -207,7 +216,7 @@ string DrvInfo::queryMetaString(const string & name)
}
NixInt DrvInfo::queryMetaInt(const string & name, NixInt def)
NixInt DrvInfo::queryMetaInt(const std::string & name, NixInt def)
{
Value * v = queryMeta(name);
if (!v) return def;
@@ -221,7 +230,7 @@ NixInt DrvInfo::queryMetaInt(const string & name, NixInt def)
return def;
}
NixFloat DrvInfo::queryMetaFloat(const string & name, NixFloat def)
NixFloat DrvInfo::queryMetaFloat(const std::string & name, NixFloat def)
{
Value * v = queryMeta(name);
if (!v) return def;
@@ -236,7 +245,7 @@ NixFloat DrvInfo::queryMetaFloat(const string & name, NixFloat def)
}
bool DrvInfo::queryMetaBool(const string & name, bool def)
bool DrvInfo::queryMetaBool(const std::string & name, bool def)
{
Value * v = queryMeta(name);
if (!v) return def;
@@ -251,7 +260,7 @@ bool DrvInfo::queryMetaBool(const string & name, bool def)
}
void DrvInfo::setMeta(const string & name, Value * v)
void DrvInfo::setMeta(const std::string & name, Value * v)
{
getMeta();
auto attrs = state->buildBindings(1 + (meta ? meta->size() : 0));
@@ -266,7 +275,7 @@ void DrvInfo::setMeta(const string & name, Value * v)
/* Cache for already considered attrsets. */
typedef set<Bindings *> Done;
typedef std::set<Bindings *> Done;
/* Evaluate value `v'. If it evaluates to a set of type `derivation',
@@ -274,11 +283,11 @@ typedef set<Bindings *> Done;
The result boolean indicates whether it makes sense
for the caller to recursively search for derivations in `v'. */
static bool getDerivation(EvalState & state, Value & v,
const string & attrPath, DrvInfos & drvs, Done & done,
const std::string & attrPath, DrvInfos & drvs, Done & done,
bool ignoreAssertionFailures)
{
try {
state.forceValue(v);
state.forceValue(v, [&]() { return v.determinePos(noPos); });
if (!state.isDerivation(v)) return true;
/* Remove spurious duplicates (e.g., a set like `rec { x =
@@ -311,7 +320,7 @@ std::optional<DrvInfo> getDerivation(EvalState & state, Value & v,
}
static string addToPath(const string & s1, const string & s2)
static std::string addToPath(const std::string & s1, const std::string & s2)
{
return s1.empty() ? s2 : s1 + "." + s2;
}
@@ -321,7 +330,7 @@ static std::regex attrRegex("[A-Za-z_][A-Za-z0-9-_+]*");
static void getDerivations(EvalState & state, Value & vIn,
const string & pathPrefix, Bindings & autoArgs,
const std::string & pathPrefix, Bindings & autoArgs,
DrvInfos & drvs, Done & done,
bool ignoreAssertionFailures)
{
@@ -346,7 +355,7 @@ static void getDerivations(EvalState & state, Value & vIn,
debug("evaluating attribute '%1%'", i->name);
if (!std::regex_match(std::string(i->name), attrRegex))
continue;
string pathPrefix2 = addToPath(pathPrefix, i->name);
std::string pathPrefix2 = addToPath(pathPrefix, i->name);
if (combineChannels)
getDerivations(state, *i->value, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
else if (getDerivation(state, *i->value, pathPrefix2, drvs, done, ignoreAssertionFailures)) {
@@ -364,7 +373,7 @@ static void getDerivations(EvalState & state, Value & vIn,
else if (v.type() == nList) {
for (auto [n, elem] : enumerate(v.listItems())) {
string pathPrefix2 = addToPath(pathPrefix, fmt("%d", n));
std::string pathPrefix2 = addToPath(pathPrefix, fmt("%d", n));
if (getDerivation(state, *elem, pathPrefix2, drvs, done, ignoreAssertionFailures))
getDerivations(state, *elem, pathPrefix2, autoArgs, drvs, done, ignoreAssertionFailures);
}
@@ -374,7 +383,7 @@ static void getDerivations(EvalState & state, Value & vIn,
}
void getDerivations(EvalState & state, Value & v, const string & pathPrefix,
void getDerivations(EvalState & state, Value & v, const std::string & pathPrefix,
Bindings & autoArgs, DrvInfos & drvs, bool ignoreAssertionFailures)
{
Done done;
+27 -25
View File
@@ -1,6 +1,7 @@
#pragma once
#include "eval.hh"
#include "path.hh"
#include <string>
#include <map>
@@ -12,16 +13,16 @@ namespace nix {
struct DrvInfo
{
public:
typedef std::map<string, Path> Outputs;
typedef std::map<std::string, StorePath> Outputs;
private:
EvalState * state;
mutable string name;
mutable string system;
mutable string drvPath;
mutable std::optional<string> outPath;
mutable string outputName;
mutable std::string name;
mutable std::string system;
mutable std::optional<std::optional<StorePath>> drvPath;
mutable std::optional<StorePath> outPath;
mutable std::string outputName;
Outputs outputs;
bool failed = false; // set if we get an AssertionError
@@ -33,36 +34,37 @@ private:
bool checkMeta(Value & v);
public:
string attrPath; /* path towards the derivation */
std::string attrPath; /* path towards the derivation */
DrvInfo(EvalState & state) : state(&state) { };
DrvInfo(EvalState & state, const string & attrPath, Bindings * attrs);
DrvInfo(EvalState & state, std::string attrPath, Bindings * attrs);
DrvInfo(EvalState & state, ref<Store> store, const std::string & drvPathWithOutputs);
string queryName() const;
string querySystem() const;
string queryDrvPath() const;
string queryOutPath() const;
string queryOutputName() const;
std::string queryName() const;
std::string querySystem() const;
std::optional<StorePath> queryDrvPath() const;
StorePath requireDrvPath() const;
StorePath queryOutPath() const;
std::string queryOutputName() const;
/** Return the list of outputs. The "outputs to install" are determined by `meta.outputsToInstall`. */
Outputs queryOutputs(bool onlyOutputsToInstall = false);
StringSet queryMetaNames();
Value * queryMeta(const string & name);
string queryMetaString(const string & name);
NixInt queryMetaInt(const string & name, NixInt def);
NixFloat queryMetaFloat(const string & name, NixFloat def);
bool queryMetaBool(const string & name, bool def);
void setMeta(const string & name, Value * v);
Value * queryMeta(const std::string & name);
std::string queryMetaString(const std::string & name);
NixInt queryMetaInt(const std::string & name, NixInt def);
NixFloat queryMetaFloat(const std::string & name, NixFloat def);
bool queryMetaBool(const std::string & name, bool def);
void setMeta(const std::string & name, Value * v);
/*
MetaInfo queryMetaInfo(EvalState & state) const;
MetaValue queryMetaInfo(EvalState & state, const string & name) const;
*/
void setName(const string & s) { name = s; }
void setDrvPath(const string & s) { drvPath = s; }
void setOutPath(const string & s) { outPath = s; }
void setName(const std::string & s) { name = s; }
void setDrvPath(StorePath path) { drvPath = {{std::move(path)}}; }
void setOutPath(StorePath path) { outPath = {{std::move(path)}}; }
void setFailed() { failed = true; };
bool hasFailed() { return failed; };
@@ -70,9 +72,9 @@ public:
#if HAVE_BOEHMGC
typedef list<DrvInfo, traceable_allocator<DrvInfo> > DrvInfos;
typedef std::list<DrvInfo, traceable_allocator<DrvInfo> > DrvInfos;
#else
typedef list<DrvInfo> DrvInfos;
typedef std::list<DrvInfo> DrvInfos;
#endif
@@ -81,7 +83,7 @@ typedef list<DrvInfo> DrvInfos;
std::optional<DrvInfo> getDerivation(EvalState & state,
Value & v, bool ignoreAssertionFailures);
void getDerivations(EvalState & state, Value & v, const string & pathPrefix,
void getDerivations(EvalState & state, Value & v, const std::string & pathPrefix,
Bindings & autoArgs, DrvInfos & drvs,
bool ignoreAssertionFailures);
+1 -1
View File
@@ -163,7 +163,7 @@ public:
}
};
void parseJSON(EvalState & state, const string & s_, Value & v)
void parseJSON(EvalState & state, const std::string_view & s_, Value & v)
{
JSONSax parser(state, v);
bool res = json::sax_parse(s_, &parser);
+1 -1
View File
@@ -8,6 +8,6 @@ namespace nix {
MakeError(JSONParseError, EvalError);
void parseJSON(EvalState & state, const string & s, Value & v);
void parseJSON(EvalState & state, const std::string_view & s, Value & v);
}
+9 -9
View File
@@ -66,7 +66,7 @@ static void adjustLoc(YYLTYPE * loc, const char * s, size_t len)
// we make use of the fact that the parser receives a private copy of the input
// string and can munge around in it.
static Expr * unescapeStr(SymbolTable & symbols, char * s, size_t length)
static StringToken unescapeStr(SymbolTable & symbols, char * s, size_t length)
{
char * result = s;
char * t = s;
@@ -89,7 +89,7 @@ static Expr * unescapeStr(SymbolTable & symbols, char * s, size_t length)
else *t = c;
t++;
}
return new ExprString(symbols.create({result, size_t(t - result)}));
return {result, size_t(t - result)};
}
@@ -176,7 +176,7 @@ or { return OR_KW; }
/* It is impossible to match strings ending with '$' with one
regex because trailing contexts are only valid at the end
of a rule. (A sane but undocumented limitation.) */
yylval->e = unescapeStr(data->symbols, yytext, yyleng);
yylval->str = unescapeStr(data->symbols, yytext, yyleng);
return STR;
}
<STRING>\$\{ { PUSH_STATE(DEFAULT); return DOLLAR_CURLY; }
@@ -191,26 +191,26 @@ or { return OR_KW; }
\'\'(\ *\n)? { PUSH_STATE(IND_STRING); return IND_STRING_OPEN; }
<IND_STRING>([^\$\']|\$[^\{\']|\'[^\'\$])+ {
yylval->e = new ExprIndStr(yytext);
yylval->str = {yytext, (size_t) yyleng, true};
return IND_STR;
}
<IND_STRING>\'\'\$ |
<IND_STRING>\$ {
yylval->e = new ExprIndStr("$");
yylval->str = {"$", 1};
return IND_STR;
}
<IND_STRING>\'\'\' {
yylval->e = new ExprIndStr("''");
yylval->str = {"''", 2};
return IND_STR;
}
<IND_STRING>\'\'\\{ANY} {
yylval->e = unescapeStr(data->symbols, yytext + 2, yyleng - 2);
yylval->str = unescapeStr(data->symbols, yytext + 2, yyleng - 2);
return IND_STR;
}
<IND_STRING>\$\{ { PUSH_STATE(DEFAULT); return DOLLAR_CURLY; }
<IND_STRING>\'\' { POP_STATE(); return IND_STRING_CLOSE; }
<IND_STRING>\' {
yylval->e = new ExprIndStr("'");
yylval->str = {"'", 1};
return IND_STR;
}
@@ -264,7 +264,7 @@ or { return OR_KW; }
PUSH_STATE(INPATH_SLASH);
else
PUSH_STATE(INPATH);
yylval->e = new ExprString(data->symbols.create(string(yytext)));
yylval->str = {yytext, (size_t) yyleng};
return STR;
}
<INPATH>{ANY} |
+18 -11
View File
@@ -16,10 +16,10 @@ std::ostream & operator << (std::ostream & str, const Expr & e)
return str;
}
static void showString(std::ostream & str, const string & s)
static void showString(std::ostream & str, std::string_view s)
{
str << '"';
for (auto c : (string) s)
for (auto c : s)
if (c == '"' || c == '\\' || c == '$') str << "\\" << c;
else if (c == '\n') str << "\\n";
else if (c == '\r') str << "\\r";
@@ -28,7 +28,7 @@ static void showString(std::ostream & str, const string & s)
str << '"';
}
static void showId(std::ostream & str, const string & s)
static void showId(std::ostream & str, std::string_view s)
{
if (s.empty())
str << "\"\"";
@@ -103,11 +103,18 @@ void ExprAttrs::show(std::ostream & str) const
{
if (recursive) str << "rec ";
str << "{ ";
for (auto & i : attrs)
if (i.second.inherited)
str << "inherit " << i.first << " " << "; ";
typedef const decltype(attrs)::value_type * Attr;
std::vector<Attr> sorted;
for (auto & i : attrs) sorted.push_back(&i);
std::sort(sorted.begin(), sorted.end(), [](Attr a, Attr b) {
return (const std::string &) a->first < (const std::string &) b->first;
});
for (auto & i : sorted) {
if (i->second.inherited)
str << "inherit " << i->first << " " << "; ";
else
str << i.first << " = " << *i.second.e << "; ";
str << i->first << " = " << *i->second.e << "; ";
}
for (auto & i : dynamicAttrs)
str << "\"${" << *i.nameExpr << "}\" = " << *i.valueExpr << "; ";
str << "}";
@@ -211,7 +218,7 @@ std::ostream & operator << (std::ostream & str, const Pos & pos)
auto f = format(ANSI_BOLD "%1%" ANSI_NORMAL ":%2%:%3%");
switch (pos.origin) {
case foFile:
f % (string) pos.file;
f % (const std::string &) pos.file;
break;
case foStdin:
case foString:
@@ -227,7 +234,7 @@ std::ostream & operator << (std::ostream & str, const Pos & pos)
}
string showAttrPath(const AttrPath & attrPath)
std::string showAttrPath(const AttrPath & attrPath)
{
std::ostringstream out;
bool first = true;
@@ -461,9 +468,9 @@ void ExprLambda::setName(Symbol & name)
}
string ExprLambda::showNamePos() const
std::string ExprLambda::showNamePos() const
{
return (format("%1% at %2%") % (name.set() ? "'" + (string) name + "'" : "anonymous function") % pos).str();
return fmt("%1% at %2%", name.set() ? "'" + (std::string) name + "'" : "anonymous function", pos);
}
+31 -25
View File
@@ -26,18 +26,21 @@ struct Pos
FileOrigin origin;
Symbol file;
unsigned int line, column;
Pos() : origin(foString), line(0), column(0) { };
Pos() : origin(foString), line(0), column(0) { }
Pos(FileOrigin origin, const Symbol & file, unsigned int line, unsigned int column)
: origin(origin), file(file), line(line), column(column) { };
: origin(origin), file(file), line(line), column(column) { }
operator bool() const
{
return line != 0;
}
bool operator < (const Pos & p2) const
{
if (!line) return p2.line;
if (!p2.line) return false;
int d = ((string) file).compare((string) p2.file);
int d = ((const std::string &) file).compare((const std::string &) p2.file);
if (d < 0) return true;
if (d > 0) return false;
if (line < p2.line) return true;
@@ -68,7 +71,7 @@ struct AttrName
typedef std::vector<AttrName> AttrPath;
string showAttrPath(const AttrPath & attrPath);
std::string showAttrPath(const AttrPath & attrPath);
/* Abstract syntax of Nix expressions. */
@@ -110,25 +113,18 @@ struct ExprFloat : Expr
struct ExprString : Expr
{
Symbol s;
std::string s;
Value v;
ExprString(const Symbol & s) : s(s) { v.mkString(s); };
ExprString(std::string s) : s(std::move(s)) { v.mkString(this->s.data()); };
COMMON_METHODS
Value * maybeThunk(EvalState & state, Env & env);
};
/* Temporary class used during parsing of indented strings. */
struct ExprIndStr : Expr
{
string s;
ExprIndStr(const string & s) : s(s) { };
};
struct ExprPath : Expr
{
string s;
std::string s;
Value v;
ExprPath(const string & s) : s(s) { v.mkPath(this->s.c_str()); };
ExprPath(std::string s) : s(std::move(s)) { v.mkPath(this->s.c_str()); };
COMMON_METHODS
Value * maybeThunk(EvalState & state, Env & env);
};
@@ -223,10 +219,25 @@ struct Formal
struct Formals
{
typedef std::list<Formal> Formals_;
typedef std::vector<Formal> Formals_;
Formals_ formals;
std::set<Symbol> argNames; // used during parsing
bool ellipsis;
bool has(Symbol arg) const {
auto it = std::lower_bound(formals.begin(), formals.end(), arg,
[] (const Formal & f, const Symbol & sym) { return f.name < sym; });
return it != formals.end() && it->name == arg;
}
std::vector<Formal> lexicographicOrder() const
{
std::vector<Formal> result(formals.begin(), formals.end());
std::sort(result.begin(), result.end(),
[] (const Formal & a, const Formal & b) {
return std::string_view(a.name) < std::string_view(b.name);
});
return result;
}
};
struct ExprLambda : Expr
@@ -239,14 +250,9 @@ struct ExprLambda : Expr
ExprLambda(const Pos & pos, const Symbol & arg, Formals * formals, Expr * body)
: pos(pos), arg(arg), formals(formals), body(body)
{
if (!arg.empty() && formals && formals->argNames.find(arg) != formals->argNames.end())
throw ParseError({
.msg = hintfmt("duplicate formal function argument '%1%'", arg),
.errPos = pos
});
};
void setName(Symbol & name);
string showNamePos() const;
std::string showNamePos() const;
inline bool hasFormals() const { return formals != nullptr; }
COMMON_METHODS
};
@@ -332,8 +338,8 @@ struct ExprConcatStrings : Expr
{
Pos pos;
bool forceString;
vector<std::pair<Pos, Expr *> > * es;
ExprConcatStrings(const Pos & pos, bool forceString, vector<std::pair<Pos, Expr *> > * es)
std::vector<std::pair<Pos, Expr *> > * es;
ExprConcatStrings(const Pos & pos, bool forceString, std::vector<std::pair<Pos, Expr *> > * es)
: pos(pos), forceString(forceString), es(es) { };
COMMON_METHODS
};
+127 -82
View File
@@ -16,6 +16,8 @@
#ifndef BISON_HEADER
#define BISON_HEADER
#include <variant>
#include "util.hh"
#include "nixexpr.hh"
@@ -39,8 +41,22 @@ namespace nix {
{ };
};
struct ParserFormals {
std::vector<Formal> formals;
bool ellipsis = false;
};
}
// using C a struct allows us to avoid having to define the special
// members that using string_view here would implicitly delete.
struct StringToken {
const char * p;
size_t l;
bool hasIndentation;
operator std::string_view() const { return {p, l}; }
};
#define YY_DECL int yylex \
(YYSTYPE * yylval_param, YYLTYPE * yylloc_param, yyscan_t yyscanner, nix::ParseData * data)
@@ -140,21 +156,46 @@ static void addAttr(ExprAttrs * attrs, AttrPath & attrPath,
}
static void addFormal(const Pos & pos, Formals * formals, const Formal & formal)
static Formals * toFormals(ParseData & data, ParserFormals * formals,
Pos pos = noPos, Symbol arg = {})
{
if (!formals->argNames.insert(formal.name).second)
std::sort(formals->formals.begin(), formals->formals.end(),
[] (const auto & a, const auto & b) {
return std::tie(a.name, a.pos) < std::tie(b.name, b.pos);
});
std::optional<std::pair<Symbol, Pos>> duplicate;
for (size_t i = 0; i + 1 < formals->formals.size(); i++) {
if (formals->formals[i].name != formals->formals[i + 1].name)
continue;
std::pair thisDup{formals->formals[i].name, formals->formals[i + 1].pos};
duplicate = std::min(thisDup, duplicate.value_or(thisDup));
}
if (duplicate)
throw ParseError({
.msg = hintfmt("duplicate formal function argument '%1%'",
formal.name),
.msg = hintfmt("duplicate formal function argument '%1%'", duplicate->first),
.errPos = duplicate->second
});
Formals result;
result.ellipsis = formals->ellipsis;
result.formals = std::move(formals->formals);
if (arg.set() && result.has(arg))
throw ParseError({
.msg = hintfmt("duplicate formal function argument '%1%'", arg),
.errPos = pos
});
formals->formals.push_front(formal);
delete formals;
return new Formals(std::move(result));
}
static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<std::pair<Pos, Expr *> > & es)
static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols,
std::vector<std::pair<Pos, std::variant<Expr *, StringToken> > > & es)
{
if (es.empty()) return new ExprString(symbols.create(""));
if (es.empty()) return new ExprString("");
/* Figure out the minimum indentation. Note that by design
whitespace-only final lines are not taken into account. (So
@@ -163,20 +204,20 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<st
size_t minIndent = 1000000;
size_t curIndent = 0;
for (auto & [i_pos, i] : es) {
ExprIndStr * e = dynamic_cast<ExprIndStr *>(i);
if (!e) {
/* Anti-quotations end the current start-of-line whitespace. */
auto * str = std::get_if<StringToken>(&i);
if (!str || !str->hasIndentation) {
/* Anti-quotations and escaped characters end the current start-of-line whitespace. */
if (atStartOfLine) {
atStartOfLine = false;
if (curIndent < minIndent) minIndent = curIndent;
}
continue;
}
for (size_t j = 0; j < e->s.size(); ++j) {
for (size_t j = 0; j < str->l; ++j) {
if (atStartOfLine) {
if (e->s[j] == ' ')
if (str->p[j] == ' ')
curIndent++;
else if (e->s[j] == '\n') {
else if (str->p[j] == '\n') {
/* Empty line, doesn't influence minimum
indentation. */
curIndent = 0;
@@ -184,7 +225,7 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<st
atStartOfLine = false;
if (curIndent < minIndent) minIndent = curIndent;
}
} else if (e->s[j] == '\n') {
} else if (str->p[j] == '\n') {
atStartOfLine = true;
curIndent = 0;
}
@@ -192,49 +233,50 @@ static Expr * stripIndentation(const Pos & pos, SymbolTable & symbols, vector<st
}
/* Strip spaces from each line. */
vector<std::pair<Pos, Expr *> > * es2 = new vector<std::pair<Pos, Expr *> >;
std::vector<std::pair<Pos, Expr *> > * es2 = new std::vector<std::pair<Pos, Expr *> >;
atStartOfLine = true;
size_t curDropped = 0;
size_t n = es.size();
for (vector<std::pair<Pos, Expr *> >::iterator i = es.begin(); i != es.end(); ++i, --n) {
ExprIndStr * e = dynamic_cast<ExprIndStr *>(i->second);
if (!e) {
atStartOfLine = false;
curDropped = 0;
es2->push_back(*i);
continue;
}
string s2;
for (size_t j = 0; j < e->s.size(); ++j) {
auto i = es.begin();
const auto trimExpr = [&] (Expr * e) {
atStartOfLine = false;
curDropped = 0;
es2->emplace_back(i->first, e);
};
const auto trimString = [&] (const StringToken & t) {
std::string s2;
for (size_t j = 0; j < t.l; ++j) {
if (atStartOfLine) {
if (e->s[j] == ' ') {
if (t.p[j] == ' ') {
if (curDropped++ >= minIndent)
s2 += e->s[j];
s2 += t.p[j];
}
else if (e->s[j] == '\n') {
else if (t.p[j] == '\n') {
curDropped = 0;
s2 += e->s[j];
s2 += t.p[j];
} else {
atStartOfLine = false;
curDropped = 0;
s2 += e->s[j];
s2 += t.p[j];
}
} else {
s2 += e->s[j];
if (e->s[j] == '\n') atStartOfLine = true;
s2 += t.p[j];
if (t.p[j] == '\n') atStartOfLine = true;
}
}
/* Remove the last line if it is empty and consists only of
spaces. */
if (n == 1) {
string::size_type p = s2.find_last_of('\n');
if (p != string::npos && s2.find_first_not_of(' ', p + 1) == string::npos)
s2 = string(s2, 0, p + 1);
std::string::size_type p = s2.find_last_of('\n');
if (p != std::string::npos && s2.find_first_not_of(' ', p + 1) == std::string::npos)
s2 = std::string(s2, 0, p + 1);
}
es2->emplace_back(i->first, new ExprString(symbols.create(s2)));
es2->emplace_back(i->first, new ExprString(s2));
};
for (; i != es.end(); ++i, --n) {
std::visit(overloaded { trimExpr, trimString }, i->second);
}
/* If this is a single string, then don't do a concatenation. */
@@ -269,22 +311,17 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParseData * data, const char * err
nix::Expr * e;
nix::ExprList * list;
nix::ExprAttrs * attrs;
nix::Formals * formals;
nix::ParserFormals * formals;
nix::Formal * formal;
nix::NixInt n;
nix::NixFloat nf;
// using C a struct allows us to avoid having to define the special
// members that using string_view here would implicitly delete.
struct StringToken {
const char * p;
size_t l;
operator std::string_view() const { return {p, l}; }
};
StringToken id; // !!! -> Symbol
StringToken path;
StringToken uri;
StringToken str;
std::vector<nix::AttrName> * attrNames;
std::vector<std::pair<nix::Pos, nix::Expr *> > * string_parts;
std::vector<std::pair<nix::Pos, std::variant<nix::Expr *, StringToken> > > * ind_string_parts;
}
%type <e> start expr expr_function expr_if expr_op
@@ -294,11 +331,12 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParseData * data, const char * err
%type <formals> formals
%type <formal> formal
%type <attrNames> attrs attrpath
%type <string_parts> string_parts_interpolated ind_string_parts
%type <string_parts> string_parts_interpolated
%type <ind_string_parts> ind_string_parts
%type <e> path_start string_parts string_attr
%type <id> attr
%token <id> ID ATTRPATH
%token <e> STR IND_STR
%token <str> STR IND_STR
%token <n> INT
%token <nf> FLOAT
%token <path> PATH HPATH SPATH PATH_END
@@ -331,11 +369,17 @@ expr_function
: ID ':' expr_function
{ $$ = new ExprLambda(CUR_POS, data->symbols.create($1), 0, $3); }
| '{' formals '}' ':' expr_function
{ $$ = new ExprLambda(CUR_POS, data->symbols.create(""), $2, $5); }
{ $$ = new ExprLambda(CUR_POS, data->symbols.create(""), toFormals(*data, $2), $5); }
| '{' formals '}' '@' ID ':' expr_function
{ $$ = new ExprLambda(CUR_POS, data->symbols.create($5), $2, $7); }
{
Symbol arg = data->symbols.create($5);
$$ = new ExprLambda(CUR_POS, arg, toFormals(*data, $2, CUR_POS, arg), $7);
}
| ID '@' '{' formals '}' ':' expr_function
{ $$ = new ExprLambda(CUR_POS, data->symbols.create($1), $4, $7); }
{
Symbol arg = data->symbols.create($1);
$$ = new ExprLambda(CUR_POS, arg, toFormals(*data, $4, CUR_POS, arg), $7);
}
| ASSERT expr ';' expr_function
{ $$ = new ExprAssert(CUR_POS, $2, $4); }
| WITH expr ';' expr_function
@@ -371,7 +415,7 @@ expr_op
| expr_op UPDATE expr_op { $$ = new ExprOpUpdate(CUR_POS, $1, $3); }
| expr_op '?' attrpath { $$ = new ExprOpHasAttr($1, *$3); }
| expr_op '+' expr_op
{ $$ = new ExprConcatStrings(CUR_POS, false, new vector<std::pair<Pos, Expr *> >({{makeCurPos(@1, data), $1}, {makeCurPos(@3, data), $3}})); }
{ $$ = new ExprConcatStrings(CUR_POS, false, new std::vector<std::pair<Pos, Expr *> >({{makeCurPos(@1, data), $1}, {makeCurPos(@3, data), $3}})); }
| expr_op '-' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__sub")), {$1, $3}); }
| expr_op '*' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__mul")), {$1, $3}); }
| expr_op '/' expr_op { $$ = new ExprCall(CUR_POS, new ExprVar(data->symbols.create("__div")), {$1, $3}); }
@@ -422,11 +466,11 @@ expr_simple
$$ = new ExprConcatStrings(CUR_POS, false, $2);
}
| SPATH {
string path($1.p + 1, $1.l - 2);
std::string path($1.p + 1, $1.l - 2);
$$ = new ExprCall(CUR_POS,
new ExprVar(data->symbols.create("__findFile")),
{new ExprVar(data->symbols.create("__nixPath")),
new ExprString(data->symbols.create(path))});
new ExprString(path)});
}
| URI {
static bool noURLLiterals = settings.isExperimentalFeatureEnabled(Xp::NoUrlLiterals);
@@ -435,7 +479,7 @@ expr_simple
.msg = hintfmt("URL literals are disabled"),
.errPos = CUR_POS
});
$$ = new ExprString(data->symbols.create($1));
$$ = new ExprString(std::string($1));
}
| '(' expr ')' { $$ = $2; }
/* Let expressions `let {..., body = ...}' are just desugared
@@ -450,18 +494,19 @@ expr_simple
;
string_parts
: STR
: STR { $$ = new ExprString(std::string($1)); }
| string_parts_interpolated { $$ = new ExprConcatStrings(CUR_POS, true, $1); }
| { $$ = new ExprString(data->symbols.create("")); }
| { $$ = new ExprString(""); }
;
string_parts_interpolated
: string_parts_interpolated STR { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $2); }
: string_parts_interpolated STR
{ $$ = $1; $1->emplace_back(makeCurPos(@2, data), new ExprString(std::string($2))); }
| string_parts_interpolated DOLLAR_CURLY expr '}' { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $3); }
| DOLLAR_CURLY expr '}' { $$ = new vector<std::pair<Pos, Expr *> >; $$->emplace_back(makeCurPos(@1, data), $2); }
| DOLLAR_CURLY expr '}' { $$ = new std::vector<std::pair<Pos, Expr *> >; $$->emplace_back(makeCurPos(@1, data), $2); }
| STR DOLLAR_CURLY expr '}' {
$$ = new vector<std::pair<Pos, Expr *> >;
$$->emplace_back(makeCurPos(@1, data), $1);
$$ = new std::vector<std::pair<Pos, Expr *> >;
$$->emplace_back(makeCurPos(@1, data), new ExprString(std::string($1)));
$$->emplace_back(makeCurPos(@2, data), $3);
}
;
@@ -475,7 +520,7 @@ path_start
$$ = new ExprPath(path);
}
| HPATH {
Path path(getHome() + string($1.p + 1, $1.l - 1));
Path path(getHome() + std::string($1.p + 1, $1.l - 1));
$$ = new ExprPath(path);
}
;
@@ -483,7 +528,7 @@ path_start
ind_string_parts
: ind_string_parts IND_STR { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $2); }
| ind_string_parts DOLLAR_CURLY expr '}' { $$ = $1; $1->emplace_back(makeCurPos(@2, data), $3); }
| { $$ = new vector<std::pair<Pos, Expr *> >; }
| { $$ = new std::vector<std::pair<Pos, std::variant<Expr *, StringToken> > >; }
;
binds
@@ -515,7 +560,7 @@ attrs
{ $$ = $1;
ExprString * str = dynamic_cast<ExprString *>($2);
if (str) {
$$->push_back(AttrName(str->s));
$$->push_back(AttrName(data->symbols.create(str->s)));
delete str;
} else
throw ParseError({
@@ -532,17 +577,17 @@ attrpath
{ $$ = $1;
ExprString * str = dynamic_cast<ExprString *>($3);
if (str) {
$$->push_back(AttrName(str->s));
$$->push_back(AttrName(data->symbols.create(str->s)));
delete str;
} else
$$->push_back(AttrName($3));
}
| attr { $$ = new vector<AttrName>; $$->push_back(AttrName(data->symbols.create($1))); }
| attr { $$ = new std::vector<AttrName>; $$->push_back(AttrName(data->symbols.create($1))); }
| string_attr
{ $$ = new vector<AttrName>;
{ $$ = new std::vector<AttrName>;
ExprString *str = dynamic_cast<ExprString *>($1);
if (str) {
$$->push_back(AttrName(str->s));
$$->push_back(AttrName(data->symbols.create(str->s)));
delete str;
} else
$$->push_back(AttrName($1));
@@ -566,13 +611,13 @@ expr_list
formals
: formal ',' formals
{ $$ = $3; addFormal(CUR_POS, $$, *$1); }
{ $$ = $3; $$->formals.push_back(*$1); }
| formal
{ $$ = new Formals; addFormal(CUR_POS, $$, *$1); $$->ellipsis = false; }
{ $$ = new ParserFormals; $$->formals.push_back(*$1); $$->ellipsis = false; }
|
{ $$ = new Formals; $$->ellipsis = false; }
{ $$ = new ParserFormals; $$->ellipsis = false; }
| ELLIPSIS
{ $$ = new Formals; $$->ellipsis = true; }
{ $$ = new ParserFormals; $$->ellipsis = true; }
;
formal
@@ -598,7 +643,7 @@ namespace nix {
Expr * EvalState::parse(char * text, size_t length, FileOrigin origin,
const Path & path, const Path & basePath, StaticEnv & staticEnv)
const PathView path, const PathView basePath, StaticEnv & staticEnv)
{
yyscan_t scanner;
ParseData data(*this);
@@ -693,40 +738,40 @@ Expr * EvalState::parseStdin()
}
void EvalState::addToSearchPath(const string & s)
void EvalState::addToSearchPath(const std::string & s)
{
size_t pos = s.find('=');
string prefix;
std::string prefix;
Path path;
if (pos == string::npos) {
if (pos == std::string::npos) {
path = s;
} else {
prefix = string(s, 0, pos);
path = string(s, pos + 1);
prefix = std::string(s, 0, pos);
path = std::string(s, pos + 1);
}
searchPath.emplace_back(prefix, path);
}
Path EvalState::findFile(const string & path)
Path EvalState::findFile(const std::string_view path)
{
return findFile(searchPath, path);
}
Path EvalState::findFile(SearchPath & searchPath, const string & path, const Pos & pos)
Path EvalState::findFile(SearchPath & searchPath, const std::string_view path, const Pos & pos)
{
for (auto & i : searchPath) {
std::string suffix;
if (i.first.empty())
suffix = "/" + path;
suffix = concatStrings("/", path);
else {
auto s = i.first.size();
if (path.compare(0, s, i.first) != 0 ||
(path.size() > s && path[s] != '/'))
continue;
suffix = path.size() == s ? "" : "/" + string(path, s);
suffix = path.size() == s ? "" : concatStrings("/", path.substr(s));
}
auto r = resolveSearchPathElem(i);
if (!r.first) continue;
@@ -735,7 +780,7 @@ Path EvalState::findFile(SearchPath & searchPath, const string & path, const Pos
}
if (hasPrefix(path, "nix/"))
return corepkgsPrefix + path.substr(4);
return concatStrings(corepkgsPrefix, path.substr(4));
throw ThrownError({
.msg = hintfmt(evalSettings.pureEval
+105 -93
View File
@@ -92,8 +92,6 @@ StringMap EvalState::realiseContext(const PathSet & context)
}
struct RealisePathFlags {
// Whether to check whether the path is a valid absolute path
bool requireAbsolutePath = true;
// Whether to check that the path is allowed in pure eval mode
bool checkForPureEval = true;
};
@@ -105,9 +103,7 @@ static Path realisePath(EvalState & state, const Pos & pos, Value & v, const Rea
auto path = [&]()
{
try {
return flags.requireAbsolutePath
? state.coerceToPath(pos, v, context)
: state.coerceToString(pos, v, context, false, false);
return state.coerceToPath(pos, v, context);
} catch (Error & e) {
e.addTrace(pos, "while realising the context of a path");
throw;
@@ -145,7 +141,7 @@ static void mkOutputString(
BindingsBuilder & attrs,
const StorePath & drvPath,
const BasicDerivation & drv,
const std::pair<string, DerivationOutput> & o)
const std::pair<std::string, DerivationOutput> & o)
{
auto optOutputPath = o.second.path(*state.store, drv.name, o.first);
attrs.alloc(o.first).mkString(
@@ -214,7 +210,7 @@ static void import(EvalState & state, const Pos & pos, Value & vPath, Value * vS
if (!vScope)
state.evalFile(path, v);
else {
state.forceAttrs(*vScope);
state.forceAttrs(*vScope, pos);
Env * env = &state.allocEnv(vScope->attrs->size());
env->up = &state.baseEnv;
@@ -318,7 +314,7 @@ void prim_importNative(EvalState & state, const Pos & pos, Value * * args, Value
{
auto path = realisePath(state, pos, *args[0]);
string sym = state.forceStringNoCtx(*args[1], pos);
std::string sym(state.forceStringNoCtx(*args[1], pos));
void *handle = dlopen(path.c_str(), RTLD_LAZY | RTLD_LOCAL);
if (!handle)
@@ -354,10 +350,11 @@ void prim_exec(EvalState & state, const Pos & pos, Value * * args, Value & v)
});
}
PathSet context;
auto program = state.coerceToString(pos, *elems[0], context, false, false);
auto program = state.coerceToString(pos, *elems[0], context, false, false).toOwned();
Strings commandArgs;
for (unsigned int i = 1; i < args[0]->listSize(); ++i)
commandArgs.emplace_back(state.coerceToString(pos, *elems[i], context, false, false));
for (unsigned int i = 1; i < args[0]->listSize(); ++i) {
commandArgs.push_back(state.coerceToString(pos, *elems[i], context, false, false).toOwned());
}
try {
auto _ = state.realiseContext(context); // FIXME: Handle CA derivations
} catch (InvalidPathError & e) {
@@ -389,7 +386,7 @@ void prim_exec(EvalState & state, const Pos & pos, Value * * args, Value & v)
static void prim_typeOf(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
state.forceValue(*args[0], pos);
string t;
std::string t;
switch (args[0]->type()) {
case nInt: t = "int"; break;
case nBool: t = "bool"; break;
@@ -577,9 +574,9 @@ struct CompareValues
#if HAVE_BOEHMGC
typedef list<Value *, gc_allocator<Value *> > ValueList;
typedef std::list<Value *, gc_allocator<Value *> > ValueList;
#else
typedef list<Value *> ValueList;
typedef std::list<Value *> ValueList;
#endif
@@ -657,7 +654,7 @@ static void prim_genericClosure(EvalState & state, const Pos & pos, Value * * ar
// `doneKeys' doesn't need to be a GC root, because its values are
// reachable from res.
auto cmp = CompareValues(state);
set<Value *, decltype(cmp)> doneKeys(cmp);
std::set<Value *, decltype(cmp)> doneKeys(cmp);
while (!workSet.empty()) {
Value * e = *(workSet.begin());
workSet.pop_front();
@@ -710,7 +707,7 @@ static RegisterPrimOp primop_abort({
.fun = [](EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string s = state.coerceToString(pos, *args[0], context);
auto s = state.coerceToString(pos, *args[0], context).toOwned();
throw Abort("evaluation aborted with the following error message: '%1%'", s);
}
});
@@ -728,7 +725,7 @@ static RegisterPrimOp primop_throw({
.fun = [](EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string s = state.coerceToString(pos, *args[0], context);
auto s = state.coerceToString(pos, *args[0], context).toOwned();
throw ThrownError(s);
}
});
@@ -740,7 +737,7 @@ static void prim_addErrorContext(EvalState & state, const Pos & pos, Value * * a
v = *args[1];
} catch (Error & e) {
PathSet context;
e.addTrace(std::nullopt, state.coerceToString(pos, *args[0], context));
e.addTrace(std::nullopt, state.coerceToString(pos, *args[0], context).toOwned());
throw;
}
}
@@ -829,7 +826,7 @@ static RegisterPrimOp primop_tryEval({
/* Return an environment variable. Use with care. */
static void prim_getEnv(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string name = state.forceStringNoCtx(*args[0], pos);
std::string name(state.forceStringNoCtx(*args[0], pos));
v.mkString(evalSettings.restrictEval || evalSettings.pureEval ? "" : getEnv(name).value_or(""));
}
@@ -938,7 +935,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
pos
);
string drvName;
std::string drvName;
Pos & posDrvName(*attr->pos);
try {
drvName = state.forceStringNoCtx(*attr->value, pos);
@@ -976,10 +973,10 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
for (auto & i : args[0]->attrs->lexicographicOrder()) {
if (i->name == state.sIgnoreNulls) continue;
const string & key = i->name;
const std::string & key = i->name;
vomit("processing attribute '%1%'", key);
auto handleHashMode = [&](const std::string & s) {
auto handleHashMode = [&](const std::string_view s) {
if (s == "recursive") ingestionMethod = FileIngestionMethod::Recursive;
else if (s == "flat") ingestionMethod = FileIngestionMethod::Flat;
else
@@ -1034,7 +1031,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
else if (i->name == state.sArgs) {
state.forceList(*i->value, pos);
for (auto elem : i->value->listItems()) {
string s = state.coerceToString(posDrvName, *elem, context, true);
auto s = state.coerceToString(posDrvName, *elem, context, true).toOwned();
drv.args.push_back(s);
}
}
@@ -1070,7 +1067,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
}
} else {
auto s = state.coerceToString(*i->pos, *i->value, context, true);
auto s = state.coerceToString(*i->pos, *i->value, context, true).toOwned();
drv.env.emplace(key, s);
if (i->name == state.sBuilder) drv.builder = std::move(s);
else if (i->name == state.sSystem) drv.platform = std::move(s);
@@ -1121,7 +1118,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
/* Handle derivation outputs of the form ‘!<name>!<path>’. */
else if (path.at(0) == '!') {
std::pair<string, string> ctx = decodeContext(path);
auto ctx = decodeContext(path);
drv.inputDrvs[state.store->parseStorePath(ctx.first)].insert(ctx.second);
}
@@ -1183,7 +1180,7 @@ static void prim_derivationStrict(EvalState & state, const Pos & pos, Value * *
drv.outputs.insert_or_assign(i, DerivationOutput {
.output = DerivationOutputCAFloating {
.method = ingestionMethod,
.hashType = std::move(ht),
.hashType = ht,
},
});
}
@@ -1273,7 +1270,7 @@ static RegisterPrimOp primop_derivationStrict(RegisterPrimOp::Info {
substituted by the corresponding output path at build time. For
example, 'placeholder "out"' returns the string
/1rz4g4znpzjwh1xymhjpm42vipw92pr73vdgl6xs1hycac8kf2n9. At build
time, any occurence of this string in an derivation attribute will
time, any occurrence of this string in an derivation attribute will
be replaced with the concrete path in the Nix store of the output
‘out’. */
static void prim_placeholder(EvalState & state, const Pos & pos, Value * * args, Value & v)
@@ -1403,7 +1400,7 @@ static RegisterPrimOp primop_pathExists({
static void prim_baseNameOf(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
v.mkString(baseNameOf(state.coerceToString(pos, *args[0], context, false, false)), context);
v.mkString(baseNameOf(*state.coerceToString(pos, *args[0], context, false, false)), context);
}
static RegisterPrimOp primop_baseNameOf({
@@ -1423,7 +1420,8 @@ static RegisterPrimOp primop_baseNameOf({
static void prim_dirOf(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
Path dir = dirOf(state.coerceToString(pos, *args[0], context, false, false));
auto path = state.coerceToString(pos, *args[0], context, false, false);
auto dir = dirOf(*path);
if (args[0]->type() == nPath) v.mkPath(dir); else v.mkString(dir, context);
}
@@ -1442,8 +1440,8 @@ static RegisterPrimOp primop_dirOf({
static void prim_readFile(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
auto path = realisePath(state, pos, *args[0]);
string s = readFile(path);
if (s.find((char) 0) != string::npos)
auto s = readFile(path);
if (s.find((char) 0) != std::string::npos)
throw Error("the contents of the file '%1%' cannot be represented as a Nix string", path);
StorePathSet refs;
if (state.store->isInStore(path)) {
@@ -1476,7 +1474,7 @@ static void prim_findFile(EvalState & state, const Pos & pos, Value * * args, Va
for (auto v2 : args[0]->listItems()) {
state.forceAttrs(*v2, pos);
string prefix;
std::string prefix;
Bindings::iterator i = v2->attrs->find(state.sPrefix);
if (i != v2->attrs->end())
prefix = state.forceStringNoCtx(*i->value, pos);
@@ -1489,12 +1487,24 @@ static void prim_findFile(EvalState & state, const Pos & pos, Value * * args, Va
pos
);
auto path = realisePath(state, pos, *i->value, { .requireAbsolutePath = false });
PathSet context;
auto path = state.coerceToString(pos, *i->value, context, false, false).toOwned();
try {
auto rewrites = state.realiseContext(context);
path = rewriteStrings(path, rewrites);
} catch (InvalidPathError & e) {
throw EvalError({
.msg = hintfmt("cannot find '%1%', since path '%2%' is not valid", path, e.path),
.errPos = pos
});
}
searchPath.emplace_back(prefix, path);
}
string path = state.forceStringNoCtx(*args[1], pos);
auto path = state.forceStringNoCtx(*args[1], pos);
v.mkPath(state.checkSourcePath(state.findFile(searchPath, path, pos)));
}
@@ -1508,7 +1518,7 @@ static RegisterPrimOp primop_findFile(RegisterPrimOp::Info {
/* Return the cryptographic hash of a file in base-16. */
static void prim_hashFile(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string type = state.forceStringNoCtx(*args[0], pos);
auto type = state.forceStringNoCtx(*args[0], pos);
std::optional<HashType> ht = parseHashType(type);
if (!ht)
throw Error({
@@ -1715,7 +1725,7 @@ static RegisterPrimOp primop_toJSON({
/* Parse a JSON string to a value. */
static void prim_fromJSON(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string s = state.forceStringNoCtx(*args[0], pos);
auto s = state.forceStringNoCtx(*args[0], pos);
try {
parseJSON(state, s, v);
} catch (JSONParseError &e) {
@@ -1744,8 +1754,8 @@ static RegisterPrimOp primop_fromJSON({
static void prim_toFile(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string name = state.forceStringNoCtx(*args[0], pos);
string contents = state.forceString(*args[1], context, pos);
std::string name(state.forceStringNoCtx(*args[0], pos));
std::string contents(state.forceString(*args[1], context, pos));
StorePathSet refs;
@@ -1853,7 +1863,7 @@ static RegisterPrimOp primop_toFile({
static void addPath(
EvalState & state,
const Pos & pos,
const string & name,
const std::string & name,
Path path,
Value * filterFun,
FileIngestionMethod method,
@@ -1909,20 +1919,15 @@ static void addPath(
if (expectedHash)
expectedStorePath = state.store->makeFixedOutputPath(method, *expectedHash, name);
Path dstPath;
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
dstPath = state.store->printStorePath(settings.readOnlyMode
StorePath dstPath = settings.readOnlyMode
? state.store->computeStorePathForPath(name, path, method, htSHA256, filter).first
: state.store->addToStore(name, path, method, htSHA256, filter, state.repair, refs));
if (expectedHash && expectedStorePath != state.store->parseStorePath(dstPath))
: state.store->addToStore(name, path, method, htSHA256, filter, state.repair, refs);
if (expectedHash && expectedStorePath != dstPath)
throw Error("store path mismatch in (possibly filtered) path added from '%s'", path);
state.allowAndSetStorePathString(dstPath, v);
} else
dstPath = state.store->printStorePath(*expectedStorePath);
v.mkString(dstPath, {dstPath});
state.allowPath(dstPath);
state.allowAndSetStorePathString(*expectedStorePath, v);
} catch (Error & e) {
e.addTrace(pos, "while adding path '%s'", path);
throw;
@@ -2006,14 +2011,14 @@ static void prim_path(EvalState & state, const Pos & pos, Value * * args, Value
{
state.forceAttrs(*args[0], pos);
Path path;
string name;
std::string name;
Value * filterFun = nullptr;
auto method = FileIngestionMethod::Recursive;
std::optional<Hash> expectedHash;
PathSet context;
for (auto & attr : *args[0]->attrs) {
const string & n(attr.name);
auto & n(attr.name);
if (n == "path")
path = state.coerceToPath(*attr.pos, *attr.value, context);
else if (attr.name == state.sName)
@@ -2145,7 +2150,7 @@ static RegisterPrimOp primop_attrValues({
/* Dynamic version of the `.' operator. */
void prim_getAttr(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string attr = state.forceStringNoCtx(*args[0], pos);
auto attr = state.forceStringNoCtx(*args[0], pos);
state.forceAttrs(*args[1], pos);
Bindings::iterator i = getAttr(
state,
@@ -2175,7 +2180,7 @@ static RegisterPrimOp primop_getAttr({
/* Return position information of the specified attribute. */
static void prim_unsafeGetAttrPos(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string attr = state.forceStringNoCtx(*args[0], pos);
auto attr = state.forceStringNoCtx(*args[0], pos);
state.forceAttrs(*args[1], pos);
Bindings::iterator i = args[1]->attrs->find(state.symbols.create(attr));
if (i == args[1]->attrs->end())
@@ -2193,7 +2198,7 @@ static RegisterPrimOp primop_unsafeGetAttrPos(RegisterPrimOp::Info {
/* Dynamic version of the `?' operator. */
static void prim_hasAttr(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string attr = state.forceStringNoCtx(*args[0], pos);
auto attr = state.forceStringNoCtx(*args[0], pos);
state.forceAttrs(*args[1], pos);
v.mkBool(args[1]->attrs->find(state.symbols.create(attr)) != args[1]->attrs->end());
}
@@ -2271,7 +2276,7 @@ static RegisterPrimOp primop_removeAttrs({
/* Builds a set from a list specifying (name, value) pairs. To be
precise, a list [{name = "name1"; value = value1;} ... {name =
"nameN"; value = valueN;}] is transformed to {name1 = value1;
... nameN = valueN;}. In case of duplicate occurences of the same
... nameN = valueN;}. In case of duplicate occurrences of the same
name, the first takes precedence. */
static void prim_listToAttrs(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
@@ -2292,7 +2297,7 @@ static void prim_listToAttrs(EvalState & state, const Pos & pos, Value * * args,
pos
);
string name = state.forceStringNoCtx(*j->value, *j->pos);
auto name = state.forceStringNoCtx(*j->value, *j->pos);
Symbol sym = state.symbols.create(name);
if (seen.insert(sym).second) {
@@ -2492,7 +2497,7 @@ static void prim_zipAttrsWith(EvalState & state, const Pos & pos, Value * * args
for (unsigned int n = 0; n < listSize; ++n) {
Value * vElem = listElems[n];
try {
state.forceAttrs(*vElem);
state.forceAttrs(*vElem, noPos);
for (auto & attr : *vElem->attrs)
attrsSeen[attr.name].first++;
} catch (TypeError & e) {
@@ -3024,7 +3029,7 @@ static void prim_groupBy(EvalState & state, const Pos & pos, Value * * args, Val
for (auto vElem : args[1]->listItems()) {
Value res;
state.callFunction(*args[0], *vElem, res, pos);
string name = state.forceStringNoCtx(res, pos);
auto name = state.forceStringNoCtx(res, pos);
Symbol sym = state.symbols.create(name);
auto vector = attrs.try_emplace(sym, ValueVector()).first;
vector->second.push_back(vElem);
@@ -3280,8 +3285,8 @@ static RegisterPrimOp primop_lessThan({
static void prim_toString(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string s = state.coerceToString(pos, *args[0], context, true, false);
v.mkString(s, context);
auto s = state.coerceToString(pos, *args[0], context, true, false);
v.mkString(*s, context);
}
static RegisterPrimOp primop_toString({
@@ -3317,7 +3322,7 @@ static void prim_substring(EvalState & state, const Pos & pos, Value * * args, V
int start = state.forceInt(*args[0], pos);
int len = state.forceInt(*args[1], pos);
PathSet context;
string s = state.coerceToString(pos, *args[2], context);
auto s = state.coerceToString(pos, *args[2], context);
if (start < 0)
throw EvalError({
@@ -3325,7 +3330,7 @@ static void prim_substring(EvalState & state, const Pos & pos, Value * * args, V
.errPos = pos
});
v.mkString((unsigned int) start >= s.size() ? "" : string(s, start, len), context);
v.mkString((unsigned int) start >= s->size() ? "" : s->substr(start, len), context);
}
static RegisterPrimOp primop_substring({
@@ -3351,8 +3356,8 @@ static RegisterPrimOp primop_substring({
static void prim_stringLength(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string s = state.coerceToString(pos, *args[0], context);
v.mkInt(s.size());
auto s = state.coerceToString(pos, *args[0], context);
v.mkInt(s->size());
}
static RegisterPrimOp primop_stringLength({
@@ -3368,7 +3373,7 @@ static RegisterPrimOp primop_stringLength({
/* Return the cryptographic hash of a string in base-16. */
static void prim_hashString(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string type = state.forceStringNoCtx(*args[0], pos);
auto type = state.forceStringNoCtx(*args[0], pos);
std::optional<HashType> ht = parseHashType(type);
if (!ht)
throw Error({
@@ -3377,7 +3382,7 @@ static void prim_hashString(EvalState & state, const Pos & pos, Value * * args,
});
PathSet context; // discarded
string s = state.forceString(*args[1], context, pos);
auto s = state.forceString(*args[1], context, pos);
v.mkString(hashString(*ht, s).to_string(Base16, false));
}
@@ -3395,7 +3400,18 @@ static RegisterPrimOp primop_hashString({
struct RegexCache
{
std::unordered_map<std::string, std::regex> cache;
// TODO use C++20 transparent comparison when available
std::unordered_map<std::string_view, std::regex> cache;
std::list<std::string> keys;
std::regex get(std::string_view re)
{
auto it = cache.find(re);
if (it != cache.end())
return it->second;
keys.emplace_back(re);
return cache.emplace(keys.back(), std::regex(keys.back(), std::regex::extended)).first->second;
}
};
std::shared_ptr<RegexCache> makeRegexCache()
@@ -3409,15 +3425,13 @@ void prim_match(EvalState & state, const Pos & pos, Value * * args, Value & v)
try {
auto regex = state.regexCache->cache.find(re);
if (regex == state.regexCache->cache.end())
regex = state.regexCache->cache.emplace(re, std::regex(re, std::regex::extended)).first;
auto regex = state.regexCache->get(re);
PathSet context;
const std::string str = state.forceString(*args[1], context, pos);
const auto str = state.forceString(*args[1], context, pos);
std::smatch match;
if (!std::regex_match(str, match, regex->second)) {
std::cmatch match;
if (!std::regex_match(str.begin(), str.end(), match, regex)) {
v.mkNull();
return;
}
@@ -3492,15 +3506,13 @@ void prim_split(EvalState & state, const Pos & pos, Value * * args, Value & v)
try {
auto regex = state.regexCache->cache.find(re);
if (regex == state.regexCache->cache.end())
regex = state.regexCache->cache.emplace(re, std::regex(re, std::regex::extended)).first;
auto regex = state.regexCache->get(re);
PathSet context;
const std::string str = state.forceString(*args[1], context, pos);
const auto str = state.forceString(*args[1], context, pos);
auto begin = std::sregex_iterator(str.begin(), str.end(), regex->second);
auto end = std::sregex_iterator();
auto begin = std::cregex_iterator(str.begin(), str.end(), regex);
auto end = std::cregex_iterator();
// Any matches results are surrounded by non-matching results.
const size_t len = std::distance(begin, end);
@@ -3512,9 +3524,9 @@ void prim_split(EvalState & state, const Pos & pos, Value * * args, Value & v)
return;
}
for (std::sregex_iterator i = begin; i != end; ++i) {
for (auto i = begin; i != end; ++i) {
assert(idx <= 2 * len + 1 - 3);
std::smatch match = *i;
auto match = *i;
// Add a string for non-matched characters.
(v.listElems()[idx++] = state.allocValue())->mkString(match.prefix().str());
@@ -3599,13 +3611,13 @@ static void prim_concatStringsSep(EvalState & state, const Pos & pos, Value * *
auto sep = state.forceString(*args[0], context, pos);
state.forceList(*args[1], pos);
string res;
std::string res;
res.reserve((args[1]->listSize() + 32) * sep.size());
bool first = true;
for (auto elem : args[1]->listItems()) {
if (first) first = false; else res += sep;
res += state.coerceToString(pos, *elem, context);
res += *state.coerceToString(pos, *elem, context);
}
v.mkString(res, context);
@@ -3632,23 +3644,23 @@ static void prim_replaceStrings(EvalState & state, const Pos & pos, Value * * ar
.errPos = pos
});
vector<string> from;
std::vector<std::string> from;
from.reserve(args[0]->listSize());
for (auto elem : args[0]->listItems())
from.push_back(state.forceString(*elem, pos));
from.emplace_back(state.forceString(*elem, pos));
vector<std::pair<string, PathSet>> to;
std::vector<std::pair<std::string, PathSet>> to;
to.reserve(args[1]->listSize());
for (auto elem : args[1]->listItems()) {
PathSet ctx;
auto s = state.forceString(*elem, ctx, pos);
to.push_back(std::make_pair(std::move(s), std::move(ctx)));
to.emplace_back(s, std::move(ctx));
}
PathSet context;
auto s = state.forceString(*args[2], context, pos);
string res;
std::string res;
// Loops one past last character to handle the case where 'from' contains an empty string.
for (size_t p = 0; p <= s.size(); ) {
bool found = false;
@@ -3704,7 +3716,7 @@ static RegisterPrimOp primop_replaceStrings({
static void prim_parseDrvName(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string name = state.forceStringNoCtx(*args[0], pos);
auto name = state.forceStringNoCtx(*args[0], pos);
DrvName parsed(name);
auto attrs = state.buildBindings(2);
attrs.alloc(state.sName).mkString(parsed.name);
@@ -3728,8 +3740,8 @@ static RegisterPrimOp primop_parseDrvName({
static void prim_compareVersions(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string version1 = state.forceStringNoCtx(*args[0], pos);
string version2 = state.forceStringNoCtx(*args[1], pos);
auto version1 = state.forceStringNoCtx(*args[0], pos);
auto version2 = state.forceStringNoCtx(*args[1], pos);
v.mkInt(compareVersions(version1, version2));
}
@@ -3748,14 +3760,14 @@ static RegisterPrimOp primop_compareVersions({
static void prim_splitVersion(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
string version = state.forceStringNoCtx(*args[0], pos);
auto version = state.forceStringNoCtx(*args[0], pos);
auto iter = version.cbegin();
Strings components;
while (iter != version.cend()) {
auto component = nextComponent(iter, version.cend());
if (component.empty())
break;
components.emplace_back(std::move(component));
components.emplace_back(component);
}
state.mkList(v, components.size());
for (const auto & [n, component] : enumerate(components))
+10 -9
View File
@@ -7,7 +7,8 @@ namespace nix {
static void prim_unsafeDiscardStringContext(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
v.mkString(state.coerceToString(pos, *args[0], context));
auto s = state.coerceToString(pos, *args[0], context);
v.mkString(*s);
}
static RegisterPrimOp primop_unsafeDiscardStringContext("__unsafeDiscardStringContext", 1, prim_unsafeDiscardStringContext);
@@ -32,13 +33,13 @@ static RegisterPrimOp primop_hasContext("__hasContext", 1, prim_hasContext);
static void prim_unsafeDiscardOutputDependency(EvalState & state, const Pos & pos, Value * * args, Value & v)
{
PathSet context;
string s = state.coerceToString(pos, *args[0], context);
auto s = state.coerceToString(pos, *args[0], context);
PathSet context2;
for (auto & p : context)
context2.insert(p.at(0) == '=' ? string(p, 1) : p);
context2.insert(p.at(0) == '=' ? std::string(p, 1) : p);
v.mkString(s, context2);
v.mkString(*s, context2);
}
static RegisterPrimOp primop_unsafeDiscardOutputDependency("__unsafeDiscardOutputDependency", 1, prim_unsafeDiscardOutputDependency);
@@ -75,13 +76,13 @@ static void prim_getContext(EvalState & state, const Pos & pos, Value * * args,
auto contextInfos = std::map<Path, ContextInfo>();
for (const auto & p : context) {
Path drv;
string output;
std::string output;
const Path * path = &p;
if (p.at(0) == '=') {
drv = string(p, 1);
drv = std::string(p, 1);
path = &drv;
} else if (p.at(0) == '!') {
std::pair<string, string> ctx = decodeContext(p);
std::pair<std::string, std::string> ctx = decodeContext(p);
drv = ctx.first;
output = ctx.second;
path = &drv;
@@ -165,7 +166,7 @@ static void prim_appendContext(EvalState & state, const Pos & pos, Value * * arg
.errPos = *i.pos
});
}
context.insert("=" + string(i.name));
context.insert("=" + std::string(i.name));
}
}
@@ -180,7 +181,7 @@ static void prim_appendContext(EvalState & state, const Pos & pos, Value * * arg
}
for (auto elem : iter->value->listItems()) {
auto name = state.forceStringNoCtx(*elem, *iter->pos);
context.insert("!" + name + "!" + string(i.name));
context.insert(concatStrings("!", name, "!", i.name));
}
}
}
+6 -6
View File
@@ -12,7 +12,7 @@ static void prim_fetchMercurial(EvalState & state, const Pos & pos, Value * * ar
std::string url;
std::optional<Hash> rev;
std::optional<std::string> ref;
std::string name = "source";
std::string_view name = "source";
PathSet context;
state.forceValue(*args[0], pos);
@@ -22,14 +22,14 @@ static void prim_fetchMercurial(EvalState & state, const Pos & pos, Value * * ar
state.forceAttrs(*args[0], pos);
for (auto & attr : *args[0]->attrs) {
string n(attr.name);
std::string_view n(attr.name);
if (n == "url")
url = state.coerceToString(*attr.pos, *attr.value, context, false, false);
url = state.coerceToString(*attr.pos, *attr.value, context, false, false).toOwned();
else if (n == "rev") {
// Ugly: unlike fetchGit, here the "rev" attribute can
// be both a revision or a branch/tag name.
auto value = state.forceStringNoCtx(*attr.value, *attr.pos);
if (std::regex_match(value, revRegex))
if (std::regex_match(value.begin(), value.end(), revRegex))
rev = Hash::parseAny(value, htSHA1);
else
ref = value;
@@ -50,7 +50,7 @@ static void prim_fetchMercurial(EvalState & state, const Pos & pos, Value * * ar
});
} else
url = state.coerceToString(pos, *args[0], context, false, false);
url = state.coerceToString(pos, *args[0], context, false, false).toOwned();
// FIXME: git externals probably can be used to bypass the URI
// whitelist. Ah well.
@@ -62,7 +62,7 @@ static void prim_fetchMercurial(EvalState & state, const Pos & pos, Value * * ar
fetchers::Attrs attrs;
attrs.insert_or_assign("type", "hg");
attrs.insert_or_assign("url", url.find("://") != std::string::npos ? url : "file://" + url);
attrs.insert_or_assign("name", name);
attrs.insert_or_assign("name", std::string(name));
if (ref) attrs.insert_or_assign("ref", *ref);
if (rev) attrs.insert_or_assign("rev", rev->gitRev());
auto input = fetchers::Input::fromAttrs(std::move(attrs));
+23 -11
View File
@@ -19,7 +19,7 @@ void emitTreeAttrs(
bool emptyRevFallback,
bool forceDirty)
{
assert(input.isImmutable());
assert(input.isLocked());
auto attrs = state.buildBindings(8);
@@ -125,7 +125,7 @@ static void fetchTree(
if (attr.name == state.sType) continue;
state.forceValue(*attr.value, *attr.pos);
if (attr.value->type() == nPath || attr.value->type() == nString) {
auto s = state.coerceToString(*attr.pos, *attr.value, context, false, false);
auto s = state.coerceToString(*attr.pos, *attr.value, context, false, false).toOwned();
attrs.emplace(attr.name,
attr.name == "url"
? type == "git"
@@ -151,7 +151,7 @@ static void fetchTree(
input = fetchers::Input::fromAttrs(std::move(attrs));
} else {
auto url = state.coerceToString(pos, *args[0], context, false, false);
auto url = state.coerceToString(pos, *args[0], context, false, false).toOwned();
if (type == "git") {
fetchers::Attrs attrs;
@@ -166,8 +166,8 @@ static void fetchTree(
if (!evalSettings.pureEval && !input.isDirect())
input = lookupInRegistries(state.store, input).first;
if (evalSettings.pureEval && !input.isImmutable())
throw Error("in pure evaluation mode, 'fetchTree' requires an immutable input, at %s", pos);
if (evalSettings.pureEval && !input.isLocked())
throw Error("in pure evaluation mode, 'fetchTree' requires a locked input, at %s", pos);
auto [tree, input2] = input.fetch(state.store);
@@ -186,7 +186,7 @@ static void prim_fetchTree(EvalState & state, const Pos & pos, Value * * args, V
static RegisterPrimOp primop_fetchTree("fetchTree", 1, prim_fetchTree);
static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
const string & who, bool unpack, std::string name)
const std::string & who, bool unpack, std::string name)
{
std::optional<std::string> url;
std::optional<Hash> expectedHash;
@@ -198,7 +198,7 @@ static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
state.forceAttrs(*args[0], pos);
for (auto & attr : *args[0]->attrs) {
string n(attr.name);
std::string n(attr.name);
if (n == "url")
url = state.forceStringNoCtx(*attr.value, *attr.pos);
else if (n == "sha256")
@@ -230,6 +230,21 @@ static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
if (evalSettings.pureEval && !expectedHash)
throw Error("in pure evaluation mode, '%s' requires a 'sha256' argument", who);
// early exit if pinned and already in the store
if (expectedHash && expectedHash->type == htSHA256) {
auto expectedPath =
unpack
? state.store->makeFixedOutputPath(FileIngestionMethod::Recursive, *expectedHash, name, {})
: state.store->makeFixedOutputPath(FileIngestionMethod::Flat, *expectedHash, name, {});
if (state.store->isValidPath(expectedPath)) {
state.allowAndSetStorePathString(expectedPath, v);
return;
}
}
// TODO: fetching may fail, yet the path may be substitutable.
// https://github.com/NixOS/nix/issues/4313
auto storePath =
unpack
? fetchers::downloadTarball(state.store, *url, name, (bool) expectedHash).first.storePath
@@ -244,10 +259,7 @@ static void fetch(EvalState & state, const Pos & pos, Value * * args, Value & v,
*url, expectedHash->to_string(Base32, true), hash.to_string(Base32, true));
}
state.allowPath(storePath);
auto path = state.store->printStorePath(storePath);
v.mkString(path, PathSet({path}));
state.allowAndSetStorePathString(storePath, v);
}
static void prim_fetchurl(EvalState & state, const Pos & pos, Value * * args, Value & v)
+1 -1
View File
@@ -9,7 +9,7 @@ static void prim_fromTOML(EvalState & state, const Pos & pos, Value * * args, Va
{
auto toml = state.forceStringNoCtx(*args[0], pos);
std::istringstream tomlStream(toml);
std::istringstream tomlStream(std::string{toml});
std::function<void(Value &, toml::value)> visit;
+4 -4
View File
@@ -17,8 +17,8 @@ namespace nix {
class Symbol
{
private:
const string * s; // pointer into SymbolTable
Symbol(const string * s) : s(s) { };
const std::string * s; // pointer into SymbolTable
Symbol(const std::string * s) : s(s) { };
friend class SymbolTable;
public:
@@ -72,7 +72,7 @@ class SymbolTable
{
private:
std::unordered_map<std::string_view, Symbol> symbols;
std::list<string> store;
std::list<std::string> store;
public:
Symbol create(std::string_view s)
@@ -84,7 +84,7 @@ public:
auto it = symbols.find(s);
if (it != symbols.end()) return it->second;
const string & rawSym = store.emplace_back(s);
auto & rawSym = store.emplace_back(s);
return symbols.emplace(rawSym, Symbol(&rawSym)).first->second;
}
+2 -2
View File
@@ -9,7 +9,7 @@
namespace nix {
static XMLAttrs singletonAttrs(const string & name, const string & value)
static XMLAttrs singletonAttrs(const std::string & name, const std::string & value)
{
XMLAttrs attrs;
attrs[name] = value;
@@ -142,7 +142,7 @@ static void printValueAsXML(EvalState & state, bool strict, bool location,
if (!v.lambda.fun->arg.empty()) attrs["name"] = v.lambda.fun->arg;
if (v.lambda.fun->formals->ellipsis) attrs["ellipsis"] = "1";
XMLOpenElement _(doc, "attrspat", attrs);
for (auto & i : v.lambda.fun->formals->formals)
for (auto & i : v.lambda.fun->formals->lexicographicOrder())
doc.writeEmptyElement("attr", singletonAttrs("name", i.name));
} else
doc.writeEmptyElement("varpat", singletonAttrs("name", v.lambda.fun->arg));
+11 -8
View File
@@ -77,20 +77,20 @@ class ExternalValueBase
public:
/* Return a simple string describing the type */
virtual string showType() const = 0;
virtual std::string showType() const = 0;
/* Return a string to be used in builtins.typeOf */
virtual string typeOf() const = 0;
virtual std::string typeOf() const = 0;
/* Coerce the value to a string. Defaults to uncoercable, i.e. throws an
* error
* error.
*/
virtual string coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const;
virtual std::string coerceToString(const Pos & pos, PathSet & context, bool copyMore, bool copyToStore) const;
/* Compare to another value of the same type. Defaults to uncomparable,
* i.e. always false.
*/
virtual bool operator==(const ExternalValueBase & b) const;
virtual bool operator ==(const ExternalValueBase & b) const;
/* Print the value as JSON. Defaults to unconvertable, i.e. throws an error */
virtual void printValueAsJSON(EvalState & state, bool strict,
@@ -114,11 +114,14 @@ struct Value
private:
InternalType internalType;
friend std::string showType(const Value & v);
friend void printValue(std::ostream & str, std::set<const Value *> & active, const Value & v);
friend std::string showType(const Value & v);
void print(std::ostream & str, std::set<const void *> * seen) const;
public:
void print(std::ostream & str, bool showRepeated = false) const;
// Functions needed to distinguish the type
// These should be removed eventually, by putting the functionality that's
// needed by callers into methods of this type
@@ -361,7 +364,7 @@ public:
return internalType == tList1 ? 1 : internalType == tList2 ? 2 : bigList.size;
}
Pos determinePos(const Pos &pos) const;
Pos determinePos(const Pos & pos) const;
/* Check whether forcing this value requires a trivial amount of
computation. In particular, function applications are
+4 -4
View File
@@ -52,13 +52,13 @@ struct CacheImpl : Cache
const Attrs & inAttrs,
const Attrs & infoAttrs,
const StorePath & storePath,
bool immutable) override
bool locked) override
{
_state.lock()->add.use()
(attrsToJSON(inAttrs).dump())
(attrsToJSON(infoAttrs).dump())
(store->printStorePath(storePath))
(immutable)
(locked)
(time(0)).exec();
}
@@ -91,7 +91,7 @@ struct CacheImpl : Cache
auto infoJSON = stmt.getStr(0);
auto storePath = store->parseStorePath(stmt.getStr(1));
auto immutable = stmt.getInt(2) != 0;
auto locked = stmt.getInt(2) != 0;
auto timestamp = stmt.getInt(3);
store->addTempRoot(storePath);
@@ -105,7 +105,7 @@ struct CacheImpl : Cache
inAttrsJSON, infoJSON, store->printStorePath(storePath));
return Result {
.expired = !immutable && (settings.tarballTtl.get() == 0 || timestamp + settings.tarballTtl < time(0)),
.expired = !locked && (settings.tarballTtl.get() == 0 || timestamp + settings.tarballTtl < time(0)),
.infoAttrs = jsonToAttrs(nlohmann::json::parse(infoJSON)),
.storePath = std::move(storePath)
};
+1 -1
View File
@@ -13,7 +13,7 @@ struct Cache
const Attrs & inAttrs,
const Attrs & infoAttrs,
const StorePath & storePath,
bool immutable) = 0;
bool locked) = 0;
virtual std::optional<std::pair<Attrs, StorePath>> lookup(
ref<Store> store,
+13
View File
@@ -0,0 +1,13 @@
#include "fetch-settings.hh"
namespace nix {
FetchSettings::FetchSettings()
{
}
FetchSettings fetchSettings;
static GlobalConfig::Register rFetchSettings(&fetchSettings);
}
+93
View File
@@ -0,0 +1,93 @@
#pragma once
#include "types.hh"
#include "config.hh"
#include "util.hh"
#include <map>
#include <limits>
#include <sys/types.h>
namespace nix {
struct FetchSettings : public Config
{
FetchSettings();
Setting<StringMap> accessTokens{this, {}, "access-tokens",
R"(
Access tokens used to access protected GitHub, GitLab, or
other locations requiring token-based authentication.
Access tokens are specified as a string made up of
space-separated `host=token` values. The specific token
used is selected by matching the `host` portion against the
"host" specification of the input. The actual use of the
`token` value is determined by the type of resource being
accessed:
* Github: the token value is the OAUTH-TOKEN string obtained
as the Personal Access Token from the Github server (see
https://docs.github.com/en/developers/apps/building-oauth-apps/authorizing-oauth-apps).
* Gitlab: the token value is either the OAuth2 token or the
Personal Access Token (these are different types tokens
for gitlab, see
https://docs.gitlab.com/12.10/ee/api/README.html#authentication).
The `token` value should be `type:tokenstring` where
`type` is either `OAuth2` or `PAT` to indicate which type
of token is being specified.
Example `~/.config/nix/nix.conf`:
```
access-tokens = github.com=23ac...b289 gitlab.mycompany.com=PAT:A123Bp_Cd..EfG gitlab.com=OAuth2:1jklw3jk
```
Example `~/code/flake.nix`:
```nix
input.foo = {
type = "gitlab";
host = "gitlab.mycompany.com";
owner = "mycompany";
repo = "pro";
};
```
This example specifies three tokens, one each for accessing
github.com, gitlab.mycompany.com, and sourceforge.net.
The `input.foo` uses the "gitlab" fetcher, which might
requires specifying the token type along with the token
value.
)"};
Setting<bool> allowDirty{this, true, "allow-dirty",
"Whether to allow dirty Git/Mercurial trees."};
Setting<bool> warnDirty{this, true, "warn-dirty",
"Whether to warn about dirty Git/Mercurial trees."};
Setting<std::string> flakeRegistry{this, "https://github.com/NixOS/flake-registry/raw/master/flake-registry.json", "flake-registry",
"Path or URI of the global flake registry."};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references."};
Setting<bool> acceptFlakeConfig{this, false, "accept-flake-config",
"Whether to accept nix configuration from a flake without prompting."};
Setting<std::string> commitLockFileSummary{
this, "", "commit-lockfile-summary",
R"(
The commit summary to use when committing changed flake lock files. If
empty, the summary is generated based on the action performed.
)"};
};
// FIXME: don't use a global variable.
extern FetchSettings fetchSettings;
}
+10 -10
View File
@@ -24,11 +24,11 @@ static void fixupInput(Input & input)
input.getType();
input.getRef();
if (input.getRev())
input.immutable = true;
input.locked = true;
input.getRevCount();
input.getLastModified();
if (input.getNarHash())
input.immutable = true;
input.locked = true;
}
Input Input::fromURL(const ParsedURL & url)
@@ -124,15 +124,13 @@ std::pair<Tree, Input> Input::fetch(ref<Store> store) const
debug("using substituted/cached input '%s' in '%s'",
to_string(), store->printStorePath(storePath));
auto actualPath = store->toRealPath(storePath);
return {fetchers::Tree(std::move(actualPath), std::move(storePath)), *this};
return {Tree { .actualPath = store->toRealPath(storePath), .storePath = std::move(storePath) }, *this};
} catch (Error & e) {
debug("substitution of input '%s' failed: %s", to_string(), e.what());
}
}
auto [tree, input] = [&]() -> std::pair<Tree, Input> {
auto [storePath, input] = [&]() -> std::pair<StorePath, Input> {
try {
return scheme->fetch(store, *this);
} catch (Error & e) {
@@ -141,8 +139,10 @@ std::pair<Tree, Input> Input::fetch(ref<Store> store) const
}
}();
if (tree.actualPath == "")
tree.actualPath = store->toRealPath(tree.storePath);
Tree tree {
.actualPath = store->toRealPath(storePath),
.storePath = storePath,
};
auto narHash = store->queryPathInfo(tree.storePath)->narHash;
input.attrs.insert_or_assign("narHash", narHash.to_string(SRI, true));
@@ -165,7 +165,7 @@ std::pair<Tree, Input> Input::fetch(ref<Store> store) const
input.to_string(), *prevRevCount);
}
input.immutable = true;
input.locked = true;
assert(input.hasAllInfo());
@@ -209,7 +209,7 @@ StorePath Input::computeStorePath(Store & store) const
{
auto narHash = getNarHash();
if (!narHash)
throw Error("cannot compute store path for mutable input '%s'", to_string());
throw Error("cannot compute store path for unlocked input '%s'", to_string());
return store.makeFixedOutputPath(FileIngestionMethod::Recursive, *narHash, getName());
}
+8 -7
View File
@@ -16,7 +16,6 @@ struct Tree
{
Path actualPath;
StorePath storePath;
Tree(Path && actualPath, StorePath && storePath) : actualPath(actualPath), storePath(std::move(storePath)) {}
};
struct InputScheme;
@@ -35,7 +34,7 @@ struct Input
std::shared_ptr<InputScheme> scheme; // note: can be null
Attrs attrs;
bool immutable = false;
bool locked = false;
bool direct = true;
/* path of the parent of this input, used for relative path resolution */
@@ -60,9 +59,9 @@ public:
one that goes through a registry. */
bool isDirect() const { return direct; }
/* Check whether this is an "immutable" input, that is,
/* Check whether this is a "locked" input, that is,
one that contains a commit hash or content hash. */
bool isImmutable() const { return immutable; }
bool isLocked() const { return locked; }
bool hasAllInfo() const;
@@ -70,6 +69,8 @@ public:
bool contains(const Input & other) const;
/* Fetch the input into the Nix store, returning the location in
the Nix store and the locked input. */
std::pair<Tree, Input> fetch(ref<Store> store) const;
Input applyOverrides(
@@ -131,7 +132,7 @@ struct InputScheme
virtual void markChangedFile(const Input & input, std::string_view file, std::optional<std::string> commitMsg);
virtual std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) = 0;
virtual std::pair<StorePath, Input> fetch(ref<Store> store, const Input & input) = 0;
};
void registerInputScheme(std::shared_ptr<InputScheme> && fetcher);
@@ -147,14 +148,14 @@ DownloadFileResult downloadFile(
ref<Store> store,
const std::string & url,
const std::string & name,
bool immutable,
bool locked,
const Headers & headers = {});
std::pair<Tree, time_t> downloadTarball(
ref<Store> store,
const std::string & url,
const std::string & name,
bool immutable,
bool locked,
const Headers & headers = {});
}
+15 -19
View File
@@ -6,6 +6,8 @@
#include "url-parts.hh"
#include "pathlocks.hh"
#include "fetch-settings.hh"
#include <sys/time.h>
#include <sys/wait.h>
@@ -172,7 +174,7 @@ struct GitInputScheme : InputScheme
return {isLocal, isLocal ? url.path : url.base};
}
std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & _input) override
{
Input input(_input);
@@ -187,7 +189,7 @@ struct GitInputScheme : InputScheme
if (submodules) cacheType += "-submodules";
if (allRefs) cacheType += "-all-refs";
auto getImmutableAttrs = [&]()
auto getLockedAttrs = [&]()
{
return Attrs({
{"type", cacheType},
@@ -197,21 +199,18 @@ struct GitInputScheme : InputScheme
};
auto makeResult = [&](const Attrs & infoAttrs, StorePath && storePath)
-> std::pair<Tree, Input>
-> std::pair<StorePath, Input>
{
assert(input.getRev());
assert(!_input.getRev() || _input.getRev() == input.getRev());
if (!shallow)
input.attrs.insert_or_assign("revCount", getIntAttr(infoAttrs, "revCount"));
input.attrs.insert_or_assign("lastModified", getIntAttr(infoAttrs, "lastModified"));
return {
Tree(store->toRealPath(storePath), std::move(storePath)),
input
};
return {std::move(storePath), input};
};
if (input.getRev()) {
if (auto res = getCache()->lookup(store, getImmutableAttrs()))
if (auto res = getCache()->lookup(store, getLockedAttrs()))
return makeResult(res->first, std::move(res->second));
}
@@ -249,10 +248,10 @@ struct GitInputScheme : InputScheme
/* This is an unclean working tree. So copy all tracked files. */
if (!settings.allowDirty)
if (!fetchSettings.allowDirty)
throw Error("Git tree '%s' is dirty", actualUrl);
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("Git tree '%s' is dirty", actualUrl);
auto gitOpts = Strings({ "-C", actualUrl, "ls-files", "-z" });
@@ -285,16 +284,13 @@ struct GitInputScheme : InputScheme
"lastModified",
haveCommits ? std::stoull(runProgram("git", true, { "-C", actualUrl, "log", "-1", "--format=%ct", "--no-show-signature", "HEAD" })) : 0);
return {
Tree(store->toRealPath(storePath), std::move(storePath)),
input
};
return {std::move(storePath), input};
}
}
if (!input.getRef()) input.attrs.insert_or_assign("ref", isLocal ? readHead(actualUrl) : "master");
Attrs mutableAttrs({
Attrs unlockedAttrs({
{"type", cacheType},
{"name", name},
{"url", actualUrl},
@@ -313,7 +309,7 @@ struct GitInputScheme : InputScheme
} else {
if (auto res = getCache()->lookup(store, mutableAttrs)) {
if (auto res = getCache()->lookup(store, unlockedAttrs)) {
auto rev2 = Hash::parseAny(getStrAttr(res->first, "rev"), htSHA1);
if (!input.getRev() || input.getRev() == rev2) {
input.attrs.insert_or_assign("rev", rev2.gitRev());
@@ -410,7 +406,7 @@ struct GitInputScheme : InputScheme
/* Now that we know the ref, check again whether we have it in
the store. */
if (auto res = getCache()->lookup(store, getImmutableAttrs()))
if (auto res = getCache()->lookup(store, getLockedAttrs()))
return makeResult(res->first, std::move(res->second));
Path tmpDir = createTempDir();
@@ -482,14 +478,14 @@ struct GitInputScheme : InputScheme
if (!_input.getRev())
getCache()->add(
store,
mutableAttrs,
unlockedAttrs,
infoAttrs,
storePath,
false);
getCache()->add(
store,
getImmutableAttrs(),
getLockedAttrs(),
infoAttrs,
storePath,
true);
+100 -12
View File
@@ -1,13 +1,16 @@
#include "filetransfer.hh"
#include "cache.hh"
#include "fetchers.hh"
#include "globals.hh"
#include "store-api.hh"
#include "types.hh"
#include "url-parts.hh"
#include "fetchers.hh"
#include "fetch-settings.hh"
#include <optional>
#include <nlohmann/json.hpp>
#include <fstream>
namespace nix::fetchers {
@@ -17,7 +20,7 @@ struct DownloadUrl
Headers headers;
};
// A github or gitlab host
// A github, gitlab, or sourcehut host
const static std::string hostRegexS = "[a-zA-Z0-9.]*"; // FIXME: check
std::regex hostRegex(hostRegexS, std::regex::ECMAScript);
@@ -156,7 +159,7 @@ struct GitArchiveInputScheme : InputScheme
std::optional<std::string> getAccessToken(const std::string & host) const
{
auto tokens = settings.accessTokens.get();
auto tokens = fetchSettings.accessTokens.get();
if (auto token = get(tokens, host))
return *token;
return {};
@@ -180,7 +183,7 @@ struct GitArchiveInputScheme : InputScheme
virtual DownloadUrl getDownloadUrl(const Input & input) const = 0;
std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & _input) override
{
Input input(_input);
@@ -192,17 +195,14 @@ struct GitArchiveInputScheme : InputScheme
input.attrs.erase("ref");
input.attrs.insert_or_assign("rev", rev->gitRev());
Attrs immutableAttrs({
Attrs lockedAttrs({
{"type", "git-tarball"},
{"rev", rev->gitRev()},
});
if (auto res = getCache()->lookup(store, immutableAttrs)) {
if (auto res = getCache()->lookup(store, lockedAttrs)) {
input.attrs.insert_or_assign("lastModified", getIntAttr(res->first, "lastModified"));
return {
Tree(store->toRealPath(res->second), std::move(res->second)),
input
};
return {std::move(res->second), input};
}
auto url = getDownloadUrl(input);
@@ -213,7 +213,7 @@ struct GitArchiveInputScheme : InputScheme
getCache()->add(
store,
immutableAttrs,
lockedAttrs,
{
{"rev", rev->gitRev()},
{"lastModified", uint64_t(lastModified)}
@@ -221,7 +221,7 @@ struct GitArchiveInputScheme : InputScheme
tree.storePath,
true);
return {std::move(tree), input};
return {std::move(tree.storePath), input};
}
};
@@ -348,7 +348,95 @@ struct GitLabInputScheme : GitArchiveInputScheme
}
};
struct SourceHutInputScheme : GitArchiveInputScheme
{
std::string type() override { return "sourcehut"; }
std::optional<std::pair<std::string, std::string>> accessHeaderFromToken(const std::string & token) const override
{
// SourceHut supports both PAT and OAuth2. See
// https://man.sr.ht/meta.sr.ht/oauth.md
return std::pair<std::string, std::string>("Authorization", fmt("Bearer %s", token));
// Note: This currently serves no purpose, as this kind of authorization
// does not allow for downloading tarballs on sourcehut private repos.
// Once it is implemented, however, should work as expected.
}
Hash getRevFromRef(nix::ref<Store> store, const Input & input) const override
{
// TODO: In the future, when the sourcehut graphql API is implemented for mercurial
// and with anonymous access, this method should use it instead.
auto ref = *input.getRef();
auto host = maybeGetStrAttr(input.attrs, "host").value_or("git.sr.ht");
auto base_url = fmt("https://%s/%s/%s",
host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo"));
Headers headers = makeHeadersWithAuthTokens(host);
std::string ref_uri;
if (ref == "HEAD") {
auto file = store->toRealPath(
downloadFile(store, fmt("%s/HEAD", base_url), "source", false, headers).storePath);
std::ifstream is(file);
std::string line;
getline(is, line);
auto ref_index = line.find("ref: ");
if (ref_index == std::string::npos) {
throw BadURL("in '%d', couldn't resolve HEAD ref '%d'", input.to_string(), ref);
}
ref_uri = line.substr(ref_index+5, line.length()-1);
} else
ref_uri = fmt("refs/heads/%s", ref);
auto file = store->toRealPath(
downloadFile(store, fmt("%s/info/refs", base_url), "source", false, headers).storePath);
std::ifstream is(file);
std::string line;
std::string id;
while(getline(is, line)) {
auto index = line.find(ref_uri);
if (index != std::string::npos) {
id = line.substr(0, index-1);
break;
}
}
if(id.empty())
throw BadURL("in '%d', couldn't find ref '%d'", input.to_string(), ref);
auto rev = Hash::parseAny(id, htSHA1);
debug("HEAD revision for '%s' is %s", fmt("%s/%s", base_url, ref), rev.gitRev());
return rev;
}
DownloadUrl getDownloadUrl(const Input & input) const override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("git.sr.ht");
auto url = fmt("https://%s/%s/%s/archive/%s.tar.gz",
host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo"),
input.getRev()->to_string(Base16, false));
Headers headers = makeHeadersWithAuthTokens(host);
return DownloadUrl { url, headers };
}
void clone(const Input & input, const Path & destDir) override
{
auto host = maybeGetStrAttr(input.attrs, "host").value_or("git.sr.ht");
Input::fromURL(fmt("git+https://%s/%s/%s",
host, getStrAttr(input.attrs, "owner"), getStrAttr(input.attrs, "repo")))
.applyOverrides(input.getRef(), input.getRev())
.clone(destDir);
}
};
static auto rGitHubInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GitHubInputScheme>()); });
static auto rGitLabInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GitLabInputScheme>()); });
static auto rSourceHutInputScheme = OnStartup([] { registerInputScheme(std::make_unique<SourceHutInputScheme>()); });
}
+1 -1
View File
@@ -94,7 +94,7 @@ struct IndirectInputScheme : InputScheme
return input;
}
std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & input) override
{
throw Error("indirect input '%s' cannot be fetched directly", input.to_string());
}
+17 -21
View File
@@ -5,6 +5,8 @@
#include "store-api.hh"
#include "url-parts.hh"
#include "fetch-settings.hh"
#include <sys/time.h>
using namespace std::string_literals;
@@ -26,7 +28,7 @@ static RunOptions hgOptions(const Strings & args)
}
// runProgram wrapper that uses hgOptions instead of stock RunOptions.
static string runHg(const Strings & args, const std::optional<std::string> & input = {})
static std::string runHg(const Strings & args, const std::optional<std::string> & input = {})
{
RunOptions opts = hgOptions(args);
opts.input = input;
@@ -143,7 +145,7 @@ struct MercurialInputScheme : InputScheme
return {isLocal, isLocal ? url.path : url.base};
}
std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & _input) override
{
Input input(_input);
@@ -165,10 +167,10 @@ struct MercurialInputScheme : InputScheme
/* This is an unclean working tree. So copy all tracked
files. */
if (!settings.allowDirty)
if (!fetchSettings.allowDirty)
throw Error("Mercurial tree '%s' is unclean", actualUrl);
if (settings.warnDirty)
if (fetchSettings.warnDirty)
warn("Mercurial tree '%s' is unclean", actualUrl);
input.attrs.insert_or_assign("ref", chomp(runHg({ "branch", "-R", actualUrl })));
@@ -193,16 +195,13 @@ struct MercurialInputScheme : InputScheme
auto storePath = store->addToStore(input.getName(), actualUrl, FileIngestionMethod::Recursive, htSHA256, filter);
return {
Tree(store->toRealPath(storePath), std::move(storePath)),
input
};
return {std::move(storePath), input};
}
}
if (!input.getRef()) input.attrs.insert_or_assign("ref", "default");
auto getImmutableAttrs = [&]()
auto getLockedAttrs = [&]()
{
return Attrs({
{"type", "hg"},
@@ -212,32 +211,29 @@ struct MercurialInputScheme : InputScheme
};
auto makeResult = [&](const Attrs & infoAttrs, StorePath && storePath)
-> std::pair<Tree, Input>
-> std::pair<StorePath, Input>
{
assert(input.getRev());
assert(!_input.getRev() || _input.getRev() == input.getRev());
input.attrs.insert_or_assign("revCount", getIntAttr(infoAttrs, "revCount"));
return {
Tree(store->toRealPath(storePath), std::move(storePath)),
input
};
return {std::move(storePath), input};
};
if (input.getRev()) {
if (auto res = getCache()->lookup(store, getImmutableAttrs()))
if (auto res = getCache()->lookup(store, getLockedAttrs()))
return makeResult(res->first, std::move(res->second));
}
auto revOrRef = input.getRev() ? input.getRev()->gitRev() : *input.getRef();
Attrs mutableAttrs({
Attrs unlockedAttrs({
{"type", "hg"},
{"name", name},
{"url", actualUrl},
{"ref", *input.getRef()},
});
if (auto res = getCache()->lookup(store, mutableAttrs)) {
if (auto res = getCache()->lookup(store, unlockedAttrs)) {
auto rev2 = Hash::parseAny(getStrAttr(res->first, "rev"), htSHA1);
if (!input.getRev() || input.getRev() == rev2) {
input.attrs.insert_or_assign("rev", rev2.gitRev());
@@ -260,7 +256,7 @@ struct MercurialInputScheme : InputScheme
runHg({ "pull", "-R", cacheDir, "--", actualUrl });
}
catch (ExecError & e) {
string transJournal = cacheDir + "/.hg/store/journal";
auto transJournal = cacheDir + "/.hg/store/journal";
/* hg throws "abandoned transaction" error only if this file exists */
if (pathExists(transJournal)) {
runHg({ "recover", "-R", cacheDir });
@@ -283,7 +279,7 @@ struct MercurialInputScheme : InputScheme
auto revCount = std::stoull(tokens[1]);
input.attrs.insert_or_assign("ref", tokens[2]);
if (auto res = getCache()->lookup(store, getImmutableAttrs()))
if (auto res = getCache()->lookup(store, getLockedAttrs()))
return makeResult(res->first, std::move(res->second));
Path tmpDir = createTempDir();
@@ -303,14 +299,14 @@ struct MercurialInputScheme : InputScheme
if (!_input.getRev())
getCache()->add(
store,
mutableAttrs,
unlockedAttrs,
infoAttrs,
storePath,
false);
getCache()->add(
store,
getImmutableAttrs(),
getLockedAttrs(),
infoAttrs,
storePath,
true);
+2 -5
View File
@@ -80,7 +80,7 @@ struct PathInputScheme : InputScheme
// nothing to do
}
std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & input) override
{
std::string absPath;
auto path = getStrAttr(input.attrs, "path");
@@ -115,10 +115,7 @@ struct PathInputScheme : InputScheme
// FIXME: try to substitute storePath.
storePath = store->addToStore("source", absPath);
return {
Tree(store->toRealPath(*storePath), std::move(*storePath)),
input
};
return {std::move(*storePath), input};
}
};
+3 -1
View File
@@ -5,6 +5,8 @@
#include "store-api.hh"
#include "local-fs-store.hh"
#include "fetch-settings.hh"
#include <nlohmann/json.hpp>
namespace nix::fetchers {
@@ -150,7 +152,7 @@ void overrideRegistry(
static std::shared_ptr<Registry> getGlobalRegistry(ref<Store> store)
{
static auto reg = [&]() {
auto path = settings.flakeRegistry.get();
auto path = fetchSettings.flakeRegistry.get();
if (!hasPrefix(path, "/")) {
auto storePath = downloadFile(store, path, "flake-registry.json", false).storePath;
+11 -11
View File
@@ -13,7 +13,7 @@ DownloadFileResult downloadFile(
ref<Store> store,
const std::string & url,
const std::string & name,
bool immutable,
bool locked,
const Headers & headers)
{
// FIXME: check store
@@ -88,7 +88,7 @@ DownloadFileResult downloadFile(
inAttrs,
infoAttrs,
*storePath,
immutable);
locked);
if (url != res.effectiveUri)
getCache()->add(
@@ -100,7 +100,7 @@ DownloadFileResult downloadFile(
},
infoAttrs,
*storePath,
immutable);
locked);
return {
.storePath = std::move(*storePath),
@@ -113,7 +113,7 @@ std::pair<Tree, time_t> downloadTarball(
ref<Store> store,
const std::string & url,
const std::string & name,
bool immutable,
bool locked,
const Headers & headers)
{
Attrs inAttrs({
@@ -126,11 +126,11 @@ std::pair<Tree, time_t> downloadTarball(
if (cached && !cached->expired)
return {
Tree(store->toRealPath(cached->storePath), std::move(cached->storePath)),
Tree { .actualPath = store->toRealPath(cached->storePath), .storePath = std::move(cached->storePath) },
getIntAttr(cached->infoAttrs, "lastModified")
};
auto res = downloadFile(store, url, name, immutable, headers);
auto res = downloadFile(store, url, name, locked, headers);
std::optional<StorePath> unpackedStorePath;
time_t lastModified;
@@ -160,10 +160,10 @@ std::pair<Tree, time_t> downloadTarball(
inAttrs,
infoAttrs,
*unpackedStorePath,
immutable);
locked);
return {
Tree(store->toRealPath(*unpackedStorePath), std::move(*unpackedStorePath)),
Tree { .actualPath = store->toRealPath(*unpackedStorePath), .storePath = std::move(*unpackedStorePath) },
lastModified,
};
}
@@ -202,7 +202,7 @@ struct TarballInputScheme : InputScheme
Input input;
input.attrs = attrs;
//input.immutable = (bool) maybeGetStrAttr(input.attrs, "hash");
//input.locked = (bool) maybeGetStrAttr(input.attrs, "hash");
return input;
}
@@ -225,10 +225,10 @@ struct TarballInputScheme : InputScheme
return true;
}
std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) override
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & input) override
{
auto tree = downloadTarball(store, getStrAttr(input.attrs, "url"), input.getName(), false).first;
return {std::move(tree), input};
return {std::move(tree.storePath), input};
}
};
+1 -1
View File
@@ -4,7 +4,7 @@
namespace nix {
MixCommonArgs::MixCommonArgs(const string & programName)
MixCommonArgs::MixCommonArgs(const std::string & programName)
: programName(programName)
{
addFlag({
+2 -2
View File
@@ -11,8 +11,8 @@ class MixCommonArgs : public virtual Args
{
void initialFlagsProcessed() override;
public:
string programName;
MixCommonArgs(const string & programName);
std::string programName;
MixCommonArgs(const std::string & programName);
protected:
virtual void pluginsInited() {}
};
+8 -7
View File
@@ -1,6 +1,7 @@
#include "globals.hh"
#include "shared.hh"
#include "store-api.hh"
#include "gc-store.hh"
#include "util.hh"
#include "loggers.hh"
@@ -94,7 +95,7 @@ void printMissing(ref<Store> store, const StorePathSet & willBuild,
}
string getArg(const string & opt,
std::string getArg(const std::string & opt,
Strings::iterator & i, const Strings::iterator & end)
{
++i;
@@ -322,14 +323,14 @@ void parseCmdLine(int argc, char * * argv,
}
void parseCmdLine(const string & programName, const Strings & args,
void parseCmdLine(const std::string & programName, const Strings & args,
std::function<bool(Strings::iterator & arg, const Strings::iterator & end)> parseArg)
{
LegacyArgs(programName, parseArg).parseCmdline(args);
}
void printVersion(const string & programName)
void printVersion(const std::string & programName)
{
std::cout << format("%1% (Nix) %2%") % programName % nixVersion << std::endl;
if (verbosity > lvlInfo) {
@@ -352,7 +353,7 @@ void printVersion(const string & programName)
}
void showManPage(const string & name)
void showManPage(const std::string & name)
{
restoreProcessContext();
setenv("MANPATH", settings.nixManDir.c_str(), 1);
@@ -361,13 +362,13 @@ void showManPage(const string & name)
}
int handleExceptions(const string & programName, std::function<void()> fun)
int handleExceptions(const std::string & programName, std::function<void()> fun)
{
ReceiveInterrupts receiveInterrupts; // FIXME: need better place for this
ErrorInfo::programName = baseNameOf(programName);
string error = ANSI_RED "error:" ANSI_NORMAL " ";
std::string error = ANSI_RED "error:" ANSI_NORMAL " ";
try {
try {
fun();
@@ -407,7 +408,7 @@ RunPager::RunPager()
if (!isatty(STDOUT_FILENO)) return;
char * pager = getenv("NIX_PAGER");
if (!pager) pager = getenv("PAGER");
if (pager && ((string) pager == "" || (string) pager == "cat")) return;
if (pager && ((std::string) pager == "" || (std::string) pager == "cat")) return;
Pipe toPager;
toPager.create();
+7 -7
View File
@@ -22,7 +22,7 @@ public:
virtual ~Exit();
};
int handleExceptions(const string & programName, std::function<void()> fun);
int handleExceptions(const std::string & programName, std::function<void()> fun);
/* Don't forget to call initPlugins() after settings are initialized! */
void initNix();
@@ -30,10 +30,10 @@ void initNix();
void parseCmdLine(int argc, char * * argv,
std::function<bool(Strings::iterator & arg, const Strings::iterator & end)> parseArg);
void parseCmdLine(const string & programName, const Strings & args,
void parseCmdLine(const std::string & programName, const Strings & args,
std::function<bool(Strings::iterator & arg, const Strings::iterator & end)> parseArg);
void printVersion(const string & programName);
void printVersion(const std::string & programName);
/* Ugh. No better place to put this. */
void printGCWarning();
@@ -50,10 +50,10 @@ void printMissing(ref<Store> store, const StorePathSet & willBuild,
const StorePathSet & willSubstitute, const StorePathSet & unknown,
uint64_t downloadSize, uint64_t narSize, Verbosity lvl = lvlInfo);
string getArg(const string & opt,
std::string getArg(const std::string & opt,
Strings::iterator & i, const Strings::iterator & end);
template<class N> N getIntArg(const string & opt,
template<class N> N getIntArg(const std::string & opt,
Strings::iterator & i, const Strings::iterator & end, bool allowUnit)
{
++i;
@@ -76,7 +76,7 @@ struct LegacyArgs : public MixCommonArgs
/* Show the manual page for the specified program. */
void showManPage(const string & name);
void showManPage(const std::string & name);
/* The constructor of this class starts a pager if stdout is a
terminal and $PAGER is set. Stdout is redirected to the pager. */
@@ -96,7 +96,7 @@ extern volatile ::sig_atomic_t blockInt;
/* GC helpers. */
string showBytes(uint64_t bytes);
std::string showBytes(uint64_t bytes);
struct GCResults;
+14 -5
View File
@@ -307,7 +307,7 @@ void BinaryCacheStore::addToStore(const ValidPathInfo & info, Source & narSource
}});
}
StorePath BinaryCacheStore::addToStoreFromDump(Source & dump, const string & name,
StorePath BinaryCacheStore::addToStoreFromDump(Source & dump, std::string_view name,
FileIngestionMethod method, HashType hashAlgo, RepairFlag repair, const StorePathSet & references)
{
if (method != FileIngestionMethod::Recursive || hashAlgo != htSHA256)
@@ -385,8 +385,14 @@ void BinaryCacheStore::queryPathInfoUncached(const StorePath & storePath,
}});
}
StorePath BinaryCacheStore::addToStore(const string & name, const Path & srcPath,
FileIngestionMethod method, HashType hashAlgo, PathFilter & filter, RepairFlag repair, const StorePathSet & references)
StorePath BinaryCacheStore::addToStore(
std::string_view name,
const Path & srcPath,
FileIngestionMethod method,
HashType hashAlgo,
PathFilter & filter,
RepairFlag repair,
const StorePathSet & references)
{
/* FIXME: Make BinaryCacheStore::addToStoreCommon support
non-recursive+sha256 so we can just use the default
@@ -418,8 +424,11 @@ StorePath BinaryCacheStore::addToStore(const string & name, const Path & srcPath
})->path;
}
StorePath BinaryCacheStore::addTextToStore(const string & name, const string & s,
const StorePathSet & references, RepairFlag repair)
StorePath BinaryCacheStore::addTextToStore(
std::string_view name,
std::string_view s,
const StorePathSet & references,
RepairFlag repair)
{
auto textHash = hashString(htSHA256, s);
auto path = makeTextPath(name, textHash, references);
+15 -7
View File
@@ -98,15 +98,23 @@ public:
void addToStore(const ValidPathInfo & info, Source & narSource,
RepairFlag repair, CheckSigsFlag checkSigs) override;
StorePath addToStoreFromDump(Source & dump, const string & name,
FileIngestionMethod method, HashType hashAlgo, RepairFlag repair, const StorePathSet & references ) override;
StorePath addToStoreFromDump(Source & dump, std::string_view name,
FileIngestionMethod method, HashType hashAlgo, RepairFlag repair, const StorePathSet & references) override;
StorePath addToStore(const string & name, const Path & srcPath,
FileIngestionMethod method, HashType hashAlgo,
PathFilter & filter, RepairFlag repair, const StorePathSet & references) override;
StorePath addToStore(
std::string_view name,
const Path & srcPath,
FileIngestionMethod method,
HashType hashAlgo,
PathFilter & filter,
RepairFlag repair,
const StorePathSet & references) override;
StorePath addTextToStore(const string & name, const string & s,
const StorePathSet & references, RepairFlag repair) override;
StorePath addTextToStore(
std::string_view name,
std::string_view s,
const StorePathSet & references,
RepairFlag repair) override;
void registerDrvOutput(const Realisation & info) override;
+77
View File
@@ -0,0 +1,77 @@
#pragma once
#include "realisation.hh"
#include <string>
#include <chrono>
namespace nix {
struct BuildResult
{
/* Note: don't remove status codes, and only add new status codes
at the end of the list, to prevent client/server
incompatibilities in the nix-store --serve protocol. */
enum Status {
Built = 0,
Substituted,
AlreadyValid,
PermanentFailure,
InputRejected,
OutputRejected,
TransientFailure, // possibly transient
CachedFailure, // no longer used
TimedOut,
MiscFailure,
DependencyFailed,
LogLimitExceeded,
NotDeterministic,
ResolvesToAlreadyValid,
} status = MiscFailure;
std::string errorMsg;
std::string toString() const {
auto strStatus = [&]() {
switch (status) {
case Built: return "Built";
case Substituted: return "Substituted";
case AlreadyValid: return "AlreadyValid";
case PermanentFailure: return "PermanentFailure";
case InputRejected: return "InputRejected";
case OutputRejected: return "OutputRejected";
case TransientFailure: return "TransientFailure";
case CachedFailure: return "CachedFailure";
case TimedOut: return "TimedOut";
case MiscFailure: return "MiscFailure";
case DependencyFailed: return "DependencyFailed";
case LogLimitExceeded: return "LogLimitExceeded";
case NotDeterministic: return "NotDeterministic";
case ResolvesToAlreadyValid: return "ResolvesToAlreadyValid";
default: return "Unknown";
};
}();
return strStatus + ((errorMsg == "") ? "" : " : " + errorMsg);
}
/* How many times this build was performed. */
unsigned int timesBuilt = 0;
/* If timesBuilt > 1, whether some builds did not produce the same
result. (Note that 'isNonDeterministic = false' does not mean
the build is deterministic, just that we don't have evidence of
non-determinism.) */
bool isNonDeterministic = false;
DrvOutputs builtOutputs;
/* The start/stop times of the build (or one of the rounds, if it
was repeated). */
time_t startTime = 0, stopTime = 0;
bool success() {
return status == Built || status == Substituted || status == AlreadyValid || status == ResolvesToAlreadyValid;
}
};
}
+21 -11
View File
@@ -116,7 +116,7 @@ DerivationGoal::~DerivationGoal()
}
string DerivationGoal::key()
std::string DerivationGoal::key()
{
/* Ensure that derivations get built in order of their name,
i.e. a derivation named "aardvark" always comes before
@@ -1013,7 +1013,7 @@ HookReply DerivationGoal::tryBuildHook()
/* Read the first line of input, which should be a word indicating
whether the hook wishes to perform the build. */
string reply;
std::string reply;
while (true) {
auto s = [&]() {
try {
@@ -1025,8 +1025,8 @@ HookReply DerivationGoal::tryBuildHook()
}();
if (handleJSONLogMessage(s, worker.act, worker.hook->activities, true))
;
else if (string(s, 0, 2) == "# ") {
reply = string(s, 2);
else if (s.substr(0, 2) == "# ") {
reply = s.substr(2);
break;
}
else {
@@ -1091,7 +1091,7 @@ HookReply DerivationGoal::tryBuildHook()
/* Create the log file and pipe. */
Path logFile = openLogFile();
set<int> fds;
std::set<int> fds;
fds.insert(hook->fromHook.readSide.get());
fds.insert(hook->builderOut.readSide.get());
worker.childStarted(shared_from_this(), fds, false, false);
@@ -1140,10 +1140,10 @@ Path DerivationGoal::openLogFile()
logDir = localStore->logDir;
else
logDir = settings.nixLogDir;
Path dir = fmt("%s/%s/%s/", logDir, LocalFSStore::drvsLogDir, string(baseName, 0, 2));
Path dir = fmt("%s/%s/%s/", logDir, LocalFSStore::drvsLogDir, baseName.substr(0, 2));
createDirs(dir);
Path logFileName = fmt("%s/%s%s", dir, string(baseName, 2),
Path logFileName = fmt("%s/%s%s", dir, baseName.substr(2),
settings.compressLog ? ".bz2" : "");
fdLogFile = open(logFileName.c_str(), O_CREAT | O_WRONLY | O_TRUNC | O_CLOEXEC, 0666);
@@ -1175,10 +1175,11 @@ bool DerivationGoal::isReadDesc(int fd)
return fd == hook->builderOut.readSide.get();
}
void DerivationGoal::handleChildOutput(int fd, const string & data)
void DerivationGoal::handleChildOutput(int fd, std::string_view data)
{
if (isReadDesc(fd))
// local & `ssh://`-builds are dealt with here.
auto isWrittenToLog = isReadDesc(fd);
if (isWrittenToLog)
{
logSize += data.size();
if (settings.maxLogSize && logSize > settings.maxLogSize) {
@@ -1207,7 +1208,16 @@ void DerivationGoal::handleChildOutput(int fd, const string & data)
if (hook && fd == hook->fromHook.readSide.get()) {
for (auto c : data)
if (c == '\n') {
handleJSONLogMessage(currentHookLine, worker.act, hook->activities, true);
auto json = parseJSONMessage(currentHookLine);
if (json) {
auto s = handleJSONLogMessage(*json, worker.act, hook->activities, true);
// ensure that logs from a builder using `ssh-ng://` as protocol
// are also available to `nix log`.
if (s && !isWrittenToLog && logSink && (*json)["type"] == resBuildLogLine) {
auto f = (*json)["fields"];
(*logSink)((f.size() > 0 ? f.at(0).get<std::string>() : "") + "\n");
}
}
currentHookLine.clear();
} else
currentHookLine += c;
+3 -2
View File
@@ -2,6 +2,7 @@
#include "parsed-derivations.hh"
#include "lock.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "pathlocks.hh"
#include "goal.hh"
@@ -145,7 +146,7 @@ struct DerivationGoal : public Goal
void timedOut(Error && ex) override;
string key() override;
std::string key() override;
void work() override;
@@ -200,7 +201,7 @@ struct DerivationGoal : public Goal
virtual bool isReadDesc(int fd);
/* Callback used by the worker to write to the log. */
void handleChildOutput(int fd, const string & data) override;
void handleChildOutput(int fd, std::string_view data) override;
void handleEOF(int fd) override;
void flushLine();
@@ -137,7 +137,7 @@ void DrvOutputSubstitutionGoal::finished()
amDone(ecSuccess);
}
string DrvOutputSubstitutionGoal::key()
std::string DrvOutputSubstitutionGoal::key()
{
/* "a$" ensures substitution goals happen before derivation
goals. */
@@ -51,7 +51,7 @@ public:
void timedOut(Error && ex) override { abort(); };
string key() override;
std::string key() override;
void work() override;
void handleEOF(int fd) override;
+2 -2
View File
@@ -5,8 +5,8 @@ namespace nix {
bool CompareGoalPtrs::operator() (const GoalPtr & a, const GoalPtr & b) const {
string s1 = a->key();
string s2 = b->key();
std::string s1 = a->key();
std::string s2 = b->key();
return s1 < s2;
}
Loaded 100 of 251 files, more files were not shown because too many files have changed in this diff. Show more