Compare commits

...
Author SHA1 Message Date
regnat c2c0dba792 Remove the right socket before starting the daemon
For some reason, an old socket occasionally stays here on OSX, causing
the subsequent tests to fail
2021-07-27 17:06:11 +02:00
regnat addacfce4a Allow running all the tests with the daemon
When `NIX_DAEMON_PACKAGE` is set, make all the tests use the Nix daemon.
That way we can test every piece of Nix functionality both with and
without the daemon.

Tests for which using the daemon isn’t possible or doesn’t make sens can
selectively be disabled with `needLocalStore`
2021-07-27 17:06:11 +02:00
Eelco Dolstra 706777a4a8 Merge pull request #5055 from NixOS/fix-manpages-install
Fix the generation of nix3 manpages
2021-07-27 14:53:16 +02:00
regnat 441be5d59a Remove dead code in doc’s local.mk 2021-07-27 14:37:55 +02:00
regnat cd6e9eb024 Fix the generation of nix3 manpages
- Separate the generation of the manpages from their installation
- Make sure that `make` generates the manpages
- Make sure that `make install` installs them

Fix #5051
2021-07-27 13:47:53 +02:00
Eelco Dolstra c000cec27f Merge pull request #5048 from tweag/flox-eval-store
--eval-store and faster closure copying
2021-07-27 12:20:32 +02:00
Eelco Dolstra 29e4913f79 Add --eval-store test 2021-07-27 11:17:56 +02:00
Eelco Dolstra 47002108d1 nix-instantiate: Fix --eval-store 2021-07-27 11:16:47 +02:00
Eelco Dolstra f52fa47e16 Merge pull request #5046 from ncfavier/patch-1
Fix --no-gc-warning
2021-07-27 09:55:25 +02:00
Eelco Dolstra 9deeab6337 Merge pull request #5043 from alyssais/vexing
libutil: use uniform initialization in _deletePath
2021-07-27 09:54:24 +02:00
Eelco Dolstra da9b2cd74e Merge pull request #5049 from Pamplemousse/less_globals
libexpr: Remove unused code
2021-07-27 09:53:12 +02:00
Pamplemousse c2d7c0cdb9 libexpr: Remove unused code
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-26 09:14:34 -07:00
Eelco Dolstra da55210403 Merge pull request #5037 from Pamplemousse/less_globals
Avoid global counters
2021-07-26 14:29:03 +02:00
Eelco Dolstra 72c5bac39d Revert no longer necessary change 2021-07-26 13:50:18 +02:00
Eelco Dolstra fe1f34fa60 Low-latency closure copy
This adds a new store operation 'addMultipleToStore' that reads a
number of NARs and ValidPathInfos from a Source, allowing any number
of store paths to be copied in a single call. This is much faster on
high-latency links when copying a lot of small files, like .drv
closures.

For example, on a connection with an 50 ms delay:

Before:

  $ nix copy --to 'unix:///tmp/proxy-socket?root=/tmp/dest-chroot' \
    /nix/store/90jjw94xiyg5drj70whm9yll6xjj0ca9-hello-2.10.drv \
    --derivation --no-check-sigs
  real    0m57.868s
  user    0m0.103s
  sys     0m0.056s

After:

  real    0m0.690s
  user    0m0.017s
  sys     0m0.011s
2021-07-26 13:31:09 +02:00
Naïm Favier cefa8b673f Fix --no-gc-warning
Broken by 8e758d402b
2021-07-25 16:06:55 +02:00
Alyssa Ross 369ed71858 libutil: use uniform initialization in _deletePath
Otherwise I get a compiler error when building for NetBSD:

src/libutil/util.cc: In function 'void nix::_deletePath(const Path&, uint64_t&)':
src/libutil/util.cc:438:17: error: base operand of '->' is not a pointer
  438 |     AutoCloseFD dirfd(open(dir.c_str(), O_RDONLY));
      |                 ^~~~~
src/libutil/util.cc:439:10: error: 'dirfd' was not declared in this scope
  439 |     if (!dirfd) {
      |          ^~~~~
src/libutil/util.cc:444:17: error: 'dirfd' was not declared in this scope
  444 |     _deletePath(dirfd.get(), path, bytesFreed);
      |                 ^~~~~
2021-07-24 09:19:48 +00:00
Domen Kožar 97be92569c Merge pull request #5040 from SuperSandro2000/patch-1
installer: fix --no-modify-profile help text
2021-07-23 20:40:45 +02:00
Domen Kožar 7cd330bc46 Merge pull request #5041 from SuperSandro2000/patch-2
installer: update global nix.conf location
2021-07-23 20:40:03 +02:00
Sandro 5c4cc5e0d6 installer: update global nix.conf location 2021-07-23 19:56:30 +02:00
Sandro 142c966691 installer: fix --no-modify-profile help text 2021-07-23 19:51:37 +02:00
Eelco Dolstra 9957315ce0 RemoteStore::buildPaths(): Handle evalStore 2021-07-22 22:50:48 +02:00
Eelco Dolstra a7b7fcfb16 Remove redundant RealisedPath::closure() call 2021-07-22 22:43:18 +02:00
Eelco Dolstra 96c62fb66c Fix formatting error in 'nix store' manpage 2021-07-22 10:11:04 +02:00
Eelco Dolstra eb6db4fd38 buildPaths(): Add an evalStore argument
With this, we don't have to copy the entire .drv closure to the
destination store ahead of time (or at all). Instead, buildPaths()
reads .drv files from the eval store and copies inputSrcs to the
destination store if it needs to build a derivation.

Issue #5025.
2021-07-22 09:59:51 +02:00
Eelco Dolstra 668abd3e57 copyPaths: Pass store by reference 2021-07-22 09:59:51 +02:00
Eelco Dolstra 8d9f7048cd Use eval-store in more places
In particular, this now works:

  $ nix path-info --eval-store auto --store https://cache.nixos.org nixpkgs#hello

Previously this would fail as it would try to upload the hello .drv to
cache.nixos.org. Now the .drv is instantiated in the local store, and
then we check for the existence of the outputs in cache.nixos.org.
2021-07-22 09:59:51 +02:00
Eelco Dolstra 95e915a993 DummyStore: Remove redundant method 2021-07-22 09:59:51 +02:00
Eelco Dolstra 7321657746 nix-shell: Handle --eval-store correctly 2021-07-22 09:59:51 +02:00
Eelco Dolstra e9848beca7 nix-build: Copy drv closure between eval store and build store 2021-07-22 09:59:51 +02:00
Eelco Dolstra 2ff3035cf4 Support --eval-store in nix-instantiate and nix-build 2021-07-22 09:59:51 +02:00
Eelco Dolstra 3d9de41a5b Hacky fast closure copying mechanism 2021-07-22 09:59:51 +02:00
Eelco Dolstra bef40c2949 Add --eval-store option 2021-07-22 09:59:51 +02:00
Eelco Dolstra dfda499326 Downgrade warning message
If a store doesn't support GC, then we don't need to warn about the
inability to create roots.
2021-07-22 09:58:50 +02:00
Pamplemousse c1c5dd7449 Avoid global counters
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-21 16:49:52 -07:00
Eelco Dolstra 3bb8667a17 Tweak --no/use-registries doc strings 2021-07-21 14:27:37 +02:00
Eelco Dolstra fc248cf59e Merge branch 'balsoft/no-registries' of https://github.com/serokell/nix 2021-07-21 10:11:40 +02:00
Eelco Dolstra 140ccf1368 deletePath(): Return ENFILE instead of EBADF when out of file descriptors
Also remove an erroneous comment.
2021-07-20 20:59:45 +02:00
Eelco Dolstra c5fafca5a4 Merge remote-tracking branch 'origin/repl-flake-support' 2021-07-20 20:06:56 +02:00
Eelco Dolstra dd7bcf3e1c Merge pull request #5030 from NixOS/lock-ca-derivations
Properly lock the builds of CA derivations
2021-07-20 14:39:21 +02:00
regnat 8707773965 Properly lock the builds of CA derivations
Make sure that we can’t build the same derivation twice at the same
time.

Fix https://github.com/NixOS/nix/issues/5029
2021-07-20 06:57:56 +02:00
Eelco Dolstra 1af3f63be5 Merge pull request #5028 from hercules-ci/edit-package-not-derivation
nix repl: Update :edit doc to remove inaccurate use of "derivation"
2021-07-19 20:15:17 +02:00
Eelco Dolstra def94953c9 Merge remote-tracking branch 'origin/cleaner-nix3-manpages-install' 2021-07-19 20:11:48 +02:00
Robert Hensing 3bbf5558e0 nix repl: Update :edit help text
It supports functions as well. Also change `package` to
`derivation` because it operates at the language level and does
not open the derivation (which would be useful but not nearly
as much).
2021-07-19 18:13:26 +02:00
Robert Hensing ad24921de8 Rename findDerivationFilename -> findPackageFilename
It does not operate on a derivation and does not return a
derivation path. Instead it works at the language level,
where a distinct term "package" is more appropriate to
distinguish the parent object of `meta.position`; an
attribute which doesn't even make it into the derivation.
2021-07-19 18:10:10 +02:00
regnat 765a3a20cb Add a :load-flake command to the nix repl
`:lf ref` does essentially the same thing as
`:a (builtins.getFlake "ref")` (but cannonicalising `ref` if needs be,
so that `:lf .` works)

Fix #3803
2021-07-19 17:52:41 +02:00
regnat c5f5d615a6 Cleanup the generation of the nix3 manpages
Use a dedicated make target for the man page rather than bundling the
generation as part of `install`.
Also make sure that `make install` is a fixpoint by

- Removing the generated markdown files from `MANUAL_SRCS`
- Not having the manpage generation write in its source directory so as
  to not update its timestamp (it would run each time otherwise)
2021-07-19 16:55:56 +02:00
Eelco Dolstra 58cb411db6 Merge pull request #5020 from NixOS/make-the-repl-reuse-the-whole-nix-config
Forward the whole Nix config to the repl subprocesses
2021-07-19 16:02:02 +02:00
Eelco Dolstra 2cf21f2829 Merge pull request #5022 from NixOS/more-lenient-realisation-compatibility-check
Be more lenient when realisations have a conflicting dependency set
2021-07-19 16:01:38 +02:00
Alexander BantyevandEelco Dolstra 3e57e3480b Add use-registries config option (and deprecate --no-registries flag)
Some people want to avoid using registries at all on their system; Instead
of having to add --no-registries to every command, this commit allows to
set use-registries = false in the config. --no-registries is still allowed
everywhere it was allowed previously, but is now deprecated.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2021-07-16 23:26:04 +03:00
regnat a4ec6cb1da Be more lenient when realisations have a conflicting dependency set
- This can legitimately happen (for example because of a non-determinism
  causing a build-time dependency to be kept or not as a runtime
  reference)
- Because of older Nix versions, it can happen that we encounter a
  realisation with an (erroneously) empty set of dependencies, in which
  case we don’t want to fail, but just warn the user and try to fix it.
2021-07-16 11:48:41 +02:00
regnat c05bdef020 Forward the whole Nix config to the repl subprocesses
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the nix subprocesses in the repl
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available.

This is required for example to make `nix repl` work with a custom
`--store`
2021-07-16 09:48:45 +02:00
Eelco Dolstra db4d4cf4ba Merge pull request #5019 from NixOS/make-the-post-hook-reuse-the-process-config
Forward the whole Nix config to the post-build-hook
2021-07-15 19:30:18 +02:00
regnat 9b1f3cbc13 Forward the whole Nix config to the post-build-hook
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the post-build-hook.
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available to the hook
2021-07-15 18:41:56 +02:00
Eelco Dolstra bdc24efc87 Merge pull request #5016 from edolstra/nlohmann
nlohmann_json: Update to 3.9.1, fix use of internal copy
2021-07-15 14:24:30 +02:00
Eelco Dolstra 5e3c6bd89a Fix perl bindings build 2021-07-15 13:51:12 +02:00
Eelco Dolstra 307977963c nlohmann_json: Update to 3.9.1, fix use of internal copy 2021-07-15 12:25:53 +02:00
Eelco Dolstra 77d5b37da3 Merge pull request #5011 from Pamplemousse/fix_adjustLoc
libexpr: Fix read out-of-bound on the heap
2021-07-15 09:23:59 +02:00
Pamplemousse 99f8fc995b libexpr: Fix read out-of-bound on the heap
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-14 09:09:42 -07:00
Eelco Dolstra bee71d692a Merge pull request #5008 from NixOS/flake-devShells-attribute
Add a `devShells` attribute to the flake schema
2021-07-13 23:27:55 +02:00
regnat 797e260e3a Make nix flake check aware of devShells 2021-07-13 17:25:27 +02:00
regnat 43d5c5f87b Make nix flake show display the devShells 2021-07-13 17:25:27 +02:00
regnat 037c86ee04 nix develop: Search in devShells.${system} by default
Make `nix develop .#foo` search `.#devShells.${system}.foo` first
2021-07-13 17:25:27 +02:00
Eelco Dolstra eb4788954d Merge pull request #5006 from illustris/nscd
fixed-output derivations: fix incorrect responses for getpwuid
2021-07-13 17:20:37 +02:00
illustris 7bc17a903b fixed output derivations: fix incorrect responses for getpwuid
Passing nscd socket into the build environment causes unexpected behavior in programs that make getpwuid and other related calls.

relevant threads:
- https://github.com/NixOS/nix/issues/4991
- https://discourse.nixos.org/t/haunted-nix-build-breaks-isolation/13869
2021-07-13 15:34:14 +05:30
Eelco Dolstra 099df07e1e Merge pull request #5000 from nielsegberts/master
toString also coerces a set with an outPath attribute to a string
2021-07-12 17:35:32 +02:00
Eelco Dolstra 02dff9e529 Style 2021-07-12 17:32:23 +02:00
Eelco Dolstra e06c272c12 Merge branch 'structured-attrs-shell' of https://github.com/Ma27/nix 2021-07-12 17:13:05 +02:00
Maximilian Bosch 04cd2da84c Merge branch 'master' into structured-attrs-shell
Conflicts:
        src/nix/develop.cc
        src/nix/get-env.sh
        tests/shell.nix
2021-07-12 15:49:39 +02:00
Eelco Dolstra 91d2e8d5ad Error -> UsageError 2021-07-12 15:04:46 +02:00
Eelco Dolstra 8d97030bfd Merge branch 'days' of https://github.com/nielsegberts/nix 2021-07-12 15:02:24 +02:00
Niels Egberts 2cf14db857 Throw on unexpected input for --delete-older-than
'--delete-older-than 10' deletes the generations older than a single day, and '--delete-older-than 12m' deletes all generations older than 12 days.

This changes makes it throw on those invalid inputs, and gives an example of a valid input.
2021-07-11 11:24:43 +01:00
Niels Egberts ae0ed53b09 toString also coerces a set with an outPath attribute to a string
nix-repl> builtins.toString { outPath = "somestring"; }
"somestring"
2021-07-09 21:50:10 +01:00
Eelco Dolstra ceda58d112 Formatting 2021-07-09 14:03:51 +02:00
Eelco Dolstra 07790fdddf ref: Add equality operators 2021-07-09 14:03:48 +02:00
Eelco Dolstra 9cf991f421 Merge pull request #4998 from edolstra/nix-print-dev-env-json
nix print-dev-env: Add --json flag
2021-07-09 13:55:44 +02:00
Eelco Dolstra 86fb01c4be nix print-dev-env: Add --json flag 2021-07-09 12:10:48 +02:00
Eelco Dolstra 223e0569ff Merge pull request #4997 from edolstra/nix-develop-arrays
nix develop: Make bash environment parsing more robust
2021-07-09 11:16:31 +02:00
Eelco Dolstra e50408bd31 nix develop: Add a test for bash functions 2021-07-09 01:18:44 +02:00
Eelco Dolstra 9fc7da1e08 Add test for #4992 2021-07-09 01:18:44 +02:00
Eelco Dolstra 5f6375a816 nix develop: Filter some bash magic variables 2021-07-09 01:18:44 +02:00
Eelco Dolstra b1f1347ade nix develop: Don't parse bash environment with regexes
Instead have get-env.sh dump the bash environment as JSON. This should
be a lot less error-prone.

Fixes #4992.
2021-07-09 01:18:37 +02:00
Eelco Dolstra b1cfe8f984 nix develop: Add a test for arrays 2021-07-09 00:32:55 +02:00
Eelco Dolstra d2b8b23ae9 Merge pull request #4995 from NixOS/fetchgit-name-attribute
Fix the `name` attribute in builtins.fetchGit
2021-07-09 00:32:13 +02:00
regnat a654c1d81c Restore the possibility to use a name parameter in fetchGit
Accidentally broken by 7e5c79a2d2
2021-07-08 14:57:48 +02:00
Eelco Dolstra 8648143120 Merge pull request #4988 from NixOS/fetchgit-name-attribute
Add a name attribute to the fetchers
2021-07-08 14:33:49 +02:00
regnat 7e5c79a2d2 Forbid the name attribute for fetchTree
We need to support it for the “old” fetch* functions for backwards
compatibility, but we don’t need it for fetchTree (as it’s a new
function).
Given that changing the `name` messes-up the content hashing, we can
just forbid passing a custom `name` argument to it
2021-07-08 13:53:52 +02:00
Eelco Dolstra 2172e60f7a Merge pull request #4935 from alyssais/host_os
Apply OS checks to host platform, not build
2021-07-08 12:40:54 +02:00
Eelco Dolstra 156666de3d Merge pull request #4969 from serokell/balsoft/fix-nixConfig-flake-registry
flake.nixConfig: fix flake-registry config settings
2021-07-08 12:35:08 +02:00
Eelco Dolstra 6060ea1b0e Add tests/dummy 2021-07-07 11:30:35 +02:00
Eelco Dolstra 02dd6bb610 tests/check.sh: Fix a race
Fixes this random failure:

    error: hash mismatch in fixed-output derivation '/tmp/nix-shell.EUgAVU/nix-test/tests/check/store/sfps3l3c5n7dabpx34kigxnfhmrwk2h6-dummy.drv':
             specified: sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=
                got:    sha256-0qhPS4tlCTfsj3PNi+LHSt1akRumTfJ0WO2CKdqASiY=

which happens because multiple tests were writing to ./dummy.
2021-07-07 10:48:47 +02:00
Eelco Dolstra 2c853e2a58 Merge branch 'master' of github.com:NixOS/nix 2021-07-07 10:03:04 +02:00
Eelco Dolstra 3b3e6bb1e5 Style tweaks 2021-07-07 10:02:55 +02:00
Eelco Dolstra e37ecd1282 Merge branch 'balsoft/registry' of https://github.com/serokell/nix 2021-07-07 09:57:15 +02:00
regnat 2c8240677e allow fetchFromGitHub to take a name argument 2021-07-06 08:43:19 +02:00
regnat e4b082a52b allow fetchMercurial to take a name argument 2021-07-06 08:43:06 +02:00
regnat a487a652ed allow fetchTarball to take a name argument 2021-07-06 08:42:47 +02:00
regnat 83615fcf8f Allow fetchGit to take a name argument
Fix #3388
2021-07-06 08:41:52 +02:00
Eelco Dolstra e700ecb901 Merge pull request #4986 from jkeifer/master
Fix wrong hash var for aarch64-darwin binary
2021-07-05 21:51:04 +02:00
Eelco Dolstra 0a8845720e Merge pull request #4980 from Ma27/file-attr-completion
libcmd/installables: implement completion for Nix expressions passed via `-f`
2021-07-05 21:50:15 +02:00
Maximilian Bosch 3c5f69bb60 completeInstallable: also match for already typed prefixes 2021-07-05 21:37:33 +02:00
Jarrett Keifer c8a80e4dbe Fix wrong hash var for aarch64-darwin binary 2021-07-05 07:48:53 -07:00
Eelco Dolstra 1f93084149 Merge pull request #4978 from NixOS/scp-uris
Allow scp-style uris in `fetchgit`
2021-07-05 14:07:04 +02:00
Eelco Dolstra f1ecd30bd5 Merge pull request #4773 from alyssais/dl
doc: builtins: use a definition list
2021-07-05 14:03:15 +02:00
Eelco Dolstra d65342d226 Merge pull request #4873 from jtojnar/fish-completion
Add a fish completion script
2021-07-05 14:02:04 +02:00
Eelco Dolstra cee426cc01 Merge pull request #4938 from tomcur/store-uri-parsing
Improve machine store URI parsing
2021-07-05 14:00:35 +02:00
Eelco Dolstra 24bc935462 Merge branch 'disable_gc' of https://github.com/Pamplemousse/nix 2021-07-05 13:53:52 +02:00
Eelco Dolstra c053aecff4 Merge pull request #4983 from manveru/fix-devShell-env
Fix devShell handling of env values including @ and %
2021-07-05 13:22:32 +02:00
Michael Fellinger 087c5f5325 Fix devShell handling of env values including @ and % 2021-07-05 12:05:06 +02:00
Maximilian Bosch 70cb2ffacc libcmd/installables: implement completion for Nix expressions passed via -f 2021-07-03 14:19:10 +02:00
Domen Kožar f9d72855ae Merge pull request #4967 from Pamplemousse/specific_errors
Prefer to throw specific errors
2021-07-03 10:34:56 +02:00
regnat ec2c6bd470 Allow scp-style uris in fetchgit
Fix #5303
2021-07-02 19:20:07 +02:00
Eelco Dolstra 7a3f43cd58 Merge pull request #4971 from ylh/master
Respect TERM=dumb more consistently
2021-07-02 17:50:23 +02:00
Yestin L. Harrison 0fe84bef72 Add $NO_COLOR check to ANSI escape conditions 2021-07-02 09:33:54 -06:00
Eelco Dolstra 44086071e5 Merge pull request #4974 from domenkozar/upload-release-aarch64-darwin
upload-release.pl: add aarch64-darwin
2021-07-02 15:32:33 +02:00
Alexander Bantyev d8ad6f1c10 Add tests for --registry and second arg for nix registry pin 2021-07-02 16:10:57 +03:00
Domen Kožar 74838deeb8 upload-release.pl: add aarch64-darwin 2021-07-02 14:12:00 +02:00
Yestin L. Harrison 20cce079f2 Respect TERM=dumb more consistently 2021-07-01 18:19:01 -06:00
Pamplemousse 4a7a8b87cd Prefer to throw specific errors
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-07-01 11:09:31 -07:00
Alexander Bantyev e756a59c72 fixup! flake.nixConfig: fix flake-registry config settings 2021-07-01 17:54:22 +03:00
Alexander Bantyev ef1e7ab840 flake.nixConfig: fix flake-registry config settings
Before this commit, nixConfig.flake-registry didn't have any real effect
on the evaluation, since config was applied after inputs were evaluated.
Change this behavior: apply the config in the beginning of flake::lockFile.
2021-07-01 00:23:47 +03:00
Alexander Bantyev 811f3e8605 nix registry pin: add a way to pin to a custom locked 2021-06-30 22:16:40 +03:00
Alexander Bantyev 093ed47636 nix registry: add --registry flag 2021-06-30 22:13:32 +03:00
Pamplemousse 5be17a4b96 Allow to compile after ./configure --enable-gc=no
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-06-30 09:45:21 -07:00
Eelco Dolstra 139f7af5ec Merge pull request #4958 from NixOS/nix-develop-remove-override-input-warning
develop: Discard the input{Overrides,Updates} when getting bash
2021-06-30 17:57:01 +02:00
Domen Kožar 69eb65403a Merge pull request #4963 from matthewbauer/cross-fix
Restrict jobs built on hydra
2021-06-30 08:03:07 +02:00
Matthew Bauer 2200f315da Disable -pie on static nix
This should resolve the failing build.

See https://github.com/NixOS/nixpkgs/pull/128674 for a better fix.
2021-06-29 21:48:07 -05:00
Matthew Bauer 7351656b82 Only cross compile from x86_64-linux
This is broken on aarch64-linux / x86_64-darwin, so might as well just disable it for now.
2021-06-29 21:47:35 -05:00
Eelco Dolstra c92fbdb654 Merge pull request #4960 from NixOS/nix-repl-experimental-features
Forward the experimental features to the nix repl subprocesses
2021-06-29 15:10:02 +02:00
regnat 7daf0c6ef1 Forward the experimental features to the nix repl subprocesses
Pass the current experimental features using `NIX_CONFIG` to the various
Nix subprocesses that `nix repl` invokes.

This is quite a hack, but having `nix repl` call Nix with a subprocess
is a hack already, so I guess that’s fine.
2021-06-29 14:52:46 +02:00
Eelco Dolstra 5a18e2a533 Merge pull request #4959 from NixOS/override-non-flake-inputs
Keep the `isFlake` attribute for overriden inputs
2021-06-29 14:23:50 +02:00
regnat 4cff8188a5 Keep the isFlake attribute for overriden inputs
When overriding an input that isn’t a flake, mark the override as also not
being a flake.

Fix #3774
2021-06-29 13:56:48 +02:00
Eelco Dolstra f14c3b6f68 Merge pull request #4944 from hercules-ci/fix-gc-crash
Fix gc crash
2021-06-29 13:52:14 +02:00
Domen Kožar 81535022dc Merge pull request #2667 from matthewbauer/cross-jobs
Add armv6l-linux & armv7l-linux as cross jobs
2021-06-29 11:39:49 +02:00
regnat bf7960a4ed develop: Discard the input{Overrides,Updates} when getting bash
`nix develop` is getting bash from an (assumed existing) `nixpkgs`
flake. However, when doing so, it reuses the `lockFlags` passed to the
current flake, including the `--input-overrides` and `--input-update`
which generally don’t make sense anymore at that point (and trigger a
warning because of that)

Clear these overrides before getting the nixpkgs flake to get rid of the
warning.
2021-06-29 10:13:42 +02:00
Matthew Bauer 6c13a3f735 Support binaryTarballCross in gha 2021-06-28 15:08:17 -05:00
Matthew Bauer 580583e0b3 Build cross-compilation in gha 2021-06-26 00:14:54 -05:00
Matthew Bauer c906d6530d Support cross-compiling binaryTarball 2021-06-26 00:12:03 -05:00
Matthew Bauer c3a929349f Merge remote-tracking branch 'origin/master' into cross-jobs 2021-06-25 15:51:02 -05:00
Robert Hensing 5c58d84a76 boehmgc: Remove unused code from patch 2021-06-25 17:45:48 +02:00
Robert Hensing 57409244ec boehmgc: Crude support for coroutines
Fixes the problem where a stack pointer outside the original
thread causes the collector to crash.

It could be made more accurate by recording the stack pointer
every time we switch to a coroutine. We can use this information
to update our own coroutine stacks like normal data. When the
stack pointer is on a thread, we can add a field to GC_thread
"fallback_sp" to be used when the thread sp is outside the original
thread range.
2021-06-24 18:17:03 +02:00
Maximilian Bosch 644415d391 Perform input rewrites only in LocalDerivationGoal 2021-06-24 15:25:21 +02:00
Robert Hensing bec83a6f95 BoehmGCStackAllocator: ignore stack protection page
This fixes a crash that looks like:

```
Thread 1 "nix-build" received signal SIGSEGV, Segmentation fault.
0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
(gdb) bt
0  0x00007ffff7ad22a0 in GC_push_all_eager () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
1  0x00007ffff7adeefb in GC_push_all_stacks () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
2  0x00007ffff7ad5ac7 in GC_mark_some () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
3  0x00007ffff7ad77bd in GC_stopped_mark () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
4  0x00007ffff7adbe3a in GC_try_to_collect_inner.part.0 () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
5  0x00007ffff7adc2a2 in GC_collect_or_expand () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
6  0x00007ffff7adc4f8 in GC_allocobj () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
7  0x00007ffff7adc88f in GC_generic_malloc_inner () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
8  0x00007ffff7ae1a04 in GC_generic_malloc_many () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
9  0x00007ffff7ae1c72 in GC_malloc_kind () from /nix/store/p1z58l18klf88iijpd0qi8yd2n9lhlk4-boehm-gc-8.0.4/lib/libgc.so.1
10 0x00007ffff7e003d6 in nix::EvalState::allocValue() () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
11 0x00007ffff7e04b9c in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
12 0x00007ffff7e0a773 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
13 0x00007ffff7e0a91d in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
14 0x00007ffff7e0a8f8 in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
15 0x00007ffff7e0e0e8 in nix::ExprOpNEq::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
16 0x00007ffff7e0d708 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
17 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
18 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
19 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
20 0x00007ffff7e0d695 in nix::ExprOpOr::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
21 0x00007ffff7e09e19 in nix::ExprOpNot::eval(nix::EvalState&, nix::Env&, nix::Value&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
22 0x00007ffff7e0a792 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
23 0x00007ffff7e8cba0 in nix::addPath(nix::EvalState&, nix::Pos const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value*, nix::FileIngestionMethod, std::optional<nix::Hash>, nix::Value&)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)#1}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixexpr.so
24 0x00007ffff752e6f9 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
25 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
26 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
27 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
28 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
29 0x00007ffff752e8e2 in nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
30 0x00007ffff757f8c0 in void boost::context::detail::fiber_entry<boost::context::detail::fiber_record<boost::context::fiber, nix::VirtualStackAllocator, boost::coroutines2::detail::pull_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::control_block::control_block<nix::VirtualStackAllocator, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}>(boost::context::preallocated, nix::VirtualStackAllocator&&, nix::sinkToSource(std::function<void (nix::Sink&)>, std::function<void ()>)::SinkToSource::read(char*, unsigned long)::{lambda(boost::coroutines2::detail::push_coroutine<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >&)#1}&&)::{lambda(boost::context::fiber&&)#1}> >(boost::context::detail::transfer_t) () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libnixutil.so
31 0x00007ffff6f331ef in make_fcontext () from /nix/store/hzdzcv9d3bc8rlsaphh7x54zsf0x8nx6-nix-2.4pre20210601_5985b8b/lib/libboost_context.so.1.69.0
32 0x0000000000000000 in ?? ()
```
2021-06-24 12:20:39 +02:00
Thomas Churchman e3d11f9a9c Improve machine store URI parsing 2021-06-23 22:09:22 +01:00
Jan Tojnar bee71e1bb1 Add a fish completion script
This is only rudimentary support as allowed by `NIX_GET_COMPLETIONS`.

In the future, we could use complete’s `--wraps` argument to autocomplete arguments for programs after `nix shell -c`.
2021-06-23 19:59:58 +02:00
Alyssa Ross 4f80464645 Apply OS checks to host platform, not build
Previously, the build system used uname(1) output when it wanted to
check the operating system it was being built for, which meant that it
didn't take into-account cross-compilation when the build and host
operating systems were different.

To fix this, instead of consulting uname output, we consult the host
triple, specifically the third "kernel" part.

For "kernel"s with stable ABIs, like Linux or Cygwin, we can use a
simple ifeq to test whether we're compiling for that system, but for
other platforms, like Darwin, FreeBSD, or Solaris, we have to use a
more complicated check to take into account the version numbers at the
end of the "kernel"s.  I couldn't find a way to just strip these
version numbers in GNU Make without shelling out, which would be even
more ugly IMO.  Because these checks differ between kernels, and the
patsubst ones are quite fiddly, I've added variables for each host OS
we might want to check to make them easier to reuse.
2021-06-23 15:00:36 +00:00
Maximilian Bosch 6f206549ba Move writeStructuredAttrsShell out of ParsedDerivation class 2021-06-22 21:14:20 +02:00
Maximilian Bosch 27ce722638 Prefix env vars for attrs.* files with NIX_ 2021-06-22 19:45:08 +02:00
Maximilian Bosch a92245b110 Remove now-obsolete typedef 2021-06-22 19:15:57 +02:00
Maximilian Bosch 3504c811a5 Add testcase for nix develop with __structuredAttrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch f1e281c4fe Split shell & json creation for build environments with structured attrs 2021-06-22 19:15:57 +02:00
Maximilian Bosch 447928bdb5 Fix usage of structured attrs for nix develop 2021-06-22 19:15:57 +02:00
Maximilian Bosch 3944a120ec Set environment variables for .attrs.json & .attrs.sh
This way no derivation has to expect that these files are in the `cwd`
during the build. This is problematic for `nix-shell` where these files
would have to be inserted into the nix-shell's `cwd` which can become
problematic with e.g. recursive `nix-shell`.

To remain backwards-compatible, the location inside the build sandbox
will be kept, however using these files directly should be deprecated
from now on.
2021-06-22 19:15:57 +02:00
Maximilian Bosch 3b5429aec1 Source complete env in nix-shell with __structuredAttrs = true;
This is needed to push the adoption of structured attrs[1] forward. It's
now checked if a `__json` exists in the environment-map of the derivation
to be openend in a `nix-shell`.

Derivations with structured attributes enabled also make use of a file
named `.attrs.json` containing every environment variable represented as
JSON which is useful for e.g. `exportReferencesGraph`[2]. To
provide an environment similar to the build sandbox, `nix-shell` now
adds a `.attrs.json` to `cwd` (which is mostly equal to the one in the
build sandbox) and removes it using an exit hook when closing the shell.

To avoid leaking internals of the build-process to the `nix-shell`, the
entire logic to generate JSON and shell code for structured attrs was
moved into the `ParsedDerivation` class.

[1] https://nixos.mayflower.consulting/blog/2020/01/20/structured-attrs/
[2] https://nixos.org/manual/nix/unstable/expressions/advanced-attributes.html#advanced-attributes
2021-06-22 19:15:57 +02:00
Alyssa Ross 5f07f2ff2b doc: builtins: use a definition list
This looks a lot better (and is also more semantically meaningful).

Since this list is generated in a Nix expression, I don't think using
HTML here is going to be the thing that puts people off modifying this
part of the documentation!
2021-05-05 14:34:46 +00:00
Matthew Bauer d5fd0f4745 Merge branch 'master' into cross-jobs 2021-03-09 11:40:16 -06:00
Matthew Bauer 4fefe26717 Re-enable armv6l support
This fixes the libatomic detection.
2021-02-05 18:22:34 -06:00
Matthew Bauer c6d878609d Merge remote-tracking branch 'origin/master' into cross-jobs 2021-02-05 18:07:48 -06:00
Matthew Bauer f6ea56dfac Get shellcheck from buildPackages 2019-02-06 23:04:40 -05:00
Matthew Bauer e9072ded97 Use nativeBuildInputs 2019-02-06 22:43:28 -05:00
Matthew Bauer 1996af425a Use buildPackages for native dependencies
Unfortunately, releaseTools.nixBuild does not separate native and
non-native build inputs. As an alternative, we can just use
buildPackages to get the native version of some packages like:

- pkgconfig
- git
- curl
- utillinux
2019-02-06 21:43:47 -05:00
Matthew Bauer 15833516a4 Add armv6l-linux & armv7l-linux as cross jobs
This is a cheap way to get 32-bit ARM working. We don’t support it
officially but lots of people have raspberry pis and similar hardware
they want to install the Nix package manager on.
2019-02-05 16:42:45 -05:00
153 changed files with 12714 additions and 6411 deletions

No files matched your search

+1 -1
View File
@@ -15,6 +15,7 @@ perl/Makefile.config
/doc/manual/*.1 /doc/manual/*.1
/doc/manual/*.5 /doc/manual/*.5
/doc/manual/*.8 /doc/manual/*.8
/doc/manual/generated/*
/doc/manual/nix.json /doc/manual/nix.json
/doc/manual/conf-file.json /doc/manual/conf-file.json
/doc/manual/builtins.json /doc/manual/builtins.json
@@ -76,7 +77,6 @@ perl/Makefile.config
# /tests/ # /tests/
/tests/test-tmp /tests/test-tmp
/tests/common.sh /tests/common.sh
/tests/dummy
/tests/result* /tests/result*
/tests/restricted-innocent /tests/restricted-innocent
/tests/shell /tests/shell
+2 -1
View File
@@ -12,6 +12,7 @@ makefiles = \
src/resolve-system-dependencies/local.mk \ src/resolve-system-dependencies/local.mk \
scripts/local.mk \ scripts/local.mk \
misc/bash/local.mk \ misc/bash/local.mk \
misc/fish/local.mk \
misc/zsh/local.mk \ misc/zsh/local.mk \
misc/systemd/local.mk \ misc/systemd/local.mk \
misc/launchd/local.mk \ misc/launchd/local.mk \
@@ -32,4 +33,4 @@ endif
include mk/lib.mk include mk/lib.mk
GLOBAL_CXXFLAGS += -g -Wall -include config.h -std=c++17 GLOBAL_CXXFLAGS += -g -Wall -include config.h -std=c++17 -I src
+1
View File
@@ -1,3 +1,4 @@
HOST_OS = @host_os@
AR = @AR@ AR = @AR@
BDW_GC_LIBS = @BDW_GC_LIBS@ BDW_GC_LIBS = @BDW_GC_LIBS@
BOOST_LDFLAGS = @BOOST_LDFLAGS@ BOOST_LDFLAGS = @BOOST_LDFLAGS@
+42
View File
@@ -0,0 +1,42 @@
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
index 1cee6a0b..46c3acd9 100644
--- a/pthread_stop_world.c
+++ b/pthread_stop_world.c
@@ -674,6 +674,8 @@ GC_INNER void GC_push_all_stacks(void)
struct GC_traced_stack_sect_s *traced_stack_sect;
pthread_t self = pthread_self();
word total_size = 0;
+ size_t stack_limit;
+ pthread_attr_t pattr;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -723,6 +725,28 @@ GC_INNER void GC_push_all_stacks(void)
hi = p->altstack + p->altstack_size;
/* FIXME: Need to scan the normal stack too, but how ? */
/* FIXME: Assume stack grows down */
+ } else {
+ if (pthread_getattr_np(p->id, &pattr)) {
+ ABORT("GC_push_all_stacks: pthread_getattr_np failed!");
+ }
+ if (pthread_attr_getstacksize(&pattr, &stack_limit)) {
+ ABORT("GC_push_all_stacks: pthread_attr_getstacksize failed!");
+ }
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ #ifndef STACK_GROWS_UP
+ if (lo >= hi || lo < hi - stack_limit) { // sp outside stack
+ lo = hi - stack_limit;
+ }
+ #else
+ #error "STACK_GROWS_UP not supported in boost_coroutine2 (as of june 2021), so we don't support it in Nix."
+ #endif
}
GC_push_all_stack_sections(lo, hi, traced_stack_sect);
# ifdef STACK_GROWS_UP
+1 -1
View File
@@ -231,7 +231,7 @@ AC_SUBST(HAVE_SECCOMP, [$have_seccomp])
# Look for aws-cpp-sdk-s3. # Look for aws-cpp-sdk-s3.
AC_LANG_PUSH(C++) AC_LANG_PUSH(C++)
AC_CHECK_HEADERS([aws/s3/S3Client.h], AC_CHECK_HEADERS([aws/s3/S3Client.h],
[AC_DEFINE([ENABLE_S3], [1], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=1], [AC_DEFINE([ENABLE_S3], [1], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=1],
[AC_DEFINE([ENABLE_S3], [0], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=]) [AC_DEFINE([ENABLE_S3], [0], [Whether to enable S3 support via aws-sdk-cpp.]) enable_s3=])
AC_SUBST(ENABLE_S3, [$enable_s3]) AC_SUBST(ENABLE_S3, [$enable_s3])
AC_LANG_POP(C++) AC_LANG_POP(C++)
+6 -4
View File
@@ -6,9 +6,11 @@ builtins:
concatStrings (map concatStrings (map
(name: (name:
let builtin = builtins.${name}; in let builtin = builtins.${name}; in
" - `builtins.${name}` " + concatStringsSep " " (map (s: "*${s}*") builtin.args) "<dt><code>${name} "
+ " \n\n" + concatStringsSep " " (map (s: "<var>${s}</var>") builtin.args)
+ concatStrings (map (s: " ${s}\n") (splitLines builtin.doc)) + "\n\n" + "</code></dt>"
+ "<dd>\n\n"
+ builtin.doc
+ "\n\n</dd>"
) )
(attrNames builtins)) (attrNames builtins))
+17 -6
View File
@@ -1,7 +1,5 @@
ifeq ($(doc_generate),yes) ifeq ($(doc_generate),yes)
MANUAL_SRCS := $(call rwildcard, $(d)/src, *.md)
# Generate man pages. # Generate man pages.
man-pages := $(foreach n, \ man-pages := $(foreach n, \
nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 \ nix-env.1 nix-build.1 nix-shell.1 nix-store.1 nix-instantiate.1 \
@@ -64,6 +62,7 @@ $(d)/conf-file.json: $(bindir)/nix
$(d)/src/expressions/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/expressions/builtins-prefix.md $(bindir)/nix $(d)/src/expressions/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/expressions/builtins-prefix.md $(bindir)/nix
@cat doc/manual/src/expressions/builtins-prefix.md > $@.tmp @cat doc/manual/src/expressions/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp $(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp
@cat doc/manual/src/expressions/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@ @mv $@.tmp $@
$(d)/builtins.json: $(bindir)/nix $(d)/builtins.json: $(bindir)/nix
@@ -74,14 +73,26 @@ $(d)/builtins.json: $(bindir)/nix
install: $(docdir)/manual/index.html install: $(docdir)/manual/index.html
# Generate 'nix' manpages. # Generate 'nix' manpages.
install: $(d)/src/command-ref/new-cli install: $(mandir)/man1/nix3-manpages
man: doc/manual/generated/man1/nix3-manpages
all: doc/manual/generated/man1/nix3-manpages
$(mandir)/man1/nix3-manpages: doc/manual/generated/man1/nix3-manpages
@mkdir -p $$(dirname $@)
$(trace-install) install -m 0644 $$(dirname $<)/* $$(dirname $@)
doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
@mkdir -p $$(dirname $@)
$(trace-gen) for i in doc/manual/src/command-ref/new-cli/*.md; do \ $(trace-gen) for i in doc/manual/src/command-ref/new-cli/*.md; do \
name=$$(basename $$i .md); \ name=$$(basename $$i .md); \
tmpFile=$$(mktemp); \
if [[ $$name = SUMMARY ]]; then continue; fi; \ if [[ $$name = SUMMARY ]]; then continue; fi; \
printf "Title: %s\n\n" "$$name" > $$i.tmp; \ printf "Title: %s\n\n" "$$name" > $$tmpFile; \
cat $$i >> $$i.tmp; \ cat $$i >> $$tmpFile; \
lowdown -sT man -M section=1 $$i.tmp -o $(mandir)/man1/$$name.1; \ lowdown -sT man -M section=1 $$tmpFile -o $$(dirname $@)/$$name.1; \
rm $$tmpFile; \
done done
touch $@
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/command-ref/conf-file.md $(d)/src/expressions/builtins.md $(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/command-ref/conf-file.md $(d)/src/expressions/builtins.md
$(trace-gen) RUST_LOG=warn mdbook build doc/manual -d $(docdir)/manual $(trace-gen) RUST_LOG=warn mdbook build doc/manual -d $(docdir)/manual
@@ -9,7 +9,8 @@ scope. Instead, you can access them through the `builtins` built-in
value, which is a set that contains all built-in functions and values. value, which is a set that contains all built-in functions and values.
For instance, `derivation` is also available as `builtins.derivation`. For instance, `derivation` is also available as `builtins.derivation`.
- `derivation` *attrs*; `builtins.derivation` *attrs*\ <dl>
<dt><code>derivation <var>attrs</var></code>;
`derivation` is described in [its own section](derivations.md). <code>builtins.derivation <var>attrs</var></code></dt>
<dd><p><var>derivation</var> in described in
<a href="derivations.md">its own section</a>.</p></dd>
@@ -0,0 +1 @@
</dl>
+150 -87
View File
@@ -20,6 +20,8 @@
linuxSystems = linux64BitSystems ++ [ "i686-linux" ]; linuxSystems = linux64BitSystems ++ [ "i686-linux" ];
systems = linuxSystems ++ [ "x86_64-darwin" "aarch64-darwin" ]; systems = linuxSystems ++ [ "x86_64-darwin" "aarch64-darwin" ];
crossSystems = [ "armv6l-linux" "armv7l-linux" ];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system); forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
# Memoize nixpkgs for different platforms for efficiency. # Memoize nixpkgs for different platforms for efficiency.
@@ -79,7 +81,7 @@
buildPackages.mercurial buildPackages.mercurial
buildPackages.jq buildPackages.jq
] ]
++ lib.optionals stdenv.isLinux [(pkgs.util-linuxMinimal or pkgs.utillinuxMinimal)]; ++ lib.optionals stdenv.hostPlatform.isLinux [(buildPackages.util-linuxMinimal or buildPackages.utillinuxMinimal)];
buildDeps = buildDeps =
[ curl [ curl
@@ -87,13 +89,12 @@
openssl sqlite openssl sqlite
libarchive libarchive
boost boost
nlohmann_json
lowdown lowdown
gmock gmock
] ]
++ lib.optionals stdenv.isLinux [libseccomp] ++ lib.optionals stdenv.isLinux [libseccomp]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isx86_64 libcpuid; ++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid;
awsDeps = lib.optional (stdenv.isLinux || stdenv.isDarwin) awsDeps = lib.optional (stdenv.isLinux || stdenv.isDarwin)
(aws-sdk-cpp.override { (aws-sdk-cpp.override {
@@ -102,7 +103,13 @@
}); });
propagatedDeps = propagatedDeps =
[ (boehmgc.override { enableLargeConfig = true; }) [ ((boehmgc.override {
enableLargeConfig = true;
}).overrideAttrs(o: {
patches = (o.patches or []) ++ [
./boehmgc-coroutine-sp-fallback.diff
];
}))
]; ];
perlDeps = perlDeps =
@@ -133,10 +140,11 @@
substitute ${./scripts/install.in} $out/install \ substitute ${./scripts/install.in} $out/install \
${pkgs.lib.concatMapStrings ${pkgs.lib.concatMapStrings
(system: (system: let
'' \ tarball = if builtins.elem system crossSystems then self.hydraJobs.binaryTarballCross.x86_64-linux.${system} else self.hydraJobs.binaryTarball.${system};
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${self.hydraJobs.binaryTarball.${system}}/*.tar.xz) \ in '' \
--replace '@tarballPath_${system}@' $(tarballPath ${self.hydraJobs.binaryTarball.${system}}/*.tar.xz) \ --replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
'' ''
) )
systems systems
@@ -174,6 +182,77 @@
}; };
binaryTarball = buildPackages: nix: pkgs: let
inherit (pkgs) cacert;
installerClosureInfo = buildPackages.closureInfo { rootPaths = [ nix cacert ]; };
in
buildPackages.runCommand "nix-binary-tarball-${version}"
{ #nativeBuildInputs = lib.optional (system != "aarch64-linux") shellcheck;
meta.description = "Distribution-independent Nix bootstrap binaries for ${pkgs.system}";
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
cp ${./scripts/create-darwin-volume.sh} $TMPDIR/create-darwin-volume.sh
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
# SC1090: Don't worry about not being able to find
# $nix/etc/profile.d/nix.sh
shellcheck --exclude SC1090 $TMPDIR/install
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
# SC1091: Don't panic about not being able to source
# /etc/profile
# SC2002: Ignore "useless cat" "error", when loading
# .reginfo, as the cat is a much cleaner
# implementation, even though it is "useless"
# SC2116: Allow ROOT_HOME=$(echo ~root) for resolving
# root's home directory
shellcheck --external-sources \
--exclude SC1091,SC2002,SC2116 $TMPDIR/install-multi-user
fi
chmod +x $TMPDIR/install
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${pkgs.system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install \
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
'';
in { in {
# A Nixpkgs overlay that overrides the 'nix' and # A Nixpkgs overlay that overrides the 'nix' and
@@ -254,7 +333,6 @@
xz xz
pkgs.perl pkgs.perl
boost boost
nlohmann_json
] ]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium ++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security; ++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
@@ -285,7 +363,7 @@
outputs = [ "out" "bin" "dev" ]; outputs = [ "out" "bin" "dev" ];
nativeBuildInputs = [ which ]; nativeBuildInputs = [ buildPackages.which ];
configurePhase = '' configurePhase = ''
${if (stdenv.isDarwin && stdenv.isAarch64) then "echo \"HAVE_SANDBOX_INIT=false\" > configure.local" else ""} ${if (stdenv.isDarwin && stdenv.isAarch64) then "echo \"HAVE_SANDBOX_INIT=false\" > configure.local" else ""}
@@ -304,92 +382,33 @@
buildStatic = nixpkgs.lib.genAttrs linux64BitSystems (system: self.packages.${system}.nix-static); buildStatic = nixpkgs.lib.genAttrs linux64BitSystems (system: self.packages.${system}.nix-static);
buildCross = nixpkgs.lib.genAttrs crossSystems (crossSystem:
nixpkgs.lib.genAttrs ["x86_64-linux"] (system: self.packages.${system}."nix-${crossSystem}"));
# Perl bindings for various platforms. # Perl bindings for various platforms.
perlBindings = nixpkgs.lib.genAttrs systems (system: self.packages.${system}.nix.perl-bindings); perlBindings = nixpkgs.lib.genAttrs systems (system: self.packages.${system}.nix.perl-bindings);
# Binary tarball for various platforms, containing a Nix store # Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of # with the closure of 'nix' package, and the second half of
# the installation script. # the installation script.
binaryTarball = nixpkgs.lib.genAttrs systems (system: binaryTarball = nixpkgs.lib.genAttrs systems (system: binaryTarball nixpkgsFor.${system} nixpkgsFor.${system}.nix nixpkgsFor.${system});
with nixpkgsFor.${system}; binaryTarballCross = nixpkgs.lib.genAttrs ["x86_64-linux"] (system: builtins.listToAttrs (map (crossSystem: {
name = crossSystem;
let value = let
installerClosureInfo = closureInfo { rootPaths = [ nix cacert ]; }; nixpkgsCross = import nixpkgs {
in inherit system crossSystem;
overlays = [ self.overlay ];
runCommand "nix-binary-tarball-${version}" };
{ #nativeBuildInputs = lib.optional (system != "aarch64-linux") shellcheck; in binaryTarball nixpkgsFor.${system} self.packages.${system}."nix-${crossSystem}" nixpkgsCross;
meta.description = "Distribution-independent Nix bootstrap binaries for ${system}"; }) crossSystems));
}
''
cp ${installerClosureInfo}/registration $TMPDIR/reginfo
cp ${./scripts/create-darwin-volume.sh} $TMPDIR/create-darwin-volume.sh
substitute ${./scripts/install-nix-from-closure.sh} $TMPDIR/install \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-darwin-multi-user.sh} $TMPDIR/install-darwin-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${./scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
# SC1090: Don't worry about not being able to find
# $nix/etc/profile.d/nix.sh
shellcheck --exclude SC1090 $TMPDIR/install
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
# SC1091: Don't panic about not being able to source
# /etc/profile
# SC2002: Ignore "useless cat" "error", when loading
# .reginfo, as the cat is a much cleaner
# implementation, even though it is "useless"
# SC2116: Allow ROOT_HOME=$(echo ~root) for resolving
# root's home directory
shellcheck --external-sources \
--exclude SC1091,SC2002,SC2116 $TMPDIR/install-multi-user
fi
chmod +x $TMPDIR/install
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${system}
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--absolute-names \
--hard-dereference \
--transform "s,$TMPDIR/install,$dir/install," \
--transform "s,$TMPDIR/create-darwin-volume.sh,$dir/create-darwin-volume.sh," \
--transform "s,$TMPDIR/reginfo,$dir/.reginfo," \
--transform "s,$NIX_STORE,$dir/store,S" \
$TMPDIR/install \
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
'');
# The first half of the installation script. This is uploaded # The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary # to https://nixos.org/nix/install. It downloads the binary
# tarball for the user's system and calls the second half of the # tarball for the user's system and calls the second half of the
# installation script. # installation script.
installerScript = installScriptFor [ "x86_64-linux" "i686-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ]; installerScript = installScriptFor [ "x86_64-linux" "i686-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" "armv6l-linux" "armv7l-linux" ];
installerScriptForGHA = installScriptFor [ "x86_64-linux" "x86_64-darwin" ]; installerScriptForGHA = installScriptFor [ "x86_64-linux" "x86_64-darwin" "armv6l-linux" "armv7l-linux"];
# Line coverage analysis. # Line coverage analysis.
coverage = coverage =
@@ -483,11 +502,14 @@
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work # `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable; # againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out"; } "touch $out";
}); } // (if system == "x86_64-linux" then (builtins.listToAttrs (map (crossSystem: {
name = "binaryTarball-${crossSystem}";
value = self.hydraJobs.binaryTarballCross.${system}.${crossSystem};
}) crossSystems)) else {}));
packages = forAllSystems (system: { packages = forAllSystems (system: {
inherit (nixpkgsFor.${system}) nix; inherit (nixpkgsFor.${system}) nix;
} // nixpkgs.lib.optionalAttrs (builtins.elem system linux64BitSystems) { } // (nixpkgs.lib.optionalAttrs (builtins.elem system linux64BitSystems) {
nix-static = let nix-static = let
nixpkgs = nixpkgsFor.${system}.pkgsStatic; nixpkgs = nixpkgsFor.${system}.pkgsStatic;
in with commonDeps nixpkgs; nixpkgs.stdenv.mkDerivation { in with commonDeps nixpkgs; nixpkgs.stdenv.mkDerivation {
@@ -525,8 +547,49 @@
stripAllList = ["bin"]; stripAllList = ["bin"];
strictDeps = true; strictDeps = true;
hardeningDisable = [ "pie" ];
}; };
}); } // builtins.listToAttrs (map (crossSystem: {
name = "nix-${crossSystem}";
value = let
nixpkgsCross = import nixpkgs {
inherit system crossSystem;
overlays = [ self.overlay ];
};
in with commonDeps nixpkgsCross; nixpkgsCross.stdenv.mkDerivation {
name = "nix-${version}";
src = self;
VERSION_SUFFIX = versionSuffix;
outputs = [ "out" "dev" "doc" ];
nativeBuildInputs = nativeBuildDeps;
buildInputs = buildDeps ++ propagatedDeps;
configureFlags = [ "--sysconfdir=/etc" "--disable-doc-gen" ];
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d";
doCheck = true;
installFlags = "sysconfdir=$(out)/etc";
postInstall = ''
mkdir -p $doc/nix-support
echo "doc manual $doc/share/doc/nix/manual" >> $doc/nix-support/hydra-build-products
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
'';
doInstallCheck = true;
installCheckFlags = "sysconfdir=$(out)/etc";
};
}) crossSystems)));
defaultPackage = forAllSystems (system: self.packages.${system}.nix); defaultPackage = forAllSystems (system: self.packages.${system}.nix);
+4
View File
@@ -110,6 +110,9 @@ downloadFile("binaryTarball.i686-linux", "1");
downloadFile("binaryTarball.x86_64-linux", "1"); downloadFile("binaryTarball.x86_64-linux", "1");
downloadFile("binaryTarball.aarch64-linux", "1"); downloadFile("binaryTarball.aarch64-linux", "1");
downloadFile("binaryTarball.x86_64-darwin", "1"); downloadFile("binaryTarball.x86_64-darwin", "1");
downloadFile("binaryTarball.aarch64-darwin", "1");
downloadFile("binaryTarballCross.x86_64-linux.armv6l-linux", "1");
downloadFile("binaryTarballCross.x86_64-linux.armv7l-linux", "1");
downloadFile("installerScript", "1"); downloadFile("installerScript", "1");
for my $fn (glob "$tmpDir/*") { for my $fn (glob "$tmpDir/*") {
@@ -153,6 +156,7 @@ write_file("$nixpkgsDir/nixos/modules/installer/tools/nix-fallback-paths.nix",
" i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" . " i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" . " aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" . " x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" .
" aarch64-darwin = \"" . getStorePath("build.aarch64-darwin") . "\";\n" .
"}\n"); "}\n");
system("cd $nixpkgsDir && git commit -a -m 'nix-fallback-paths.nix: Update to $version'") == 0 or die; system("cd $nixpkgsDir && git commit -a -m 'nix-fallback-paths.nix: Update to $version'") == 0 or die;
+37
View File
@@ -0,0 +1,37 @@
function _nix_complete
# Get the current command up to a cursor.
# - Behaves correctly even with pipes and nested in commands like env.
# - TODO: Returns the command verbatim (does not interpolate variables).
# That might not be optimal for arguments like -f.
set -l nix_args (commandline --current-process --tokenize --cut-at-cursor)
# --cut-at-cursor with --tokenize removes the current token so we need to add it separately.
# https://github.com/fish-shell/fish-shell/issues/7375
# Can be an empty string.
set -l current_token (commandline --current-token --cut-at-cursor)
# Nix wants the index of the argv item to complete but the $nix_args variable
# also contains the program name (argv[0]) so we would need to subtract 1.
# But the variable also misses the current token so it cancels out.
set -l nix_arg_to_complete (count $nix_args)
env NIX_GET_COMPLETIONS=$nix_arg_to_complete $nix_args $current_token
end
function _nix_accepts_files
set -l response (_nix_complete)
# First line is either filenames or no-filenames.
test $response[1] = 'filenames'
end
function _nix
set -l response (_nix_complete)
# Skip the first line since it handled by _nix_accepts_files.
# Tail lines each contain a command followed by a tab character and, optionally, a description.
# This is also the format fish expects.
string collect -- $response[2..-1]
end
# Disable file path completion if paths do not belong in the current context.
complete --command nix --condition 'not _nix_accepts_files' --no-files
complete --command nix --arguments '(_nix)'
+1
View File
@@ -0,0 +1 @@
$(eval $(call install-file-as, $(d)/completion.fish, $(datarootdir)/fish/vendor_completions.d/nix.fish, 0644))
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
$(eval $(call install-data-in, $(d)/org.nixos.nix-daemon.plist, $(prefix)/Library/LaunchDaemons)) $(eval $(call install-data-in, $(d)/org.nixos.nix-daemon.plist, $(prefix)/Library/LaunchDaemons))
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Linux) ifdef HOST_LINUX
$(foreach n, nix-daemon.socket nix-daemon.service, $(eval $(call install-file-in, $(d)/$(n), $(prefix)/lib/systemd/system, 0644))) $(foreach n, nix-daemon.socket nix-daemon.service, $(eval $(call install-file-in, $(d)/$(n), $(prefix)/lib/systemd/system, 0644)))
+1 -1
View File
@@ -1,4 +1,4 @@
ifeq ($(OS), Linux) ifdef HOST_LINUX
$(foreach n, nix-daemon.conf, $(eval $(call install-file-in, $(d)/$(n), $(sysconfdir)/init, 0644))) $(foreach n, nix-daemon.conf, $(eval $(call install-file-in, $(d)/$(n), $(sysconfdir)/init, 0644)))
+22 -5
View File
@@ -10,8 +10,25 @@ bin-scripts :=
noinst-scripts := noinst-scripts :=
man-pages := man-pages :=
install-tests := install-tests :=
OS = $(shell uname -s)
ifdef HOST_OS
HOST_KERNEL = $(firstword $(subst -, ,$(HOST_OS)))
ifeq ($(HOST_KERNEL), cygwin)
HOST_CYGWIN = 1
endif
ifeq ($(patsubst darwin%,,$(HOST_KERNEL)),)
HOST_DARWIN = 1
endif
ifeq ($(patsubst freebsd%,,$(HOST_KERNEL)),)
HOST_FREEBSD = 1
endif
ifeq ($(HOST_KERNEL), linux)
HOST_LINUX = 1
endif
ifeq ($(patsubst solaris%,,$(HOST_KERNEL)),)
HOST_SOLARIS = 1
endif
endif
# Hack to define a literal space. # Hack to define a literal space.
space := space :=
@@ -50,16 +67,16 @@ endif
BUILD_SHARED_LIBS ?= 1 BUILD_SHARED_LIBS ?= 1
ifeq ($(BUILD_SHARED_LIBS), 1) ifeq ($(BUILD_SHARED_LIBS), 1)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS))) ifdef HOST_CYGWIN
GLOBAL_CFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE GLOBAL_CFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
GLOBAL_CXXFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE GLOBAL_CXXFLAGS += -U__STRICT_ANSI__ -D_GNU_SOURCE
else else
GLOBAL_CFLAGS += -fPIC GLOBAL_CFLAGS += -fPIC
GLOBAL_CXXFLAGS += -fPIC GLOBAL_CXXFLAGS += -fPIC
endif endif
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
ifneq ($(OS), SunOS) ifndef HOST_SOLARIS
ifneq ($(OS), FreeBSD) ifndef HOST_FREEBSD
GLOBAL_LDFLAGS += -Wl,--no-copy-dt-needed-entries GLOBAL_LDFLAGS += -Wl,--no-copy-dt-needed-entries
endif endif
endif endif
+9 -9
View File
@@ -1,9 +1,9 @@
libs-list := libs-list :=
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
SO_EXT = dylib SO_EXT = dylib
else else
ifeq (CYGWIN,$(findstring CYGWIN,$(OS))) ifdef HOST_CYGWIN
SO_EXT = dll SO_EXT = dll
else else
SO_EXT = so SO_EXT = so
@@ -59,7 +59,7 @@ define build-library
$(1)_OBJS := $$(addprefix $(buildprefix), $$(addsuffix .o, $$(basename $$(_srcs)))) $(1)_OBJS := $$(addprefix $(buildprefix), $$(addsuffix .o, $$(basename $$(_srcs))))
_libs := $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_PATH)) _libs := $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_PATH))
ifeq (CYGWIN,$(findstring CYGWIN,$(OS))) ifdef HOST_CYGWIN
$(1)_INSTALL_DIR ?= $$(bindir) $(1)_INSTALL_DIR ?= $$(bindir)
else else
$(1)_INSTALL_DIR ?= $$(libdir) $(1)_INSTALL_DIR ?= $$(libdir)
@@ -73,18 +73,18 @@ define build-library
ifeq ($(BUILD_SHARED_LIBS), 1) ifeq ($(BUILD_SHARED_LIBS), 1)
ifdef $(1)_ALLOW_UNDEFINED ifdef $(1)_ALLOW_UNDEFINED
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
$(1)_LDFLAGS += -undefined suppress -flat_namespace $(1)_LDFLAGS += -undefined suppress -flat_namespace
endif endif
else else
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
ifneq (CYGWIN,$(findstring CYGWIN,$(OS))) ifndef HOST_CYGWIN
$(1)_LDFLAGS += -Wl,-z,defs $(1)_LDFLAGS += -Wl,-z,defs
endif endif
endif endif
endif endif
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
$(1)_LDFLAGS += -Wl,-soname=$$($(1)_NAME).$(SO_EXT) $(1)_LDFLAGS += -Wl,-soname=$$($(1)_NAME).$(SO_EXT)
endif endif
@@ -93,7 +93,7 @@ define build-library
$$($(1)_PATH): $$($(1)_OBJS) $$(_libs) | $$(_d)/ $$($(1)_PATH): $$($(1)_OBJS) $$(_libs) | $$(_d)/
$$(trace-ld) $(CXX) -o $$(abspath $$@) -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE)) $$($(1)_LDFLAGS_UNINSTALLED) $$(trace-ld) $(CXX) -o $$(abspath $$@) -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE)) $$($(1)_LDFLAGS_UNINSTALLED)
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
$(1)_LDFLAGS_USE += -Wl,-rpath,$$(abspath $$(_d)) $(1)_LDFLAGS_USE += -Wl,-rpath,$$(abspath $$(_d))
endif endif
$(1)_LDFLAGS_USE += -L$$(_d) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME))) $(1)_LDFLAGS_USE += -L$$(_d) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME)))
@@ -108,7 +108,7 @@ define build-library
$$(trace-ld) $(CXX) -o $$@ -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED)) $$(trace-ld) $(CXX) -o $$@ -shared $$(LDFLAGS) $$(GLOBAL_LDFLAGS) $$($(1)_OBJS) $$($(1)_LDFLAGS) $$($(1)_LDFLAGS_PROPAGATED) $$(foreach lib, $$($(1)_LIBS), $$($$(lib)_LDFLAGS_USE_INSTALLED))
$(1)_LDFLAGS_USE_INSTALLED += -L$$(DESTDIR)$$($(1)_INSTALL_DIR) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME))) $(1)_LDFLAGS_USE_INSTALLED += -L$$(DESTDIR)$$($(1)_INSTALL_DIR) -l$$(patsubst lib%,%,$$(strip $$($(1)_NAME)))
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
ifeq ($(SET_RPATH_TO_LIBS), 1) ifeq ($(SET_RPATH_TO_LIBS), 1)
$(1)_LDFLAGS_USE_INSTALLED += -Wl,-rpath,$$($(1)_INSTALL_DIR) $(1)_LDFLAGS_USE_INSTALLED += -Wl,-rpath,$$($(1)_INSTALL_DIR)
else else
+4 -4
View File
@@ -11,12 +11,12 @@ libnixrust_INSTALL_PATH := $(libdir)/libnixrust.$(SO_EXT)
libnixrust_LDFLAGS_USE := -L$(d)/target/$(RUST_DIR) -lnixrust libnixrust_LDFLAGS_USE := -L$(d)/target/$(RUST_DIR) -lnixrust
libnixrust_LDFLAGS_USE_INSTALLED := -L$(libdir) -lnixrust libnixrust_LDFLAGS_USE_INSTALLED := -L$(libdir) -lnixrust
ifeq ($(OS), Linux) ifdef HOST_LINUX
libnixrust_LDFLAGS_USE += -ldl libnixrust_LDFLAGS_USE += -ldl
libnixrust_LDFLAGS_USE_INSTALLED += -ldl libnixrust_LDFLAGS_USE_INSTALLED += -ldl
endif endif
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
libnixrust_BUILD_FLAGS = NIX_LDFLAGS="-undefined dynamic_lookup" libnixrust_BUILD_FLAGS = NIX_LDFLAGS="-undefined dynamic_lookup"
else else
libnixrust_LDFLAGS_USE += -Wl,-rpath,$(abspath $(d)/target/$(RUST_DIR)) libnixrust_LDFLAGS_USE += -Wl,-rpath,$(abspath $(d)/target/$(RUST_DIR))
@@ -31,7 +31,7 @@ $(libnixrust_PATH): $(call rwildcard, $(d)/src, *.rs) $(d)/Cargo.toml
$(libnixrust_INSTALL_PATH): $(libnixrust_PATH) $(libnixrust_INSTALL_PATH): $(libnixrust_PATH)
$(target-gen) cp $^ $@ $(target-gen) cp $^ $@
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
install_name_tool -id $@ $@ install_name_tool -id $@ $@
endif endif
@@ -40,7 +40,7 @@ clean: clean-rust
clean-rust: clean-rust:
$(suppress) rm -rfv nix-rust/target $(suppress) rm -rfv nix-rust/target
ifneq ($(OS), Darwin) ifndef HOST_DARWIN
check: rust-tests check: rust-tests
rust-tests: rust-tests:
+1 -1
View File
@@ -1,6 +1,6 @@
makefiles = local.mk makefiles = local.mk
GLOBAL_CXXFLAGS += -g -Wall -std=c++17 GLOBAL_CXXFLAGS += -g -Wall -std=c++17 -I ../src
-include Makefile.config -include Makefile.config
+1
View File
@@ -1,3 +1,4 @@
HOST_OS = @host_os@
CC = @CC@ CC = @CC@
CFLAGS = @CFLAGS@ CFLAGS = @CFLAGS@
CXX = @CXX@ CXX = @CXX@
+2
View File
@@ -7,6 +7,8 @@ CXXFLAGS=
AC_PROG_CC AC_PROG_CC
AC_PROG_CXX AC_PROG_CXX
AC_CANONICAL_HOST
# Use 64-bit file system calls so that we can support files > 2 GiB. # Use 64-bit file system calls so that we can support files > 2 GiB.
AC_SYS_LARGEFILE AC_SYS_LARGEFILE
+1 -1
View File
@@ -28,7 +28,7 @@ Store_CXXFLAGS = \
Store_LDFLAGS := $(SODIUM_LIBS) $(NIX_LIBS) Store_LDFLAGS := $(SODIUM_LIBS) $(NIX_LIBS)
ifeq (CYGWIN,$(findstring CYGWIN,$(OS))) ifdef HOST_CYGWIN
archlib = $(shell perl -E 'use Config; print $$Config{archlib};') archlib = $(shell perl -E 'use Config; print $$Config{archlib};')
libperl = $(shell perl -E 'use Config; print $$Config{libperl};') libperl = $(shell perl -E 'use Config; print $$Config{libperl};')
Store_LDFLAGS += $(shell find ${archlib} -name ${libperl}) Store_LDFLAGS += $(shell find ${archlib} -name ${libperl})
+2 -3
View File
@@ -106,12 +106,11 @@ while [ $# -gt 0 ]; do
echo "" echo ""
echo " --no-channel-add: Don't add any channels. nixpkgs-unstable is installed by default." echo " --no-channel-add: Don't add any channels. nixpkgs-unstable is installed by default."
echo "" echo ""
echo " --no-modify-profile: Skip channel installation. When not provided nixpkgs-unstable" echo " --no-modify-profile: Don't modify the user profile to automatically load nix."
echo " is installed by default."
echo "" echo ""
echo " --daemon-user-count: Number of build users to create. Defaults to 32." echo " --daemon-user-count: Number of build users to create. Defaults to 32."
echo "" echo ""
echo " --nix-extra-conf-file: Path to nix.conf to prepend when installing /etc/nix.conf" echo " --nix-extra-conf-file: Path to nix.conf to prepend when installing /etc/nix/nix.conf"
echo "" echo ""
if [ -n "${INVOKED_FROM_INSTALL_IN:-}" ]; then if [ -n "${INVOKED_FROM_INSTALL_IN:-}" ]; then
echo " --tarball-url-prefix URL: Base URL to download the Nix tarball from." echo " --tarball-url-prefix URL: Base URL to download the Nix tarball from."
+11 -1
View File
@@ -40,13 +40,23 @@ case "$(uname -s).$(uname -m)" in
path=@tarballPath_aarch64-linux@ path=@tarballPath_aarch64-linux@
system=aarch64-linux system=aarch64-linux
;; ;;
Linux.armv6l_linux)
hash=@tarballHash_armv6l-linux@
path=@tarballPath_armv6l-linux@
system=armv6l-linux
;;
Linux.armv7l_linux)
hash=@tarballHash_armv7l-linux@
path=@tarballPath_armv7l-linux@
system=armv7l-linux
;;
Darwin.x86_64) Darwin.x86_64)
hash=@tarballHash_x86_64-darwin@ hash=@tarballHash_x86_64-darwin@
path=@tarballPath_x86_64-darwin@ path=@tarballPath_x86_64-darwin@
system=x86_64-darwin system=x86_64-darwin
;; ;;
Darwin.arm64|Darwin.aarch64) Darwin.arm64|Darwin.aarch64)
hash=@binaryTarball_aarch64-darwin@ hash=@tarballHash_aarch64-darwin@
path=@tarballPath_aarch64-darwin@ path=@tarballPath_aarch64-darwin@
system=aarch64-darwin system=aarch64-darwin
;; ;;
+2 -2
View File
@@ -270,7 +270,7 @@ connected:
{ {
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri)); Activity act(*logger, lvlTalkative, actUnknown, fmt("copying dependencies to '%s'", storeUri));
copyPaths(store, ref<Store>(sshStore), store->parseStorePathSet(inputs), NoRepair, NoCheckSigs, substitute); copyPaths(*store, *sshStore, store->parseStorePathSet(inputs), NoRepair, NoCheckSigs, substitute);
} }
uploadLock = -1; uploadLock = -1;
@@ -321,7 +321,7 @@ connected:
if (auto localStore = store.dynamic_pointer_cast<LocalStore>()) if (auto localStore = store.dynamic_pointer_cast<LocalStore>())
for (auto & path : missingPaths) for (auto & path : missingPaths)
localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */ localStore->locksHeld.insert(store->printStorePath(path)); /* FIXME: ugly */
copyPaths(ref<Store>(sshStore), store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute); copyPaths(*sshStore, *store, missingPaths, NoRepair, NoCheckSigs, NoSubstitute);
} }
// XXX: Should be done as part of `copyPaths` // XXX: Should be done as part of `copyPaths`
for (auto & realisation : missingRealisations) { for (auto & realisation : missingRealisations) {
+26 -2
View File
@@ -54,6 +54,30 @@ void StoreCommand::run()
run(getStore()); run(getStore());
} }
EvalCommand::EvalCommand()
{
}
EvalCommand::~EvalCommand()
{
if (evalState)
evalState->printStats();
}
ref<Store> EvalCommand::getEvalStore()
{
if (!evalStore)
evalStore = evalStoreUrl ? openStore(*evalStoreUrl) : getStore();
return ref<Store>(evalStore);
}
ref<EvalState> EvalCommand::getEvalState()
{
if (!evalState)
evalState = std::make_shared<EvalState>(searchPath, getEvalStore(), getStore());
return ref<EvalState>(evalState);
}
BuiltPathsCommand::BuiltPathsCommand(bool recursive) BuiltPathsCommand::BuiltPathsCommand(bool recursive)
: recursive(recursive) : recursive(recursive)
{ {
@@ -91,7 +115,7 @@ void BuiltPathsCommand::run(ref<Store> store)
for (auto & p : store->queryAllValidPaths()) for (auto & p : store->queryAllValidPaths())
paths.push_back(BuiltPath::Opaque{p}); paths.push_back(BuiltPath::Opaque{p});
} else { } else {
paths = toBuiltPaths(store, realiseMode, operateOn, installables); paths = toBuiltPaths(getEvalStore(), store, realiseMode, operateOn, installables);
if (recursive) { if (recursive) {
// XXX: This only computes the store path closure, ignoring // XXX: This only computes the store path closure, ignoring
// intermediate realisations // intermediate realisations
@@ -188,7 +212,7 @@ void MixProfile::updateProfile(const BuiltPaths & buildables)
} }
if (result.size() != 1) if (result.size() != 1)
throw Error("'--profile' requires that the arguments produce a single store path, but there are %d", result.size()); throw UsageError("'--profile' requires that the arguments produce a single store path, but there are %d", result.size());
updateProfile(result[0]); updateProfile(result[0]);
} }
+22 -8
View File
@@ -45,11 +45,18 @@ private:
struct EvalCommand : virtual StoreCommand, MixEvalArgs struct EvalCommand : virtual StoreCommand, MixEvalArgs
{ {
ref<EvalState> getEvalState(); EvalCommand();
std::shared_ptr<EvalState> evalState;
~EvalCommand(); ~EvalCommand();
ref<Store> getEvalStore();
ref<EvalState> getEvalState();
private:
std::shared_ptr<Store> evalStore;
std::shared_ptr<EvalState> evalState;
}; };
struct MixFlakeOptions : virtual Args, EvalCommand struct MixFlakeOptions : virtual Args, EvalCommand
@@ -216,15 +223,21 @@ static RegisterCommand registerCommand2(std::vector<std::string> && name)
return RegisterCommand(std::move(name), [](){ return make_ref<T>(); }); return RegisterCommand(std::move(name), [](){ return make_ref<T>(); });
} }
BuiltPaths build(ref<Store> store, Realise mode, BuiltPaths build(ref<Store> evalStore, ref<Store> store, Realise mode,
std::vector<std::shared_ptr<Installable>> installables, BuildMode bMode = bmNormal); std::vector<std::shared_ptr<Installable>> installables, BuildMode bMode = bmNormal);
std::set<StorePath> toStorePaths(ref<Store> store, std::set<StorePath> toStorePaths(
Realise mode, OperateOn operateOn, ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::vector<std::shared_ptr<Installable>> installables); std::vector<std::shared_ptr<Installable>> installables);
StorePath toStorePath(ref<Store> store, StorePath toStorePath(
Realise mode, OperateOn operateOn, ref<Store> evalStore,
ref<Store> store,
Realise mode,
OperateOn operateOn,
std::shared_ptr<Installable> installable); std::shared_ptr<Installable> installable);
std::set<StorePath> toDerivations(ref<Store> store, std::set<StorePath> toDerivations(ref<Store> store,
@@ -232,6 +245,7 @@ std::set<StorePath> toDerivations(ref<Store> store,
bool useDeriver = false); bool useDeriver = false);
BuiltPaths toBuiltPaths( BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store, ref<Store> store,
Realise mode, Realise mode,
OperateOn operateOn, OperateOn operateOn,
+75 -42
View File
@@ -58,9 +58,13 @@ MixFlakeOptions::MixFlakeOptions()
addFlag({ addFlag({
.longName = "no-registries", .longName = "no-registries",
.description = "Don't allow lookups in the flake registries.", .description =
"Don't allow lookups in the flake registries. This option is deprecated; use `--no-use-registries`.",
.category = category, .category = category,
.handler = {&lockFlags.useRegistries, false} .handler = {[&]() {
lockFlags.useRegistries = false;
warn("'--no-registries' is deprecated; use '--no-use-registries'");
}}
}); });
addFlag({ addFlag({
@@ -171,14 +175,50 @@ Strings SourceExprCommand::getDefaultFlakeAttrPathPrefixes()
void SourceExprCommand::completeInstallable(std::string_view prefix) void SourceExprCommand::completeInstallable(std::string_view prefix)
{ {
if (file) return; // FIXME if (file) {
evalSettings.pureEval = false;
auto state = getEvalState();
Expr *e = state->parseExprFromFile(
resolveExprPath(state->checkSourcePath(lookupFileArg(*state, *file)))
);
completeFlakeRefWithFragment( Value root;
getEvalState(), state->eval(e, root);
lockFlags,
getDefaultFlakeAttrPathPrefixes(), auto autoArgs = getAutoArgs(*state);
getDefaultFlakeAttrPaths(),
prefix); std::string prefix_ = std::string(prefix);
auto sep = prefix_.rfind('.');
std::string searchWord;
if (sep != std::string::npos) {
searchWord = prefix_.substr(sep, std::string::npos);
prefix_ = prefix_.substr(0, sep);
} else {
searchWord = prefix_;
prefix_ = "";
}
Value &v1(*findAlongAttrPath(*state, prefix_, *autoArgs, root).first);
state->forceValue(v1);
Value v2;
state->autoCallFunction(*autoArgs, v1, v2);
if (v2.type() == nAttrs) {
for (auto & i : *v2.attrs) {
std::string name = i.name;
if (name.find(searchWord) == 0) {
completions->add(i.name);
}
}
}
} else {
completeFlakeRefWithFragment(
getEvalState(),
lockFlags,
getDefaultFlakeAttrPathPrefixes(),
getDefaultFlakeAttrPaths(),
prefix);
}
} }
void completeFlakeRefWithFragment( void completeFlakeRefWithFragment(
@@ -249,19 +289,6 @@ void completeFlakeRefWithFragment(
completeFlakeRef(evalState->store, prefix); completeFlakeRef(evalState->store, prefix);
} }
ref<EvalState> EvalCommand::getEvalState()
{
if (!evalState)
evalState = std::make_shared<EvalState>(searchPath, getStore());
return ref<EvalState>(evalState);
}
EvalCommand::~EvalCommand()
{
if (evalState)
evalState->printStats();
}
void completeFlakeRef(ref<Store> store, std::string_view prefix) void completeFlakeRef(ref<Store> store, std::string_view prefix)
{ {
if (prefix == "") if (prefix == "")
@@ -351,6 +378,7 @@ DerivedPaths InstallableValue::toDerivedPaths()
DerivedPaths res; DerivedPaths res;
std::map<StorePath, std::set<std::string>> drvsToOutputs; std::map<StorePath, std::set<std::string>> drvsToOutputs;
RealisedPath::Set drvsToCopy;
// Group by derivation, helps with .all in particular // Group by derivation, helps with .all in particular
for (auto & drv : toDerivations()) { for (auto & drv : toDerivations()) {
@@ -358,6 +386,7 @@ DerivedPaths InstallableValue::toDerivedPaths()
if (outputName == "") if (outputName == "")
throw Error("derivation '%s' lacks an 'outputName' attribute", state->store->printStorePath(drv.drvPath)); throw Error("derivation '%s' lacks an 'outputName' attribute", state->store->printStorePath(drv.drvPath));
drvsToOutputs[drv.drvPath].insert(outputName); drvsToOutputs[drv.drvPath].insert(outputName);
drvsToCopy.insert(drv.drvPath);
} }
for (auto & i : drvsToOutputs) for (auto & i : drvsToOutputs)
@@ -573,10 +602,10 @@ InstallableFlake::getCursors(EvalState & state)
std::shared_ptr<flake::LockedFlake> InstallableFlake::getLockedFlake() const std::shared_ptr<flake::LockedFlake> InstallableFlake::getLockedFlake() const
{ {
flake::LockFlags lockFlagsApplyConfig = lockFlags;
lockFlagsApplyConfig.applyNixConfig = true;
if (!_lockedFlake) { if (!_lockedFlake) {
_lockedFlake = std::make_shared<flake::LockedFlake>(lockFlake(*state, flakeRef, lockFlags)); _lockedFlake = std::make_shared<flake::LockedFlake>(lockFlake(*state, flakeRef, lockFlagsApplyConfig));
_lockedFlake->flake.config.apply();
// FIXME: send new config to the daemon.
} }
return _lockedFlake; return _lockedFlake;
} }
@@ -672,10 +701,10 @@ std::shared_ptr<Installable> SourceExprCommand::parseInstallable(
return installables.front(); return installables.front();
} }
BuiltPaths getBuiltPaths(ref<Store> store, DerivedPaths hopefullyBuiltPaths) BuiltPaths getBuiltPaths(ref<Store> evalStore, ref<Store> store, const DerivedPaths & hopefullyBuiltPaths)
{ {
BuiltPaths res; BuiltPaths res;
for (auto& b : hopefullyBuiltPaths) for (auto & b : hopefullyBuiltPaths)
std::visit( std::visit(
overloaded{ overloaded{
[&](DerivedPath::Opaque bo) { [&](DerivedPath::Opaque bo) {
@@ -683,14 +712,13 @@ BuiltPaths getBuiltPaths(ref<Store> store, DerivedPaths hopefullyBuiltPaths)
}, },
[&](DerivedPath::Built bfd) { [&](DerivedPath::Built bfd) {
OutputPathMap outputs; OutputPathMap outputs;
auto drv = store->readDerivation(bfd.drvPath); auto drv = evalStore->readDerivation(bfd.drvPath);
auto outputHashes = staticOutputHashes(*store, drv); auto outputHashes = staticOutputHashes(*evalStore, drv); // FIXME: expensive
auto drvOutputs = drv.outputsAndOptPaths(*store); auto drvOutputs = drv.outputsAndOptPaths(*store);
for (auto& output : bfd.outputs) { for (auto & output : bfd.outputs) {
if (!outputHashes.count(output)) if (!outputHashes.count(output))
throw Error( throw Error(
"the derivation '%s' doesn't have an output " "the derivation '%s' doesn't have an output named '%s'",
"named '%s'",
store->printStorePath(bfd.drvPath), output); store->printStorePath(bfd.drvPath), output);
if (settings.isExperimentalFeatureEnabled( if (settings.isExperimentalFeatureEnabled(
"ca-derivations")) { "ca-derivations")) {
@@ -722,7 +750,7 @@ BuiltPaths getBuiltPaths(ref<Store> store, DerivedPaths hopefullyBuiltPaths)
return res; return res;
} }
BuiltPaths build(ref<Store> store, Realise mode, BuiltPaths build(ref<Store> evalStore, ref<Store> store, Realise mode,
std::vector<std::shared_ptr<Installable>> installables, BuildMode bMode) std::vector<std::shared_ptr<Installable>> installables, BuildMode bMode)
{ {
if (mode == Realise::Nothing) if (mode == Realise::Nothing)
@@ -738,20 +766,21 @@ BuiltPaths build(ref<Store> store, Realise mode,
if (mode == Realise::Nothing) if (mode == Realise::Nothing)
printMissing(store, pathsToBuild, lvlError); printMissing(store, pathsToBuild, lvlError);
else if (mode == Realise::Outputs) else if (mode == Realise::Outputs)
store->buildPaths(pathsToBuild, bMode); store->buildPaths(pathsToBuild, bMode, evalStore);
return getBuiltPaths(store, pathsToBuild); return getBuiltPaths(evalStore, store, pathsToBuild);
} }
BuiltPaths toBuiltPaths( BuiltPaths toBuiltPaths(
ref<Store> evalStore,
ref<Store> store, ref<Store> store,
Realise mode, Realise mode,
OperateOn operateOn, OperateOn operateOn,
std::vector<std::shared_ptr<Installable>> installables) std::vector<std::shared_ptr<Installable>> installables)
{ {
if (operateOn == OperateOn::Output) { if (operateOn == OperateOn::Output)
return build(store, mode, installables); return build(evalStore, store, mode, installables);
} else { else {
if (mode == Realise::Nothing) if (mode == Realise::Nothing)
settings.readOnlyMode = true; settings.readOnlyMode = true;
@@ -762,23 +791,27 @@ BuiltPaths toBuiltPaths(
} }
} }
StorePathSet toStorePaths(ref<Store> store, StorePathSet toStorePaths(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn, Realise mode, OperateOn operateOn,
std::vector<std::shared_ptr<Installable>> installables) std::vector<std::shared_ptr<Installable>> installables)
{ {
StorePathSet outPaths; StorePathSet outPaths;
for (auto & path : toBuiltPaths(store, mode, operateOn, installables)) { for (auto & path : toBuiltPaths(evalStore, store, mode, operateOn, installables)) {
auto thisOutPaths = path.outPaths(); auto thisOutPaths = path.outPaths();
outPaths.insert(thisOutPaths.begin(), thisOutPaths.end()); outPaths.insert(thisOutPaths.begin(), thisOutPaths.end());
} }
return outPaths; return outPaths;
} }
StorePath toStorePath(ref<Store> store, StorePath toStorePath(
ref<Store> evalStore,
ref<Store> store,
Realise mode, OperateOn operateOn, Realise mode, OperateOn operateOn,
std::shared_ptr<Installable> installable) std::shared_ptr<Installable> installable)
{ {
auto paths = toStorePaths(store, mode, operateOn, {installable}); auto paths = toStorePaths(evalStore, store, mode, operateOn, {installable});
if (paths.size() != 1) if (paths.size() != 1)
throw Error("argument '%s' should evaluate to one store path", installable->what()); throw Error("argument '%s' should evaluate to one store path", installable->what());
+1 -1
View File
@@ -26,7 +26,7 @@ struct App
struct UnresolvedApp struct UnresolvedApp
{ {
App unresolved; App unresolved;
App resolve(ref<Store>); App resolve(ref<Store> evalStore, ref<Store> store);
}; };
struct Installable struct Installable
+4 -4
View File
@@ -19,7 +19,7 @@ static Strings parseAttrPath(std::string_view s)
++i; ++i;
while (1) { while (1) {
if (i == s.end()) if (i == s.end())
throw Error("missing closing quote in selection path '%1%'", s); throw ParseError("missing closing quote in selection path '%1%'", s);
if (*i == '"') break; if (*i == '"') break;
cur.push_back(*i++); cur.push_back(*i++);
} }
@@ -100,7 +100,7 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
} }
Pos findDerivationFilename(EvalState & state, Value & v, std::string what) Pos findPackageFilename(EvalState & state, Value & v, std::string what)
{ {
Value * v2; Value * v2;
try { try {
@@ -116,14 +116,14 @@ Pos findDerivationFilename(EvalState & state, Value & v, std::string what)
auto colon = pos.rfind(':'); auto colon = pos.rfind(':');
if (colon == std::string::npos) if (colon == std::string::npos)
throw Error("cannot parse meta.position attribute '%s'", pos); throw ParseError("cannot parse meta.position attribute '%s'", pos);
std::string filename(pos, 0, colon); std::string filename(pos, 0, colon);
unsigned int lineno; unsigned int lineno;
try { try {
lineno = std::stoi(std::string(pos, colon + 1)); lineno = std::stoi(std::string(pos, colon + 1));
} catch (std::invalid_argument & e) { } catch (std::invalid_argument & e) {
throw Error("cannot parse line number '%s'", pos); throw ParseError("cannot parse line number '%s'", pos);
} }
Symbol file = state.symbols.create(filename); Symbol file = state.symbols.create(filename);
+1 -1
View File
@@ -14,7 +14,7 @@ std::pair<Value *, Pos> findAlongAttrPath(EvalState & state, const string & attr
Bindings & autoArgs, Value & vIn); Bindings & autoArgs, Value & vIn);
/* Heuristic to find the filename and lineno or a nix value. */ /* Heuristic to find the filename and lineno or a nix value. */
Pos findDerivationFilename(EvalState & state, Value & v, std::string what); Pos findPackageFilename(EvalState & state, Value & v, std::string what);
std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s); std::vector<Symbol> parseAttrPath(EvalState & state, std::string_view s);
+8
View File
@@ -61,6 +61,14 @@ MixEvalArgs::MixEvalArgs()
fetchers::overrideRegistry(from.input, to.input, extraAttrs); fetchers::overrideRegistry(from.input, to.input, extraAttrs);
}} }}
}); });
addFlag({
.longName = "eval-store",
.description = "The Nix store to use for evaluations.",
.category = category,
.labels = {"store-url"},
.handler = {&evalStoreUrl},
});
} }
Bindings * MixEvalArgs::getAutoArgs(EvalState & state) Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
+2 -1
View File
@@ -16,8 +16,9 @@ struct MixEvalArgs : virtual Args
Strings searchPath; Strings searchPath;
private: std::optional<std::string> evalStoreUrl;
private:
std::map<std::string, std::string> autoArgs; std::map<std::string, std::string> autoArgs;
}; };
+35 -27
View File
@@ -64,7 +64,11 @@ static char * dupStringWithLen(const char * s, size_t size)
RootValue allocRootValue(Value * v) RootValue allocRootValue(Value * v)
{ {
#if HAVE_BOEHMGC
return std::allocate_shared<Value *>(traceable_allocator<Value *>(), v); return std::allocate_shared<Value *>(traceable_allocator<Value *>(), v);
#else
return std::make_shared<Value *>(v);
#endif
} }
@@ -233,22 +237,34 @@ static void * oomHandler(size_t requested)
} }
class BoehmGCStackAllocator : public StackAllocator { class BoehmGCStackAllocator : public StackAllocator {
boost::coroutines2::protected_fixedsize_stack stack { boost::coroutines2::protected_fixedsize_stack stack {
// We allocate 8 MB, the default max stack size on NixOS. // We allocate 8 MB, the default max stack size on NixOS.
// A smaller stack might be quicker to allocate but reduces the stack // A smaller stack might be quicker to allocate but reduces the stack
// depth available for source filter expressions etc. // depth available for source filter expressions etc.
std::max(boost::context::stack_traits::default_size(), static_cast<std::size_t>(8 * 1024 * 1024)) std::max(boost::context::stack_traits::default_size(), static_cast<std::size_t>(8 * 1024 * 1024))
}; };
// This is specific to boost::coroutines2::protected_fixedsize_stack.
// The stack protection page is included in sctx.size, so we have to
// subtract one page size from the stack size.
std::size_t pfss_usable_stack_size(boost::context::stack_context &sctx) {
return sctx.size - boost::context::stack_traits::page_size();
}
public: public:
boost::context::stack_context allocate() override { boost::context::stack_context allocate() override {
auto sctx = stack.allocate(); auto sctx = stack.allocate();
GC_add_roots(static_cast<char *>(sctx.sp) - sctx.size, sctx.sp);
// Stacks generally start at a high address and grow to lower addresses.
// Architectures that do the opposite are rare; in fact so rare that
// boost_routine does not implement it.
// So we subtract the stack size.
GC_add_roots(static_cast<char *>(sctx.sp) - pfss_usable_stack_size(sctx), sctx.sp);
return sctx; return sctx;
} }
void deallocate(boost::context::stack_context sctx) override { void deallocate(boost::context::stack_context sctx) override {
GC_remove_roots(static_cast<char *>(sctx.sp) - sctx.size, sctx.sp); GC_remove_roots(static_cast<char *>(sctx.sp) - pfss_usable_stack_size(sctx), sctx.sp);
stack.deallocate(sctx); stack.deallocate(sctx);
} }
@@ -362,7 +378,10 @@ static Strings parseNixPath(const string & s)
} }
EvalState::EvalState(const Strings & _searchPath, ref<Store> store) EvalState::EvalState(
const Strings & _searchPath,
ref<Store> store,
std::shared_ptr<Store> buildStore)
: sWith(symbols.create("<with>")) : sWith(symbols.create("<with>"))
, sOutPath(symbols.create("outPath")) , sOutPath(symbols.create("outPath"))
, sDrvPath(symbols.create("drvPath")) , sDrvPath(symbols.create("drvPath"))
@@ -395,6 +414,7 @@ EvalState::EvalState(const Strings & _searchPath, ref<Store> store)
, sEpsilon(symbols.create("")) , sEpsilon(symbols.create(""))
, repair(NoRepair) , repair(NoRepair)
, store(store) , store(store)
, buildStore(buildStore ? buildStore : store)
, regexCache(makeRegexCache()) , regexCache(makeRegexCache())
, baseEnv(allocEnv(128)) , baseEnv(allocEnv(128))
, staticBaseEnv(false, 0) , staticBaseEnv(false, 0)
@@ -760,18 +780,10 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
} }
std::atomic<uint64_t> nrValuesFreed{0};
void finalizeValue(void * obj, void * data)
{
nrValuesFreed++;
}
Value * EvalState::allocValue() Value * EvalState::allocValue()
{ {
nrValues++; nrValues++;
auto v = (Value *) allocBytes(sizeof(Value)); auto v = (Value *) allocBytes(sizeof(Value));
//GC_register_finalizer_no_order(v, finalizeValue, nullptr, nullptr, nullptr);
return v; return v;
} }
@@ -838,39 +850,37 @@ Value * Expr::maybeThunk(EvalState & state, Env & env)
} }
unsigned long nrAvoided = 0;
Value * ExprVar::maybeThunk(EvalState & state, Env & env) Value * ExprVar::maybeThunk(EvalState & state, Env & env)
{ {
Value * v = state.lookupVar(&env, *this, true); Value * v = state.lookupVar(&env, *this, true);
/* The value might not be initialised in the environment yet. /* The value might not be initialised in the environment yet.
In that case, ignore it. */ In that case, ignore it. */
if (v) { nrAvoided++; return v; } if (v) { state.nrAvoided++; return v; }
return Expr::maybeThunk(state, env); return Expr::maybeThunk(state, env);
} }
Value * ExprString::maybeThunk(EvalState & state, Env & env) Value * ExprString::maybeThunk(EvalState & state, Env & env)
{ {
nrAvoided++; state.nrAvoided++;
return &v; return &v;
} }
Value * ExprInt::maybeThunk(EvalState & state, Env & env) Value * ExprInt::maybeThunk(EvalState & state, Env & env)
{ {
nrAvoided++; state.nrAvoided++;
return &v; return &v;
} }
Value * ExprFloat::maybeThunk(EvalState & state, Env & env) Value * ExprFloat::maybeThunk(EvalState & state, Env & env)
{ {
nrAvoided++; state.nrAvoided++;
return &v; return &v;
} }
Value * ExprPath::maybeThunk(EvalState & state, Env & env) Value * ExprPath::maybeThunk(EvalState & state, Env & env)
{ {
nrAvoided++; state.nrAvoided++;
return &v; return &v;
} }
@@ -908,7 +918,7 @@ void EvalState::evalFile(const Path & path_, Value & v, bool mustBeTrivial)
// computation. // computation.
if (mustBeTrivial && if (mustBeTrivial &&
!(dynamic_cast<ExprAttrs *>(e))) !(dynamic_cast<ExprAttrs *>(e)))
throw Error("file '%s' must be an attribute set", path); throw EvalError("file '%s' must be an attribute set", path);
eval(e, v); eval(e, v);
} catch (Error & e) { } catch (Error & e) {
addErrorTrace(e, "while evaluating the file '%1%':", path2); addErrorTrace(e, "while evaluating the file '%1%':", path2);
@@ -1125,8 +1135,6 @@ static string showAttrPath(EvalState & state, Env & env, const AttrPath & attrPa
} }
unsigned long nrLookups = 0;
void ExprSelect::eval(EvalState & state, Env & env, Value & v) void ExprSelect::eval(EvalState & state, Env & env, Value & v)
{ {
Value vTmp; Value vTmp;
@@ -1138,7 +1146,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
try { try {
for (auto & i : attrPath) { for (auto & i : attrPath) {
nrLookups++; state.nrLookups++;
Bindings::iterator j; Bindings::iterator j;
Symbol name = getName(i, state, env); Symbol name = getName(i, state, env);
if (def) { if (def) {
+15 -1
View File
@@ -94,8 +94,12 @@ public:
Value vEmptySet; Value vEmptySet;
/* Store used to materialise .drv files. */
const ref<Store> store; const ref<Store> store;
/* Store used to build stuff. */
const ref<Store> buildStore;
private: private:
SrcToStore srcToStore; SrcToStore srcToStore;
@@ -128,7 +132,10 @@ private:
public: public:
EvalState(const Strings & _searchPath, ref<Store> store); EvalState(
const Strings & _searchPath,
ref<Store> store,
std::shared_ptr<Store> buildStore = nullptr);
~EvalState(); ~EvalState();
void addToSearchPath(const string & s); void addToSearchPath(const string & s);
@@ -316,8 +323,10 @@ private:
unsigned long nrValuesInEnvs = 0; unsigned long nrValuesInEnvs = 0;
unsigned long nrValues = 0; unsigned long nrValues = 0;
unsigned long nrListElems = 0; unsigned long nrListElems = 0;
unsigned long nrLookups = 0;
unsigned long nrAttrsets = 0; unsigned long nrAttrsets = 0;
unsigned long nrAttrsInAttrsets = 0; unsigned long nrAttrsInAttrsets = 0;
unsigned long nrAvoided = 0;
unsigned long nrOpUpdates = 0; unsigned long nrOpUpdates = 0;
unsigned long nrOpUpdateValuesCopied = 0; unsigned long nrOpUpdateValuesCopied = 0;
unsigned long nrListConcats = 0; unsigned long nrListConcats = 0;
@@ -339,6 +348,11 @@ private:
friend struct ExprOpUpdate; friend struct ExprOpUpdate;
friend struct ExprOpConcatLists; friend struct ExprOpConcatLists;
friend struct ExprVar;
friend struct ExprString;
friend struct ExprInt;
friend struct ExprFloat;
friend struct ExprPath;
friend struct ExprSelect; friend struct ExprSelect;
friend void prim_getAttr(EvalState & state, const Pos & pos, Value * * args, Value & v); friend void prim_getAttr(EvalState & state, const Pos & pos, Value * * args, Value & v);
friend void prim_match(EvalState & state, const Pos & pos, Value * * args, Value & v); friend void prim_match(EvalState & state, const Pos & pos, Value * * args, Value & v);
+19 -6
View File
@@ -1,4 +1,5 @@
#include "flake.hh" #include "flake.hh"
#include "eval.hh"
#include "lockfile.hh" #include "lockfile.hh"
#include "primops.hh" #include "primops.hh"
#include "eval-inline.hh" #include "eval-inline.hh"
@@ -296,7 +297,14 @@ LockedFlake lockFlake(
FlakeCache flakeCache; FlakeCache flakeCache;
auto flake = getFlake(state, topRef, lockFlags.useRegistries, flakeCache); auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto flake = getFlake(state, topRef, useRegistries, flakeCache);
if (lockFlags.applyNixConfig) {
flake.config.apply();
// FIXME: send new config to the daemon.
}
try { try {
@@ -359,7 +367,12 @@ LockedFlake lockFlake(
ancestors? */ ancestors? */
auto i = overrides.find(inputPath); auto i = overrides.find(inputPath);
bool hasOverride = i != overrides.end(); bool hasOverride = i != overrides.end();
if (hasOverride) overridesUsed.insert(inputPath); if (hasOverride) {
overridesUsed.insert(inputPath);
// Respect the “flakeness” of the input even if we
// override it
i->second.isFlake = input2.isFlake;
}
auto & input = hasOverride ? i->second : input2; auto & input = hasOverride ? i->second : input2;
/* Resolve 'follows' later (since it may refer to an input /* Resolve 'follows' later (since it may refer to an input
@@ -454,7 +467,7 @@ LockedFlake lockFlake(
throw Error("cannot update flake input '%s' in pure mode", inputPathS); throw Error("cannot update flake input '%s' in pure mode", inputPathS);
if (input.isFlake) { if (input.isFlake) {
auto inputFlake = getFlake(state, *input.ref, lockFlags.useRegistries, flakeCache); auto inputFlake = getFlake(state, *input.ref, useRegistries, flakeCache);
/* Note: in case of an --override-input, we use /* Note: in case of an --override-input, we use
the *original* ref (input2.ref) for the the *original* ref (input2.ref) for the
@@ -489,7 +502,7 @@ LockedFlake lockFlake(
else { else {
auto [sourceInfo, resolvedRef, lockedRef] = fetchOrSubstituteTree( auto [sourceInfo, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, *input.ref, lockFlags.useRegistries, flakeCache); state, *input.ref, useRegistries, flakeCache);
node->inputs.insert_or_assign(id, node->inputs.insert_or_assign(id,
std::make_shared<LockedNode>(lockedRef, *input.ref, false)); std::make_shared<LockedNode>(lockedRef, *input.ref, false));
} }
@@ -563,7 +576,7 @@ LockedFlake lockFlake(
also just clear the 'rev' field... */ also just clear the 'rev' field... */
auto prevLockedRef = flake.lockedRef; auto prevLockedRef = flake.lockedRef;
FlakeCache dummyCache; FlakeCache dummyCache;
flake = getFlake(state, topRef, lockFlags.useRegistries, dummyCache); flake = getFlake(state, topRef, useRegistries, dummyCache);
if (lockFlags.commitLockFile && if (lockFlags.commitLockFile &&
flake.lockedRef.input.getRev() && flake.lockedRef.input.getRev() &&
@@ -633,7 +646,7 @@ static void prim_getFlake(EvalState & state, const Pos & pos, Value * * args, Va
lockFlake(state, flakeRef, lockFlake(state, flakeRef,
LockFlags { LockFlags {
.updateLockFile = false, .updateLockFile = false,
.useRegistries = !evalSettings.pureEval, .useRegistries = !evalSettings.pureEval && !settings.useRegistries,
.allowMutable = !evalSettings.pureEval, .allowMutable = !evalSettings.pureEval,
}), }),
v); v);
+5 -1
View File
@@ -102,7 +102,11 @@ struct LockFlags
/* Whether to use the registries to lookup indirect flake /* Whether to use the registries to lookup indirect flake
references like 'nixpkgs'. */ references like 'nixpkgs'. */
bool useRegistries = true; std::optional<bool> useRegistries = std::nullopt;
/* Whether to apply flake's nixConfig attribute to the configuration */
bool applyNixConfig = false;
/* Whether mutable flake references (i.e. those without a Git /* Whether mutable flake references (i.e. those without a Git
revision or similar) without a corresponding lock are revision or similar) without a corresponding lock are
+4 -2
View File
@@ -38,11 +38,13 @@ static void adjustLoc(YYLTYPE * loc, const char * s, size_t len)
loc->first_line = loc->last_line; loc->first_line = loc->last_line;
loc->first_column = loc->last_column; loc->first_column = loc->last_column;
while (len--) { for (size_t i = 0; i < len; i++) {
switch (*s++) { switch (*s++) {
case '\r': case '\r':
if (*s == '\n') /* cr/lf */ if (*s == '\n') { /* cr/lf */
i++;
s++; s++;
}
/* fall through */ /* fall through */
case '\n': case '\n':
++loc->last_line; ++loc->last_line;
+1 -1
View File
@@ -16,7 +16,7 @@ libexpr_CXXFLAGS += -I src/libutil -I src/libstore -I src/libfetchers -I src/lib
libexpr_LIBS = libutil libstore libfetchers libexpr_LIBS = libutil libstore libfetchers
libexpr_LDFLAGS = -lboost_context libexpr_LDFLAGS = -lboost_context
ifeq ($(OS), Linux) ifdef HOST_LINUX
libexpr_LDFLAGS += -ldl libexpr_LDFLAGS += -ldl
endif endif
+1 -1
View File
@@ -3109,7 +3109,7 @@ static RegisterPrimOp primop_toString({
- A path (e.g., `toString /foo/bar` yields `"/foo/bar"`. - A path (e.g., `toString /foo/bar` yields `"/foo/bar"`.
- A set containing `{ __toString = self: ...; }`. - A set containing `{ __toString = self: ...; }` or `{ outPath = ...; }`.
- An integer. - An integer.
+1
View File
@@ -62,6 +62,7 @@ static void prim_fetchMercurial(EvalState & state, const Pos & pos, Value * * ar
fetchers::Attrs attrs; fetchers::Attrs attrs;
attrs.insert_or_assign("type", "hg"); attrs.insert_or_assign("type", "hg");
attrs.insert_or_assign("url", url.find("://") != std::string::npos ? url : "file://" + url); attrs.insert_or_assign("url", url.find("://") != std::string::npos ? url : "file://" + url);
attrs.insert_or_assign("name", name);
if (ref) attrs.insert_or_assign("ref", *ref); if (ref) attrs.insert_or_assign("ref", *ref);
if (rev) attrs.insert_or_assign("rev", rev->gitRev()); if (rev) attrs.insert_or_assign("rev", rev->gitRev());
auto input = fetchers::Input::fromAttrs(std::move(attrs)); auto input = fetchers::Input::fromAttrs(std::move(attrs));
+32 -9
View File
@@ -7,6 +7,7 @@
#include <ctime> #include <ctime>
#include <iomanip> #include <iomanip>
#include <regex>
namespace nix { namespace nix {
@@ -60,10 +61,19 @@ void emitTreeAttrs(
v.attrs->sort(); v.attrs->sort();
} }
std::string fixURI(std::string uri, EvalState &state) std::string fixURI(std::string uri, EvalState &state, const std::string & defaultScheme = "file")
{ {
state.checkURI(uri); state.checkURI(uri);
return uri.find("://") != std::string::npos ? uri : "file://" + uri; return uri.find("://") != std::string::npos ? uri : defaultScheme + "://" + uri;
}
std::string fixURIForGit(std::string uri, EvalState & state)
{
static std::regex scp_uri("([^/].*)@(.*):(.*)");
if (uri[0] != '/' && std::regex_match(uri, scp_uri))
return fixURI(std::regex_replace(uri, scp_uri, "$1@$2/$3"), state, "ssh");
else
return fixURI(uri, state);
} }
void addURI(EvalState &state, fetchers::Attrs &attrs, Symbol name, std::string v) void addURI(EvalState &state, fetchers::Attrs &attrs, Symbol name, std::string v)
@@ -72,13 +82,18 @@ void addURI(EvalState &state, fetchers::Attrs &attrs, Symbol name, std::string v
attrs.emplace(name, n == "url" ? fixURI(v, state) : v); attrs.emplace(name, n == "url" ? fixURI(v, state) : v);
} }
struct FetchTreeParams {
bool emptyRevFallback = false;
bool allowNameArgument = false;
};
static void fetchTree( static void fetchTree(
EvalState &state, EvalState &state,
const Pos &pos, const Pos &pos,
Value **args, Value **args,
Value &v, Value &v,
const std::optional<std::string> type, const std::optional<std::string> type,
bool emptyRevFallback = false const FetchTreeParams & params = FetchTreeParams{}
) { ) {
fetchers::Input input; fetchers::Input input;
PathSet context; PathSet context;
@@ -119,17 +134,25 @@ static void fetchTree(
.errPos = pos .errPos = pos
}); });
if (!params.allowNameArgument)
if (auto nameIter = attrs.find("name"); nameIter != attrs.end())
throw Error({
.msg = hintfmt("attribute 'name' isn’t supported in call to 'fetchTree'"),
.errPos = pos
});
input = fetchers::Input::fromAttrs(std::move(attrs)); input = fetchers::Input::fromAttrs(std::move(attrs));
} else { } else {
auto url = fixURI(state.coerceToString(pos, *args[0], context, false, false), state); auto url = state.coerceToString(pos, *args[0], context, false, false);
if (type == "git") { if (type == "git") {
fetchers::Attrs attrs; fetchers::Attrs attrs;
attrs.emplace("type", "git"); attrs.emplace("type", "git");
attrs.emplace("url", url); attrs.emplace("url", fixURIForGit(url, state));
input = fetchers::Input::fromAttrs(std::move(attrs)); input = fetchers::Input::fromAttrs(std::move(attrs));
} else { } else {
input = fetchers::Input::fromURL(url); input = fetchers::Input::fromURL(fixURI(url, state));
} }
} }
@@ -144,13 +167,13 @@ static void fetchTree(
if (state.allowedPaths) if (state.allowedPaths)
state.allowedPaths->insert(tree.actualPath); state.allowedPaths->insert(tree.actualPath);
emitTreeAttrs(state, tree, input2, v, emptyRevFallback); emitTreeAttrs(state, tree, input2, v, params.emptyRevFallback);
} }
static void prim_fetchTree(EvalState & state, const Pos & pos, Value * * args, Value & v) static void prim_fetchTree(EvalState & state, const Pos & pos, Value * * args, Value & v)
{ {
settings.requireExperimentalFeature("flakes"); settings.requireExperimentalFeature("flakes");
fetchTree(state, pos, args, v, std::nullopt); fetchTree(state, pos, args, v, std::nullopt, FetchTreeParams { .allowNameArgument = false });
} }
// FIXME: document // FIXME: document
@@ -292,7 +315,7 @@ static RegisterPrimOp primop_fetchTarball({
static void prim_fetchGit(EvalState &state, const Pos &pos, Value **args, Value &v) static void prim_fetchGit(EvalState &state, const Pos &pos, Value **args, Value &v)
{ {
fetchTree(state, pos, args, v, "git", true); fetchTree(state, pos, args, v, "git", FetchTreeParams { .emptyRevFallback = true, .allowNameArgument = true });
} }
static RegisterPrimOp primop_fetchGit({ static RegisterPrimOp primop_fetchGit({
+6 -1
View File
@@ -200,12 +200,17 @@ void Input::markChangedFile(
return scheme->markChangedFile(*this, file, commitMsg); return scheme->markChangedFile(*this, file, commitMsg);
} }
std::string Input::getName() const
{
return maybeGetStrAttr(attrs, "name").value_or("source");
}
StorePath Input::computeStorePath(Store & store) const StorePath Input::computeStorePath(Store & store) const
{ {
auto narHash = getNarHash(); auto narHash = getNarHash();
if (!narHash) if (!narHash)
throw Error("cannot compute store path for mutable input '%s'", to_string()); throw Error("cannot compute store path for mutable input '%s'", to_string());
return store.makeFixedOutputPath(FileIngestionMethod::Recursive, *narHash, "source"); return store.makeFixedOutputPath(FileIngestionMethod::Recursive, *narHash, getName());
} }
std::string Input::getType() const std::string Input::getType() const
+2
View File
@@ -81,6 +81,8 @@ public:
std::string_view file, std::string_view file,
std::optional<std::string> commitMsg) const; std::optional<std::string> commitMsg) const;
std::string getName() const;
StorePath computeStorePath(Store & store) const; StorePath computeStorePath(Store & store) const;
// Convenience functions for common attributes. // Convenience functions for common attributes.
+4 -4
View File
@@ -60,7 +60,7 @@ struct GitInputScheme : InputScheme
if (maybeGetStrAttr(attrs, "type") != "git") return {}; if (maybeGetStrAttr(attrs, "type") != "git") return {};
for (auto & [name, value] : attrs) for (auto & [name, value] : attrs)
if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "shallow" && name != "submodules" && name != "lastModified" && name != "revCount" && name != "narHash" && name != "allRefs") if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "shallow" && name != "submodules" && name != "lastModified" && name != "revCount" && name != "narHash" && name != "allRefs" && name != "name")
throw Error("unsupported Git input attribute '%s'", name); throw Error("unsupported Git input attribute '%s'", name);
parseURL(getStrAttr(attrs, "url")); parseURL(getStrAttr(attrs, "url"));
@@ -167,10 +167,10 @@ struct GitInputScheme : InputScheme
std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override
{ {
auto name = "source";
Input input(_input); Input input(_input);
std::string name = input.getName();
bool shallow = maybeGetBoolAttr(input.attrs, "shallow").value_or(false); bool shallow = maybeGetBoolAttr(input.attrs, "shallow").value_or(false);
bool submodules = maybeGetBoolAttr(input.attrs, "submodules").value_or(false); bool submodules = maybeGetBoolAttr(input.attrs, "submodules").value_or(false);
bool allRefs = maybeGetBoolAttr(input.attrs, "allRefs").value_or(false); bool allRefs = maybeGetBoolAttr(input.attrs, "allRefs").value_or(false);
@@ -270,7 +270,7 @@ struct GitInputScheme : InputScheme
return files.count(file); return files.count(file);
}; };
auto storePath = store->addToStore("source", actualUrl, FileIngestionMethod::Recursive, htSHA256, filter); auto storePath = store->addToStore(input.getName(), actualUrl, FileIngestionMethod::Recursive, htSHA256, filter);
// FIXME: maybe we should use the timestamp of the last // FIXME: maybe we should use the timestamp of the last
// modified dirty file? // modified dirty file?
+1 -1
View File
@@ -207,7 +207,7 @@ struct GitArchiveInputScheme : InputScheme
auto url = getDownloadUrl(input); auto url = getDownloadUrl(input);
auto [tree, lastModified] = downloadTarball(store, url.url, "source", true, url.headers); auto [tree, lastModified] = downloadTarball(store, url.url, input.getName(), true, url.headers);
input.attrs.insert_or_assign("lastModified", uint64_t(lastModified)); input.attrs.insert_or_assign("lastModified", uint64_t(lastModified));
+4 -4
View File
@@ -74,7 +74,7 @@ struct MercurialInputScheme : InputScheme
if (maybeGetStrAttr(attrs, "type") != "hg") return {}; if (maybeGetStrAttr(attrs, "type") != "hg") return {};
for (auto & [name, value] : attrs) for (auto & [name, value] : attrs)
if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "revCount" && name != "narHash") if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "revCount" && name != "narHash" && name != "name")
throw Error("unsupported Mercurial input attribute '%s'", name); throw Error("unsupported Mercurial input attribute '%s'", name);
parseURL(getStrAttr(attrs, "url")); parseURL(getStrAttr(attrs, "url"));
@@ -147,10 +147,10 @@ struct MercurialInputScheme : InputScheme
std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override std::pair<Tree, Input> fetch(ref<Store> store, const Input & _input) override
{ {
auto name = "source";
Input input(_input); Input input(_input);
auto name = input.getName();
auto [isLocal, actualUrl_] = getActualUrl(input); auto [isLocal, actualUrl_] = getActualUrl(input);
auto actualUrl = actualUrl_; // work around clang bug auto actualUrl = actualUrl_; // work around clang bug
@@ -193,7 +193,7 @@ struct MercurialInputScheme : InputScheme
return files.count(file); return files.count(file);
}; };
auto storePath = store->addToStore("source", actualUrl, FileIngestionMethod::Recursive, htSHA256, filter); auto storePath = store->addToStore(input.getName(), actualUrl, FileIngestionMethod::Recursive, htSHA256, filter);
return { return {
Tree(store->toRealPath(storePath), std::move(storePath)), Tree(store->toRealPath(storePath), std::move(storePath)),
+7
View File
@@ -124,6 +124,13 @@ std::shared_ptr<Registry> getUserRegistry()
return userRegistry; return userRegistry;
} }
std::shared_ptr<Registry> getCustomRegistry(const Path & p)
{
static auto customRegistry =
Registry::read(p, Registry::Custom);
return customRegistry;
}
static std::shared_ptr<Registry> flagRegistry = static std::shared_ptr<Registry> flagRegistry =
std::make_shared<Registry>(Registry::Flag); std::make_shared<Registry>(Registry::Flag);
+3
View File
@@ -14,6 +14,7 @@ struct Registry
User = 1, User = 1,
System = 2, System = 2,
Global = 3, Global = 3,
Custom = 4,
}; };
RegistryType type; RegistryType type;
@@ -48,6 +49,8 @@ typedef std::vector<std::shared_ptr<Registry>> Registries;
std::shared_ptr<Registry> getUserRegistry(); std::shared_ptr<Registry> getUserRegistry();
std::shared_ptr<Registry> getCustomRegistry(const Path & p);
Path getUserRegistryPath(); Path getUserRegistryPath();
Registries getRegistries(ref<Store> store); Registries getRegistries(ref<Store> store);
+2 -2
View File
@@ -196,7 +196,7 @@ struct TarballInputScheme : InputScheme
if (maybeGetStrAttr(attrs, "type") != "tarball") return {}; if (maybeGetStrAttr(attrs, "type") != "tarball") return {};
for (auto & [name, value] : attrs) for (auto & [name, value] : attrs)
if (name != "type" && name != "url" && /* name != "hash" && */ name != "narHash") if (name != "type" && name != "url" && /* name != "hash" && */ name != "narHash" && name != "name")
throw Error("unsupported tarball input attribute '%s'", name); throw Error("unsupported tarball input attribute '%s'", name);
Input input; Input input;
@@ -226,7 +226,7 @@ struct TarballInputScheme : InputScheme
std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) override std::pair<Tree, Input> fetch(ref<Store> store, const Input & input) override
{ {
auto tree = downloadTarball(store, getStrAttr(input.attrs, "url"), "source", false).first; auto tree = downloadTarball(store, getStrAttr(input.attrs, "url"), input.getName(), false).first;
return {std::move(tree), input}; return {std::move(tree), input};
} }
}; };
+1 -1
View File
@@ -484,7 +484,7 @@ Logger * makeProgressBar(bool printBuildLogs)
{ {
return new ProgressBar( return new ProgressBar(
printBuildLogs, printBuildLogs,
isatty(STDERR_FILENO) && getEnv("TERM").value_or("dumb") != "dumb" shouldANSI()
); );
} }
+1 -1
View File
@@ -238,7 +238,7 @@ LegacyArgs::LegacyArgs(const std::string & programName,
addFlag({ addFlag({
.longName = "no-gc-warning", .longName = "no-gc-warning",
.description = "Disable warnings about not using `--add-root`.", .description = "Disable warnings about not using `--add-root`.",
.handler = {&gcWarning, true}, .handler = {&gcWarning, false},
}); });
addFlag({ addFlag({
+26 -47
View File
@@ -143,7 +143,6 @@ void DerivationGoal::work()
(this->*state)(); (this->*state)();
} }
void DerivationGoal::addWantedOutputs(const StringSet & outputs) void DerivationGoal::addWantedOutputs(const StringSet & outputs)
{ {
/* If we already want all outputs, there is nothing to do. */ /* If we already want all outputs, there is nothing to do. */
@@ -166,7 +165,7 @@ void DerivationGoal::getDerivation()
/* The first thing to do is to make sure that the derivation /* The first thing to do is to make sure that the derivation
exists. If it doesn't, it may be created through a exists. If it doesn't, it may be created through a
substitute. */ substitute. */
if (buildMode == bmNormal && worker.store.isValidPath(drvPath)) { if (buildMode == bmNormal && worker.evalStore.isValidPath(drvPath)) {
loadDerivation(); loadDerivation();
return; return;
} }
@@ -189,12 +188,12 @@ void DerivationGoal::loadDerivation()
/* `drvPath' should already be a root, but let's be on the safe /* `drvPath' should already be a root, but let's be on the safe
side: if the user forgot to make it a root, we wouldn't want side: if the user forgot to make it a root, we wouldn't want
things being garbage collected while we're busy. */ things being garbage collected while we're busy. */
worker.store.addTempRoot(drvPath); worker.evalStore.addTempRoot(drvPath);
assert(worker.store.isValidPath(drvPath)); assert(worker.evalStore.isValidPath(drvPath));
/* Get the derivation. */ /* Get the derivation. */
drv = std::make_unique<Derivation>(worker.store.derivationFromPath(drvPath)); drv = std::make_unique<Derivation>(worker.evalStore.derivationFromPath(drvPath));
haveDerivation(); haveDerivation();
} }
@@ -213,8 +212,8 @@ void DerivationGoal::haveDerivation()
if (i.second.second) if (i.second.second)
worker.store.addTempRoot(*i.second.second); worker.store.addTempRoot(*i.second.second);
auto outputHashes = staticOutputHashes(worker.store, *drv); auto outputHashes = staticOutputHashes(worker.evalStore, *drv);
for (auto &[outputName, outputHash] : outputHashes) for (auto & [outputName, outputHash] : outputHashes)
initialOutputs.insert({ initialOutputs.insert({
outputName, outputName,
InitialOutput{ InitialOutput{
@@ -338,6 +337,15 @@ void DerivationGoal::gaveUpOnSubstitution()
for (auto & i : dynamic_cast<Derivation *>(drv.get())->inputDrvs) for (auto & i : dynamic_cast<Derivation *>(drv.get())->inputDrvs)
addWaitee(worker.makeDerivationGoal(i.first, i.second, buildMode == bmRepair ? bmRepair : bmNormal)); addWaitee(worker.makeDerivationGoal(i.first, i.second, buildMode == bmRepair ? bmRepair : bmNormal));
/* Copy the input sources from the eval store to the build
store. */
if (&worker.evalStore != &worker.store) {
RealisedPath::Set inputSrcs;
for (auto & i : drv->inputSrcs)
inputSrcs.insert(i);
copyClosure(worker.evalStore, worker.store, inputSrcs);
}
for (auto & i : drv->inputSrcs) { for (auto & i : drv->inputSrcs) {
if (worker.store.isValidPath(i)) continue; if (worker.store.isValidPath(i)) continue;
if (!settings.useSubstitutes) if (!settings.useSubstitutes)
@@ -479,8 +487,8 @@ void DerivationGoal::inputsRealised()
/* Add the relevant output closures of the input derivation /* Add the relevant output closures of the input derivation
`i' as input paths. Only add the closures of output paths `i' as input paths. Only add the closures of output paths
that are specified as inputs. */ that are specified as inputs. */
assert(worker.store.isValidPath(drvPath)); assert(worker.evalStore.isValidPath(drvPath));
auto outputs = worker.store.queryPartialDerivationOutputMap(depDrvPath); auto outputs = worker.evalStore.queryPartialDerivationOutputMap(depDrvPath);
for (auto & j : wantedDepOutputs) { for (auto & j : wantedDepOutputs) {
if (outputs.count(j) > 0) { if (outputs.count(j) > 0) {
auto optRealizedInput = outputs.at(j); auto optRealizedInput = outputs.at(j);
@@ -545,7 +553,7 @@ void DerivationGoal::tryToBuild()
PathSet lockFiles; PathSet lockFiles;
/* FIXME: Should lock something like the drv itself so we don't build same /* FIXME: Should lock something like the drv itself so we don't build same
CA drv concurrently */ CA drv concurrently */
if (dynamic_cast<LocalStore *>(&worker.store)) if (dynamic_cast<LocalStore *>(&worker.store)) {
/* If we aren't a local store, we might need to use the local store as /* If we aren't a local store, we might need to use the local store as
a build remote, but that would cause a deadlock. */ a build remote, but that would cause a deadlock. */
/* FIXME: Make it so we can use ourselves as a build remote even if we /* FIXME: Make it so we can use ourselves as a build remote even if we
@@ -553,9 +561,15 @@ void DerivationGoal::tryToBuild()
/* FIXME: find some way to lock for scheduling for the other stores so /* FIXME: find some way to lock for scheduling for the other stores so
a forking daemon with --store still won't farm out redundant builds. a forking daemon with --store still won't farm out redundant builds.
*/ */
for (auto & i : drv->outputsAndOptPaths(worker.store)) for (auto & i : drv->outputsAndOptPaths(worker.store)) {
if (i.second.second) if (i.second.second)
lockFiles.insert(worker.store.Store::toRealPath(*i.second.second)); lockFiles.insert(worker.store.Store::toRealPath(*i.second.second));
else
lockFiles.insert(
worker.store.Store::toRealPath(drvPath) + "!" + i.first
);
}
}
if (!outputLocks.lockPaths(lockFiles, "", false)) { if (!outputLocks.lockPaths(lockFiles, "", false)) {
if (!actLock) if (!actLock)
@@ -758,6 +772,7 @@ void runPostBuildHook(
hookEnvironment.emplace("DRV_PATH", store.printStorePath(drvPath)); hookEnvironment.emplace("DRV_PATH", store.printStorePath(drvPath));
hookEnvironment.emplace("OUT_PATHS", chomp(concatStringsSep(" ", store.printStorePathSet(outputPaths)))); hookEnvironment.emplace("OUT_PATHS", chomp(concatStringsSep(" ", store.printStorePathSet(outputPaths))));
hookEnvironment.emplace("NIX_CONFIG", globalConfig.toKeyValue());
RunOptions opts(settings.postBuildHook, {}); RunOptions opts(settings.postBuildHook, {});
opts.environment = hookEnvironment; opts.environment = hookEnvironment;
@@ -1074,42 +1089,6 @@ HookReply DerivationGoal::tryBuildHook()
} }
StorePathSet DerivationGoal::exportReferences(const StorePathSet & storePaths)
{
StorePathSet paths;
for (auto & storePath : storePaths) {
if (!inputPaths.count(storePath))
throw BuildError("cannot export references of path '%s' because it is not in the input closure of the derivation", worker.store.printStorePath(storePath));
worker.store.computeFSClosure({storePath}, paths);
}
/* If there are derivations in the graph, then include their
outputs as well. This is useful if you want to do things
like passing all build-time dependencies of some path to a
derivation that builds a NixOS DVD image. */
auto paths2 = paths;
for (auto & j : paths2) {
if (j.isDerivation()) {
Derivation drv = worker.store.derivationFromPath(j);
for (auto & k : drv.outputsAndOptPaths(worker.store)) {
if (!k.second.second)
/* FIXME: I am confused why we are calling
`computeFSClosure` on the output path, rather than
derivation itself. That doesn't seem right to me, so I
won't try to implemented this for CA derivations. */
throw UnimplementedError("exportReferences on CA derivations is not yet implemented");
worker.store.computeFSClosure(*k.second.second, paths);
}
}
}
return paths;
}
void DerivationGoal::registerOutputs() void DerivationGoal::registerOutputs()
{ {
/* When using a build hook, the build hook can register the output /* When using a build hook, the build hook can register the output
+5 -5
View File
@@ -6,9 +6,9 @@
namespace nix { namespace nix {
void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMode) void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMode, std::shared_ptr<Store> evalStore)
{ {
Worker worker(*this); Worker worker(*this, evalStore ? *evalStore : *this);
Goals goals; Goals goals;
for (auto & br : reqs) { for (auto & br : reqs) {
@@ -51,7 +51,7 @@ void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMod
BuildResult Store::buildDerivation(const StorePath & drvPath, const BasicDerivation & drv, BuildResult Store::buildDerivation(const StorePath & drvPath, const BasicDerivation & drv,
BuildMode buildMode) BuildMode buildMode)
{ {
Worker worker(*this); Worker worker(*this, *this);
auto goal = worker.makeBasicDerivationGoal(drvPath, drv, {}, buildMode); auto goal = worker.makeBasicDerivationGoal(drvPath, drv, {}, buildMode);
BuildResult result; BuildResult result;
@@ -93,7 +93,7 @@ void Store::ensurePath(const StorePath & path)
/* If the path is already valid, we're done. */ /* If the path is already valid, we're done. */
if (isValidPath(path)) return; if (isValidPath(path)) return;
Worker worker(*this); Worker worker(*this, *this);
GoalPtr goal = worker.makePathSubstitutionGoal(path); GoalPtr goal = worker.makePathSubstitutionGoal(path);
Goals goals = {goal}; Goals goals = {goal};
@@ -111,7 +111,7 @@ void Store::ensurePath(const StorePath & path)
void LocalStore::repairPath(const StorePath & path) void LocalStore::repairPath(const StorePath & path)
{ {
Worker worker(*this); Worker worker(*this, *this);
GoalPtr goal = worker.makePathSubstitutionGoal(path, Repair); GoalPtr goal = worker.makePathSubstitutionGoal(path, Repair);
Goals goals = {goal}; Goals goals = {goal};
+22 -105
View File
@@ -518,7 +518,7 @@ void LocalDerivationGoal::startBuilder()
/* Write closure info to <fileName>. */ /* Write closure info to <fileName>. */
writeFile(tmpDir + "/" + fileName, writeFile(tmpDir + "/" + fileName,
worker.store.makeValidityRegistration( worker.store.makeValidityRegistration(
exportReferences({storePath}), false, false)); worker.store.exportReferences({storePath}, inputPaths), false, false));
} }
} }
@@ -1084,113 +1084,28 @@ void LocalDerivationGoal::initEnv()
} }
static std::regex shVarName("[A-Za-z_][A-Za-z0-9_]*");
void LocalDerivationGoal::writeStructuredAttrs() void LocalDerivationGoal::writeStructuredAttrs()
{ {
auto structuredAttrs = parsedDrv->getStructuredAttrs(); if (auto structAttrsJson = parsedDrv->prepareStructuredAttrs(worker.store, inputPaths)) {
if (!structuredAttrs) return; auto json = structAttrsJson.value();
nlohmann::json rewritten;
for (auto & [i, v] : json["outputs"].get<nlohmann::json::object_t>()) {
/* The placeholder must have a rewrite, so we use it to cover both the
cases where we know or don't know the output path ahead of time. */
rewritten[i] = rewriteStrings(v, inputRewrites);
}
auto json = *structuredAttrs; json["outputs"] = rewritten;
/* Add an "outputs" object containing the output paths. */ auto jsonSh = writeStructuredAttrsShell(json);
nlohmann::json outputs;
for (auto & i : drv->outputs) { writeFile(tmpDir + "/.attrs.sh", rewriteStrings(jsonSh, inputRewrites));
/* The placeholder must have a rewrite, so we use it to cover both the chownToBuilder(tmpDir + "/.attrs.sh");
cases where we know or don't know the output path ahead of time. */ env["NIX_ATTRS_SH_FILE"] = tmpDir + "/.attrs.sh";
outputs[i.first] = rewriteStrings(hashPlaceholder(i.first), inputRewrites); writeFile(tmpDir + "/.attrs.json", rewriteStrings(json.dump(), inputRewrites));
chownToBuilder(tmpDir + "/.attrs.json");
env["NIX_ATTRS_JSON_FILE"] = tmpDir + "/.attrs.json";
} }
json["outputs"] = outputs;
/* Handle exportReferencesGraph. */
auto e = json.find("exportReferencesGraph");
if (e != json.end() && e->is_object()) {
for (auto i = e->begin(); i != e->end(); ++i) {
std::ostringstream str;
{
JSONPlaceholder jsonRoot(str, true);
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(worker.store.parseStorePath(p.get<std::string>()));
worker.store.pathInfoToJSON(jsonRoot,
exportReferences(storePaths), false, true);
}
json[i.key()] = nlohmann::json::parse(str.str()); // urgh
}
}
writeFile(tmpDir + "/.attrs.json", rewriteStrings(json.dump(), inputRewrites));
chownToBuilder(tmpDir + "/.attrs.json");
/* As a convenience to bash scripts, write a shell file that
maps all attributes that are representable in bash -
namely, strings, integers, nulls, Booleans, and arrays and
objects consisting entirely of those values. (So nested
arrays or objects are not supported.) */
auto handleSimpleType = [](const nlohmann::json & value) -> std::optional<std::string> {
if (value.is_string())
return shellEscape(value);
if (value.is_number()) {
auto f = value.get<float>();
if (std::ceil(f) == f)
return std::to_string(value.get<int>());
}
if (value.is_null())
return std::string("''");
if (value.is_boolean())
return value.get<bool>() ? std::string("1") : std::string("");
return {};
};
std::string jsonSh;
for (auto i = json.begin(); i != json.end(); ++i) {
if (!std::regex_match(i.key(), shVarName)) continue;
auto & value = i.value();
auto s = handleSimpleType(value);
if (s)
jsonSh += fmt("declare %s=%s\n", i.key(), *s);
else if (value.is_array()) {
std::string s2;
bool good = true;
for (auto i = value.begin(); i != value.end(); ++i) {
auto s3 = handleSimpleType(i.value());
if (!s3) { good = false; break; }
s2 += *s3; s2 += ' ';
}
if (good)
jsonSh += fmt("declare -a %s=(%s)\n", i.key(), s2);
}
else if (value.is_object()) {
std::string s2;
bool good = true;
for (auto i = value.begin(); i != value.end(); ++i) {
auto s3 = handleSimpleType(i.value());
if (!s3) { good = false; break; }
s2 += fmt("[%s]=%s ", shellEscape(i.key()), *s3);
}
if (good)
jsonSh += fmt("declare -A %s=(%s)\n", i.key(), s2);
}
}
writeFile(tmpDir + "/.attrs.sh", rewriteStrings(jsonSh, inputRewrites));
chownToBuilder(tmpDir + "/.attrs.sh");
} }
@@ -1339,8 +1254,10 @@ struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual Lo
return next->queryRealisation(id); return next->queryRealisation(id);
} }
void buildPaths(const std::vector<DerivedPath> & paths, BuildMode buildMode) override void buildPaths(const std::vector<DerivedPath> & paths, BuildMode buildMode, std::shared_ptr<Store> evalStore) override
{ {
assert(!evalStore);
if (buildMode != bmNormal) throw Error("unsupported build mode"); if (buildMode != bmNormal) throw Error("unsupported build mode");
StorePathSet newPaths; StorePathSet newPaths;
@@ -1754,7 +1671,7 @@ void LocalDerivationGoal::runChild()
/* N.B. it is realistic that these paths might not exist. It /* N.B. it is realistic that these paths might not exist. It
happens when testing Nix building fixed-output derivations happens when testing Nix building fixed-output derivations
within a pure derivation. */ within a pure derivation. */
for (auto & path : { "/etc/resolv.conf", "/etc/services", "/etc/hosts", "/var/run/nscd/socket" }) for (auto & path : { "/etc/resolv.conf", "/etc/services", "/etc/hosts" })
if (pathExists(path)) if (pathExists(path))
ss.push_back(path); ss.push_back(path);
} }
+1 -1
View File
@@ -204,7 +204,7 @@ void PathSubstitutionGoal::tryToRun()
Activity act(*logger, actSubstitute, Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()}); Activity act(*logger, actSubstitute, Logger::Fields{worker.store.printStorePath(storePath), sub->getUri()});
PushActivity pact(act.id); PushActivity pact(act.id);
copyStorePath(ref<Store>(sub), ref<Store>(worker.store.shared_from_this()), copyStorePath(*sub, worker.store,
subPath ? *subPath : storePath, repair, sub->isTrusted ? NoCheckSigs : CheckSigs); subPath ? *subPath : storePath, repair, sub->isTrusted ? NoCheckSigs : CheckSigs);
promise.set_value(); promise.set_value();
+2 -1
View File
@@ -9,11 +9,12 @@
namespace nix { namespace nix {
Worker::Worker(Store & store) Worker::Worker(Store & store, Store & evalStore)
: act(*logger, actRealise) : act(*logger, actRealise)
, actDerivations(*logger, actBuilds) , actDerivations(*logger, actBuilds)
, actSubstitutions(*logger, actCopyPaths) , actSubstitutions(*logger, actCopyPaths)
, store(store) , store(store)
, evalStore(evalStore)
{ {
/* Debugging: prevent recursive workers. */ /* Debugging: prevent recursive workers. */
nrLocalBuilds = 0; nrLocalBuilds = 0;
+2 -1
View File
@@ -110,6 +110,7 @@ public:
bool checkMismatch; bool checkMismatch;
Store & store; Store & store;
Store & evalStore;
std::unique_ptr<HookInstance> hook; std::unique_ptr<HookInstance> hook;
@@ -131,7 +132,7 @@ public:
it answers with "decline-permanently", we don't try again. */ it answers with "decline-permanently", we don't try again. */
bool tryBuildHook = true; bool tryBuildHook = true;
Worker(Store & store); Worker(Store & store, Store & evalStore);
~Worker(); ~Worker();
/* Make a goal (with caching). */ /* Make a goal (with caching). */
+17 -21
View File
@@ -243,23 +243,6 @@ struct ClientSettings
} }
}; };
static void writeValidPathInfo(
ref<Store> store,
unsigned int clientVersion,
Sink & to,
std::shared_ptr<const ValidPathInfo> info)
{
to << (info->deriver ? store->printStorePath(*info->deriver) : "")
<< info->narHash.to_string(Base16, false);
worker_proto::write(*store, to, info->references);
to << info->registrationTime << info->narSize;
if (GET_PROTOCOL_MINOR(clientVersion) >= 16) {
to << info->ultimate
<< info->sigs
<< renderContentAddress(info->ca);
}
}
static std::vector<DerivedPath> readDerivedPaths(Store & store, unsigned int clientVersion, Source & from) static std::vector<DerivedPath> readDerivedPaths(Store & store, unsigned int clientVersion, Source & from)
{ {
std::vector<DerivedPath> reqs; std::vector<DerivedPath> reqs;
@@ -422,9 +405,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
}(); }();
logger->stopWork(); logger->stopWork();
to << store->printStorePath(pathInfo->path); pathInfo->write(to, *store, GET_PROTOCOL_MINOR(clientVersion));
writeValidPathInfo(store, clientVersion, to, pathInfo);
} else { } else {
HashType hashAlgo; HashType hashAlgo;
std::string baseName; std::string baseName;
@@ -471,6 +452,21 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break; break;
} }
case wopAddMultipleToStore: {
bool repair, dontCheckSigs;
from >> repair >> dontCheckSigs;
if (!trusted && dontCheckSigs)
dontCheckSigs = false;
logger->startWork();
FramedSource source(from);
store->addMultipleToStore(source,
RepairFlag{repair},
dontCheckSigs ? NoCheckSigs : CheckSigs);
logger->stopWork();
break;
}
case wopAddTextToStore: { case wopAddTextToStore: {
string suffix = readString(from); string suffix = readString(from);
string s = readString(from); string s = readString(from);
@@ -770,7 +766,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
if (info) { if (info) {
if (GET_PROTOCOL_MINOR(clientVersion) >= 17) if (GET_PROTOCOL_MINOR(clientVersion) >= 17)
to << 1; to << 1;
writeValidPathInfo(store, clientVersion, to, info); info->write(to, *store, GET_PROTOCOL_MINOR(clientVersion), false);
} else { } else {
assert(GET_PROTOCOL_MINOR(clientVersion) >= 17); assert(GET_PROTOCOL_MINOR(clientVersion) >= 17);
to << 0; to << 0;
+1 -1
View File
@@ -568,7 +568,7 @@ DrvHashModulo hashDerivationModulo(Store & store, const Derivation & drv, bool m
} }
std::map<std::string, Hash> staticOutputHashes(Store& store, const Derivation& drv) std::map<std::string, Hash> staticOutputHashes(Store & store, const Derivation & drv)
{ {
std::map<std::string, Hash> res; std::map<std::string, Hash> res;
std::visit(overloaded { std::visit(overloaded {
-5
View File
@@ -43,11 +43,6 @@ struct DummyStore : public virtual DummyStoreConfig, public virtual Store
RepairFlag repair, CheckSigsFlag checkSigs) override RepairFlag repair, CheckSigsFlag checkSigs) override
{ unsupported("addToStore"); } { unsupported("addToStore"); }
StorePath addToStore(const string & name, const Path & srcPath,
FileIngestionMethod method, HashType hashAlgo,
PathFilter & filter, RepairFlag repair) override
{ unsupported("addToStore"); }
StorePath addTextToStore(const string & name, const string & s, StorePath addTextToStore(const string & name, const string & s,
const StorePathSet & references, RepairFlag repair) override const StorePathSet & references, RepairFlag repair) override
{ unsupported("addTextToStore"); } { unsupported("addTextToStore"); }
+3
View File
@@ -956,6 +956,9 @@ public:
resolves to a different location from that of the build machine. You resolves to a different location from that of the build machine. You
can enable this setting if you are sure you're not going to do that. can enable this setting if you are sure you're not going to do that.
)"}; )"};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references."};
}; };
+4 -1
View File
@@ -267,8 +267,11 @@ public:
return status; return status;
} }
void buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode) override void buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore) override
{ {
if (evalStore && evalStore.get() != this)
throw Error("building on an SSH store is incompatible with '--eval-store'");
auto conn(connections->get()); auto conn(connections->get());
conn->to << cmdBuildPaths; conn->to << cmdBuildPaths;
+4 -4
View File
@@ -342,7 +342,7 @@ LocalStore::LocalStore(const Params & params)
if (settings.isExperimentalFeatureEnabled("ca-derivations")) { if (settings.isExperimentalFeatureEnabled("ca-derivations")) {
state->stmts->RegisterRealisedOutput.create(state->db, state->stmts->RegisterRealisedOutput.create(state->db,
R"( R"(
insert or replace into Realisations (drvPath, outputName, outputPath, signatures) insert into Realisations (drvPath, outputName, outputPath, signatures)
values (?, ?, (select id from ValidPaths where path = ?), ?) values (?, ?, (select id from ValidPaths where path = ?), ?)
; ;
)"); )");
@@ -379,7 +379,7 @@ LocalStore::LocalStore(const Params & params)
R"( R"(
insert or replace into RealisationsRefs (referrer, realisationReference) insert or replace into RealisationsRefs (referrer, realisationReference)
values ( values (
?, (select id from Realisations where drvPath = ? and outputName = ?),
(select id from Realisations where drvPath = ? and outputName = ?)); (select id from Realisations where drvPath = ? and outputName = ?));
)"); )");
} }
@@ -748,7 +748,6 @@ void LocalStore::registerDrvOutput(const Realisation & info)
(concatStringsSep(" ", info.signatures)) (concatStringsSep(" ", info.signatures))
.exec(); .exec();
} }
uint64_t myId = state->db.getLastInsertedRowId();
for (auto & [outputId, depPath] : info.dependentRealisations) { for (auto & [outputId, depPath] : info.dependentRealisations) {
auto localRealisation = queryRealisationCore_(*state, outputId); auto localRealisation = queryRealisationCore_(*state, outputId);
if (!localRealisation) if (!localRealisation)
@@ -761,7 +760,8 @@ void LocalStore::registerDrvOutput(const Realisation & info)
"match what we have locally", "match what we have locally",
info.id.to_string(), outputId.to_string()); info.id.to_string(), outputId.to_string());
state->stmts->AddRealisationReference.use() state->stmts->AddRealisationReference.use()
(myId) (info.id.strHash())
(info.id.outputName)
(outputId.strHash()) (outputId.strHash())
(outputId.outputName) (outputId.outputName)
.exec(); .exec();
+3 -3
View File
@@ -9,11 +9,11 @@ libstore_SOURCES := $(wildcard $(d)/*.cc $(d)/builtins/*.cc $(d)/build/*.cc)
libstore_LIBS = libutil libstore_LIBS = libutil
libstore_LDFLAGS = $(SQLITE3_LIBS) -lbz2 $(LIBCURL_LIBS) $(SODIUM_LIBS) -pthread libstore_LDFLAGS = $(SQLITE3_LIBS) -lbz2 $(LIBCURL_LIBS) $(SODIUM_LIBS) -pthread
ifeq ($(OS), Linux) ifdef HOST_LINUX
libstore_LDFLAGS += -ldl libstore_LDFLAGS += -ldl
endif endif
ifeq ($(OS), Darwin) ifdef HOST_DARWIN
libstore_FILES = sandbox-defaults.sb sandbox-minimal.sb sandbox-network.sb libstore_FILES = sandbox-defaults.sb sandbox-minimal.sb sandbox-network.sb
endif endif
@@ -23,7 +23,7 @@ ifeq ($(ENABLE_S3), 1)
libstore_LDFLAGS += -laws-cpp-sdk-transfer -laws-cpp-sdk-s3 -laws-cpp-sdk-core libstore_LDFLAGS += -laws-cpp-sdk-transfer -laws-cpp-sdk-s3 -laws-cpp-sdk-core
endif endif
ifeq ($(OS), SunOS) ifdef HOST_SOLARIS
libstore_LDFLAGS += -lsocket libstore_LDFLAGS += -lsocket
endif endif
+11 -6
View File
@@ -16,13 +16,18 @@ Machine::Machine(decltype(storeUri) storeUri,
decltype(mandatoryFeatures) mandatoryFeatures, decltype(mandatoryFeatures) mandatoryFeatures,
decltype(sshPublicHostKey) sshPublicHostKey) : decltype(sshPublicHostKey) sshPublicHostKey) :
storeUri( storeUri(
// Backwards compatibility: if the URI is a hostname, // Backwards compatibility: if the URI is schemeless, is not a path,
// prepend ssh://. // and is not one of the special store connection words, prepend
// ssh://.
storeUri.find("://") != std::string::npos storeUri.find("://") != std::string::npos
|| hasPrefix(storeUri, "local") || storeUri.find("/") != std::string::npos
|| hasPrefix(storeUri, "remote") || storeUri == "auto"
|| hasPrefix(storeUri, "auto") || storeUri == "daemon"
|| hasPrefix(storeUri, "/") || storeUri == "local"
|| hasPrefix(storeUri, "auto?")
|| hasPrefix(storeUri, "daemon?")
|| hasPrefix(storeUri, "local?")
|| hasPrefix(storeUri, "?")
? storeUri ? storeUri
: "ssh://" + storeUri), : "ssh://" + storeUri),
systemTypes(systemTypes), systemTypes(systemTypes),
+105
View File
@@ -1,6 +1,8 @@
#include "parsed-derivations.hh" #include "parsed-derivations.hh"
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
#include <regex>
#include "json.hh"
namespace nix { namespace nix {
@@ -123,4 +125,107 @@ bool ParsedDerivation::substitutesAllowed() const
return getBoolAttr("allowSubstitutes", true); return getBoolAttr("allowSubstitutes", true);
} }
static std::regex shVarName("[A-Za-z_][A-Za-z0-9_]*");
std::optional<nlohmann::json> ParsedDerivation::prepareStructuredAttrs(Store & store, const StorePathSet & inputPaths)
{
auto structuredAttrs = getStructuredAttrs();
if (!structuredAttrs) return std::nullopt;
auto json = *structuredAttrs;
/* Add an "outputs" object containing the output paths. */
nlohmann::json outputs;
for (auto & i : drv.outputs)
outputs[i.first] = hashPlaceholder(i.first);
json["outputs"] = outputs;
/* Handle exportReferencesGraph. */
auto e = json.find("exportReferencesGraph");
if (e != json.end() && e->is_object()) {
for (auto i = e->begin(); i != e->end(); ++i) {
std::ostringstream str;
{
JSONPlaceholder jsonRoot(str, true);
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(store.parseStorePath(p.get<std::string>()));
store.pathInfoToJSON(jsonRoot,
store.exportReferences(storePaths, inputPaths), false, true);
}
json[i.key()] = nlohmann::json::parse(str.str()); // urgh
}
}
return json;
}
/* As a convenience to bash scripts, write a shell file that
maps all attributes that are representable in bash -
namely, strings, integers, nulls, Booleans, and arrays and
objects consisting entirely of those values. (So nested
arrays or objects are not supported.) */
std::string writeStructuredAttrsShell(const nlohmann::json & json)
{
auto handleSimpleType = [](const nlohmann::json & value) -> std::optional<std::string> {
if (value.is_string())
return shellEscape(value);
if (value.is_number()) {
auto f = value.get<float>();
if (std::ceil(f) == f)
return std::to_string(value.get<int>());
}
if (value.is_null())
return std::string("''");
if (value.is_boolean())
return value.get<bool>() ? std::string("1") : std::string("");
return {};
};
std::string jsonSh;
for (auto & [key, value] : json.items()) {
if (!std::regex_match(key, shVarName)) continue;
auto s = handleSimpleType(value);
if (s)
jsonSh += fmt("declare %s=%s\n", key, *s);
else if (value.is_array()) {
std::string s2;
bool good = true;
for (auto & value2 : value) {
auto s3 = handleSimpleType(value2);
if (!s3) { good = false; break; }
s2 += *s3; s2 += ' ';
}
if (good)
jsonSh += fmt("declare -a %s=(%s)\n", key, s2);
}
else if (value.is_object()) {
std::string s2;
bool good = true;
for (auto & [key2, value2] : value.items()) {
auto s3 = handleSimpleType(value2);
if (!s3) { good = false; break; }
s2 += fmt("[%s]=%s ", shellEscape(key2), *s3);
}
if (good)
jsonSh += fmt("declare -A %s=(%s)\n", key, s2);
}
}
return jsonSh;
}
} }
+4
View File
@@ -36,6 +36,10 @@ public:
bool willBuildLocally(Store & localStore) const; bool willBuildLocally(Store & localStore) const;
bool substitutesAllowed() const; bool substitutesAllowed() const;
std::optional<nlohmann::json> prepareStructuredAttrs(Store & store, const StorePathSet & inputPaths);
}; };
std::string writeStructuredAttrsShell(const nlohmann::json & json);
} }
+46
View File
@@ -0,0 +1,46 @@
#include "path-info.hh"
#include "worker-protocol.hh"
namespace nix {
ValidPathInfo ValidPathInfo::read(Source & source, const Store & store, unsigned int format)
{
return read(source, store, format, store.parseStorePath(readString(source)));
}
ValidPathInfo ValidPathInfo::read(Source & source, const Store & store, unsigned int format, StorePath && path)
{
auto deriver = readString(source);
auto narHash = Hash::parseAny(readString(source), htSHA256);
ValidPathInfo info(path, narHash);
if (deriver != "") info.deriver = store.parseStorePath(deriver);
info.references = worker_proto::read(store, source, Phantom<StorePathSet> {});
source >> info.registrationTime >> info.narSize;
if (format >= 16) {
source >> info.ultimate;
info.sigs = readStrings<StringSet>(source);
info.ca = parseContentAddressOpt(readString(source));
}
return info;
}
void ValidPathInfo::write(
Sink & sink,
const Store & store,
unsigned int format,
bool includePath) const
{
if (includePath)
sink << store.printStorePath(path);
sink << (deriver ? store.printStorePath(*deriver) : "")
<< narHash.to_string(Base16, false);
worker_proto::write(store, sink, references);
sink << registrationTime << narSize;
if (format >= 16) {
sink << ultimate
<< sigs
<< renderContentAddress(ca);
}
}
}
+5
View File
@@ -105,6 +105,11 @@ struct ValidPathInfo
ValidPathInfo(const StorePath & path, Hash narHash) : path(path), narHash(narHash) { }; ValidPathInfo(const StorePath & path, Hash narHash) : path(path), narHash(narHash) { };
virtual ~ValidPathInfo() { } virtual ~ValidPathInfo() { }
static ValidPathInfo read(Source & source, const Store & store, unsigned int format);
static ValidPathInfo read(Source & source, const Store & store, unsigned int format, StorePath && path);
void write(Sink & sink, const Store & store, unsigned int format, bool includePath = true) const;
}; };
typedef std::map<StorePath, ValidPathInfo> ValidPathInfos; typedef std::map<StorePath, ValidPathInfo> ValidPathInfos;
+4 -1
View File
@@ -211,12 +211,15 @@ void deleteGenerationsOlderThan(const Path & profile, time_t t, bool dryRun)
void deleteGenerationsOlderThan(const Path & profile, const string & timeSpec, bool dryRun) void deleteGenerationsOlderThan(const Path & profile, const string & timeSpec, bool dryRun)
{ {
if (timeSpec.empty() || timeSpec[timeSpec.size() - 1] != 'd')
throw UsageError("invalid number of days specifier '%1%', expected something like '14d'", timeSpec);
time_t curTime = time(0); time_t curTime = time(0);
string strDays = string(timeSpec, 0, timeSpec.size() - 1); string strDays = string(timeSpec, 0, timeSpec.size() - 1);
auto days = string2Int<int>(strDays); auto days = string2Int<int>(strDays);
if (!days || *days < 1) if (!days || *days < 1)
throw Error("invalid number of days specifier '%1%'", timeSpec); throw UsageError("invalid number of days specifier '%1%'", timeSpec);
time_t oldTime = curTime - *days * 24 * 3600; time_t oldTime = curTime - *days * 24 * 3600;
+10 -2
View File
@@ -144,8 +144,16 @@ bool Realisation::isCompatibleWith(const Realisation & other) const
{ {
assert (id == other.id); assert (id == other.id);
if (outPath == other.outPath) { if (outPath == other.outPath) {
assert(dependentRealisations == other.dependentRealisations); if (dependentRealisations.empty() != other.dependentRealisations.empty()) {
return true; warn(
"Encountered a realisation for '%s' with an empty set of "
"dependencies. This is likely an artifact from an older Nix. "
"I’ll try to fix the realisation if I can",
id.to_string());
return true;
} else if (dependentRealisations == other.dependentRealisations) {
return true;
}
} }
return false; return false;
} }
+1 -1
View File
@@ -45,7 +45,7 @@ struct Realisation {
size_t checkSignatures(const PublicKeys & publicKeys) const; size_t checkSignatures(const PublicKeys & publicKeys) const;
static std::set<Realisation> closure(Store &, const std::set<Realisation> &); static std::set<Realisation> closure(Store &, const std::set<Realisation> &);
static void closure(Store &, const std::set<Realisation> &, std::set<Realisation>& res); static void closure(Store &, const std::set<Realisation> &, std::set<Realisation> & res);
bool isCompatibleWith(const Realisation & other) const; bool isCompatibleWith(const Realisation & other) const;
+37 -25
View File
@@ -386,23 +386,6 @@ void RemoteStore::querySubstitutablePathInfos(const StorePathCAMap & pathsMap, S
} }
ref<const ValidPathInfo> RemoteStore::readValidPathInfo(ConnectionHandle & conn, const StorePath & path)
{
auto deriver = readString(conn->from);
auto narHash = Hash::parseAny(readString(conn->from), htSHA256);
auto info = make_ref<ValidPathInfo>(path, narHash);
if (deriver != "") info->deriver = parseStorePath(deriver);
info->references = worker_proto::read(*this, conn->from, Phantom<StorePathSet> {});
conn->from >> info->registrationTime >> info->narSize;
if (GET_PROTOCOL_MINOR(conn->daemonVersion) >= 16) {
conn->from >> info->ultimate;
info->sigs = readStrings<StringSet>(conn->from);
info->ca = parseContentAddressOpt(readString(conn->from));
}
return info;
}
void RemoteStore::queryPathInfoUncached(const StorePath & path, void RemoteStore::queryPathInfoUncached(const StorePath & path,
Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept
{ {
@@ -423,7 +406,8 @@ void RemoteStore::queryPathInfoUncached(const StorePath & path,
bool valid; conn->from >> valid; bool valid; conn->from >> valid;
if (!valid) throw InvalidPath("path '%s' is not valid", printStorePath(path)); if (!valid) throw InvalidPath("path '%s' is not valid", printStorePath(path));
} }
info = readValidPathInfo(conn, path); info = std::make_shared<ValidPathInfo>(
ValidPathInfo::read(conn->from, *this, GET_PROTOCOL_MINOR(conn->daemonVersion), StorePath{path}));
} }
callback(std::move(info)); callback(std::move(info));
} catch (...) { callback.rethrow(); } } catch (...) { callback.rethrow(); }
@@ -525,8 +509,8 @@ ref<const ValidPathInfo> RemoteStore::addCAToStore(
}); });
} }
auto path = parseStorePath(readString(conn->from)); return make_ref<ValidPathInfo>(
return readValidPathInfo(conn, path); ValidPathInfo::read(conn->from, *this, GET_PROTOCOL_MINOR(conn->daemonVersion)));
} }
else { else {
if (repair) throw Error("repairing is not supported when building through the Nix daemon protocol < 1.25"); if (repair) throw Error("repairing is not supported when building through the Nix daemon protocol < 1.25");
@@ -642,6 +626,25 @@ void RemoteStore::addToStore(const ValidPathInfo & info, Source & source,
} }
void RemoteStore::addMultipleToStore(
Source & source,
RepairFlag repair,
CheckSigsFlag checkSigs)
{
if (GET_PROTOCOL_MINOR(getConnection()->daemonVersion) >= 32) {
auto conn(getConnection());
conn->to
<< wopAddMultipleToStore
<< repair
<< !checkSigs;
conn.withFramedSink([&](Sink & sink) {
source.drainInto(sink);
});
} else
Store::addMultipleToStore(source, repair, checkSigs);
}
StorePath RemoteStore::addTextToStore(const string & name, const string & s, StorePath RemoteStore::addTextToStore(const string & name, const string & s,
const StorePathSet & references, RepairFlag repair) const StorePathSet & references, RepairFlag repair)
{ {
@@ -705,8 +708,18 @@ static void writeDerivedPaths(RemoteStore & store, ConnectionHandle & conn, cons
} }
} }
void RemoteStore::buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode) void RemoteStore::buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore)
{ {
if (evalStore && evalStore.get() != this) {
/* The remote doesn't have a way to access evalStore, so copy
the .drvs. */
RealisedPath::Set drvPaths2;
for (auto & i : drvPaths)
if (auto p = std::get_if<DerivedPath::Built>(&i))
drvPaths2.insert(p->drvPath);
copyClosure(*evalStore, *this, drvPaths2);
}
auto conn(getConnection()); auto conn(getConnection());
conn->to << wopBuildPaths; conn->to << wopBuildPaths;
assert(GET_PROTOCOL_MINOR(conn->daemonVersion) >= 13); assert(GET_PROTOCOL_MINOR(conn->daemonVersion) >= 13);
@@ -1001,14 +1014,14 @@ std::exception_ptr RemoteStore::Connection::processStderr(Sink * sink, Source *
return nullptr; return nullptr;
} }
void ConnectionHandle::withFramedSink(std::function<void(Sink &sink)> fun) void ConnectionHandle::withFramedSink(std::function<void(Sink & sink)> fun)
{ {
(*this)->to.flush(); (*this)->to.flush();
std::exception_ptr ex; std::exception_ptr ex;
/* Handle log messages / exceptions from the remote on a /* Handle log messages / exceptions from the remote on a separate
separate thread. */ thread. */
std::thread stderrThread([&]() std::thread stderrThread([&]()
{ {
try { try {
@@ -1041,7 +1054,6 @@ void ConnectionHandle::withFramedSink(std::function<void(Sink &sink)> fun)
stderrThread.join(); stderrThread.join();
if (ex) if (ex)
std::rethrow_exception(ex); std::rethrow_exception(ex);
} }
} }
+6 -3
View File
@@ -78,6 +78,11 @@ public:
void addToStore(const ValidPathInfo & info, Source & nar, void addToStore(const ValidPathInfo & info, Source & nar,
RepairFlag repair, CheckSigsFlag checkSigs) override; RepairFlag repair, CheckSigsFlag checkSigs) override;
void addMultipleToStore(
Source & source,
RepairFlag repair,
CheckSigsFlag checkSigs) override;
StorePath addTextToStore(const string & name, const string & s, StorePath addTextToStore(const string & name, const string & s,
const StorePathSet & references, RepairFlag repair) override; const StorePathSet & references, RepairFlag repair) override;
@@ -85,7 +90,7 @@ public:
std::optional<const Realisation> queryRealisation(const DrvOutput &) override; std::optional<const Realisation> queryRealisation(const DrvOutput &) override;
void buildPaths(const std::vector<DerivedPath> & paths, BuildMode buildMode) override; void buildPaths(const std::vector<DerivedPath> & paths, BuildMode buildMode, std::shared_ptr<Store> evalStore) override;
BuildResult buildDerivation(const StorePath & drvPath, const BasicDerivation & drv, BuildResult buildDerivation(const StorePath & drvPath, const BasicDerivation & drv,
BuildMode buildMode) override; BuildMode buildMode) override;
@@ -151,8 +156,6 @@ protected:
virtual void narFromPath(const StorePath & path, Sink & sink) override; virtual void narFromPath(const StorePath & path, Sink & sink) override;
ref<const ValidPathInfo> readValidPathInfo(ConnectionHandle & conn, const StorePath & path);
private: private:
std::atomic_bool failed{false}; std::atomic_bool failed{false};
+161 -41
View File
@@ -9,6 +9,7 @@
#include "url.hh" #include "url.hh"
#include "archive.hh" #include "archive.hh"
#include "callback.hh" #include "callback.hh"
#include "remote-store.hh"
#include <regex> #include <regex>
@@ -249,6 +250,20 @@ StorePath Store::addToStore(const string & name, const Path & _srcPath,
} }
void Store::addMultipleToStore(
Source & source,
RepairFlag repair,
CheckSigsFlag checkSigs)
{
auto expected = readNum<uint64_t>(source);
for (uint64_t i = 0; i < expected; ++i) {
auto info = ValidPathInfo::read(source, *this, 16);
info.ultimate = false;
addToStore(info, source, repair, checkSigs);
}
}
/* /*
The aim of this function is to compute in one pass the correct ValidPathInfo for The aim of this function is to compute in one pass the correct ValidPathInfo for
the files that we are trying to add to the store. To accomplish that in one the files that we are trying to add to the store. To accomplish that in one
@@ -634,6 +649,42 @@ string Store::makeValidityRegistration(const StorePathSet & paths,
} }
StorePathSet Store::exportReferences(const StorePathSet & storePaths, const StorePathSet & inputPaths)
{
StorePathSet paths;
for (auto & storePath : storePaths) {
if (!inputPaths.count(storePath))
throw BuildError("cannot export references of path '%s' because it is not in the input closure of the derivation", printStorePath(storePath));
computeFSClosure({storePath}, paths);
}
/* If there are derivations in the graph, then include their
outputs as well. This is useful if you want to do things
like passing all build-time dependencies of some path to a
derivation that builds a NixOS DVD image. */
auto paths2 = paths;
for (auto & j : paths2) {
if (j.isDerivation()) {
Derivation drv = derivationFromPath(j);
for (auto & k : drv.outputsAndOptPaths(*this)) {
if (!k.second.second)
/* FIXME: I am confused why we are calling
`computeFSClosure` on the output path, rather than
derivation itself. That doesn't seem right to me, so I
won't try to implemented this for CA derivations. */
throw UnimplementedError("exportReferences on CA derivations is not yet implemented");
computeFSClosure(*k.second.second, paths);
}
}
}
return paths;
}
void Store::pathInfoToJSON(JSONPlaceholder & jsonOut, const StorePathSet & storePaths, void Store::pathInfoToJSON(JSONPlaceholder & jsonOut, const StorePathSet & storePaths,
bool includeImpureInfo, bool showClosureSize, bool includeImpureInfo, bool showClosureSize,
Base hashBase, Base hashBase,
@@ -734,30 +785,43 @@ const Store::Stats & Store::getStats()
} }
void copyStorePath(ref<Store> srcStore, ref<Store> dstStore, static std::string makeCopyPathMessage(
const StorePath & storePath, RepairFlag repair, CheckSigsFlag checkSigs) std::string_view srcUri,
std::string_view dstUri,
std::string_view storePath)
{ {
auto srcUri = srcStore->getUri(); return srcUri == "local" || srcUri == "daemon"
auto dstUri = dstStore->getUri(); ? fmt("copying path '%s' to '%s'", storePath, dstUri)
: dstUri == "local" || dstUri == "daemon"
? fmt("copying path '%s' from '%s'", storePath, srcUri)
: fmt("copying path '%s' from '%s' to '%s'", storePath, srcUri, dstUri);
}
void copyStorePath(
Store & srcStore,
Store & dstStore,
const StorePath & storePath,
RepairFlag repair,
CheckSigsFlag checkSigs)
{
auto srcUri = srcStore.getUri();
auto dstUri = dstStore.getUri();
auto storePathS = srcStore.printStorePath(storePath);
Activity act(*logger, lvlInfo, actCopyPath, Activity act(*logger, lvlInfo, actCopyPath,
srcUri == "local" || srcUri == "daemon" makeCopyPathMessage(srcUri, dstUri, storePathS),
? fmt("copying path '%s' to '%s'", srcStore->printStorePath(storePath), dstUri) {storePathS, srcUri, dstUri});
: dstUri == "local" || dstUri == "daemon"
? fmt("copying path '%s' from '%s'", srcStore->printStorePath(storePath), srcUri)
: fmt("copying path '%s' from '%s' to '%s'", srcStore->printStorePath(storePath), srcUri, dstUri),
{srcStore->printStorePath(storePath), srcUri, dstUri});
PushActivity pact(act.id); PushActivity pact(act.id);
auto info = srcStore->queryPathInfo(storePath); auto info = srcStore.queryPathInfo(storePath);
uint64_t total = 0; uint64_t total = 0;
// recompute store path on the chance dstStore does it differently // recompute store path on the chance dstStore does it differently
if (info->ca && info->references.empty()) { if (info->ca && info->references.empty()) {
auto info2 = make_ref<ValidPathInfo>(*info); auto info2 = make_ref<ValidPathInfo>(*info);
info2->path = dstStore->makeFixedOutputPathFromCA(info->path.name(), *info->ca); info2->path = dstStore.makeFixedOutputPathFromCA(info->path.name(), *info->ca);
if (dstStore->storeDir == srcStore->storeDir) if (dstStore.storeDir == srcStore.storeDir)
assert(info->path == info2->path); assert(info->path == info2->path);
info = info2; info = info2;
} }
@@ -774,17 +838,22 @@ void copyStorePath(ref<Store> srcStore, ref<Store> dstStore,
act.progress(total, info->narSize); act.progress(total, info->narSize);
}); });
TeeSink tee { sink, progressSink }; TeeSink tee { sink, progressSink };
srcStore->narFromPath(storePath, tee); srcStore.narFromPath(storePath, tee);
}, [&]() { }, [&]() {
throw EndOfFile("NAR for '%s' fetched from '%s' is incomplete", srcStore->printStorePath(storePath), srcStore->getUri()); throw EndOfFile("NAR for '%s' fetched from '%s' is incomplete", srcStore.printStorePath(storePath), srcStore.getUri());
}); });
dstStore->addToStore(*info, *source, repair, checkSigs); dstStore.addToStore(*info, *source, repair, checkSigs);
} }
std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStore, const RealisedPath::Set & paths, std::map<StorePath, StorePath> copyPaths(
RepairFlag repair, CheckSigsFlag checkSigs, SubstituteFlag substitute) Store & srcStore,
Store & dstStore,
const RealisedPath::Set & paths,
RepairFlag repair,
CheckSigsFlag checkSigs,
SubstituteFlag substitute)
{ {
StorePathSet storePaths; StorePathSet storePaths;
std::set<Realisation> toplevelRealisations; std::set<Realisation> toplevelRealisations;
@@ -802,24 +871,24 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
try { try {
// Copy the realisation closure // Copy the realisation closure
processGraph<Realisation>( processGraph<Realisation>(
pool, Realisation::closure(*srcStore, toplevelRealisations), pool, Realisation::closure(srcStore, toplevelRealisations),
[&](const Realisation& current) -> std::set<Realisation> { [&](const Realisation & current) -> std::set<Realisation> {
std::set<Realisation> children; std::set<Realisation> children;
for (const auto& [drvOutput, _] : current.dependentRealisations) { for (const auto & [drvOutput, _] : current.dependentRealisations) {
auto currentChild = srcStore->queryRealisation(drvOutput); auto currentChild = srcStore.queryRealisation(drvOutput);
if (!currentChild) if (!currentChild)
throw Error( throw Error(
"Incomplete realisation closure: '%s' is a " "incomplete realisation closure: '%s' is a "
"dependency of '%s' but isn’t registered", "dependency of '%s' but isn't registered",
drvOutput.to_string(), current.id.to_string()); drvOutput.to_string(), current.id.to_string());
children.insert(*currentChild); children.insert(*currentChild);
} }
return children; return children;
}, },
[&](const Realisation& current) -> void { [&](const Realisation& current) -> void {
dstStore->registerDrvOutput(current, checkSigs); dstStore.registerDrvOutput(current, checkSigs);
}); });
} catch (MissingExperimentalFeature& e) { } catch (MissingExperimentalFeature & e) {
// Don't fail if the remote doesn't support CA derivations is it might // Don't fail if the remote doesn't support CA derivations is it might
// not be within our control to change that, and we might still want // not be within our control to change that, and we might still want
// to at least copy the output paths. // to at least copy the output paths.
@@ -832,10 +901,15 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
return pathsMap; return pathsMap;
} }
std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStore, const StorePathSet & storePaths, std::map<StorePath, StorePath> copyPaths(
RepairFlag repair, CheckSigsFlag checkSigs, SubstituteFlag substitute) Store & srcStore,
Store & dstStore,
const StorePathSet & storePaths,
RepairFlag repair,
CheckSigsFlag checkSigs,
SubstituteFlag substitute)
{ {
auto valid = dstStore->queryValidPaths(storePaths, substitute); auto valid = dstStore.queryValidPaths(storePaths, substitute);
StorePathSet missing; StorePathSet missing;
for (auto & path : storePaths) for (auto & path : storePaths)
@@ -845,9 +919,31 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
for (auto & path : storePaths) for (auto & path : storePaths)
pathsMap.insert_or_assign(path, path); pathsMap.insert_or_assign(path, path);
Activity act(*logger, lvlInfo, actCopyPaths, fmt("copying %d paths", missing.size())); Activity act(*logger, lvlInfo, actCopyPaths, fmt("copying %d paths", missing.size()));
auto sorted = srcStore.topoSortPaths(missing);
std::reverse(sorted.begin(), sorted.end());
auto source = sinkToSource([&](Sink & sink) {
sink << sorted.size();
for (auto & storePath : sorted) {
auto srcUri = srcStore.getUri();
auto dstUri = dstStore.getUri();
auto storePathS = srcStore.printStorePath(storePath);
Activity act(*logger, lvlInfo, actCopyPath,
makeCopyPathMessage(srcUri, dstUri, storePathS),
{storePathS, srcUri, dstUri});
PushActivity pact(act.id);
auto info = srcStore.queryPathInfo(storePath);
info->write(sink, srcStore, 16);
srcStore.narFromPath(storePath, sink);
}
});
dstStore.addMultipleToStore(*source, repair, checkSigs);
#if 0
std::atomic<size_t> nrDone{0}; std::atomic<size_t> nrDone{0};
std::atomic<size_t> nrFailed{0}; std::atomic<size_t> nrFailed{0};
std::atomic<uint64_t> bytesExpected{0}; std::atomic<uint64_t> bytesExpected{0};
@@ -863,18 +959,21 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
StorePathSet(missing.begin(), missing.end()), StorePathSet(missing.begin(), missing.end()),
[&](const StorePath & storePath) { [&](const StorePath & storePath) {
auto info = srcStore->queryPathInfo(storePath); auto info = srcStore.queryPathInfo(storePath);
auto storePathForDst = storePath; auto storePathForDst = storePath;
if (info->ca && info->references.empty()) { if (info->ca && info->references.empty()) {
storePathForDst = dstStore->makeFixedOutputPathFromCA(storePath.name(), *info->ca); storePathForDst = dstStore.makeFixedOutputPathFromCA(storePath.name(), *info->ca);
if (dstStore->storeDir == srcStore->storeDir) if (dstStore.storeDir == srcStore.storeDir)
assert(storePathForDst == storePath); assert(storePathForDst == storePath);
if (storePathForDst != storePath) if (storePathForDst != storePath)
debug("replaced path '%s' to '%s' for substituter '%s'", srcStore->printStorePath(storePath), dstStore->printStorePath(storePathForDst), dstStore->getUri()); debug("replaced path '%s' to '%s' for substituter '%s'",
srcStore.printStorePath(storePath),
dstStore.printStorePath(storePathForDst),
dstStore.getUri());
} }
pathsMap.insert_or_assign(storePath, storePathForDst); pathsMap.insert_or_assign(storePath, storePathForDst);
if (dstStore->isValidPath(storePath)) { if (dstStore.isValidPath(storePath)) {
nrDone++; nrDone++;
showProgress(); showProgress();
return StorePathSet(); return StorePathSet();
@@ -889,19 +988,22 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
[&](const StorePath & storePath) { [&](const StorePath & storePath) {
checkInterrupt(); checkInterrupt();
auto info = srcStore->queryPathInfo(storePath); auto info = srcStore.queryPathInfo(storePath);
auto storePathForDst = storePath; auto storePathForDst = storePath;
if (info->ca && info->references.empty()) { if (info->ca && info->references.empty()) {
storePathForDst = dstStore->makeFixedOutputPathFromCA(storePath.name(), *info->ca); storePathForDst = dstStore.makeFixedOutputPathFromCA(storePath.name(), *info->ca);
if (dstStore->storeDir == srcStore->storeDir) if (dstStore.storeDir == srcStore.storeDir)
assert(storePathForDst == storePath); assert(storePathForDst == storePath);
if (storePathForDst != storePath) if (storePathForDst != storePath)
debug("replaced path '%s' to '%s' for substituter '%s'", srcStore->printStorePath(storePath), dstStore->printStorePath(storePathForDst), dstStore->getUri()); debug("replaced path '%s' to '%s' for substituter '%s'",
srcStore.printStorePath(storePath),
dstStore.printStorePath(storePathForDst),
dstStore.getUri());
} }
pathsMap.insert_or_assign(storePath, storePathForDst); pathsMap.insert_or_assign(storePath, storePathForDst);
if (!dstStore->isValidPath(storePathForDst)) { if (!dstStore.isValidPath(storePathForDst)) {
MaintainCount<decltype(nrRunning)> mc(nrRunning); MaintainCount<decltype(nrRunning)> mc(nrRunning);
showProgress(); showProgress();
try { try {
@@ -910,7 +1012,7 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
nrFailed++; nrFailed++;
if (!settings.keepGoing) if (!settings.keepGoing)
throw e; throw e;
logger->log(lvlError, fmt("could not copy %s: %s", dstStore->printStorePath(storePath), e.what())); logger->log(lvlError, fmt("could not copy %s: %s", dstStore.printStorePath(storePath), e.what()));
showProgress(); showProgress();
return; return;
} }
@@ -919,9 +1021,27 @@ std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStor
nrDone++; nrDone++;
showProgress(); showProgress();
}); });
#endif
return pathsMap; return pathsMap;
} }
void copyClosure(
Store & srcStore,
Store & dstStore,
const RealisedPath::Set & paths,
RepairFlag repair,
CheckSigsFlag checkSigs,
SubstituteFlag substitute)
{
if (&srcStore == &dstStore) return;
RealisedPath::Set closure;
RealisedPath::closure(srcStore, paths, closure);
copyPaths(srcStore, dstStore, closure, repair, checkSigs, substitute);
}
std::optional<ValidPathInfo> decodeValidPathInfo(const Store & store, std::istream & str, std::optional<HashResult> hashGiven) std::optional<ValidPathInfo> decodeValidPathInfo(const Store & store, std::istream & str, std::optional<HashResult> hashGiven)
{ {
std::string path; std::string path;
+33 -7
View File
@@ -440,6 +440,12 @@ public:
virtual void addToStore(const ValidPathInfo & info, Source & narSource, virtual void addToStore(const ValidPathInfo & info, Source & narSource,
RepairFlag repair = NoRepair, CheckSigsFlag checkSigs = CheckSigs) = 0; RepairFlag repair = NoRepair, CheckSigsFlag checkSigs = CheckSigs) = 0;
/* Import multiple paths into the store. */
virtual void addMultipleToStore(
Source & source,
RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs);
/* Copy the contents of a path to the store and register the /* Copy the contents of a path to the store and register the
validity the resulting path. The resulting path is returned. validity the resulting path. The resulting path is returned.
The function object `filter' can be used to exclude files (see The function object `filter' can be used to exclude files (see
@@ -497,7 +503,8 @@ public:
not derivations, substitute them. */ not derivations, substitute them. */
virtual void buildPaths( virtual void buildPaths(
const std::vector<DerivedPath> & paths, const std::vector<DerivedPath> & paths,
BuildMode buildMode = bmNormal); BuildMode buildMode = bmNormal,
std::shared_ptr<Store> evalStore = nullptr);
/* Build a single non-materialized derivation (i.e. not from an /* Build a single non-materialized derivation (i.e. not from an
on-disk .drv file). on-disk .drv file).
@@ -543,7 +550,7 @@ public:
/* Add a store path as a temporary root of the garbage collector. /* Add a store path as a temporary root of the garbage collector.
The root disappears as soon as we exit. */ The root disappears as soon as we exit. */
virtual void addTempRoot(const StorePath & path) virtual void addTempRoot(const StorePath & path)
{ warn("not creating temp root, store doesn't support GC"); } { debug("not creating temporary root, store doesn't support GC"); }
/* Add an indirect root, which is merely a symlink to `path' from /* Add an indirect root, which is merely a symlink to `path' from
/nix/var/nix/gcroots/auto/<hash of `path'>. `path' is supposed /nix/var/nix/gcroots/auto/<hash of `path'>. `path' is supposed
@@ -697,6 +704,11 @@ public:
const Stats & getStats(); const Stats & getStats();
/* Computes the full closure of of a set of store-paths for e.g.
derivations that need this information for `exportReferencesGraph`.
*/
StorePathSet exportReferences(const StorePathSet & storePaths, const StorePathSet & inputPaths);
/* Return the build log of the specified store path, if available, /* Return the build log of the specified store path, if available,
or null otherwise. */ or null otherwise. */
virtual std::shared_ptr<std::string> getBuildLog(const StorePath & path) virtual std::shared_ptr<std::string> getBuildLog(const StorePath & path)
@@ -746,8 +758,12 @@ protected:
/* Copy a path from one store to another. */ /* Copy a path from one store to another. */
void copyStorePath(ref<Store> srcStore, ref<Store> dstStore, void copyStorePath(
const StorePath & storePath, RepairFlag repair = NoRepair, CheckSigsFlag checkSigs = CheckSigs); Store & srcStore,
Store & dstStore,
const StorePath & storePath,
RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs);
/* Copy store paths from one store to another. The paths may be copied /* Copy store paths from one store to another. The paths may be copied
@@ -756,17 +772,27 @@ void copyStorePath(ref<Store> srcStore, ref<Store> dstStore,
of store paths is not automatically closed; use copyClosure() for of store paths is not automatically closed; use copyClosure() for
that. Returns a map of what each path was copied to the dstStore that. Returns a map of what each path was copied to the dstStore
as. */ as. */
std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStore, std::map<StorePath, StorePath> copyPaths(
Store & srcStore, Store & dstStore,
const RealisedPath::Set &, const RealisedPath::Set &,
RepairFlag repair = NoRepair, RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs, CheckSigsFlag checkSigs = CheckSigs,
SubstituteFlag substitute = NoSubstitute); SubstituteFlag substitute = NoSubstitute);
std::map<StorePath, StorePath> copyPaths(ref<Store> srcStore, ref<Store> dstStore,
const StorePathSet& paths, std::map<StorePath, StorePath> copyPaths(
Store & srcStore, Store & dstStore,
const StorePathSet & paths,
RepairFlag repair = NoRepair, RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs, CheckSigsFlag checkSigs = CheckSigs,
SubstituteFlag substitute = NoSubstitute); SubstituteFlag substitute = NoSubstitute);
/* Copy the closure of `paths` from `srcStore` to `dstStore`. */
void copyClosure(
Store & srcStore, Store & dstStore,
const RealisedPath::Set & paths,
RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs,
SubstituteFlag substitute = NoSubstitute);
/* Remove the temporary roots file for this process. Any temporary /* Remove the temporary roots file for this process. Any temporary
root becomes garbage after this point unless it has been registered root becomes garbage after this point unless it has been registered
+2 -1
View File
@@ -9,7 +9,7 @@ namespace nix {
#define WORKER_MAGIC_1 0x6e697863 #define WORKER_MAGIC_1 0x6e697863
#define WORKER_MAGIC_2 0x6478696f #define WORKER_MAGIC_2 0x6478696f
#define PROTOCOL_VERSION (1 << 8 | 31) #define PROTOCOL_VERSION (1 << 8 | 32)
#define GET_PROTOCOL_MAJOR(x) ((x) & 0xff00) #define GET_PROTOCOL_MAJOR(x) ((x) & 0xff00)
#define GET_PROTOCOL_MINOR(x) ((x) & 0x00ff) #define GET_PROTOCOL_MINOR(x) ((x) & 0x00ff)
@@ -55,6 +55,7 @@ typedef enum {
wopQueryDerivationOutputMap = 41, wopQueryDerivationOutputMap = 41,
wopRegisterDrvOutput = 42, wopRegisterDrvOutput = 42,
wopQueryRealisation = 43, wopQueryRealisation = 43,
wopAddMultipleToStore = 44,
} WorkerOp; } WorkerOp;
+20
View File
@@ -152,6 +152,16 @@ nlohmann::json Config::toJSON()
return res; return res;
} }
std::string Config::toKeyValue()
{
auto res = std::string();
for (auto & s : _settings)
if (!s.second.isAlias) {
res += fmt("%s = %s\n", s.first, s.second.setting->to_string());
}
return res;
}
void Config::convertToArgs(Args & args, const std::string & category) void Config::convertToArgs(Args & args, const std::string & category)
{ {
for (auto & s : _settings) for (auto & s : _settings)
@@ -385,6 +395,16 @@ nlohmann::json GlobalConfig::toJSON()
return res; return res;
} }
std::string GlobalConfig::toKeyValue()
{
std::string res;
std::map<std::string, Config::SettingInfo> settings;
globalConfig.getSettings(settings);
for (auto & s : settings)
res += fmt("%s = %s\n", s.first, s.second.value);
return res;
}
void GlobalConfig::convertToArgs(Args & args, const std::string & category) void GlobalConfig::convertToArgs(Args & args, const std::string & category)
{ {
for (auto & config : *configRegistrations) for (auto & config : *configRegistrations)
+10
View File
@@ -99,6 +99,12 @@ public:
*/ */
virtual nlohmann::json toJSON() = 0; virtual nlohmann::json toJSON() = 0;
/**
* Outputs all settings in a key-value pair format suitable to be used as
* `nix.conf`
*/
virtual std::string toKeyValue() = 0;
/** /**
* Converts settings to `Args` to be used on the command line interface * Converts settings to `Args` to be used on the command line interface
* - args: args to write to * - args: args to write to
@@ -169,6 +175,8 @@ public:
nlohmann::json toJSON() override; nlohmann::json toJSON() override;
std::string toKeyValue() override;
void convertToArgs(Args & args, const std::string & category) override; void convertToArgs(Args & args, const std::string & category) override;
}; };
@@ -330,6 +338,8 @@ struct GlobalConfig : public AbstractConfig
nlohmann::json toJSON() override; nlohmann::json toJSON() override;
std::string toKeyValue() override;
void convertToArgs(Args & args, const std::string & category) override; void convertToArgs(Args & args, const std::string & category) override;
struct Register struct Register
+1 -1
View File
@@ -46,7 +46,7 @@ public:
: printBuildLogs(printBuildLogs) : printBuildLogs(printBuildLogs)
{ {
systemd = getEnv("IN_SYSTEMD") == "1"; systemd = getEnv("IN_SYSTEMD") == "1";
tty = isatty(STDERR_FILENO); tty = shouldANSI();
} }
bool isVerbose() override { bool isVerbose() override {
+10
View File
@@ -73,6 +73,16 @@ public:
return ref<T2>((std::shared_ptr<T2>) p); return ref<T2>((std::shared_ptr<T2>) p);
} }
bool operator == (const ref<T> & other) const
{
return p == other.p;
}
bool operator != (const ref<T> & other) const
{
return p != other.p;
}
private: private:
template<typename T2, typename... Args> template<typename T2, typename... Args>
+1 -1
View File
@@ -32,7 +32,7 @@ ParsedURL parseURL(const std::string & url)
auto isFile = scheme.find("file") != std::string::npos; auto isFile = scheme.find("file") != std::string::npos;
if (authority && *authority != "" && isFile) if (authority && *authority != "" && isFile)
throw Error("file:// URL '%s' has unexpected authority '%s'", throw BadURL("file:// URL '%s' has unexpected authority '%s'",
url, *authority); url, *authority);
if (isFile && path.empty()) if (isFile && path.empty())
+8 -5
View File
@@ -413,7 +413,7 @@ static void _deletePath(int parentfd, const Path & path, uint64_t & bytesFreed)
} }
int fd = openat(parentfd, path.c_str(), O_RDONLY); int fd = openat(parentfd, path.c_str(), O_RDONLY);
if (!fd) if (fd == -1)
throw SysError("opening directory '%1%'", path); throw SysError("opening directory '%1%'", path);
AutoCloseDir dir(fdopendir(fd)); AutoCloseDir dir(fdopendir(fd));
if (!dir) if (!dir)
@@ -435,12 +435,9 @@ static void _deletePath(const Path & path, uint64_t & bytesFreed)
if (dir == "") if (dir == "")
dir = "/"; dir = "/";
AutoCloseFD dirfd(open(dir.c_str(), O_RDONLY)); AutoCloseFD dirfd{open(dir.c_str(), O_RDONLY)};
if (!dirfd) { if (!dirfd) {
// This really shouldn't fail silently, but it's left this way
// for backwards compatibility.
if (errno == ENOENT) return; if (errno == ENOENT) return;
throw SysError("opening directory '%1%'", path); throw SysError("opening directory '%1%'", path);
} }
@@ -1372,6 +1369,12 @@ void ignoreException()
} }
} }
bool shouldANSI()
{
return isatty(STDERR_FILENO)
&& getEnv("TERM").value_or("dumb") != "dumb"
&& !getEnv("NO_COLOR").has_value();
}
std::string filterANSIEscapes(const std::string & s, bool filterAll, unsigned int width) std::string filterANSIEscapes(const std::string & s, bool filterAll, unsigned int width)
{ {
+3
View File
@@ -482,6 +482,9 @@ constexpr char treeLast[] = "└───";
constexpr char treeLine[] = "│ "; constexpr char treeLine[] = "│ ";
constexpr char treeNull[] = " "; constexpr char treeNull[] = " ";
/* Determine whether ANSI escape sequences are appropriate for the
present output. */
bool shouldANSI();
/* Truncate a string to 'width' printable characters. If 'filterAll' /* Truncate a string to 'width' printable characters. If 'filterAll'
is true, all ANSI escape sequences are filtered out. Otherwise, is true, all ANSI escape sequences are filtered out. Otherwise,
+65 -14
View File
@@ -1,10 +1,15 @@
#include <cstring> #include <cstring>
#include <fstream> #include <fstream>
#include <iostream> #include <iostream>
#include <filesystem>
#include <regex> #include <regex>
#include <sstream> #include <sstream>
#include <vector> #include <vector>
#include <map>
#include <nlohmann/json.hpp>
#include "parsed-derivations.hh"
#include "store-api.hh" #include "store-api.hh"
#include "local-fs-store.hh" #include "local-fs-store.hh"
#include "globals.hh" #include "globals.hh"
@@ -245,8 +250,9 @@ static void main_nix_build(int argc, char * * argv)
throw UsageError("'-p' and '-E' are mutually exclusive"); throw UsageError("'-p' and '-E' are mutually exclusive");
auto store = openStore(); auto store = openStore();
auto evalStore = myArgs.evalStoreUrl ? openStore(*myArgs.evalStoreUrl) : store;
auto state = std::make_unique<EvalState>(myArgs.searchPath, store); auto state = std::make_unique<EvalState>(myArgs.searchPath, evalStore, store);
state->repair = repair; state->repair = repair;
auto autoArgs = myArgs.getAutoArgs(*state); auto autoArgs = myArgs.getAutoArgs(*state);
@@ -296,8 +302,8 @@ static void main_nix_build(int argc, char * * argv)
absolute = canonPath(absPath(i), true); absolute = canonPath(absPath(i), true);
} catch (Error & e) {}; } catch (Error & e) {};
auto [path, outputNames] = parsePathWithOutputs(absolute); auto [path, outputNames] = parsePathWithOutputs(absolute);
if (store->isStorePath(path) && hasSuffix(path, ".drv")) if (evalStore->isStorePath(path) && hasSuffix(path, ".drv"))
drvs.push_back(DrvInfo(*state, store, absolute)); drvs.push_back(DrvInfo(*state, evalStore, absolute));
else else
/* If we're in a #! script, interpret filenames /* If we're in a #! script, interpret filenames
relative to the script. */ relative to the script. */
@@ -335,7 +341,7 @@ static void main_nix_build(int argc, char * * argv)
printMissing(ref<Store>(store), willBuild, willSubstitute, unknown, downloadSize, narSize); printMissing(ref<Store>(store), willBuild, willSubstitute, unknown, downloadSize, narSize);
if (!dryRun) if (!dryRun)
store->buildPaths(paths, buildMode); store->buildPaths(paths, buildMode, evalStore);
}; };
if (runEnv) { if (runEnv) {
@@ -343,9 +349,10 @@ static void main_nix_build(int argc, char * * argv)
throw UsageError("nix-shell requires a single derivation"); throw UsageError("nix-shell requires a single derivation");
auto & drvInfo = drvs.front(); auto & drvInfo = drvs.front();
auto drv = store->derivationFromPath(store->parseStorePath(drvInfo.queryDrvPath())); auto drv = evalStore->derivationFromPath(evalStore->parseStorePath(drvInfo.queryDrvPath()));
std::vector<StorePathWithOutputs> pathsToBuild; std::vector<StorePathWithOutputs> pathsToBuild;
RealisedPath::Set pathsToCopy;
/* Figure out what bash shell to use. If $NIX_BUILD_SHELL /* Figure out what bash shell to use. If $NIX_BUILD_SHELL
is not set, then build bashInteractive from is not set, then build bashInteractive from
@@ -364,7 +371,9 @@ static void main_nix_build(int argc, char * * argv)
if (!drv) if (!drv)
throw Error("the 'bashInteractive' attribute in <nixpkgs> did not evaluate to a derivation"); throw Error("the 'bashInteractive' attribute in <nixpkgs> did not evaluate to a derivation");
pathsToBuild.push_back({store->parseStorePath(drv->queryDrvPath())}); auto bashDrv = store->parseStorePath(drv->queryDrvPath());
pathsToBuild.push_back({bashDrv});
pathsToCopy.insert(bashDrv);
shell = drv->queryOutPath() + "/bin/bash"; shell = drv->queryOutPath() + "/bin/bash";
@@ -379,9 +388,14 @@ static void main_nix_build(int argc, char * * argv)
for (const auto & input : drv.inputDrvs) for (const auto & input : drv.inputDrvs)
if (std::all_of(envExclude.cbegin(), envExclude.cend(), if (std::all_of(envExclude.cbegin(), envExclude.cend(),
[&](const string & exclude) { return !std::regex_search(store->printStorePath(input.first), std::regex(exclude)); })) [&](const string & exclude) { return !std::regex_search(store->printStorePath(input.first), std::regex(exclude)); }))
{
pathsToBuild.push_back({input.first, input.second}); pathsToBuild.push_back({input.first, input.second});
for (const auto & src : drv.inputSrcs) pathsToCopy.insert(input.first);
}
for (const auto & src : drv.inputSrcs) {
pathsToBuild.push_back({src}); pathsToBuild.push_back({src});
pathsToCopy.insert(src);
}
buildPaths(pathsToBuild); buildPaths(pathsToBuild);
@@ -428,12 +442,45 @@ static void main_nix_build(int argc, char * * argv)
} else } else
env[var.first] = var.second; env[var.first] = var.second;
std::string structuredAttrsRC;
if (env.count("__json")) {
StorePathSet inputs;
for (auto & [depDrvPath, wantedDepOutputs] : drv.inputDrvs) {
auto outputs = evalStore->queryPartialDerivationOutputMap(depDrvPath);
for (auto & i : wantedDepOutputs) {
auto o = outputs.at(i);
store->computeFSClosure(*o, inputs);
}
}
ParsedDerivation parsedDrv(
StorePath(store->parseStorePath(drvInfo.queryDrvPath())),
drv
);
if (auto structAttrs = parsedDrv.prepareStructuredAttrs(*store, inputs)) {
auto json = structAttrs.value();
structuredAttrsRC = writeStructuredAttrsShell(json);
auto attrsJSON = (Path) tmpDir + "/.attrs.json";
writeFile(attrsJSON, json.dump());
auto attrsSH = (Path) tmpDir + "/.attrs.sh";
writeFile(attrsSH, structuredAttrsRC);
env["NIX_ATTRS_SH_FILE"] = attrsSH;
env["NIX_ATTRS_JSON_FILE"] = attrsJSON;
keepTmp = true;
}
}
/* Run a shell using the derivation's environment. For /* Run a shell using the derivation's environment. For
convenience, source $stdenv/setup to setup additional convenience, source $stdenv/setup to setup additional
environment variables and shell functions. Also don't environment variables and shell functions. Also don't
lose the current $PATH directories. */ lose the current $PATH directories. */
auto rcfile = (Path) tmpDir + "/rc"; auto rcfile = (Path) tmpDir + "/rc";
writeFile(rcfile, fmt( std::string rc = fmt(
R"(_nix_shell_clean_tmpdir() { rm -rf %1%; }; )"s + R"(_nix_shell_clean_tmpdir() { rm -rf %1%; }; )"s +
(keepTmp ? (keepTmp ?
"trap _nix_shell_clean_tmpdir EXIT; " "trap _nix_shell_clean_tmpdir EXIT; "
@@ -442,8 +489,9 @@ static void main_nix_build(int argc, char * * argv)
"_nix_shell_clean_tmpdir; ") + "_nix_shell_clean_tmpdir; ") +
(pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc;") + (pure ? "" : "[ -n \"$PS1\" ] && [ -e ~/.bashrc ] && source ~/.bashrc;") +
"%2%" "%2%"
"dontAddDisableDepTrack=1; " "dontAddDisableDepTrack=1;\n"
"[ -e $stdenv/setup ] && source $stdenv/setup; " + structuredAttrsRC +
"\n[ -e $stdenv/setup ] && source $stdenv/setup; "
"%3%" "%3%"
"PATH=%4%:\"$PATH\"; " "PATH=%4%:\"$PATH\"; "
"SHELL=%5%; " "SHELL=%5%; "
@@ -461,7 +509,9 @@ static void main_nix_build(int argc, char * * argv)
shellEscape(dirOf(*shell)), shellEscape(dirOf(*shell)),
shellEscape(*shell), shellEscape(*shell),
(getenv("TZ") ? (string("export TZ=") + shellEscape(getenv("TZ")) + "; ") : ""), (getenv("TZ") ? (string("export TZ=") + shellEscape(getenv("TZ")) + "; ") : ""),
envCommand)); envCommand);
vomit("Sourcing nix-shell with file %s and contents:\n%s", rcfile, rc);
writeFile(rcfile, rc);
Strings envStrs; Strings envStrs;
for (auto & i : env) for (auto & i : env)
@@ -490,6 +540,7 @@ static void main_nix_build(int argc, char * * argv)
std::vector<StorePathWithOutputs> pathsToBuild; std::vector<StorePathWithOutputs> pathsToBuild;
std::vector<std::pair<StorePath, std::string>> pathsToBuildOrdered; std::vector<std::pair<StorePath, std::string>> pathsToBuildOrdered;
RealisedPath::Set drvsToCopy;
std::map<StorePath, std::pair<size_t, StringSet>> drvMap; std::map<StorePath, std::pair<size_t, StringSet>> drvMap;
@@ -502,13 +553,13 @@ static void main_nix_build(int argc, char * * argv)
pathsToBuild.push_back({drvPath, {outputName}}); pathsToBuild.push_back({drvPath, {outputName}});
pathsToBuildOrdered.push_back({drvPath, {outputName}}); pathsToBuildOrdered.push_back({drvPath, {outputName}});
drvsToCopy.insert(drvPath);
auto i = drvMap.find(drvPath); auto i = drvMap.find(drvPath);
if (i != drvMap.end()) if (i != drvMap.end())
i->second.second.insert(outputName); i->second.second.insert(outputName);
else { else
drvMap[drvPath] = {drvMap.size(), {outputName}}; drvMap[drvPath] = {drvMap.size(), {outputName}};
}
} }
buildPaths(pathsToBuild); buildPaths(pathsToBuild);
@@ -523,7 +574,7 @@ static void main_nix_build(int argc, char * * argv)
if (counter) if (counter)
drvPrefix += fmt("-%d", counter + 1); drvPrefix += fmt("-%d", counter + 1);
auto builtOutputs = store->queryPartialDerivationOutputMap(drvPath); auto builtOutputs = evalStore->queryPartialDerivationOutputMap(drvPath);
auto maybeOutputPath = builtOutputs.at(outputName); auto maybeOutputPath = builtOutputs.at(outputName);
assert(maybeOutputPath); assert(maybeOutputPath);
+1 -4
View File
@@ -54,10 +54,7 @@ static int main_nix_copy_closure(int argc, char ** argv)
for (auto & path : storePaths) for (auto & path : storePaths)
storePaths2.insert(from->followLinksToStorePath(path)); storePaths2.insert(from->followLinksToStorePath(path));
RealisedPath::Set closure; copyClosure(*from, *to, storePaths2, NoRepair, NoCheckSigs, useSubstitutes);
RealisedPath::closure(*from, storePaths2, closure);
copyPaths(from, to, closure, NoRepair, NoCheckSigs, useSubstitutes);
return 0; return 0;
} }
+2 -1
View File
@@ -153,8 +153,9 @@ static int main_nix_instantiate(int argc, char * * argv)
settings.readOnlyMode = true; settings.readOnlyMode = true;
auto store = openStore(); auto store = openStore();
auto evalStore = myArgs.evalStoreUrl ? openStore(*myArgs.evalStoreUrl) : store;
auto state = std::make_unique<EvalState>(myArgs.searchPath, store); auto state = std::make_unique<EvalState>(myArgs.searchPath, evalStore, store);
state->repair = repair; state->repair = repair;
Bindings & autoArgs = *myArgs.getAutoArgs(*state); Bindings & autoArgs = *myArgs.getAutoArgs(*state);
+3 -2
View File
@@ -100,7 +100,8 @@ UnresolvedApp Installable::toApp(EvalState & state)
throw Error("attribute '%s' has unsupported type '%s'", attrPath, type); throw Error("attribute '%s' has unsupported type '%s'", attrPath, type);
} }
App UnresolvedApp::resolve(ref<Store> store) // FIXME: move to libcmd
App UnresolvedApp::resolve(ref<Store> evalStore, ref<Store> store)
{ {
auto res = unresolved; auto res = unresolved;
@@ -110,7 +111,7 @@ App UnresolvedApp::resolve(ref<Store> store)
installableContext.push_back( installableContext.push_back(
std::make_shared<InstallableDerivedPath>(store, ctxElt.toDerivedPath())); std::make_shared<InstallableDerivedPath>(store, ctxElt.toDerivedPath()));
auto builtContext = build(store, Realise::Outputs, installableContext); auto builtContext = build(evalStore, store, Realise::Outputs, installableContext);
res.program = resolveString(*store, unresolved.program, builtContext); res.program = resolveString(*store, unresolved.program, builtContext);
if (!store->isInStore(res.program)) if (!store->isInStore(res.program))
throw Error("app program '%s' is not in the Nix store", res.program); throw Error("app program '%s' is not in the Nix store", res.program);
+4 -1
View File
@@ -52,7 +52,10 @@ struct CmdBuild : InstallablesCommand, MixDryRun, MixJSON, MixProfile
void run(ref<Store> store) override void run(ref<Store> store) override
{ {
auto buildables = build(store, dryRun ? Realise::Nothing : Realise::Outputs, installables, buildMode); auto buildables = build(
getEvalStore(), store,
dryRun ? Realise::Nothing : Realise::Outputs,
installables, buildMode);
if (json) logger->cout("%s", derivedPathsWithHintsToJSON(buildables, store).dump()); if (json) logger->cout("%s", derivedPathsWithHintsToJSON(buildables, store).dump());
+1 -1
View File
@@ -69,7 +69,7 @@ struct CmdBundle : InstallableCommand
{ {
auto evalState = getEvalState(); auto evalState = getEvalState();
auto app = installable->toApp(*evalState).resolve(store); auto app = installable->toApp(*evalState).resolve(getEvalStore(), store);
auto [bundlerFlakeRef, bundlerName] = parseFlakeRefWithFragment(bundler, absPath(".")); auto [bundlerFlakeRef, bundlerName] = parseFlakeRefWithFragment(bundler, absPath("."));
const flake::LockFlags lockFlags{ .writeLockFile = false }; const flake::LockFlags lockFlags{ .writeLockFile = false };
+1 -1
View File
@@ -90,7 +90,7 @@ struct CmdCopy : BuiltPathsCommand
} }
copyPaths( copyPaths(
srcStore, dstStore, stuffToCopy, NoRepair, checkSigs, substitute); *srcStore, *dstStore, stuffToCopy, NoRepair, checkSigs, substitute);
} }
}; };
+157 -110
View File
@@ -8,7 +8,7 @@
#include "affinity.hh" #include "affinity.hh"
#include "progress-bar.hh" #include "progress-bar.hh"
#include <regex> #include <nlohmann/json.hpp>
using namespace nix; using namespace nix;
@@ -25,94 +25,142 @@ static DevelopSettings developSettings;
static GlobalConfig::Register rDevelopSettings(&developSettings); static GlobalConfig::Register rDevelopSettings(&developSettings);
struct Var
{
bool exported = true;
bool associative = false;
std::string quoted; // quoted string or array
};
struct BuildEnvironment struct BuildEnvironment
{ {
std::map<std::string, Var> env; struct String
std::string bashFunctions; {
}; bool exported;
std::string value;
BuildEnvironment readEnvironment(const Path & path) bool operator == (const String & other) const
{ {
BuildEnvironment res; return exported == other.exported && value == other.value;
}
};
std::set<std::string> exported; using Array = std::vector<std::string>;
debug("reading environment file '%s'", path); using Associative = std::map<std::string, std::string>;
auto file = readFile(path); using Value = std::variant<String, Array, Associative>;
auto pos = file.cbegin(); std::map<std::string, Value> vars;
std::map<std::string, std::string> bashFunctions;
static std::string varNameRegex = static BuildEnvironment fromJSON(std::string_view in)
R"re((?:[a-zA-Z_][a-zA-Z0-9_]*))re"; {
BuildEnvironment res;
static std::string simpleStringRegex = std::set<std::string> exported;
R"re((?:[a-zA-Z0-9_/:\.\-\+=]*))re";
static std::string dquotedStringRegex = auto json = nlohmann::json::parse(in);
R"re((?:\$?"(?:[^"\\]|\\[$`"\\\n])*"))re";
static std::string squotedStringRegex = for (auto & [name, info] : json["variables"].items()) {
R"re((?:\$?(?:'(?:[^'\\]|\\[abeEfnrtv\\'"?])*'|\\')+))re"; std::string type = info["type"];
if (type == "var" || type == "exported")
static std::string indexedArrayRegex = res.vars.insert({name, BuildEnvironment::String { .exported = type == "exported", .value = info["value"] }});
R"re((?:\(( *\[[0-9]+\]="(?:[^"\\]|\\.)*")*\)))re"; else if (type == "array")
res.vars.insert({name, (Array) info["value"]});
static std::regex declareRegex( else if (type == "associative")
"^declare -a?x (" + varNameRegex + ")(=(" + res.vars.insert({name, (Associative) info["value"]});
dquotedStringRegex + "|" + indexedArrayRegex + "))?\n");
static std::regex varRegex(
"^(" + varNameRegex + ")=(" + simpleStringRegex + "|" + squotedStringRegex + "|" + indexedArrayRegex + ")\n");
/* Note: we distinguish between an indexed and associative array
using the space before the closing parenthesis. Will
undoubtedly regret this some day. */
static std::regex assocArrayRegex(
"^(" + varNameRegex + ")=" + R"re((?:\(( *\[[^\]]+\]="(?:[^"\\]|\\.)*")* *\)))re" + "\n");
static std::regex functionRegex(
"^" + varNameRegex + " \\(\\) *\n");
while (pos != file.end()) {
std::smatch match;
if (std::regex_search(pos, file.cend(), match, declareRegex, std::regex_constants::match_continuous)) {
pos = match[0].second;
exported.insert(match[1]);
} }
else if (std::regex_search(pos, file.cend(), match, varRegex, std::regex_constants::match_continuous)) { for (auto & [name, def] : json["bashFunctions"].items()) {
pos = match[0].second; res.bashFunctions.insert({name, def});
res.env.insert({match[1], Var { .exported = exported.count(match[1]) > 0, .quoted = match[2] }});
} }
else if (std::regex_search(pos, file.cend(), match, assocArrayRegex, std::regex_constants::match_continuous)) { return res;
pos = match[0].second;
res.env.insert({match[1], Var { .associative = true, .quoted = match[2] }});
}
else if (std::regex_search(pos, file.cend(), match, functionRegex, std::regex_constants::match_continuous)) {
res.bashFunctions = std::string(pos, file.cend());
break;
}
else throw Error("shell environment '%s' has unexpected line '%s'",
path, file.substr(pos - file.cbegin(), 60));
} }
res.env.erase("__output"); std::string toJSON() const
{
auto res = nlohmann::json::object();
return res; auto vars2 = nlohmann::json::object();
} for (auto & [name, value] : vars) {
auto info = nlohmann::json::object();
if (auto str = std::get_if<String>(&value)) {
info["type"] = str->exported ? "exported" : "var";
info["value"] = str->value;
}
else if (auto arr = std::get_if<Array>(&value)) {
info["type"] = "array";
info["value"] = *arr;
}
else if (auto arr = std::get_if<Associative>(&value)) {
info["type"] = "associative";
info["value"] = *arr;
}
vars2[name] = std::move(info);
}
res["variables"] = std::move(vars2);
res["bashFunctions"] = bashFunctions;
auto json = res.dump();
assert(BuildEnvironment::fromJSON(json) == *this);
return json;
}
void toBash(std::ostream & out, const std::set<std::string> & ignoreVars) const
{
for (auto & [name, value] : vars) {
if (!ignoreVars.count(name)) {
if (auto str = std::get_if<String>(&value)) {
out << fmt("%s=%s\n", name, shellEscape(str->value));
if (str->exported)
out << fmt("export %s\n", name);
}
else if (auto arr = std::get_if<Array>(&value)) {
out << "declare -a " << name << "=(";
for (auto & s : *arr)
out << shellEscape(s) << " ";
out << ")\n";
}
else if (auto arr = std::get_if<Associative>(&value)) {
out << "declare -A " << name << "=(";
for (auto & [n, v] : *arr)
out << "[" << shellEscape(n) << "]=" << shellEscape(v) << " ";
out << ")\n";
}
}
}
for (auto & [name, def] : bashFunctions) {
out << name << " ()\n{\n" << def << "}\n";
}
}
static std::string getString(const Value & value)
{
if (auto str = std::get_if<String>(&value))
return str->value;
else
throw Error("bash variable is not a string");
}
static Array getStrings(const Value & value)
{
if (auto str = std::get_if<String>(&value))
return tokenizeString<Array>(str->value);
else if (auto arr = std::get_if<Array>(&value)) {
return *arr;
} else if (auto assoc = std::get_if<Associative>(&value)) {
Array assocKeys;
std::for_each(assoc->begin(), assoc->end(), [&](auto & n) { assocKeys.push_back(n.first); });
return assocKeys;
}
else
throw Error("bash variable is not a string or array");
}
bool operator == (const BuildEnvironment & other) const
{
return vars == other.vars && bashFunctions == other.bashFunctions;
}
};
const static std::string getEnvSh = const static std::string getEnvSh =
#include "get-env.sh.gen.hh" #include "get-env.sh.gen.hh"
@@ -123,15 +171,15 @@ const static std::string getEnvSh =
modified derivation with the same dependencies and nearly the same modified derivation with the same dependencies and nearly the same
initial environment variables, that just writes the resulting initial environment variables, that just writes the resulting
environment to a file and exits. */ environment to a file and exits. */
StorePath getDerivationEnvironment(ref<Store> store, const StorePath & drvPath) static StorePath getDerivationEnvironment(ref<Store> store, ref<Store> evalStore, const StorePath & drvPath)
{ {
auto drv = store->derivationFromPath(drvPath); auto drv = evalStore->derivationFromPath(drvPath);
auto builder = baseNameOf(drv.builder); auto builder = baseNameOf(drv.builder);
if (builder != "bash") if (builder != "bash")
throw Error("'nix develop' only works on derivations that use 'bash' as their builder"); throw Error("'nix develop' only works on derivations that use 'bash' as their builder");
auto getEnvShPath = store->addTextToStore("get-env.sh", getEnvSh, {}); auto getEnvShPath = evalStore->addTextToStore("get-env.sh", getEnvSh, {});
drv.args = {store->printStorePath(getEnvShPath)}; drv.args = {store->printStorePath(getEnvShPath)};
@@ -157,7 +205,7 @@ StorePath getDerivationEnvironment(ref<Store> store, const StorePath & drvPath)
output.second = { .output = DerivationOutputInputAddressed { .path = StorePath::dummy } }; output.second = { .output = DerivationOutputInputAddressed { .path = StorePath::dummy } };
drv.env[output.first] = ""; drv.env[output.first] = "";
} }
Hash h = std::get<0>(hashDerivationModulo(*store, drv, true)); Hash h = std::get<0>(hashDerivationModulo(*evalStore, drv, true));
for (auto & output : drv.outputs) { for (auto & output : drv.outputs) {
auto outPath = store->makeOutputPath(output.first, h, drv.name); auto outPath = store->makeOutputPath(output.first, h, drv.name);
@@ -166,12 +214,12 @@ StorePath getDerivationEnvironment(ref<Store> store, const StorePath & drvPath)
} }
} }
auto shellDrvPath = writeDerivation(*store, drv); auto shellDrvPath = writeDerivation(*evalStore, drv);
/* Build the derivation. */ /* Build the derivation. */
store->buildPaths({DerivedPath::Built{shellDrvPath}}); store->buildPaths({DerivedPath::Built{shellDrvPath}}, bmNormal, evalStore);
for (auto & [_0, optPath] : store->queryPartialDerivationOutputMap(shellDrvPath)) { for (auto & [_0, optPath] : evalStore->queryPartialDerivationOutputMap(shellDrvPath)) {
assert(optPath); assert(optPath);
auto & outPath = *optPath; auto & outPath = *optPath;
assert(store->isValidPath(outPath)); assert(store->isValidPath(outPath));
@@ -185,19 +233,15 @@ StorePath getDerivationEnvironment(ref<Store> store, const StorePath & drvPath)
struct Common : InstallableCommand, MixProfile struct Common : InstallableCommand, MixProfile
{ {
std::set<string> ignoreVars{ std::set<std::string> ignoreVars{
"BASHOPTS", "BASHOPTS",
"EUID",
"HOME", // FIXME: don't ignore in pure mode? "HOME", // FIXME: don't ignore in pure mode?
"HOSTNAME",
"NIX_BUILD_TOP", "NIX_BUILD_TOP",
"NIX_ENFORCE_PURITY", "NIX_ENFORCE_PURITY",
"NIX_LOG_FD", "NIX_LOG_FD",
"NIX_REMOTE", "NIX_REMOTE",
"PPID", "PPID",
"PWD",
"SHELLOPTS", "SHELLOPTS",
"SHLVL",
"SSL_CERT_FILE", // FIXME: only want to ignore /no-cert-file.crt "SSL_CERT_FILE", // FIXME: only want to ignore /no-cert-file.crt
"TEMP", "TEMP",
"TEMPDIR", "TEMPDIR",
@@ -233,22 +277,10 @@ struct Common : InstallableCommand, MixProfile
out << "nix_saved_PATH=\"$PATH\"\n"; out << "nix_saved_PATH=\"$PATH\"\n";
for (auto & i : buildEnvironment.env) { buildEnvironment.toBash(out, ignoreVars);
if (!ignoreVars.count(i.first) && !hasPrefix(i.first, "BASH_")) {
if (i.second.associative)
out << fmt("declare -A %s=(%s)\n", i.first, i.second.quoted);
else {
out << fmt("%s=%s\n", i.first, i.second.quoted);
if (i.second.exported)
out << fmt("export %s\n", i.first);
}
}
}
out << "PATH=\"$PATH:$nix_saved_PATH\"\n"; out << "PATH=\"$PATH:$nix_saved_PATH\"\n";
out << buildEnvironment.bashFunctions << "\n";
out << "export NIX_BUILD_TOP=\"$(mktemp -d -t nix-shell.XXXXXX)\"\n"; out << "export NIX_BUILD_TOP=\"$(mktemp -d -t nix-shell.XXXXXX)\"\n";
for (auto & i : {"TMP", "TMPDIR", "TEMP", "TEMPDIR"}) for (auto & i : {"TMP", "TMPDIR", "TEMP", "TEMPDIR"})
out << fmt("export %s=\"$NIX_BUILD_TOP\"\n", i); out << fmt("export %s=\"$NIX_BUILD_TOP\"\n", i);
@@ -258,16 +290,16 @@ struct Common : InstallableCommand, MixProfile
auto script = out.str(); auto script = out.str();
/* Substitute occurrences of output paths. */ /* Substitute occurrences of output paths. */
auto outputs = buildEnvironment.env.find("outputs"); auto outputs = buildEnvironment.vars.find("outputs");
assert(outputs != buildEnvironment.env.end()); assert(outputs != buildEnvironment.vars.end());
// FIXME: properly unquote 'outputs'. // FIXME: properly unquote 'outputs'.
StringMap rewrites; StringMap rewrites;
for (auto & outputName : tokenizeString<std::vector<std::string>>(replaceStrings(outputs->second.quoted, "'", ""))) { for (auto & outputName : BuildEnvironment::getStrings(outputs->second)) {
auto from = buildEnvironment.env.find(outputName); auto from = buildEnvironment.vars.find(outputName);
assert(from != buildEnvironment.env.end()); assert(from != buildEnvironment.vars.end());
// FIXME: unquote // FIXME: unquote
rewrites.insert({from->second.quoted, outputsDir + "/" + outputName}); rewrites.insert({BuildEnvironment::getString(from->second), outputsDir + "/" + outputName});
} }
/* Substitute redirects. */ /* Substitute redirects. */
@@ -275,7 +307,7 @@ struct Common : InstallableCommand, MixProfile
auto dir = absPath(dir_); auto dir = absPath(dir_);
auto installable = parseInstallable(store, installable_); auto installable = parseInstallable(store, installable_);
auto builtPaths = toStorePaths( auto builtPaths = toStorePaths(
store, Realise::Nothing, OperateOn::Output, {installable}); getEvalStore(), store, Realise::Nothing, OperateOn::Output, {installable});
for (auto & path: builtPaths) { for (auto & path: builtPaths) {
auto from = store->printStorePath(path); auto from = store->printStorePath(path);
if (script.find(from) == std::string::npos) if (script.find(from) == std::string::npos)
@@ -294,6 +326,12 @@ struct Common : InstallableCommand, MixProfile
{ {
return {"devShell." + settings.thisSystem.get(), "defaultPackage." + settings.thisSystem.get()}; return {"devShell." + settings.thisSystem.get(), "defaultPackage." + settings.thisSystem.get()};
} }
Strings getDefaultFlakeAttrPathPrefixes() override
{
auto res = SourceExprCommand::getDefaultFlakeAttrPathPrefixes();
res.emplace_front("devShells." + settings.thisSystem.get());
return res;
}
StorePath getShellOutPath(ref<Store> store) StorePath getShellOutPath(ref<Store> store)
{ {
@@ -309,7 +347,7 @@ struct Common : InstallableCommand, MixProfile
auto & drvPath = *drvs.begin(); auto & drvPath = *drvs.begin();
return getDerivationEnvironment(store, drvPath); return getDerivationEnvironment(store, getEvalStore(), drvPath);
} }
} }
@@ -321,7 +359,9 @@ struct Common : InstallableCommand, MixProfile
updateProfile(shellOutPath); updateProfile(shellOutPath);
return {readEnvironment(strPath), strPath}; debug("reading environment file '%s'", strPath);
return {BuildEnvironment::fromJSON(readFile(store->toRealPath(shellOutPath))), strPath};
} }
}; };
@@ -443,16 +483,20 @@ struct CmdDevelop : Common, MixEnvironment
try { try {
auto state = getEvalState(); auto state = getEvalState();
auto nixpkgsLockFlags = lockFlags;
nixpkgsLockFlags.inputOverrides = {};
nixpkgsLockFlags.inputUpdates = {};
auto bashInstallable = std::make_shared<InstallableFlake>( auto bashInstallable = std::make_shared<InstallableFlake>(
this, this,
state, state,
installable->nixpkgsFlakeRef(), installable->nixpkgsFlakeRef(),
Strings{"bashInteractive"}, Strings{"bashInteractive"},
Strings{"legacyPackages." + settings.thisSystem.get() + "."}, Strings{"legacyPackages." + settings.thisSystem.get() + "."},
lockFlags); nixpkgsLockFlags);
shell = state->store->printStorePath( shell = store->printStorePath(
toStorePath(state->store, Realise::Outputs, OperateOn::Output, bashInstallable)) + "/bin/bash"; toStorePath(getEvalStore(), store, Realise::Outputs, OperateOn::Output, bashInstallable)) + "/bin/bash";
} catch (Error &) { } catch (Error &) {
ignoreException(); ignoreException();
} }
@@ -470,7 +514,7 @@ struct CmdDevelop : Common, MixEnvironment
} }
}; };
struct CmdPrintDevEnv : Common struct CmdPrintDevEnv : Common, MixJSON
{ {
std::string description() override std::string description() override
{ {
@@ -492,7 +536,10 @@ struct CmdPrintDevEnv : Common
stopProgressBar(); stopProgressBar();
std::cout << makeRcScript(store, buildEnvironment); logger->writeToStdout(
json
? buildEnvironment.toJSON()
: makeRcScript(store, buildEnvironment));
} }
}; };
+10 -1
View File
@@ -84,11 +84,20 @@ the flake's `nixConfig` attribute.
# Flake output attributes # Flake output attributes
If no flake output attribute is given, `nix run` tries the following If no flake output attribute is given, `nix develop` tries the following
flake output attributes: flake output attributes:
* `devShell.<system>` * `devShell.<system>`
* `defaultPackage.<system>` * `defaultPackage.<system>`
If a flake output *name* is given, `nix develop` tries the following flake
output attributes:
* `devShells.<system>.<name>`
* `packages.<system>.<name>`
* `legacyPackages.<system>.<name>`
)"" )""
+2 -2
View File
@@ -131,9 +131,9 @@ struct CmdDiffClosures : SourceExprCommand
void run(ref<Store> store) override void run(ref<Store> store) override
{ {
auto before = parseInstallable(store, _before); auto before = parseInstallable(store, _before);
auto beforePath = toStorePath(store, Realise::Outputs, operateOn, before); auto beforePath = toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, before);
auto after = parseInstallable(store, _after); auto after = parseInstallable(store, _after);
auto afterPath = toStorePath(store, Realise::Outputs, operateOn, after); auto afterPath = toStorePath(getEvalStore(), store, Realise::Outputs, operateOn, after);
printClosureDiff(store, beforePath, afterPath, ""); printClosureDiff(store, beforePath, afterPath, "");
} }
}; };
+1 -1
View File
@@ -31,7 +31,7 @@ struct CmdEdit : InstallableCommand
auto [v, pos] = installable->toValue(*state); auto [v, pos] = installable->toValue(*state);
try { try {
pos = findDerivationFilename(*state, *v, installable->what()); pos = findPackageFilename(*state, *v, installable->what());
} catch (NoPositionInfo &) { } catch (NoPositionInfo &) {
} }
+1
View File
@@ -33,6 +33,7 @@ The following flake output attributes must be derivations:
* `checks.`*system*`.`*name* * `checks.`*system*`.`*name*
* `defaultPackage.`*system*` * `defaultPackage.`*system*`
* `devShell.`*system*` * `devShell.`*system*`
* `devShells.`*system*`.`*name*`
* `nixosConfigurations.`*name*`.config.system.build.toplevel * `nixosConfigurations.`*name*`.config.system.build.toplevel
* `packages.`*system*`.`*name* * `packages.`*system*`.`*name*
+9 -3
View File
@@ -84,6 +84,7 @@ struct CmdFlakeUpdate : FlakeCommand
lockFlags.recreateLockFile = true; lockFlags.recreateLockFile = true;
lockFlags.writeLockFile = true; lockFlags.writeLockFile = true;
lockFlags.applyNixConfig = true;
lockFlake(); lockFlake();
} }
@@ -114,6 +115,7 @@ struct CmdFlakeLock : FlakeCommand
settings.tarballTtl = 0; settings.tarballTtl = 0;
lockFlags.writeLockFile = true; lockFlags.writeLockFile = true;
lockFlags.applyNixConfig = true;
lockFlake(); lockFlake();
} }
@@ -270,6 +272,8 @@ struct CmdFlakeCheck : FlakeCommand
settings.readOnlyMode = !build; settings.readOnlyMode = !build;
auto state = getEvalState(); auto state = getEvalState();
lockFlags.applyNixConfig = true;
auto flake = lockFlake(); auto flake = lockFlake();
bool hasErrors = false; bool hasErrors = false;
@@ -482,7 +486,7 @@ struct CmdFlakeCheck : FlakeCommand
} }
} }
else if (name == "packages") { else if (name == "packages" || name == "devShells") {
state->forceAttrs(vOutput, pos); state->forceAttrs(vOutput, pos);
for (auto & attr : *vOutput.attrs) { for (auto & attr : *vOutput.attrs) {
checkSystemName(attr.name, *attr.pos); checkSystemName(attr.name, *attr.pos);
@@ -837,7 +841,7 @@ struct CmdFlakeArchive : FlakeCommand, MixJSON, MixDryRun
if (!dryRun && !dstUri.empty()) { if (!dryRun && !dstUri.empty()) {
ref<Store> dstStore = dstUri.empty() ? openStore() : openStore(dstUri); ref<Store> dstStore = dstUri.empty() ? openStore() : openStore(dstUri);
copyPaths(store, dstStore, sources); copyPaths(*store, *dstStore, sources);
} }
} }
}; };
@@ -910,6 +914,7 @@ struct CmdFlakeShow : FlakeCommand
logger->cout("%s: %s '%s'", logger->cout("%s: %s '%s'",
headerPrefix, headerPrefix,
attrPath.size() == 2 && attrPath[0] == "devShell" ? "development environment" : attrPath.size() == 2 && attrPath[0] == "devShell" ? "development environment" :
attrPath.size() >= 2 && attrPath[0] == "devShells" ? "development environment" :
attrPath.size() == 3 && attrPath[0] == "checks" ? "derivation" : attrPath.size() == 3 && attrPath[0] == "checks" ? "derivation" :
attrPath.size() >= 1 && attrPath[0] == "hydraJobs" ? "derivation" : attrPath.size() >= 1 && attrPath[0] == "hydraJobs" ? "derivation" :
"package", "package",
@@ -928,6 +933,7 @@ struct CmdFlakeShow : FlakeCommand
|| ((attrPath.size() == 1 || attrPath.size() == 2) || ((attrPath.size() == 1 || attrPath.size() == 2)
&& (attrPath[0] == "checks" && (attrPath[0] == "checks"
|| attrPath[0] == "packages" || attrPath[0] == "packages"
|| attrPath[0] == "devShells"
|| attrPath[0] == "apps")) || attrPath[0] == "apps"))
) )
{ {
@@ -936,7 +942,7 @@ struct CmdFlakeShow : FlakeCommand
else if ( else if (
(attrPath.size() == 2 && (attrPath[0] == "defaultPackage" || attrPath[0] == "devShell")) (attrPath.size() == 2 && (attrPath[0] == "defaultPackage" || attrPath[0] == "devShell"))
|| (attrPath.size() == 3 && (attrPath[0] == "checks" || attrPath[0] == "packages")) || (attrPath.size() == 3 && (attrPath[0] == "checks" || attrPath[0] == "packages" || attrPath[0] == "devShells"))
) )
{ {
if (visitor.isDerivation()) if (visitor.isDerivation())
+115 -4
View File
@@ -8,12 +8,123 @@ if [[ -n $stdenv ]]; then
source $stdenv/setup source $stdenv/setup
fi fi
for __output in $outputs; do # Better to use compgen, but stdenv bash doesn't have it.
__vars="$(declare -p)"
__functions="$(declare -F)"
__dumpEnv() {
printf '{\n'
printf ' "bashFunctions": {\n'
local __first=1
while read __line; do
if ! [[ $__line =~ ^declare\ -f\ (.*) ]]; then continue; fi
__fun_name="${BASH_REMATCH[1]}"
__fun_body="$(type $__fun_name)"
if [[ $__fun_body =~ \{(.*)\} ]]; then
if [[ -z $__first ]]; then printf ',\n'; else __first=; fi
__fun_body="${BASH_REMATCH[1]}"
printf " "
__escapeString "$__fun_name"
printf ':'
__escapeString "$__fun_body"
else
printf "Cannot parse definition of function '%s'.\n" "$__fun_name" >&2
return 1
fi
done < <(printf "%s\n" "$__functions")
printf '\n },\n'
printf ' "variables": {\n'
local __first=1
while read __line; do
if ! [[ $__line =~ ^declare\ (-[^ ])\ ([^=]*) ]]; then continue; fi
local type="${BASH_REMATCH[1]}"
local __var_name="${BASH_REMATCH[2]}"
if [[ $__var_name =~ ^BASH_ || \
$__var_name = _ || \
$__var_name = DIRSTACK || \
$__var_name = EUID || \
$__var_name = FUNCNAME || \
$__var_name = HISTCMD || \
$__var_name = HOSTNAME || \
$__var_name = GROUPS || \
$__var_name = PIPESTATUS || \
$__var_name = PWD || \
$__var_name = RANDOM || \
$__var_name = SHLVL || \
$__var_name = SECONDS \
]]; then continue; fi
if [[ -z $__first ]]; then printf ',\n'; else __first=; fi
printf " "
__escapeString "$__var_name"
printf ': {'
# FIXME: handle -i, -r, -n.
if [[ $type == -x ]]; then
printf '"type": "exported", "value": '
__escapeString "${!__var_name}"
elif [[ $type == -- ]]; then
printf '"type": "var", "value": '
__escapeString "${!__var_name}"
elif [[ $type == -a ]]; then
printf '"type": "array", "value": ['
local __first2=1
__var_name="$__var_name[@]"
for __i in "${!__var_name}"; do
if [[ -z $__first2 ]]; then printf ', '; else __first2=; fi
__escapeString "$__i"
printf ' '
done
printf ']'
elif [[ $type == -A ]]; then
printf '"type": "associative", "value": {\n'
local __first2=1
declare -n __var_name2="$__var_name"
for __i in "${!__var_name2[@]}"; do
if [[ -z $__first2 ]]; then printf ',\n'; else __first2=; fi
printf " "
__escapeString "$__i"
printf ": "
__escapeString "${__var_name2[$__i]}"
done
printf '\n }'
else
printf '"type": "unknown"'
fi
printf "}"
done < <(printf "%s\n" "$__vars")
printf '\n }\n}'
}
__escapeString() {
local __s="$1"
__s="${__s//\\/\\\\}"
__s="${__s//\"/\\\"}"
__s="${__s//$'\n'/\\n}"
__s="${__s//$'\r'/\\r}"
__s="${__s//$'\t'/\\t}"
printf '"%s"' "$__s"
}
# In case of `__structuredAttrs = true;` the list of outputs is an associative
# array with a format like `outname => /nix/store/hash-drvname-outname`, so `__olist`
# must contain the array's keys (hence `${!...[@]}`) in this case.
if [ -e .attrs.sh ]; then
__olist="${!outputs[@]}"
else
__olist=$outputs
fi
for __output in $__olist; do
if [[ -z $__done ]]; then if [[ -z $__done ]]; then
export > ${!__output} __dumpEnv > ${!__output}
set >> ${!__output}
__done=1 __done=1
else else
echo -n >> ${!__output} echo -n >> "${!__output}"
fi fi
done done
Loaded 100 of 153 files, more files were not shown because too many files have changed in this diff. Show more