Compare commits

..
Author SHA1 Message Date
Théophane Hufschmitt 16175e07a9 develop: Allow shells to declare the minimal bash version they require
Allow derivations to export a `NIX_ENV_MIN_BASH_VERSION` that sets the
minimal (major) bash version that is required to parse their setup
script, and use that to gracefully fail if the current bash is too old.

Fix #10263 (from the Nix side at least, needs https://github.com/NixOS/nixpkgs/pull/299490 from the Nixpkgs side to be useful in practice)
2024-03-29 18:13:42 +01:00
Théophane Hufschmitt b72e1c79da Merge pull request #10299 from detroyejr/remove-repl-flake
flakes: remove experimental repl-flake
2024-03-29 15:06:23 +01:00
Jonathan De TroyeandThéophane Hufschmitt a09d0e19c1 flakes: Reword doc/manual/rl-next/remove-repl-flake.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-29 08:57:41 -04:00
Théophane Hufschmitt 90f5189451 Merge pull request #10131 from intelfx/work/notty
`TERM=dumb` fixes
2024-03-29 13:38:37 +01:00
Théophane Hufschmitt de101417eb Merge pull request #10305 from hercules-ci/addErrorContext-is-great
Always print addErrorContext
2024-03-29 09:10:09 +01:00
Robert Hensing ef26133df3 Merge pull request #10337 from Cyclic4179/patch-1
`nix shell` shebang: support O'Caml comments
2024-03-28 19:54:03 +01:00
Eelco Dolstra 7829caab49 Merge pull request #10347 from Atry/patch-2
docs: clarify that `builtins.stringLength` is counting bytes
2024-03-28 15:56:22 +01:00
Yang, Bo c39afb28db Clarify stringLength is counting bytes 2024-03-27 16:46:50 -07:00
Robert Hensing c0b6907ccd doc/local.mk: Add manual-html-open phony target for auto-opening the browser (#10308) 2024-03-27 22:28:05 +01:00
vac (Brendan) 37f8edce99 docs: add cacert in macos multi-user upgrade (#10237) 2024-03-27 22:26:50 +01:00
Eelco Dolstra 1b71a4270a Merge pull request #10329 from obsidiansystems/mingw-devops
Build system changes for MinGW support
2024-03-27 22:17:11 +01:00
John Ericson 77205b2042 Allow for ergnomically putting Unix-only files in subdirs by creating INLCUDE_$(pkg) vars
Separate platform-specific files will allow avoiding a lot of CPP.
2024-03-27 12:01:59 -04:00
John Ericson 4a61827d2d Hack to make sure the DLL linking job works with the check output 2024-03-27 12:01:59 -04:00
John Ericson ae2b2849c9 Disable GC on windows
We can build the dep and the our GC code is totally portable, but for
some reason we get link errors saying `GC_throw_bad_alloc` is missing.
2024-03-27 12:01:59 -04:00
Robert HensingandRebecca Turner d4fa0a84a5 refact: TraceKind -> TracePrint
Co-authored-by: Rebecca Turner <rbt@sent.as>
2024-03-27 16:32:49 +01:00
Robert Hensing bed541b04e error.cc: Make printTrace static 2024-03-27 16:28:04 +01:00
Robert Hensing 981c309057 Remove trace item: while calling the 'addErrorContext' builtin 2024-03-27 16:28:04 +01:00
Robert Hensing bebacc475c Always print addErrorContext traces 2024-03-27 16:28:04 +01:00
Robert Hensing 189e40214e Merge pull request #10339 from edolstra/gha-vm-tests
Run some VM tests on GitHub
2024-03-27 16:13:41 +01:00
Cyclic4179 9f7b2b93ce fixup! add flakes.sh test for shebang supported types of comments 2024-03-27 15:03:54 +01:00
Cyclic4179 6227cd06bd add flakes.sh test for shebang supported types of comments 2024-03-27 14:55:02 +01:00
Eelco Dolstra 6f3972498b Run some VM tests on GitHub 2024-03-27 14:11:06 +01:00
Cyclic4179 a2c3333b97 fix #10336 2024-03-27 07:47:36 +01:00
Théophane Hufschmitt 6c10cc0eda Merge pull request #10312 from hercules-ci/add-build-dir
Add build-dir setting
2024-03-26 13:51:50 +01:00
Théophane Hufschmitt 0bf01917ef Merge pull request #10325 from K900/ooformatting
build-remote: fix format string shenanigans
2024-03-26 11:45:32 +01:00
Théophane Hufschmitt 1394d4e9c5 Merge pull request #10306 from hercules-ci/baseNameOf
Test and document `builtins.baseNameOf`, improve internal `baseNameOf()`
2024-03-26 09:59:27 +01:00
K900 2d4edb945b build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes https://github.com/NixOS/nix/issues/10238
2024-03-26 11:01:03 +03:00
Robert Hensing 8b16cced18 Add build-dir setting 2024-03-25 19:48:59 +01:00
Robert Hensing b1fe388d33 Remove uncalled for message 2024-03-25 19:48:57 +01:00
Robert Hensing 6221770c9d tests/functional: Add count() 2024-03-25 19:46:34 +01:00
Théophane Hufschmitt 290072ca1e Merge pull request #10321 from Ericson2314/libstore-codeowner
Add @Ericson2314 as libstore codeowner
2024-03-25 18:27:20 +01:00
John Ericson bead1a1cde Add @Ericson2314 as libstore codeowner 2024-03-25 11:26:13 -04:00
Eelco Dolstra 6d90287f5a Merge pull request #10303 from hercules-ci/fix-empty-TMPDIR
fix: Treat empty TMPDIR as unset
2024-03-25 12:49:10 +01:00
Robert Hensing 641b0bd746 Merge pull request #10310 from obsidiansystems/fix-simple-git-hashing-test
Improve `git-hashing/simple.sh` test
2024-03-24 17:31:49 +01:00
John Ericson 513ba1977c Merge pull request #10307 from hercules-ci/doc-language-nix-dev
doc/language: Link to nix.dev introduction
2024-03-24 09:17:32 -04:00
John Ericson 2a44b11f55 Fix git-hashing/simple.sh
I realized it was checking NAR hashes before of added objects, which
makes little sense --- we don't really care about ancillary NAR hashes.

Now, the bottom `nix store add` tests compare the CA field with a git
hash to hashes calculated by Git. This matches top `nix hash path` ones
in using git as a source of truth.
2024-03-23 22:08:52 -04:00
Robert Hensing f891673460 Merge pull request #10293 from NixOS/fix-missing-format-arg
EvalCache: Fix missing format string argument
2024-03-24 02:28:05 +01:00
Robert Hensing fbf493758c doc/language: Link to nix.dev introduction
Closes #10283
2024-03-24 02:08:28 +01:00
Robert Hensing 9884018dfa baseNameOf(): Remove all trailing slashes 2024-03-24 01:38:22 +01:00
Robert Hensing 754a15e2db builtins.baseNameOf: Fork 2024-03-24 01:37:58 +01:00
Robert Hensing 175afc7106 Test and document builtins.baseNameOf 2024-03-24 01:26:17 +01:00
Robert Hensing 3b7f2bf997 git/dumpTree: Assert name not empty before back() 2024-03-24 00:58:08 +01:00
Robert Hensing 850c9a6caf HttpBinaryCacheStore: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing dd26f41379 local-derivation-goal.cc: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing fd31945742 local-derivation-goal.cc: Reuse defaultTempDir() 2024-03-24 00:45:15 +01:00
Robert Hensing b9e7f5aa2d fix: Treat empty XDG_RUNTIME_DIR as unset
See preceding commit. Not observed in the wild, but is sensible
and consistent with TMPDIR behavior.
2024-03-24 00:42:31 +01:00
Robert Hensing c3fb2aa1f9 fix: Treat empty TMPDIR as unset
Fixes an instance of

    nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.

... which I've been getting in one of my shells for some reason.
I have yet to find out why TMPDIR was empty, but it's no reason for
Nix to break.
2024-03-24 00:42:31 +01:00
detroyejr 50885b81c9 fix: correct remove-repl-flake.md 2024-03-23 06:31:49 -04:00
Théophane Hufschmitt 38ba96d7b0 Merge pull request #10296 from picnoir/pic/doc-lsp
Doc: explicitly specify which shell to use to get clangd
2024-03-23 08:46:13 +01:00
detroyejr b11dd58fe4 flakes: test to ensure we get an error if --file isn't used 2024-03-22 21:58:39 -04:00
detroyejr c625b45357 flakes: add remove-repl-flake.md note 2024-03-22 21:39:48 -04:00
detroyejr 717391731c flakes: remove experimental repl-flake 2024-03-22 19:30:03 -04:00
Picnoir cc29d85d06 Doc: explicitly specify which shell to use to get clangd
I was using by mistake the .#nix-clangStdenv shell to retrieve clangd.
This clangd is unusable with the project and constantly segfaults.
Let's explicitly state which shell the user should use in the docs.

I don't really understand the source of this segfault. I assume it's
related to a clang version incompatibility. (16.0.6 for
.#nix-clangStdenv 14.0.6 for .#native-clangStdenvPackages)
2024-03-22 22:36:55 +01:00
Théophane Hufschmitt 4ec2a7d3bf Merge pull request #10292 from dottharun/docBuildOutput
Docs: Documentation manual build output path correction
2024-03-22 21:45:20 +01:00
Eelco Dolstra bfd36402ac EvalCache: Fix missing format string argument
Fixes

  terminate called after throwing an instance of 'boost::wrapexcept<boost::io::too_few_args>'
    what():  boost::too_few_args: format-string referred to more arguments than were passed
  Aborted (core dumped)

for type errors in AttrCursor.
2024-03-22 18:11:24 +01:00
Tharun T 4e2f11b692 doc build output correction
Signed-off-by: Tharun T <tharun242424@gmail.com>
2024-03-22 18:34:55 +05:30
Robert Hensing a223280664 Merge pull request #10286 from hercules-ci/various
Allocate a bit less
2024-03-22 11:19:58 +01:00
Théophane Hufschmitt 57d9d0d6e4 Merge pull request #10269 from andreabedini/make-compile-commands
Create compile-commands.json with Make
2024-03-21 17:22:35 +01:00
Théophane Hufschmitt d0824f661e Document the new compile_commands.json target 2024-03-21 16:54:28 +01:00
Théophane Hufschmitt f666eda94d Merge pull request #10274 from tweag/warn-on-untrusted-setting
Daemon: warn when an untrusted user cannot override a setting
2024-03-21 16:40:24 +01:00
Théophane Hufschmitt 039b3fbdfe Merge pull request #10271 from jonringer/document-flake-multiple-outputs
Document how to build many outputs of a flake package
2024-03-21 15:58:32 +01:00
Andrea Bedini 83fc988bec Create compile-commands.json with Make 2024-03-21 15:55:44 +08:00
Robert Hensing 1fcdd1640e functionArgs: Allocate bools only once 2024-03-20 23:25:28 +01:00
Robert Hensing 8c6e0df45f value.hh: Fix warning about {struct/class} Value 2024-03-20 23:25:28 +01:00
Robert Hensing a865049c4f tryEval: Allocate true and false once 2024-03-20 23:25:28 +01:00
Robert Hensing d71e74838a readDir: Allocate type strings only once 2024-03-20 23:25:28 +01:00
Jonathan RingerandThéophane Hufschmitt 4c8a33ce46 Update src/nix/build.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-20 14:42:44 -07:00
Robert Hensing 7d2ead50e5 Merge pull request #10083 from lf-/jade/refactor-repl-input
refactor: move the repl input code to its own file
2024-03-20 22:37:40 +01:00
Robert Hensing d16d7f5f31 Merge pull request #10251 from edolstra/list-builder
Add a ListBuilder helper for constructing list values
2024-03-20 22:30:38 +01:00
Robert Hensing d4b0b7f152 createBaseEnv: Use state.vNull 2024-03-20 21:34:23 +01:00
Robert Hensing 0b08dd45b0 prim_match: Use state.vNull 2024-03-20 21:28:38 +01:00
Félix Baylac-Jacqué 40a7929c8e Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.

This is a cherry-pick of 9e0f5f803f.
The above commit has been reverted by
a59e77d9e5 to prevent spamming warnings
with experimental features, but these are now totally ignored on the
daemon side, so there's no reason for the revert any more.
2024-03-20 17:55:35 +01:00
Théophane Hufschmitt 346f92aa4c Merge pull request #10241 from bouk/resolve-ref-packed-refs
git fetcher: use resolveRef for getting revision of reference
2024-03-20 17:37:15 +01:00
Théophane Hufschmitt 4cb0847743 Merge pull request #10273 from jesses-canva/patch-1
Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in
2024-03-20 12:24:17 +01:00
Jesse Schalken c448636f7c Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in 2024-03-20 15:23:31 +11:00
Jonathan Ringer a5262fb880 Document how to build many outputs of a flake package 2024-03-19 17:37:04 -07:00
Théophane Hufschmitt 7c506432ab Merge pull request #10264 from lelgenio/document-pure-eval-store-path
Document builtins.storePath being disabled by pure-eval
2024-03-19 09:59:19 +01:00
lelgenio ad217ddbbc Document builtins.storePath being disabled by pure-eval 2024-03-18 16:23:17 -03:00
Théophane Hufschmitt 5c8983b9e6 Merge pull request #10259 from ShamrockLee/doc-primops-fix
doc: builtins.addDrvOutputDependencies: fix link target
2024-03-17 21:10:52 +01:00
Yueh-Shun Li d2b512959c builtins.addDrvOutputDependencies: fix commentary 2024-03-18 02:38:31 +08:00
Yueh-Shun Li 39b0b8452f doc: builtins.addDrvOutputDependencies: fix link target 2024-03-17 06:13:14 +08:00
Eelco Dolstra 3e6730ee62 Mark Value pointers in Value::elems as const
This catches modification of finalized values (e.g. in prim_sort).
2024-03-15 18:26:37 +01:00
Eelco Dolstra fecff520d7 Add a ListBuilder helper for constructing list values
Previously, `state.mkList()` would set the type of the value to tList
and allocate the list vector, but it would not initialize the values
in the list. This has two problems:

* If an exception occurs, the list is left in an undefined state.

* More importantly, for multithreaded evaluation, if a value
  transitions from thunk to non-thunk, it should be final (i.e. other
  threads should be able to access the value safely).

To address this, there now is a `ListBuilder` class (analogous to
`BindingsBuilder`) to build the list vector prior to the call to
`Value::mkList()`. Typical usage:

   auto list = state.buildList(size);
   for (auto & v : list)
       v = ... set value ...;
   vRes.mkList(list);
2024-03-15 18:26:37 +01:00
Eelco Dolstra 0fb5024d8d Merge pull request #10243 from DimitarNestorov/patch-1
docs: update registry examples
2024-03-15 18:13:12 +01:00
Eelco Dolstra 650161b7ae Merge pull request #10242 from fricklerhandwerk/fix-index-redirects
adjust anchor redirects to point to new pages
2024-03-15 18:12:50 +01:00
Daniel Sidhion 49e9efeaaa doc: document SRI hash format for outputHash (#10230) 2024-03-15 07:09:47 +01:00
Dimitar Nestorov a50295425e docs: update registry examples
When you run `nix flake info` you get a deprecated message
2024-03-14 16:15:52 +02:00
Valentin Gagarin 3754614b9c adjust anchor redirects to point to new pages
a previous moving of files accounted for server-side redirects, but not
client-side redirects.
2024-03-14 15:01:47 +01:00
Bouke van der Bijl 60c2d15f5a git fetcher: use resolveRef for getting revision of reference
* Add regression test
* Fix 'no repo' test so it doesn't succeed if the data is still in cache
* Use git_revparse_single inside git-utils instead of reimplementing the same logic.
2024-03-14 14:04:51 +01:00
Robert Hensing c152c2767a Merge pull request #10239 from correabuscar/patch-1
typo consant->constant in context.cc
2024-03-13 22:45:36 +01:00
Emanuel Czirai e5840d5760 typo consant->constant in context.cc 2024-03-13 20:35:24 +01:00
Jade Lovelace ea31b8a117 refactor: repl prompts are now the job of the interacter 2024-03-12 18:25:35 -07:00
Jade Lovelace 70a6ce139b refactor: move readline stuff into its own file
This is in direct preparation for an automation mode of nix repl.
2024-03-12 18:25:35 -07:00
Jade Lovelace 76aced6915 finally.hh: delete copy constructor which is a bad idea 2024-03-12 18:25:35 -07:00
Eelco Dolstra bff5c94184 Merge pull request #10205 from 9999years/final-matchers
Make `Matcher` subclasses `final`
2024-03-12 07:23:15 +01:00
Eelco Dolstra 18b89ca58c Merge pull request #10227 from tmplt/doc/nix-copy
nix-copy: document --all --from local binary cache example
2024-03-12 07:22:28 +01:00
Eelco Dolstra e3cc701d86 Merge pull request #10231 from NixOS/dependabot/github_actions/cachix/install-nix-action-26
Bump cachix/install-nix-action from 25 to 26
2024-03-12 07:16:25 +01:00
dependabot[bot] aa121dc318 Bump cachix/install-nix-action from 25 to 26
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 25 to 26.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v25...v26)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-11 22:02:01 +00:00
Eelco Dolstra 90e86bf6d3 Merge pull request #10229 from NixOS/bump-2.22.0
Bump version
2024-03-11 21:51:52 +01:00
Eelco Dolstra 222c38370f Bump version 2024-03-11 21:16:10 +01:00
Théophane Hufschmitt 18b2ef8b20 Merge pull request #10209 from 9999years/rethrow-for-top-level-errors-in-repl
Print top-level errors normally in `nix repl`
2024-03-11 21:11:28 +01:00
Viktor Sonesten db36c9ca90 nix-copy: document --all --from local binary cache example 2024-03-11 19:17:45 +01:00
tomberek 25bf671015 Merge pull request #10200 from 9999years/fix-nix-eval-for-derivations
Fix `nix eval nixpkgs#bash` segfault
2024-03-11 13:56:49 -04:00
Rebecca Turner 7f45b1c8d8 Add release note 2024-03-11 10:03:01 -07:00
Rebecca Turner db9bab2708 Matcher: Add virtual destructor 2024-03-10 12:56:07 -07:00
Rebecca Turner d13c63afa2 Print top-level errors normally in nix repl
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```

Now, errors will be displayed normally if they're emitted at the
top-level of an expression:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
       … in the condition of the assert statement
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
          267|     in commonAttrs // {
          268|       drvPath = assert condition; drv.drvPath;
             |                 ^
          269|       outPath = assert condition; drv.outPath;

       … in the left operand of the OR (||) operator
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
           27|         if lib.isDerivation value then
           28|           lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
             |                                             ^
           29|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```

Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.
2024-03-09 18:28:04 -08:00
Rebecca Turner 70e93c1e2b Make Matcher subclasses final
Fixes this very long warning, which I'll only include the first line of:

/nix/store/8wrjhrycpshhc3b41xmjwvgqr2m3yajq-libcxx-16.0.6-dev/include/c++/v1/__memory/construct_at.h:66:5: warning: destructor called on non-final 'RegexMatcher' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
    __loc->~_Tp();
2024-03-09 17:07:52 -08:00
Rebecca Turner 4910d74086 Print derivation paths in nix eval
`nix eval` forces values and prints derivations as attribute sets, so
commands that print derivations (e.g. `nix eval nixpkgs#bash`) will
infinitely loop and segfault.

Printing derivations as `.drv` paths makes `nix eval` complete as
expected. Further work is needed, but this is better than a segfault.
2024-03-08 21:43:54 -08:00
Ivan Shapovalov c6f0407103 libutil/terminal: cache isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 8c1eeb4681 treewide: shouldANSI() -> isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov d9fc4bf5c5 treewide: replace usages of isatty(STDERR_FILENO) with shouldANSI() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 950b6401f9 libmain/progress-bar: try harder to avoid escape sequences if !isTTY 2024-03-02 16:31:51 +01:00
Jade Lovelace f6158ea53b finally.hh: include works by itself; mark as nodiscard 2024-02-28 11:40:32 -08:00
145 changed files with 1501 additions and 1549 deletions

No files matched your search

+1 -1
View File
@@ -14,4 +14,4 @@
src/libexpr/primops.cc @roberth
# Libstore layer
/src/libstore @thufschmitt
/src/libstore @thufschmitt @ericson2314
+12 -4
View File
@@ -20,7 +20,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
@@ -62,7 +62,7 @@ jobs:
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v14
@@ -84,7 +84,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
@@ -114,7 +114,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
@@ -159,3 +159,11 @@ jobs:
# deprecated 2024-02-24
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes
+1
View File
@@ -142,6 +142,7 @@ GTAGS
# auto-generated compilation database
compile_commands.json
*.compile_commands.json
nix-rust/target
+1 -1
View File
@@ -1 +1 @@
2.21.5
2.22.0
+1 -5
View File
@@ -62,17 +62,13 @@ AC_CHECK_TOOL([AR], [ar])
AC_SYS_LARGEFILE
# OS-specific stuff.
# Solaris-specific stuff.
AC_STRUCT_DIRENT_D_TYPE
case "$host_os" in
solaris*)
# Solaris requires -lsocket -lnsl for network functions
LDFLAGS="-lsocket -lnsl $LDFLAGS"
;;
darwin*)
# Need to link to libsandbox.
LDFLAGS="-lsandbox $LDFLAGS"
;;
esac
+2
View File
@@ -6,6 +6,8 @@ additional-css = ["custom.css"]
additional-js = ["redirects.js"]
edit-url-template = "https://github.com/NixOS/nix/tree/master/doc/manual/{path}"
git-repository-url = "https://github.com/NixOS/nix"
fold.enable = true
fold.level = 1
[preprocessor.anchors]
renderers = ["html"]
+10
View File
@@ -175,6 +175,16 @@ $(d)/src/SUMMARY-rl-next.md: $(d)/src/release-notes/rl-next.md
# Generate the HTML manual.
.PHONY: manual-html
manual-html: $(docdir)/manual/index.html
# Open the built HTML manual in the default browser.
manual-html-open: $(docdir)/manual/index.html
@echo " OPEN " $<; \
xdg-open $< \
|| open $< \
|| { \
echo "Could not open the manual in a browser. Please open '$<'" >&2; \
false; \
}
install: $(docdir)/manual/index.html
# Generate 'nix' manpages.
+9 -9
View File
@@ -14,7 +14,7 @@
const redirects = {
"index.html": {
"part-advanced-topics": "advanced-topics/advanced-topics.html",
"part-advanced-topics": "advanced-topics/index.html",
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"chap-diff-hook": "advanced-topics/diff-hook.html",
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
@@ -22,7 +22,7 @@ const redirects = {
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"chap-writing-nix-expressions": "language/index.html",
"part-command-ref": "command-ref/command-ref.html",
"part-command-ref": "command-ref/index.html",
"conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
@@ -261,7 +261,7 @@ const redirects = {
"sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"chap-installation": "installation/installation.html",
"chap-installation": "installation/index.html",
"ch-installing-binary": "installation/installing-binary.html",
"sect-macos-installation": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
@@ -288,16 +288,16 @@ const redirects = {
"ssec-copy-closure": "package-management/copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"chap-package-management": "package-management/index.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "store/types/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "store/types/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "store/types/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "store/types/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",
"sec-relnotes": "release-notes/release-notes.html",
"sec-relnotes": "release-notes/index.html",
"ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"ch-relnotes-0.10": "release-notes/rl-0.10.html",
"ssec-relnotes-0.11": "release-notes/rl-0.11.html",
@@ -1,7 +0,0 @@
---
synopsis: Harden the user sandboxing
significance: significant
issues:
---
The build directory has been hardened against interference with the outside world by nesting it inside another directory owned by (and only readable by) the daemon user.
@@ -0,0 +1,13 @@
---
synopsis: "`nix eval` prints derivations as `.drv` paths"
prs: 10200
---
`nix eval` will now print derivations as their `.drv` paths, rather than as
attribute sets. This makes commands like `nix eval nixpkgs#bash` terminate
instead of infinitely looping into recursive self-referential attributes:
```ShellSession
$ nix eval nixpkgs#bash
«derivation /nix/store/m32cbgbd598f4w299g0hwyv7gbw6rqcg-bash-5.2p26.drv»
```
+8
View File
@@ -0,0 +1,8 @@
---
synopsis: Remove experimental repl-flake
significance: significant
issues: 10103
prs: 10299
---
The `repl-flake` experimental feature has been removed. The `nix repl` command now works like the rest of the new CLI in that `nix repl {path}` now tries to load a flake at `{path}` (or fails if the `flakes` experimental feature isn't enabled).*
-8
View File
@@ -1,8 +0,0 @@
---
synopsis: "`<nix/fetchurl.nix>` uses TLS verification"
prs: [11585]
---
Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
+1
View File
@@ -42,6 +42,7 @@
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
+1 -3
View File
@@ -27,11 +27,9 @@ and open `./result-doc/share/doc/nix/manual/index.html`.
To build the manual incrementally, [enter the development shell](./hacking.md) and run:
```console
make manual-html -j $NIX_BUILD_CORES
make manual-html-open -j $NIX_BUILD_CORES
```
and open `./outputs/out/share/doc/nix/manual/language/index.html`.
In order to reflect changes to the [Makefile for the manual], clear all generated files before re-building:
[Makefile for the manual]: https://github.com/NixOS/nix/blob/master/doc/manual/local.mk
+2 -2
View File
@@ -258,10 +258,10 @@ See [supported compilation environments](#compilation-environments) and instruct
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make clean && bear -- make -j$NIX_BUILD_CORES default check install
make compile_commands.json
```
Configure your editor to use the `clangd` from the shell, either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
Configure your editor to use the `clangd` from the `.#native-clangStdenvPackages` shell. You can do that either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
> **Note**
>
+1 -1
View File
@@ -28,7 +28,7 @@ $ sudo su
## macOS multi-user
```console
$ sudo nix-env --install --file '<nixpkgs>' --attr nix -I nixpkgs=channel:nixpkgs-unstable
$ sudo nix-env --install --file '<nixpkgs>' --attr nix cacert -I nixpkgs=channel:nixpkgs-unstable
$ sudo launchctl remove org.nixos.nix-daemon
$ sudo launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist
```
@@ -188,9 +188,13 @@ Derivations can declare some infrequently used optional attributes.
}
```
The `outputHashAlgo` attribute specifies the hash algorithm used to
compute the hash. It can currently be `"sha1"`, `"sha256"` or
`"sha512"`.
The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
The "nix32" encoding is an adaptation of base-32 encoding.
The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings, and the [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
It can currently be `"sha1"`, `"sha256"`, `"sha512"`, or `null`.
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
The `outputHashMode` attribute determines how the hash is computed.
It must be one of the following two values:
@@ -209,11 +213,6 @@ Derivations can declare some infrequently used optional attributes.
this case, the output can be anything, including a directory
tree.
The `outputHash` attribute, finally, must be a string containing
the hash in either hexadecimal or base-32 notation. (See the
[`nix-hash` command](../command-ref/nix-hash.md) for information
about converting to and from base-32 notation.)
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
> **Warning**
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
+7 -1
View File
@@ -1,7 +1,13 @@
# Nix Language
The Nix language is designed for conveniently creating and composing *derivations* – precise descriptions of how contents of existing files are used to derive new files.
It is:
> **Tip**
>
> These pages are written as a reference.
> If you are learning Nix, nix.dev has a good [introduction to the Nix language](https://nix.dev/tutorials/nix-language).
The language is:
- *domain-specific*
-6
View File
@@ -200,9 +200,3 @@
while performing various operations (including `nix develop`, `nix flake
update`, and so on). With several fixes to Nix's signal handlers, Nix
commands will now exit quickly after Ctrl-C is pressed.
- `nix copy` to a `ssh-ng` store now needs `--substitute-on-destination` (a.k.a. `-s`)
in order to substitute paths on the remote store instead of copying them.
The behavior is consistent with `nix copy` to a different kind of remote store.
Previously this behavior was controlled by the
`builders-use-substitutes` setting and `--substitute-on-destination` was ignored.
+4 -4
View File
@@ -14,7 +14,7 @@
inherit (nixpkgs) lib;
inherit (lib) fileset;
officialRelease = true;
officialRelease = false;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
@@ -165,7 +165,7 @@
nix =
let
officialRelease = true;
officialRelease = false;
versionSuffix =
if officialRelease
then ""
@@ -177,7 +177,7 @@
stdenv
versionSuffix
;
officialRelease = true;
officialRelease = false;
boehmgc = final.boehmgc-nix;
libgit2 = final.libgit2-nix;
busybox-sandbox-shell = final.busybox-sandbox-shell or final.default-busybox-sandbox-shell;
@@ -263,7 +263,7 @@
installerScriptForGHA = installScriptFor [
# Native
self.hydraJobs.binaryTarball."x86_64-linux"
self.hydraJobs.binaryTarball."aarch64-darwin"
self.hydraJobs.binaryTarball."x86_64-darwin"
# Cross
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
+1 -1
View File
@@ -7,4 +7,4 @@ $(foreach i, config.h $(wildcard src/lib*/*.hh), \
$(GCH): src/libutil/util.hh config.h
GCH_CXXFLAGS = -I src/libutil
GCH_CXXFLAGS = $(INCLUDE_libutil)
+11
View File
@@ -0,0 +1,11 @@
compile-commands-json-files :=
define write-compile-commands
_srcs := $$(sort $$(foreach src, $$($(1)_SOURCES), $$(src)))
$(1)_COMPILE_COMMANDS_JSON := $$(addprefix $(buildprefix), $$(addsuffix .compile_commands.json, $$(basename $$(_srcs))))
compile-commands-json-files += $$($(1)_COMPILE_COMMANDS_JSON)
clean-files += $$($(1)_COMPILE_COMMANDS_JSON)
endef
+8
View File
@@ -68,6 +68,7 @@ include mk/patterns.mk
include mk/templates.mk
include mk/cxx-big-literal.mk
include mk/tests.mk
include mk/compilation-database.mk
# Include all sub-Makefiles.
@@ -97,6 +98,13 @@ $(foreach test-group, $(install-tests-groups), \
$(eval $(call run-test,$(test),$(install_test_init))) \
$(eval $(test-group).test-group: $(test).test)))
# Compilation database.
$(foreach lib, $(libraries), $(eval $(call write-compile-commands,$(lib))))
$(foreach prog, $(programs), $(eval $(call write-compile-commands,$(prog))))
compile_commands.json: $(compile-commands-json-files)
@jq --slurp '.' $^ >$@
# Include makefiles requiring built programs.
$(foreach mf, $(makefiles-late), $(eval $(call include-sub-makefile,$(mf))))
+33 -3
View File
@@ -1,11 +1,41 @@
# These are the complete command lines we use to compile C and C++ files.
# - $< is the source file.
# - $1 is the object file to create.
CC_CMD=$(CC) -o $1 -c $< $(CPPFLAGS) $(GLOBAL_CFLAGS) $(CFLAGS) $($1_CFLAGS) -MMD -MF $(call filename-to-dep,$1) -MP
CXX_CMD=$(CXX) -o $1 -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($1_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep,$1) -MP
# We use COMPILE_COMMANDS_JSON_CMD to turn a compilation command (like CC_CMD
# or CXX_CMD above) into a comple_commands.json file. We rely on bash native
# word splitting to define the positional arguments.
# - $< is the source file being compiled.
COMPILE_COMMANDS_JSON_CMD=jq --null-input '{ directory: $$ENV.PWD, file: "$<", arguments: $$ARGS.positional }' --args --
$(buildprefix)%.o: %.cc
@mkdir -p "$(dir $@)"
$(trace-cxx) $(CXX) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($@_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cxx) $(call CXX_CMD,$@)
$(buildprefix)%.o: %.cpp
@mkdir -p "$(dir $@)"
$(trace-cxx) $(CXX) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($@_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cxx) $(call CXX_CMD,$@)
$(buildprefix)%.o: %.c
@mkdir -p "$(dir $@)"
$(trace-cc) $(CC) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CFLAGS) $(CFLAGS) $($@_CFLAGS) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cc) $(call CC_CMD,$@)
# In the following we need to replace the .compile_commands.json extension in $@ with .o
# to make the object file. This is needed because CC_CMD and CXX_CMD do further expansions
# based on the object file name (i.e. *_CXXFLAGS and filename-to-dep).
$(buildprefix)%.compile_commands.json: %.cc
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CXX_CMD,$(@:.compile_commands.json=.o)) > $@
$(buildprefix)%.compile_commands.json: %.cpp
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CXX_CMD,$(@:.compile_commands.json=.o)) > $@
$(buildprefix)%.compile_commands.json: %.c
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CC_CMD,$(@:.compile_commands.json=.o)) > $@
+2
View File
@@ -10,6 +10,8 @@ ifeq ($(V), 0)
trace-install = @echo " INST " $@;
trace-mkdir = @echo " MKDIR " $@;
trace-test = @echo " TEST " $@;
trace-sh = @echo " SH " $@;
trace-jq = @echo " JQ " $@;
suppress = @
+10 -3
View File
@@ -25,7 +25,6 @@
, libseccomp
, libsodium
, man
, darwin
, lowdown
, mdbook
, mdbook-linkcheck
@@ -76,7 +75,10 @@
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
, enableGC ? true
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows
# Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows
@@ -240,7 +242,6 @@ in {
gtest
rapidcheck
] ++ lib.optional stdenv.isLinux libseccomp
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin))
@@ -337,6 +338,12 @@ in {
echo "doc internal-api-docs $out/share/doc/nix/internal-api/html" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
# So the check output gets links for DLLs in the out output.
preFixup = lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs) ''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
doInstallCheck = attrs.doInstallCheck;
installCheckFlags = "sysconfdir=$(out)/etc";
+1 -2
View File
@@ -41,8 +41,7 @@ perl.pkgs.toPerlModule (stdenv.mkDerivation (finalAttrs: {
boost
]
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox;
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
# `perlPackages.Test2Harness` is marked broken for Darwin
doCheck = !stdenv.isDarwin;
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
((NEW_NIX_FIRST_BUILD_UID=351))
((NEW_NIX_FIRST_BUILD_UID=301))
id_available(){
dscl . list /Users UniqueID | grep -E '\b'$1'\b' >/dev/null
+1 -11
View File
@@ -4,17 +4,7 @@ set -eu
set -o pipefail
# System specific settings
# Notes:
# - up to macOS Big Sur we used the same GID/UIDs as Linux (30000:30001-32)
# - we changed UID to 301 because Big Sur updates failed into recovery mode
# we're targeting the 200-400 UID range for role users mentioned in the
# usage note for sysadminctl
# - we changed UID to 351 because Sequoia now uses UIDs 300-304 for its own
# daemon users
# - we changed GID to 350 alongside above just because it hides the nixbld
# group from the Users & Groups settings panel :)
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-351}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-350}"
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-301}"
export NIX_BUILD_USER_NAME_TEMPLATE="_nixbld%d"
readonly NIX_DAEMON_DEST=/Library/LaunchDaemons/org.nixos.nix-daemon.plist
+4 -2
View File
@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# each system specific installer must set these:
# NIX_FIRST_BUILD_UID
# NIX_BUILD_GROUP_ID
# NIX_BUILD_USER_NAME_TEMPLATE
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
@@ -530,7 +530,9 @@ It seems the build group $NIX_BUILD_GROUP_NAME already exists, but
with the UID $primary_group_id. This script can't really handle
that right now, so I'm going to give up.
You can export NIX_BUILD_GROUP_ID=$primary_group_id and re-run.
You can fix this by editing this script and changing the
NIX_BUILD_GROUP_ID variable near the top to from $NIX_BUILD_GROUP_ID
to $primary_group_id and re-run.
EOF
else
row " Exists" "Yes"
-1
View File
@@ -5,7 +5,6 @@ set -o pipefail
# System specific settings
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
export NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
readonly SERVICE_SRC=/lib/systemd/system/nix-daemon.service
+2 -2
View File
@@ -28,7 +28,7 @@ else
end
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
if test -n "$NIX_SSL_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt
@@ -44,7 +44,7 @@ else if test -e "$NIX_LINK/etc/ca-bundle.crt" # old cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ca-bundle.crt"
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
for i in $NIX_PROFILES
for i in (string split ' ' $NIX_PROFILES)
if test -e "$i/etc/ssl/certs/ca-bundle.crt"
set --export NIX_SSL_CERT_FILE "$i/etc/ssl/certs/ca-bundle.crt"
end
+1 -1
View File
@@ -6,7 +6,7 @@ libcmd_DIR := $(d)
libcmd_SOURCES := $(wildcard $(d)/*.cc)
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers
libcmd_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libfetchers) $(INCLUDE_libexpr) -I src/libmain
libcmd_LDFLAGS = $(EDITLINE_LIBS) $(LOWDOWN_LIBS) $(THREAD_LDFLAGS)
+1 -1
View File
@@ -50,7 +50,7 @@ std::string renderMarkdownToTerminal(std::string_view markdown)
if (!rndr_res)
throw Error("allocation error while rendering Markdown");
return filterANSIEscapes(std::string(buf->data, buf->size), !shouldANSI());
return filterANSIEscapes(std::string(buf->data, buf->size), !isTTY());
#else
return std::string(markdown);
#endif
+186
View File
@@ -0,0 +1,186 @@
#include <cstdio>
#ifdef USE_READLINE
#include <readline/history.h>
#include <readline/readline.h>
#else
// editline < 1.15.2 don't wrap their API for C++ usage
// (added in https://github.com/troglobit/editline/commit/91398ceb3427b730995357e9d120539fb9bb7461).
// This results in linker errors due to to name-mangling of editline C symbols.
// For compatibility with these versions, we wrap the API here
// (wrapping multiple times on newer versions is no problem).
extern "C" {
#include <editline.h>
}
#endif
#include "signals.hh"
#include "finally.hh"
#include "repl-interacter.hh"
#include "file-system.hh"
#include "libcmd/repl.hh"
namespace nix {
namespace {
// Used to communicate to NixRepl::getLine whether a signal occurred in ::readline.
volatile sig_atomic_t g_signal_received = 0;
void sigintHandler(int signo)
{
g_signal_received = signo;
}
};
static detail::ReplCompleterMixin * curRepl; // ugly
static char * completionCallback(char * s, int * match)
{
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
auto * res = strdup(possible.begin()->c_str() + strlen(s));
if (!res)
throw Error("allocation failure");
return res;
} else if (possible.size() > 1) {
auto checkAllHaveSameAt = [&](size_t pos) {
auto & first = *possible.begin();
for (auto & p : possible) {
if (p.size() <= pos || p[pos] != first[pos])
return false;
}
return true;
};
size_t start = strlen(s);
size_t len = 0;
while (checkAllHaveSameAt(start + len))
++len;
if (len > 0) {
*match = 1;
auto * res = strdup(std::string(*possible.begin(), start, len).c_str());
if (!res)
throw Error("allocation failure");
return res;
}
}
*match = 0;
return nullptr;
}
static int listPossibleCallback(char * s, char *** avp)
{
auto possible = curRepl->completePrefix(s);
if (possible.size() > (INT_MAX / sizeof(char *)))
throw Error("too many completions");
int ac = 0;
char ** vp = nullptr;
auto check = [&](auto * p) {
if (!p) {
if (vp) {
while (--ac >= 0)
free(vp[ac]);
free(vp);
}
throw Error("allocation failure");
}
return p;
};
vp = check((char **) malloc(possible.size() * sizeof(char *)));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
*avp = vp;
return ac;
}
ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleterMixin * repl)
{
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
try {
createDirs(dirOf(historyFile));
} catch (SystemError & e) {
logWarning(e.info());
}
#ifndef USE_READLINE
el_hist_size = 1000;
#endif
read_history(historyFile.c_str());
auto oldRepl = curRepl;
curRepl = repl;
Guard restoreRepl([oldRepl] { curRepl = oldRepl; });
#ifndef USE_READLINE
rl_set_complete_func(completionCallback);
rl_set_list_possib_func(listPossibleCallback);
#endif
return restoreRepl;
}
static constexpr const char * promptForType(ReplPromptType promptType)
{
switch (promptType) {
case ReplPromptType::ReplPrompt:
return "nix-repl> ";
case ReplPromptType::ContinuationPrompt:
return " ";
}
assert(false);
}
bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptType)
{
struct sigaction act, old;
sigset_t savedSignalMask, set;
auto setupSignals = [&]() {
act.sa_handler = sigintHandler;
sigfillset(&act.sa_mask);
act.sa_flags = 0;
if (sigaction(SIGINT, &act, &old))
throw SysError("installing handler for SIGINT");
sigemptyset(&set);
sigaddset(&set, SIGINT);
if (sigprocmask(SIG_UNBLOCK, &set, &savedSignalMask))
throw SysError("unblocking SIGINT");
};
auto restoreSignals = [&]() {
if (sigprocmask(SIG_SETMASK, &savedSignalMask, nullptr))
throw SysError("restoring signals");
if (sigaction(SIGINT, &old, 0))
throw SysError("restoring handler for SIGINT");
};
setupSignals();
char * s = readline(promptForType(promptType));
Finally doFree([&]() { free(s); });
restoreSignals();
if (g_signal_received) {
g_signal_received = 0;
input.clear();
return true;
}
if (!s)
return false;
input += s;
input += '\n';
return true;
}
ReadlineLikeInteracter::~ReadlineLikeInteracter()
{
write_history(historyFile.c_str());
}
};
+48
View File
@@ -0,0 +1,48 @@
#pragma once
/// @file
#include "finally.hh"
#include "types.hh"
#include <functional>
#include <string>
namespace nix {
namespace detail {
/** Provides the completion hooks for the repl, without exposing its complete
* internals. */
struct ReplCompleterMixin {
virtual StringSet completePrefix(const std::string & prefix) = 0;
};
};
enum class ReplPromptType {
ReplPrompt,
ContinuationPrompt,
};
class ReplInteracter
{
public:
using Guard = Finally<std::function<void()>>;
virtual Guard init(detail::ReplCompleterMixin * repl) = 0;
/** Returns a boolean of whether the interacter got EOF */
virtual bool getLine(std::string & input, ReplPromptType promptType) = 0;
virtual ~ReplInteracter(){};
};
class ReadlineLikeInteracter : public virtual ReplInteracter
{
std::string historyFile;
public:
ReadlineLikeInteracter(std::string historyFile)
: historyFile(historyFile)
{
}
virtual Guard init(detail::ReplCompleterMixin * repl) override;
virtual bool getLine(std::string & input, ReplPromptType promptType) override;
virtual ~ReadlineLikeInteracter() override;
};
};
+11 -167
View File
@@ -3,32 +3,17 @@
#include <cstring>
#include <climits>
#include <setjmp.h>
#ifdef USE_READLINE
#include <readline/history.h>
#include <readline/readline.h>
#else
// editline < 1.15.2 don't wrap their API for C++ usage
// (added in https://github.com/troglobit/editline/commit/91398ceb3427b730995357e9d120539fb9bb7461).
// This results in linker errors due to to name-mangling of editline C symbols.
// For compatibility with these versions, we wrap the API here
// (wrapping multiple times on newer versions is no problem).
extern "C" {
#include <editline.h>
}
#endif
#include "libcmd/repl-interacter.hh"
#include "repl.hh"
#include "ansicolor.hh"
#include "signals.hh"
#include "shared.hh"
#include "eval.hh"
#include "eval-cache.hh"
#include "eval-inline.hh"
#include "eval-settings.hh"
#include "attr-path.hh"
#include "signals.hh"
#include "store-api.hh"
#include "log-store.hh"
#include "common-eval-args.hh"
@@ -38,7 +23,6 @@ extern "C" {
#include "flake/flake.hh"
#include "flake/lockfile.hh"
#include "users.hh"
#include "terminal.hh"
#include "editor-for.hh"
#include "finally.hh"
#include "markdown.hh"
@@ -75,6 +59,7 @@ enum class ProcessLineResult {
struct NixRepl
: AbstractNixRepl
, detail::ReplCompleterMixin
#if HAVE_BOEHMGC
, gc
#endif
@@ -90,17 +75,16 @@ struct NixRepl
int displ;
StringSet varNames;
const Path historyFile;
std::unique_ptr<ReplInteracter> interacter;
NixRepl(const SearchPath & searchPath, nix::ref<Store> store,ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
virtual ~NixRepl();
virtual ~NixRepl() = default;
ReplExitStatus mainLoop() override;
void initEnv() override;
StringSet completePrefix(const std::string & prefix);
bool getLine(std::string & input, const std::string & prompt);
virtual StringSet completePrefix(const std::string & prefix) override;
StorePath getDerivationPath(Value & v);
ProcessLineResult processLine(std::string line);
@@ -123,7 +107,8 @@ struct NixRepl
.force = true,
.derivationPaths = true,
.maxDepth = maxDepth,
.prettyIndent = 2
.prettyIndent = 2,
.errors = ErrorPrintBehavior::ThrowTopLevel,
});
}
};
@@ -143,16 +128,10 @@ NixRepl::NixRepl(const SearchPath & searchPath, nix::ref<Store> store, ref<EvalS
, debugTraceIndex(0)
, getValues(getValues)
, staticEnv(new StaticEnv(nullptr, state->staticBaseEnv.get()))
, historyFile(getDataDir() + "/nix/repl-history")
, interacter(make_unique<ReadlineLikeInteracter>(getDataDir() + "/nix/repl-history"))
{
}
NixRepl::~NixRepl()
{
write_history(historyFile.c_str());
}
void runNix(Path program, const Strings & args,
const std::optional<std::string> & input = {})
{
@@ -169,79 +148,6 @@ void runNix(Path program, const Strings & args,
return;
}
static NixRepl * curRepl; // ugly
static char * completionCallback(char * s, int *match) {
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
auto *res = strdup(possible.begin()->c_str() + strlen(s));
if (!res) throw Error("allocation failure");
return res;
} else if (possible.size() > 1) {
auto checkAllHaveSameAt = [&](size_t pos) {
auto &first = *possible.begin();
for (auto &p : possible) {
if (p.size() <= pos || p[pos] != first[pos])
return false;
}
return true;
};
size_t start = strlen(s);
size_t len = 0;
while (checkAllHaveSameAt(start + len)) ++len;
if (len > 0) {
*match = 1;
auto *res = strdup(std::string(*possible.begin(), start, len).c_str());
if (!res) throw Error("allocation failure");
return res;
}
}
*match = 0;
return nullptr;
}
static int listPossibleCallback(char *s, char ***avp) {
auto possible = curRepl->completePrefix(s);
if (possible.size() > (INT_MAX / sizeof(char*)))
throw Error("too many completions");
int ac = 0;
char **vp = nullptr;
auto check = [&](auto *p) {
if (!p) {
if (vp) {
while (--ac >= 0)
free(vp[ac]);
free(vp);
}
throw Error("allocation failure");
}
return p;
};
vp = check((char **)malloc(possible.size() * sizeof(char*)));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
*avp = vp;
return ac;
}
namespace {
// Used to communicate to NixRepl::getLine whether a signal occurred in ::readline.
volatile sig_atomic_t g_signal_received = 0;
void sigintHandler(int signo) {
g_signal_received = signo;
}
}
static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positions, const DebugTrace & dt)
{
if (dt.isError)
@@ -281,24 +187,7 @@ ReplExitStatus NixRepl::mainLoop()
loadFiles();
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
try {
createDirs(dirOf(historyFile));
} catch (SystemError & e) {
logWarning(e.info());
}
#ifndef USE_READLINE
el_hist_size = 1000;
#endif
read_history(historyFile.c_str());
auto oldRepl = curRepl;
curRepl = this;
Finally restoreRepl([&] { curRepl = oldRepl; });
#ifndef USE_READLINE
rl_set_complete_func(completionCallback);
rl_set_list_possib_func(listPossibleCallback);
#endif
auto _guard = interacter->init(static_cast<detail::ReplCompleterMixin *>(this));
std::string input;
@@ -307,7 +196,7 @@ ReplExitStatus NixRepl::mainLoop()
logger->pause();
// When continuing input from previous lines, don't print a prompt, just align to the same
// number of chars as the prompt.
if (!getLine(input, input.empty() ? "nix-repl> " : " ")) {
if (!interacter->getLine(input, input.empty() ? ReplPromptType::ReplPrompt : ReplPromptType::ContinuationPrompt)) {
// Ctrl-D should exit the debugger.
state->debugStop = false;
logger->cout("");
@@ -350,51 +239,6 @@ ReplExitStatus NixRepl::mainLoop()
}
}
bool NixRepl::getLine(std::string & input, const std::string & prompt)
{
struct sigaction act, old;
sigset_t savedSignalMask, set;
auto setupSignals = [&]() {
act.sa_handler = sigintHandler;
sigfillset(&act.sa_mask);
act.sa_flags = 0;
if (sigaction(SIGINT, &act, &old))
throw SysError("installing handler for SIGINT");
sigemptyset(&set);
sigaddset(&set, SIGINT);
if (sigprocmask(SIG_UNBLOCK, &set, &savedSignalMask))
throw SysError("unblocking SIGINT");
};
auto restoreSignals = [&]() {
if (sigprocmask(SIG_SETMASK, &savedSignalMask, nullptr))
throw SysError("restoring signals");
if (sigaction(SIGINT, &old, 0))
throw SysError("restoring handler for SIGINT");
};
setupSignals();
char * s = readline(prompt.c_str());
Finally doFree([&]() { free(s); });
restoreSignals();
if (g_signal_received) {
g_signal_received = 0;
input.clear();
return true;
}
if (!s)
return false;
input += s;
input += '\n';
return true;
}
StringSet NixRepl::completePrefix(const std::string & prefix)
{
StringSet completions;
-5
View File
@@ -3,11 +3,6 @@
#include "eval.hh"
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW
#include <gc/gc_cpp.h>
#endif
namespace nix {
struct AbstractNixRepl
+2 -1
View File
@@ -76,9 +76,10 @@ struct EvalSettings : Config
- Restrict file system and network access to files specified by cryptographic hash
- Disable impure constants:
- [`bultins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
- [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
- [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
- [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath)
- [`builtins.storePath`](@docroot@/language/builtin-constants.md#builtins-storePath)
)"
};
+27 -14
View File
@@ -435,7 +435,14 @@ EvalState::EvalState(
static_assert(sizeof(Env) <= 16, "environment must be <= 16 bytes");
vEmptyList.mkList(0);
vEmptyList.mkList(buildList(0));
vNull.mkNull();
vTrue.mkBool(true);
vFalse.mkBool(false);
vStringRegular.mkString("regular");
vStringDirectory.mkString("directory");
vStringSymlink.mkString("symlink");
vStringUnknown.mkString("unknown");
/* Initialise the Nix expression search path. */
if (!evalSettings.pureEval) {
@@ -923,14 +930,16 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
}
}
void EvalState::mkList(Value & v, size_t size)
ListBuilder::ListBuilder(EvalState & state, size_t size)
: size(size)
, elems(size <= 2 ? inlineElems : (Value * *) allocBytes(size * sizeof(Value *)))
{
v.mkList(size);
if (size > 2)
v.bigList.elems = (Value * *) allocBytes(size * sizeof(Value *));
nrListElems += size;
state.nrListElems += size;
}
Value * EvalState::getBool(bool b) {
return b ? &vTrue : &vFalse;
}
unsigned long nrThunks = 0;
@@ -1353,9 +1362,10 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
void ExprList::eval(EvalState & state, Env & env, Value & v)
{
state.mkList(v, elems.size());
for (auto [n, v2] : enumerate(v.listItems()))
const_cast<Value * &>(v2) = elems[n]->maybeThunk(state, env);
auto list = state.buildList(elems.size());
for (const auto & [n, v2] : enumerate(list))
v2 = elems[n]->maybeThunk(state, env);
v.mkList(list);
}
@@ -1655,7 +1665,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
try {
fn->fun(*this, vCur.determinePos(noPos), args, vCur);
} catch (Error & e) {
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
if (fn->addTrace)
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
throw;
}
@@ -1703,7 +1714,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
// so the debugger allows to inspect the wrong parameters passed to the builtin.
fn->fun(*this, vCur.determinePos(noPos), vArgs, vCur);
} catch (Error & e) {
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
if (fn->addTrace)
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
throw;
}
@@ -1945,7 +1957,7 @@ void ExprOpConcatLists::eval(EvalState & state, Env & env, Value & v)
}
void EvalState::concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx)
void EvalState::concatLists(Value & v, size_t nrLists, Value * const * lists, const PosIdx pos, std::string_view errorCtx)
{
nrListConcats++;
@@ -1963,14 +1975,15 @@ void EvalState::concatLists(Value & v, size_t nrLists, Value * * lists, const Po
return;
}
mkList(v, len);
auto out = v.listElems();
auto list = buildList(len);
auto out = list.elems;
for (size_t n = 0, pos = 0; n < nrLists; ++n) {
auto l = lists[n]->listSize();
if (l)
memcpy(out + pos, lists[n]->listElems(), l * sizeof(Value *));
pos += l;
}
v.mkList(list);
}
+49 -2
View File
@@ -69,6 +69,13 @@ struct PrimOp
*/
const char * doc = nullptr;
/**
* Add a trace item, `while calling the '<name>' builtin`
*
* This is used to remove the redundant item for `builtins.addErrorContext`.
*/
bool addTrace = true;
/**
* Implementation of the primop.
*/
@@ -186,6 +193,36 @@ public:
*/
Value vEmptyList;
/**
* `null` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vNull;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vTrue;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vFalse;
/** `"regular"` */
Value vStringRegular;
/** `"directory"` */
Value vStringDirectory;
/** `"symlink"` */
Value vStringSymlink;
/** `"unknown"` */
Value vStringUnknown;
/**
* The accessor for the root filesystem.
*/
@@ -615,7 +652,16 @@ public:
return BindingsBuilder(*this, allocBindings(capacity));
}
void mkList(Value & v, size_t length);
ListBuilder buildList(size_t size)
{
return ListBuilder(*this, size);
}
/**
* Return a boolean `Value *` without allocating.
*/
Value *getBool(bool b);
void mkThunk_(Value & v, Expr * expr);
void mkPos(Value & v, PosIdx pos);
@@ -662,7 +708,7 @@ public:
const SingleDerivedPath & p,
Value & v);
void concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx);
void concatLists(Value & v, size_t nrLists, Value * const * lists, const PosIdx pos, std::string_view errorCtx);
/**
* Print statistics, if enabled.
@@ -756,6 +802,7 @@ private:
friend void prim_split(EvalState & state, const PosIdx pos, Value * * args, Value & v);
friend struct Value;
friend class ListBuilder;
};
struct DebugTraceStacker {
+4 -5
View File
@@ -57,11 +57,10 @@ class JSONSax : nlohmann::json_sax<json> {
ValueVector values;
std::unique_ptr<JSONState> resolve(EvalState & state) override
{
Value & v = parent->value(state);
state.mkList(v, values.size());
for (size_t n = 0; n < values.size(); ++n) {
v.listElems()[n] = values[n];
}
auto list = state.buildList(values.size());
for (const auto & [n, v2] : enumerate(list))
v2 = values[n];
parent->value(state).mkList(list);
return std::move(parent);
}
void add() override {
+4 -1
View File
@@ -11,8 +11,11 @@ libexpr_SOURCES := \
$(wildcard $(d)/flake/*.cc) \
$(d)/lexer-tab.cc \
$(d)/parser-tab.cc
# Not just for this library itself, but also for downstream libraries using this library
libexpr_CXXFLAGS += -I src/libutil -I src/libstore -I src/libfetchers -I src/libmain -I src/libexpr
INCLUDE_libexpr := -I $(d)
libexpr_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libfetchers) -I src/libmain $(INCLUDE_libexpr)
libexpr_LIBS = libutil libstore libfetchers
+163 -119
View File
@@ -187,13 +187,13 @@ static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * v
NixStringContextElem::DrvDeep { .drvPath = *storePath },
});
attrs.alloc(state.sName).mkString(drv.env["name"]);
auto & outputsVal = attrs.alloc(state.sOutputs);
state.mkList(outputsVal, drv.outputs.size());
auto list = state.buildList(drv.outputs.size());
for (const auto & [i, o] : enumerate(drv.outputs)) {
mkOutputString(state, attrs, *storePath, o);
(outputsVal.listElems()[i] = state.allocValue())->mkString(o.first);
(list[i] = state.allocValue())->mkString(o.first);
}
attrs.alloc(state.sOutputs).mkList(list);
auto w = state.allocValue();
w->mkAttrs(attrs);
@@ -694,10 +694,10 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
}
/* Create the result list. */
state.mkList(v, res.size());
unsigned int n = 0;
for (auto & i : res)
v.listElems()[n++] = i;
auto list = state.buildList(res.size());
for (const auto & [n, i] : enumerate(res))
list[n] = i;
v.mkList(list);
}
static RegisterPrimOp primop_genericClosure(PrimOp {
@@ -826,7 +826,7 @@ static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value * *
auto message = state.coerceToString(pos, *args[0], context,
"while evaluating the error message passed to builtins.addErrorContext",
false, false).toOwned();
e.addTrace(nullptr, HintFmt(message));
e.addTrace(nullptr, HintFmt(message), TracePrint::Always);
throw;
}
}
@@ -834,6 +834,8 @@ static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value * *
static RegisterPrimOp primop_addErrorContext(PrimOp {
.name = "__addErrorContext",
.arity = 2,
// The normal trace item is redundant
.addTrace = false,
.fun = prim_addErrorContext,
});
@@ -896,10 +898,11 @@ static void prim_tryEval(EvalState & state, const PosIdx pos, Value * * args, Va
try {
state.forceValue(*args[0], pos);
attrs.insert(state.sValue, args[0]);
attrs.alloc("success").mkBool(true);
attrs.insert(state.symbols.create("success"), &state.vTrue);
} catch (AssertionError & e) {
attrs.alloc(state.sValue).mkBool(false);
attrs.alloc("success").mkBool(false);
// `value = false;` is unfortunate but removing it is a breaking change.
attrs.insert(state.sValue, &state.vFalse);
attrs.insert(state.symbols.create("success"), &state.vFalse);
}
// restore the debugRepl pointer if we saved it earlier.
@@ -1567,23 +1570,50 @@ static RegisterPrimOp primop_pathExists({
.fun = prim_pathExists,
});
// Ideally, all trailing slashes should have been removed, but it's been like this for
// almost a decade as of writing. Changing it will affect reproducibility.
static std::string_view legacyBaseNameOf(std::string_view path)
{
if (path.empty())
return "";
auto last = path.size() - 1;
if (path[last] == '/' && last > 0)
last -= 1;
auto pos = path.rfind('/', last);
if (pos == path.npos)
pos = 0;
else
pos += 1;
return path.substr(pos, last - pos + 1);
}
/* Return the base name of the given string, i.e., everything
following the last slash. */
static void prim_baseNameOf(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
NixStringContext context;
v.mkString(baseNameOf(*state.coerceToString(pos, *args[0], context,
v.mkString(legacyBaseNameOf(*state.coerceToString(pos, *args[0], context,
"while evaluating the first argument passed to builtins.baseNameOf",
false, false)), context);
}
static RegisterPrimOp primop_baseNameOf({
.name = "baseNameOf",
.args = {"s"},
.args = {"x"},
.doc = R"(
Return the *base name* of the string *s*, that is, everything
following the final slash in the string. This is similar to the GNU
`basename` command.
Return the *base name* of either a [path value](@docroot@/language/values.md#type-path) *x* or a string *x*, depending on which type is passed, and according to the following rules.
For a path value, the *base name* is considered to be the part of the path after the last directory separator, including any file extensions.
This is the simple case, as path values don't have trailing slashes.
When the argument is a string, a more involved logic applies. If the string ends with a `/`, only this one final slash is removed.
After this, the *base name* is returned as previously described, assuming `/` as the directory separator. (Note that evaluation must be platform independent.)
This is somewhat similar to the [GNU `basename`](https://www.gnu.org/software/coreutils/manual/html_node/basename-invocation.html) command, but GNU `basename` will strip any number of trailing slashes.
)",
.fun = prim_baseNameOf,
});
@@ -1775,20 +1805,20 @@ static RegisterPrimOp primop_hashFile({
.fun = prim_hashFile,
});
static std::string_view fileTypeToString(InputAccessor::Type type)
static Value * fileTypeToString(EvalState & state, InputAccessor::Type type)
{
return
type == InputAccessor::Type::tRegular ? "regular" :
type == InputAccessor::Type::tDirectory ? "directory" :
type == InputAccessor::Type::tSymlink ? "symlink" :
"unknown";
type == InputAccessor::Type::tRegular ? &state.vStringRegular :
type == InputAccessor::Type::tDirectory ? &state.vStringDirectory :
type == InputAccessor::Type::tSymlink ? &state.vStringSymlink :
&state.vStringUnknown;
}
static void prim_readFileType(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
auto path = realisePath(state, pos, *args[0], std::nullopt);
/* Retrieve the directory entry type and stringize it. */
v.mkString(fileTypeToString(path.lstat().type));
v = *fileTypeToString(state, path.lstat().type);
}
static RegisterPrimOp primop_readFileType({
@@ -1819,8 +1849,8 @@ static void prim_readDir(EvalState & state, const PosIdx pos, Value * * args, Va
Value * readFileType = nullptr;
for (auto & [name, type] : entries) {
auto & attr = attrs.alloc(name);
if (!type) {
auto & attr = attrs.alloc(name);
// Some filesystems or operating systems may not be able to return
// detailed node info quickly in this case we produce a thunk to
// query the file type lazily.
@@ -1832,7 +1862,7 @@ static void prim_readDir(EvalState & state, const PosIdx pos, Value * * args, Va
} else {
// This branch of the conditional is much more likely.
// Here we just stringize the directory entry type.
attr.mkString(fileTypeToString(*type));
attrs.insert(state.symbols.create(name), fileTypeToString(state, *type));
}
}
@@ -2193,11 +2223,8 @@ bool EvalState::callPathFilter(
Value arg1;
arg1.mkString(pathArg);
Value arg2;
// assert that type is not "unknown"
arg2.mkString(fileTypeToString(st.type));
Value * args []{&arg1, &arg2};
Value * args []{&arg1, fileTypeToString(*this, st.type)};
Value res;
callFunction(*filterFun, 2, args, res, pos);
@@ -2423,14 +2450,15 @@ static void prim_attrNames(EvalState & state, const PosIdx pos, Value * * args,
{
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.attrNames");
state.mkList(v, args[0]->attrs->size());
auto list = state.buildList(args[0]->attrs->size());
size_t n = 0;
for (auto & i : *args[0]->attrs)
(v.listElems()[n++] = state.allocValue())->mkString(state.symbols[i.name]);
for (const auto & [n, i] : enumerate(*args[0]->attrs))
(list[n] = state.allocValue())->mkString(state.symbols[i.name]);
std::sort(v.listElems(), v.listElems() + n,
std::sort(list.begin(), list.end(),
[](Value * v1, Value * v2) { return strcmp(v1->c_str(), v2->c_str()) < 0; });
v.mkList(list);
}
static RegisterPrimOp primop_attrNames({
@@ -2450,21 +2478,22 @@ static void prim_attrValues(EvalState & state, const PosIdx pos, Value * * args,
{
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.attrValues");
state.mkList(v, args[0]->attrs->size());
auto list = state.buildList(args[0]->attrs->size());
unsigned int n = 0;
for (auto & i : *args[0]->attrs)
v.listElems()[n++] = (Value *) &i;
for (const auto & [n, i] : enumerate(*args[0]->attrs))
list[n] = (Value *) &i;
std::sort(v.listElems(), v.listElems() + n,
std::sort(list.begin(), list.end(),
[&](Value * v1, Value * v2) {
std::string_view s1 = state.symbols[((Attr *) v1)->name],
s2 = state.symbols[((Attr *) v2)->name];
return s1 < s2;
});
for (unsigned int i = 0; i < n; ++i)
v.listElems()[i] = ((Attr *) v.listElems()[i])->value;
for (auto & v : list)
v = ((Attr *) v)->value;
v.mkList(list);
}
static RegisterPrimOp primop_attrValues({
@@ -2805,9 +2834,10 @@ static void prim_catAttrs(EvalState & state, const PosIdx pos, Value * * args, V
res[found++] = i->value;
}
state.mkList(v, found);
auto list = state.buildList(found);
for (unsigned int n = 0; n < found; ++n)
v.listElems()[n] = res[n];
list[n] = res[n];
v.mkList(list);
}
static RegisterPrimOp primop_catAttrs({
@@ -2844,8 +2874,7 @@ static void prim_functionArgs(EvalState & state, const PosIdx pos, Value * * arg
auto attrs = state.buildBindings(args[0]->lambda.fun->formals->formals.size());
for (auto & i : args[0]->lambda.fun->formals->formals)
// !!! should optimise booleans (allocate only once)
attrs.alloc(i.name, i.pos).mkBool(i.def);
attrs.insert(i.name, state.getBool(i.def), i.pos);
v.mkAttrs(attrs);
}
@@ -2908,43 +2937,50 @@ static void prim_zipAttrsWith(EvalState & state, const PosIdx pos, Value * * arg
// attribute with the merge function application. this way we need not
// use (slightly slower) temporary storage the GC does not know about.
std::map<Symbol, std::pair<size_t, Value * *>> attrsSeen;
struct Item
{
size_t size = 0;
size_t pos = 0;
std::optional<ListBuilder> list;
};
std::map<Symbol, Item> attrsSeen;
state.forceFunction(*args[0], pos, "while evaluating the first argument passed to builtins.zipAttrsWith");
state.forceList(*args[1], pos, "while evaluating the second argument passed to builtins.zipAttrsWith");
const auto listSize = args[1]->listSize();
const auto listElems = args[1]->listElems();
const auto listItems = args[1]->listItems();
for (unsigned int n = 0; n < listSize; ++n) {
Value * vElem = listElems[n];
for (auto & vElem : listItems) {
state.forceAttrs(*vElem, noPos, "while evaluating a value of the list passed as second argument to builtins.zipAttrsWith");
for (auto & attr : *vElem->attrs)
attrsSeen[attr.name].first++;
attrsSeen.try_emplace(attr.name).first->second.size++;
}
for (auto & [sym, elem] : attrsSeen)
elem.list.emplace(state.buildList(elem.size));
for (auto & vElem : listItems) {
for (auto & attr : *vElem->attrs) {
auto & item = attrsSeen.at(attr.name);
(*item.list)[item.pos++] = attr.value;
}
}
auto attrs = state.buildBindings(attrsSeen.size());
for (auto & [sym, elem] : attrsSeen) {
auto & list = attrs.alloc(sym);
state.mkList(list, elem.first);
elem.second = list.listElems();
}
v.mkAttrs(attrs.alreadySorted());
for (unsigned int n = 0; n < listSize; ++n) {
Value * vElem = listElems[n];
for (auto & attr : *vElem->attrs)
*attrsSeen[attr.name].second++ = attr.value;
}
for (auto & attr : *v.attrs) {
auto name = state.allocValue();
name->mkString(state.symbols[attr.name]);
name->mkString(state.symbols[sym]);
auto call1 = state.allocValue();
call1->mkApp(args[0], name);
auto call2 = state.allocValue();
call2->mkApp(call1, attr.value);
attr.value = call2;
auto arg = state.allocValue();
arg->mkList(*elem.list);
call2->mkApp(call1, arg);
attrs.insert(sym, call2);
}
v.mkAttrs(attrs.alreadySorted());
}
static RegisterPrimOp primop_zipAttrsWith({
@@ -3055,9 +3091,10 @@ static void prim_tail(EvalState & state, const PosIdx pos, Value * * args, Value
if (args[0]->listSize() == 0)
state.error<EvalError>("'tail' called on an empty list").atPos(pos).debugThrow();
state.mkList(v, args[0]->listSize() - 1);
for (unsigned int n = 0; n < v.listSize(); ++n)
v.listElems()[n] = args[0]->listElems()[n + 1];
auto list = state.buildList(args[0]->listSize() - 1);
for (const auto & [n, v] : enumerate(list))
v = args[0]->listElems()[n + 1];
v.mkList(list);
}
static RegisterPrimOp primop_tail({
@@ -3088,10 +3125,11 @@ static void prim_map(EvalState & state, const PosIdx pos, Value * * args, Value
state.forceFunction(*args[0], pos, "while evaluating the first argument passed to builtins.map");
state.mkList(v, args[1]->listSize());
for (unsigned int n = 0; n < v.listSize(); ++n)
(v.listElems()[n] = state.allocValue())->mkApp(
auto list = state.buildList(args[1]->listSize());
for (const auto & [n, v] : enumerate(list))
(v = state.allocValue())->mkApp(
args[0], args[1]->listElems()[n]);
v.mkList(list);
}
static RegisterPrimOp primop_map({
@@ -3140,8 +3178,9 @@ static void prim_filter(EvalState & state, const PosIdx pos, Value * * args, Val
if (same)
v = *args[1];
else {
state.mkList(v, k);
for (unsigned int n = 0; n < k; ++n) v.listElems()[n] = vs[n];
auto list = state.buildList(k);
for (const auto & [n, v] : enumerate(list)) v = vs[n];
v.mkList(list);
}
}
@@ -3316,12 +3355,13 @@ static void prim_genList(EvalState & state, const PosIdx pos, Value * * args, Va
// as evaluating map without accessing any values makes little sense.
state.forceFunction(*args[0], noPos, "while evaluating the first argument passed to builtins.genList");
state.mkList(v, len);
for (unsigned int n = 0; n < (unsigned int) len; ++n) {
auto list = state.buildList(len);
for (const auto & [n, v] : enumerate(list)) {
auto arg = state.allocValue();
arg->mkInt(n);
(v.listElems()[n] = state.allocValue())->mkApp(args[0], arg);
(v = state.allocValue())->mkApp(args[0], arg);
}
v.mkList(list);
}
static RegisterPrimOp primop_genList({
@@ -3355,11 +3395,9 @@ static void prim_sort(EvalState & state, const PosIdx pos, Value * * args, Value
state.forceFunction(*args[0], pos, "while evaluating the first argument passed to builtins.sort");
state.mkList(v, len);
for (unsigned int n = 0; n < len; ++n) {
state.forceValue(*args[1]->listElems()[n], pos);
v.listElems()[n] = args[1]->listElems()[n];
}
auto list = state.buildList(len);
for (const auto & [n, v] : enumerate(list))
state.forceValue(*(v = args[1]->listElems()[n]), pos);
auto comparator = [&](Value * a, Value * b) {
/* Optimization: if the comparator is lessThan, bypass
@@ -3378,7 +3416,9 @@ static void prim_sort(EvalState & state, const PosIdx pos, Value * * args, Value
/* FIXME: std::sort can segfault if the comparator is not a strict
weak ordering. What to do? std::stable_sort() seems more
resilient, but no guarantees... */
std::stable_sort(v.listElems(), v.listElems() + len, comparator);
std::stable_sort(list.begin(), list.end(), comparator);
v.mkList(list);
}
static RegisterPrimOp primop_sort({
@@ -3424,17 +3464,17 @@ static void prim_partition(EvalState & state, const PosIdx pos, Value * * args,
auto attrs = state.buildBindings(2);
auto & vRight = attrs.alloc(state.sRight);
auto rsize = right.size();
state.mkList(vRight, rsize);
auto rlist = state.buildList(rsize);
if (rsize)
memcpy(vRight.listElems(), right.data(), sizeof(Value *) * rsize);
memcpy(rlist.elems, right.data(), sizeof(Value *) * rsize);
attrs.alloc(state.sRight).mkList(rlist);
auto & vWrong = attrs.alloc(state.sWrong);
auto wsize = wrong.size();
state.mkList(vWrong, wsize);
auto wlist = state.buildList(wsize);
if (wsize)
memcpy(vWrong.listElems(), wrong.data(), sizeof(Value *) * wsize);
memcpy(wlist.elems, wrong.data(), sizeof(Value *) * wsize);
attrs.alloc(state.sWrong).mkList(wlist);
v.mkAttrs(attrs);
}
@@ -3481,10 +3521,10 @@ static void prim_groupBy(EvalState & state, const PosIdx pos, Value * * args, Va
auto attrs2 = state.buildBindings(attrs.size());
for (auto & i : attrs) {
auto & list = attrs2.alloc(i.first);
auto size = i.second.size();
state.mkList(list, size);
memcpy(list.listElems(), i.second.data(), sizeof(Value *) * size);
auto list = state.buildList(size);
memcpy(list.elems, i.second.data(), sizeof(Value *) * size);
attrs2.alloc(i.first).mkList(list);
}
v.mkAttrs(attrs2.alreadySorted());
@@ -3531,14 +3571,15 @@ static void prim_concatMap(EvalState & state, const PosIdx pos, Value * * args,
len += lists[n].listSize();
}
state.mkList(v, len);
auto out = v.listElems();
auto list = state.buildList(len);
auto out = list.elems;
for (unsigned int n = 0, pos = 0; n < nrLists; ++n) {
auto l = lists[n].listSize();
if (l)
memcpy(out + pos, lists[n].listElems(), l * sizeof(Value *));
pos += l;
}
v.mkList(list);
}
static RegisterPrimOp primop_concatMap({
@@ -3820,7 +3861,7 @@ static RegisterPrimOp primop_stringLength({
.name = "__stringLength",
.args = {"e"},
.doc = R"(
Return the length of the string *e*. If *e* is not a string,
Return the number of bytes of the string *e*. If *e* is not a string,
evaluation is aborted.
)",
.fun = prim_stringLength,
@@ -3986,14 +4027,13 @@ void prim_match(EvalState & state, const PosIdx pos, Value * * args, Value & v)
}
// the first match is the whole string
const size_t len = match.size() - 1;
state.mkList(v, len);
for (size_t i = 0; i < len; ++i) {
if (!match[i+1].matched)
(v.listElems()[i] = state.allocValue())->mkNull();
auto list = state.buildList(match.size() - 1);
for (const auto & [i, v2] : enumerate(list))
if (!match[i + 1].matched)
v2 = &state.vNull;
else
(v.listElems()[i] = state.allocValue())->mkString(match[i + 1].str());
}
(v2 = state.allocValue())->mkString(match[i + 1].str());
v.mkList(list);
} catch (std::regex_error & e) {
if (e.code() == std::regex_constants::error_space) {
@@ -4062,11 +4102,12 @@ void prim_split(EvalState & state, const PosIdx pos, Value * * args, Value & v)
// Any matches results are surrounded by non-matching results.
const size_t len = std::distance(begin, end);
state.mkList(v, 2 * len + 1);
auto list = state.buildList(2 * len + 1);
size_t idx = 0;
if (len == 0) {
v.listElems()[idx++] = args[1];
list[0] = args[1];
v.mkList(list);
return;
}
@@ -4075,28 +4116,31 @@ void prim_split(EvalState & state, const PosIdx pos, Value * * args, Value & v)
auto match = *i;
// Add a string for non-matched characters.
(v.listElems()[idx++] = state.allocValue())->mkString(match.prefix().str());
(list[idx++] = state.allocValue())->mkString(match.prefix().str());
// Add a list for matched substrings.
const size_t slen = match.size() - 1;
auto elem = v.listElems()[idx++] = state.allocValue();
// Start at 1, beacause the first match is the whole string.
state.mkList(*elem, slen);
for (size_t si = 0; si < slen; ++si) {
auto list2 = state.buildList(slen);
for (const auto & [si, v2] : enumerate(list2)) {
if (!match[si + 1].matched)
(elem->listElems()[si] = state.allocValue())->mkNull();
v2 = &state.vNull;
else
(elem->listElems()[si] = state.allocValue())->mkString(match[si + 1].str());
(v2 = state.allocValue())->mkString(match[si + 1].str());
}
(list[idx++] = state.allocValue())->mkList(list2);
// Add a string for non-matched suffix characters.
if (idx == 2 * len)
(v.listElems()[idx++] = state.allocValue())->mkString(match.suffix().str());
(list[idx++] = state.allocValue())->mkString(match.suffix().str());
}
assert(idx == 2 * len + 1);
v.mkList(list);
} catch (std::regex_error & e) {
if (e.code() == std::regex_constants::error_space) {
// limit is _GLIBCXX_REGEX_STATE_LIMIT for libstdc++
@@ -4316,9 +4360,10 @@ static void prim_splitVersion(EvalState & state, const PosIdx pos, Value * * arg
break;
components.emplace_back(component);
}
state.mkList(v, components.size());
auto list = state.buildList(components.size());
for (const auto & [n, component] : enumerate(components))
(v.listElems()[n] = state.allocValue())->mkString(std::move(component));
(list[n] = state.allocValue())->mkString(std::move(component));
v.mkList(list);
}
static RegisterPrimOp primop_splitVersion({
@@ -4411,8 +4456,7 @@ void EvalState::createBaseEnv()
)",
});
v.mkNull();
addConstant("null", v, {
addConstant("null", &vNull, {
.type = nNull,
.doc = R"(
Primitive value.
@@ -4559,14 +4603,14 @@ void EvalState::createBaseEnv()
});
/* Add a value containing the current Nix expression search path. */
mkList(v, searchPath.elements.size());
int n = 0;
for (auto & i : searchPath.elements) {
auto list = buildList(searchPath.elements.size());
for (const auto & [n, i] : enumerate(searchPath.elements)) {
auto attrs = buildBindings(2);
attrs.alloc("path").mkString(i.path.s);
attrs.alloc("prefix").mkString(i.prefix.s);
(v.listElems()[n++] = allocValue())->mkAttrs(attrs);
(list[n] = allocValue())->mkAttrs(attrs);
}
v.mkList(list);
addConstant("__nixPath", v, {
.type = nList,
.doc = R"(
+4 -4
View File
@@ -137,7 +137,7 @@ static RegisterPrimOp primop_addDrvOutputDependencies({
.name = "__addDrvOutputDependencies",
.args = {"s"},
.doc = R"(
Create a copy of the given string where a single consant string context element is turned into a "derivation deep" string context element.
Create a copy of the given string where a single constant string context element is turned into a "derivation deep" string context element.
The store path that is the constant string context element should point to a valid derivation, and end in `.drv`.
@@ -207,10 +207,10 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value * * args,
if (info.second.allOutputs)
infoAttrs.alloc(sAllOutputs).mkBool(true);
if (!info.second.outputs.empty()) {
auto & outputsVal = infoAttrs.alloc(state.sOutputs);
state.mkList(outputsVal, info.second.outputs.size());
auto list = state.buildList(info.second.outputs.size());
for (const auto & [i, output] : enumerate(info.second.outputs))
(outputsVal.listElems()[i] = state.allocValue())->mkString(output);
(list[i] = state.allocValue())->mkString(output);
infoAttrs.alloc(state.sOutputs).mkList(list);
}
attrs.alloc(state.store->printStorePath(info.first)).mkAttrs(infoAttrs);
}
+4 -4
View File
@@ -38,10 +38,10 @@ static void prim_fromTOML(EvalState & state, const PosIdx pos, Value * * args, V
{
auto array = toml::get<std::vector<toml::value>>(t);
size_t size = array.size();
state.mkList(v, size);
for (size_t i = 0; i < size; ++i)
visit(*(v.listElems()[i] = state.allocValue()), array[i]);
auto list = state.buildList(array.size());
for (const auto & [n, v] : enumerate(list))
visit(*(v = state.allocValue()), array[n]);
v.mkList(list);
}
break;;
case toml::value_t::boolean:
+29 -1
View File
@@ -8,6 +8,29 @@
namespace nix {
/**
* How errors should be handled when printing values.
*/
enum class ErrorPrintBehavior {
/**
* Print the first line of the error in brackets: `«error: oh no!»`
*/
Print,
/**
* Throw the error to the code that attempted to print the value, instead
* of suppressing it it.
*/
Throw,
/**
* Only throw the error if encountered at the top level of the expression.
*
* This will cause expressions like `builtins.throw "uh oh!"` to throw
* errors, but will print attribute sets and other nested structures
* containing values that error (like `nixpkgs`) normally.
*/
ThrowTopLevel,
};
/**
* Options for printing Nix values.
*/
@@ -68,6 +91,11 @@ struct PrintOptions
*/
size_t prettyIndent = 0;
/**
* How to handle errors encountered while printing values.
*/
ErrorPrintBehavior errors = ErrorPrintBehavior::Print;
/**
* True if pretty-printing is enabled.
*/
@@ -86,7 +114,7 @@ static PrintOptions errorPrintOptions = PrintOptions {
.maxDepth = 10,
.maxAttrs = 10,
.maxListItems = 10,
.maxStringLength = 1024
.maxStringLength = 1024,
};
}
+60 -60
View File
@@ -271,25 +271,21 @@ private:
void printDerivation(Value & v)
{
try {
Bindings::iterator i = v.attrs->find(state.sDrvPath);
NixStringContext context;
std::string storePath;
if (i != v.attrs->end())
storePath = state.store->printStorePath(state.coerceToStorePath(i->pos, *i->value, context, "while evaluating the drvPath of a derivation"));
Bindings::iterator i = v.attrs->find(state.sDrvPath);
NixStringContext context;
std::string storePath;
if (i != v.attrs->end())
storePath = state.store->printStorePath(state.coerceToStorePath(i->pos, *i->value, context, "while evaluating the drvPath of a derivation"));
if (options.ansiColors)
output << ANSI_GREEN;
output << "«derivation";
if (!storePath.empty()) {
output << " " << storePath;
}
output << "»";
if (options.ansiColors)
output << ANSI_NORMAL;
} catch (Error & e) {
printError_(e);
if (options.ansiColors)
output << ANSI_GREEN;
output << "«derivation";
if (!storePath.empty()) {
output << " " << storePath;
}
output << "»";
if (options.ansiColors)
output << ANSI_NORMAL;
}
bool shouldPrettyPrintAttrs(AttrVec & v)
@@ -510,64 +506,68 @@ private:
output.flush();
checkInterrupt();
if (options.force) {
try {
try {
if (options.force) {
state.forceValue(v, v.determinePos(noPos));
} catch (Error & e) {
printError_(e);
return;
}
}
switch (v.type()) {
switch (v.type()) {
case nInt:
printInt(v);
break;
case nInt:
printInt(v);
break;
case nFloat:
printFloat(v);
break;
case nFloat:
printFloat(v);
break;
case nBool:
printBool(v);
break;
case nBool:
printBool(v);
break;
case nString:
printString(v);
break;
case nString:
printString(v);
break;
case nPath:
printPath(v);
break;
case nPath:
printPath(v);
break;
case nNull:
printNull();
break;
case nNull:
printNull();
break;
case nAttrs:
printAttrs(v, depth);
break;
case nAttrs:
printAttrs(v, depth);
break;
case nList:
printList(v, depth);
break;
case nList:
printList(v, depth);
break;
case nFunction:
printFunction(v);
break;
case nFunction:
printFunction(v);
break;
case nThunk:
printThunk(v);
break;
case nThunk:
printThunk(v);
break;
case nExternal:
printExternal(v);
break;
case nExternal:
printExternal(v);
break;
default:
printUnknown();
break;
default:
printUnknown();
break;
}
} catch (Error & e) {
if (options.errors == ErrorPrintBehavior::Throw
|| (options.errors == ErrorPrintBehavior::ThrowTopLevel
&& depth == 0)) {
throw;
}
printError_(e);
}
}
+40 -7
View File
@@ -18,6 +18,7 @@
namespace nix {
struct Value;
class BindingsBuilder;
@@ -134,6 +135,34 @@ class ExternalValueBase
std::ostream & operator << (std::ostream & str, const ExternalValueBase & v);
class ListBuilder
{
const size_t size;
Value * inlineElems[2] = {nullptr, nullptr};
public:
Value * * elems;
ListBuilder(EvalState & state, size_t size);
ListBuilder(ListBuilder && x)
: size(x.size)
, inlineElems{x.inlineElems[0], x.inlineElems[1]}
, elems(size <= 2 ? inlineElems : x.elems)
{ }
Value * & operator [](size_t n)
{
return elems[n];
}
typedef Value * * iterator;
iterator begin() { return &elems[0]; }
iterator end() { return &elems[size]; }
friend struct Value;
};
struct Value
{
private:
@@ -217,7 +246,7 @@ public:
Bindings * attrs;
struct {
size_t size;
Value * * elems;
Value * const * elems;
} bigList;
Value * smallList[2];
ClosureThunk thunk;
@@ -323,16 +352,20 @@ public:
Value & mkAttrs(BindingsBuilder & bindings);
inline void mkList(size_t size)
void mkList(const ListBuilder & builder)
{
clearValue();
if (size == 1)
if (builder.size == 1) {
smallList[0] = builder.inlineElems[0];
internalType = tList1;
else if (size == 2)
} else if (builder.size == 2) {
smallList[0] = builder.inlineElems[0];
smallList[1] = builder.inlineElems[1];
internalType = tList2;
else {
} else {
bigList.size = builder.size;
bigList.elems = builder.elems;
internalType = tListN;
bigList.size = size;
}
}
@@ -392,7 +425,7 @@ public:
return internalType == tList1 || internalType == tList2 || internalType == tListN;
}
Value * * listElems()
Value * const * listElems()
{
return internalType == tList1 || internalType == tList2 ? smallList : bigList.elems;
}
+1 -16
View File
@@ -78,6 +78,7 @@ struct FetchSettings : public Config
)",
{}, true, Xp::Flakes};
Setting<bool> useRegistries{this, true, "use-registries",
"Whether to use flake registries to resolve flake references.",
{}, true, Xp::Flakes};
@@ -93,22 +94,6 @@ struct FetchSettings : public Config
empty, the summary is generated based on the action performed.
)",
{}, true, Xp::Flakes};
Setting<bool> trustTarballsFromGitForges{
this, true, "trust-tarballs-from-git-forges",
R"(
If enabled (the default), Nix will consider tarballs from
GitHub and similar Git forges to be locked if a Git revision
is specified,
e.g. `github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f`.
This requires Nix to trust that the provider will return the
correct contents for the specified Git revision.
If disabled, such tarballs are only considered locked if a
`narHash` attribute is specified,
e.g. `github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f?narHash=sha256-PPXqKY2hJng4DBVE0I4xshv/vGLUskL7jl53roB8UdU%3D`.
)"};
};
// FIXME: don't use a global variable.
+11 -27
View File
@@ -21,6 +21,7 @@
#include <git2/refs.h>
#include <git2/remote.h>
#include <git2/repository.h>
#include <git2/revparse.h>
#include <git2/status.h>
#include <git2/submodule.h>
#include <git2/tree.h>
@@ -197,35 +198,12 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
return git_repository_is_shallow(*this);
}
void setRemote(const std::string & name, const std::string & url) override
{
if (git_remote_set_url(*this, name.c_str(), url.c_str()))
throw Error("setting remote '%s' URL to '%s': %s", name, url, git_error_last()->message);
}
Hash resolveRef(std::string ref) override
{
// Handle revisions used as refs.
{
git_oid oid;
if (git_oid_fromstr(&oid, ref.c_str()) == 0)
return toHash(oid);
}
// Resolve short names like 'master'.
Reference ref2;
if (!git_reference_dwim(Setter(ref2), *this, ref.c_str()))
ref = git_reference_name(ref2.get());
// Resolve full references like 'refs/heads/master'.
Reference ref3;
if (git_reference_lookup(Setter(ref3), *this, ref.c_str()))
Object object;
if (git_revparse_single(Setter(object), *this, ref.c_str()))
throw Error("resolving Git reference '%s': %s", ref, git_error_last()->message);
auto oid = git_reference_target(ref3.get());
if (!oid)
throw Error("cannot get OID for Git reference '%s'", git_reference_name(ref3.get()));
auto oid = git_object_id(object.get());
return toHash(*oid);
}
@@ -324,7 +302,9 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
std::vector<std::tuple<Submodule, Hash>> getSubmodules(const Hash & rev, bool exportIgnore) override;
std::string resolveSubmoduleUrl(const std::string & url) override
std::string resolveSubmoduleUrl(
const std::string & url,
const std::string & base) override
{
git_buf buf = GIT_BUF_INIT;
if (git_submodule_resolve_url(&buf, *this, url.c_str()))
@@ -332,6 +312,10 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Finally cleanup = [&]() { git_buf_dispose(&buf); };
std::string res(buf.ptr);
if (!hasPrefix(res, "/") && res.find("://") == res.npos)
res = parseURL(base + "/" + res).canonicalise().to_string();
return res;
}
+3 -3
View File
@@ -32,8 +32,6 @@ struct GitRepo
/* Return the commit hash to which a ref points. */
virtual Hash resolveRef(std::string ref) = 0;
virtual void setRemote(const std::string & name, const std::string & url) = 0;
/**
* Info about a submodule.
*/
@@ -71,7 +69,9 @@ struct GitRepo
*/
virtual std::vector<std::tuple<Submodule, Hash>> getSubmodules(const Hash & rev, bool exportIgnore) = 0;
virtual std::string resolveSubmoduleUrl(const std::string & url) = 0;
virtual std::string resolveSubmoduleUrl(
const std::string & url,
const std::string & base) = 0;
virtual bool hasObject(const Hash & oid) = 0;
+2 -10
View File
@@ -523,9 +523,6 @@ struct GitInputScheme : InputScheme
auto repo = GitRepo::openRepo(cacheDir, true, true);
// We need to set the origin so resolving submodule URLs works
repo->setRemote("origin", repoInfo.url);
Path localRefFile =
ref.compare(0, 5, "refs/") == 0
? cacheDir + "/" + ref
@@ -588,7 +585,7 @@ struct GitInputScheme : InputScheme
repoInfo.url
);
} else
input.attrs.insert_or_assign("rev", Hash::parseAny(chomp(readFile(localRefFile)), HashAlgorithm::SHA1).gitRev());
input.attrs.insert_or_assign("rev", repo->resolveRef(ref).gitRev());
// cache dir lock is removed at scope end; we will only use read-only operations on specific revisions in the remainder
}
@@ -629,7 +626,7 @@ struct GitInputScheme : InputScheme
std::map<CanonPath, nix::ref<InputAccessor>> mounts;
for (auto & [submodule, submoduleRev] : repo->getSubmodules(rev, exportIgnore)) {
auto resolved = repo->resolveSubmoduleUrl(submodule.url);
auto resolved = repo->resolveSubmoduleUrl(submodule.url, repoInfo.url);
debug("Git submodule %s: %s %s %s -> %s",
submodule.path, submodule.url, submodule.branch, submoduleRev.gitRev(), resolved);
fetchers::Attrs attrs;
@@ -639,8 +636,6 @@ struct GitInputScheme : InputScheme
attrs.insert_or_assign("ref", submodule.branch);
attrs.insert_or_assign("rev", submoduleRev.gitRev());
attrs.insert_or_assign("exportIgnore", Explicit<bool>{ exportIgnore });
attrs.insert_or_assign("submodules", Explicit<bool>{ true });
attrs.insert_or_assign("allRefs", Explicit<bool>{ true });
auto submoduleInput = fetchers::Input::fromAttrs(std::move(attrs));
auto [submoduleAccessor, submoduleInput2] =
submoduleInput.getAccessor(store);
@@ -692,9 +687,6 @@ struct GitInputScheme : InputScheme
attrs.insert_or_assign("type", "git");
attrs.insert_or_assign("url", submodulePath.abs());
attrs.insert_or_assign("exportIgnore", Explicit<bool>{ exportIgnore });
attrs.insert_or_assign("submodules", Explicit<bool>{ true });
// TODO: fall back to getAccessorFromCommit-like fetch when submodules aren't checked out
// attrs.insert_or_assign("allRefs", Explicit<bool>{ true });
auto submoduleInput = fetchers::Input::fromAttrs(std::move(attrs));
auto [submoduleAccessor, submoduleInput2] =
+1 -8
View File
@@ -248,15 +248,10 @@ struct GitArchiveInputScheme : InputScheme
getFileTransfer()->download(std::move(req), sink);
});
auto act = std::make_unique<Activity>(*logger, lvlInfo, actUnknown,
fmt("unpacking '%s' into the Git cache", input.to_string()));
TarArchive archive { *source };
auto parseSink = getTarballCache()->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
act.reset();
TarballInfo tarballInfo {
.treeHash = parseSink->sync(),
.lastModified = lastModified
@@ -299,9 +294,7 @@ struct GitArchiveInputScheme : InputScheme
Git revision alone, we also require a NAR hash for
locking. FIXME: in the future, we may want to require a Git
tree hash instead of a NAR hash. */
return input.getRev().has_value()
&& (fetchSettings.trustTarballsFromGitForges ||
input.getNarHash().has_value());
return input.getRev().has_value() && input.getNarHash().has_value();
}
std::optional<ExperimentalFeature> experimentalFeature() const override
+5 -1
View File
@@ -6,7 +6,11 @@ libfetchers_DIR := $(d)
libfetchers_SOURCES := $(wildcard $(d)/*.cc)
libfetchers_CXXFLAGS += -I src/libutil -I src/libstore
# Not just for this library itself, but also for downstream libraries using this library
INCLUDE_libfetchers := -I $(d)
libfetchers_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libfetchers)
libfetchers_LDFLAGS += $(THREAD_LDFLAGS) $(LIBGIT2_LIBS) -larchive
+1 -24
View File
@@ -156,35 +156,12 @@ DownloadTarballResult downloadTarball(
// TODO: fall back to cached value if download fails.
auto act = std::make_unique<Activity>(*logger, lvlInfo, actUnknown,
fmt("unpacking '%s' into the Git cache", url));
AutoDelete cleanupTemp;
/* Note: if the download is cached, `importTarball()` will receive
no data, which causes it to import an empty tarball. */
auto archive =
hasSuffix(toLower(parseURL(url).path), ".zip")
? ({
/* In streaming mode, libarchive doesn't handle
symlinks in zip files correctly (#10649). So write
the entire file to disk so libarchive can access it
in random-access mode. */
auto [fdTemp, path] = createTempFile("nix-zipfile");
cleanupTemp.reset(path);
debug("downloading '%s' into '%s'...", url, path);
{
FdSink sink(fdTemp.get());
source->drainInto(sink);
}
TarArchive{path};
})
: TarArchive{*source};
TarArchive archive { *source };
auto parseSink = getTarballCache()->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
act.reset();
auto res(_res->lock());
Attrs infoAttrs;
+1 -1
View File
@@ -6,7 +6,7 @@ libmain_DIR := $(d)
libmain_SOURCES := $(wildcard $(d)/*.cc)
libmain_CXXFLAGS += -I src/libutil -I src/libstore
libmain_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore)
libmain_LDFLAGS += $(OPENSSL_LIBS)
+12 -10
View File
@@ -123,14 +123,18 @@ public:
}
void pause() override {
state_.lock()->paused = true;
writeToStderr("\r\e[K");
auto state (state_.lock());
state->paused = true;
if (state->active)
writeToStderr("\r\e[K");
}
void resume() override {
state_.lock()->paused = false;
writeToStderr("\r\e[K");
state_.lock()->haveUpdate = true;
auto state (state_.lock());
state->paused = false;
if (state->active)
writeToStderr("\r\e[K");
state->haveUpdate = true;
updateCV.notify_one();
}
@@ -162,9 +166,7 @@ public:
writeToStderr("\r\e[K" + filterANSIEscapes(s, !isTTY) + ANSI_NORMAL "\n");
draw(state);
} else {
auto s2 = s + ANSI_NORMAL "\n";
if (!isTTY) s2 = filterANSIEscapes(s2, true);
writeToStderr(s2);
writeToStderr(filterANSIEscapes(s, !isTTY) + "\n");
}
}
@@ -519,7 +521,7 @@ public:
std::optional<char> ask(std::string_view msg) override
{
auto state(state_.lock());
if (!state->active || !isatty(STDIN_FILENO)) return {};
if (!state->active) return {};
std::cerr << fmt("\r\e[K%s ", msg);
auto s = trim(readLine(STDIN_FILENO));
if (s.size() != 1) return {};
@@ -535,7 +537,7 @@ public:
Logger * makeProgressBar()
{
return new ProgressBar(shouldANSI());
return new ProgressBar(isTTY());
}
void startProgressBar()
+142 -169
View File
@@ -26,7 +26,6 @@
#include <regex>
#include <queue>
#include <sys/stat.h>
#include <sys/un.h>
#include <fcntl.h>
#include <termios.h>
@@ -58,10 +57,6 @@
#if __APPLE__
#include <spawn.h>
#include <sys/sysctl.h>
#include <sandbox.h>
/* This definition is undocumented but depended upon by all major browsers. */
extern "C" int sandbox_init_with_parameters(const char *profile, uint64_t flags, const char *const parameters[], char **errorbuf);
#endif
#include <pwd.h>
@@ -401,30 +396,20 @@ void LocalDerivationGoal::cleanupPostOutputsRegisteredModeNonCheck()
static void doBind(const Path & source, const Path & target, bool optional = false) {
debug("bind mounting '%1%' to '%2%'", source, target);
struct stat st;
auto bindMount = [&]() {
if (mount(source.c_str(), target.c_str(), "", MS_BIND | MS_REC, 0) == -1)
throw SysError("bind mount from '%1%' to '%2%' failed", source, target);
};
if (lstat(source.c_str(), &st) == -1) {
if (stat(source.c_str(), &st) == -1) {
if (optional && errno == ENOENT)
return;
else
throw SysError("getting attributes of path '%1%'", source);
}
if (S_ISDIR(st.st_mode)) {
if (S_ISDIR(st.st_mode))
createDirs(target);
bindMount();
} else if (S_ISLNK(st.st_mode)) {
// Symlinks can (apparently) not be bind-mounted, so just copy it
createDirs(dirOf(target));
copyFile(source, target, /* andDelete */ false);
} else {
else {
createDirs(dirOf(target));
writeFile(target, "");
bindMount();
}
if (mount(source.c_str(), target.c_str(), "", MS_BIND | MS_REC, 0) == -1)
throw SysError("bind mount from '%1%' to '%2%' failed", source, target);
};
#endif
@@ -503,24 +488,8 @@ void LocalDerivationGoal::startBuilder()
/* Create a temporary directory where the build will take
place. */
topTmpDir = createTempDir("", "nix-build-" + std::string(drvPath.name()), false, false, 0700);
#if __APPLE__
if (false) {
#else
if (useChroot) {
#endif
/* If sandboxing is enabled, put the actual TMPDIR underneath
an inaccessible root-owned directory, to prevent outside
access.
tmpDir = createTempDir(settings.buildDir.get().value_or(""), "nix-build-" + std::string(drvPath.name()), false, false, 0700);
On macOS, we don't use an actual chroot, so this isn't
possible. Any mitigation along these lines would have to be
done directly in the sandbox profile. */
tmpDir = topTmpDir + "/build";
createDir(tmpDir, 0700);
} else {
tmpDir = topTmpDir;
}
chownToBuilder(tmpDir);
for (auto & [outputName, status] : initialOutputs) {
@@ -688,19 +657,15 @@ void LocalDerivationGoal::startBuilder()
environment using bind-mounts. We put it in the Nix store
so that the build outputs can be moved efficiently from the
chroot to their final location. */
chrootParentDir = worker.store.Store::toRealPath(drvPath) + ".chroot";
deletePath(chrootParentDir);
chrootRootDir = worker.store.Store::toRealPath(drvPath) + ".chroot";
deletePath(chrootRootDir);
/* Clean up the chroot directory automatically. */
autoDelChroot = std::make_shared<AutoDelete>(chrootParentDir);
autoDelChroot = std::make_shared<AutoDelete>(chrootRootDir);
printMsg(lvlChatty, "setting up chroot environment in '%1%'", chrootParentDir);
if (mkdir(chrootParentDir.c_str(), 0700) == -1)
throw SysError("cannot create '%s'", chrootRootDir);
chrootRootDir = chrootParentDir + "/root";
printMsg(lvlChatty, "setting up chroot environment in '%1%'", chrootRootDir);
// FIXME: make this 0700
if (mkdir(chrootRootDir.c_str(), buildUser && buildUser->getUIDCount() != 1 ? 0755 : 0750) == -1)
throw SysError("cannot create '%1%'", chrootRootDir);
@@ -1738,20 +1703,13 @@ void LocalDerivationGoal::runChild()
bool setUser = true;
/* Make the contents of netrc and the CA certificate bundle
available to builtin:fetchurl (which may run under a
different uid and/or in a sandbox). */
/* Make the contents of netrc available to builtin:fetchurl
(which may run under a different uid and/or in a sandbox). */
std::string netrcData;
std::string caFileData;
if (drv->isBuiltin() && drv->builder == "builtin:fetchurl") {
try {
netrcData = readFile(settings.netrcFile);
} catch (SystemError &) { }
try {
caFileData = readFile(settings.caFile);
} catch (SystemError &) { }
}
try {
if (drv->isBuiltin() && drv->builder == "builtin:fetchurl")
netrcData = readFile(settings.netrcFile);
} catch (SystemError &) { }
#if __linux__
if (useChroot) {
@@ -2017,129 +1975,154 @@ void LocalDerivationGoal::runChild()
throw SysError("setuid failed");
}
/* Fill in the arguments. */
Strings args;
std::string builder = "invalid";
if (drv->isBuiltin()) {
;
}
#if __APPLE__
/* This has to appear before import statements. */
std::string sandboxProfile = "(version 1)\n";
else {
/* This has to appear before import statements. */
std::string sandboxProfile = "(version 1)\n";
if (useChroot) {
if (useChroot) {
/* Lots and lots and lots of file functions freak out if they can't stat their full ancestry */
PathSet ancestry;
/* Lots and lots and lots of file functions freak out if they can't stat their full ancestry */
PathSet ancestry;
/* We build the ancestry before adding all inputPaths to the store because we know they'll
all have the same parents (the store), and there might be lots of inputs. This isn't
particularly efficient... I doubt it'll be a bottleneck in practice */
for (auto & i : pathsInChroot) {
Path cur = i.first;
while (cur.compare("/") != 0) {
cur = dirOf(cur);
ancestry.insert(cur);
/* We build the ancestry before adding all inputPaths to the store because we know they'll
all have the same parents (the store), and there might be lots of inputs. This isn't
particularly efficient... I doubt it'll be a bottleneck in practice */
for (auto & i : pathsInChroot) {
Path cur = i.first;
while (cur.compare("/") != 0) {
cur = dirOf(cur);
ancestry.insert(cur);
}
}
}
/* And we want the store in there regardless of how empty pathsInChroot. We include the innermost
path component this time, since it's typically /nix/store and we care about that. */
Path cur = worker.store.storeDir;
while (cur.compare("/") != 0) {
ancestry.insert(cur);
cur = dirOf(cur);
}
/* And we want the store in there regardless of how empty pathsInChroot. We include the innermost
path component this time, since it's typically /nix/store and we care about that. */
Path cur = worker.store.storeDir;
while (cur.compare("/") != 0) {
ancestry.insert(cur);
cur = dirOf(cur);
}
/* Add all our input paths to the chroot */
for (auto & i : inputPaths) {
auto p = worker.store.printStorePath(i);
pathsInChroot[p] = p;
}
/* Add all our input paths to the chroot */
for (auto & i : inputPaths) {
auto p = worker.store.printStorePath(i);
pathsInChroot[p] = p;
}
/* Violations will go to the syslog if you set this. Unfortunately the destination does not appear to be configurable */
if (settings.darwinLogSandboxViolations) {
sandboxProfile += "(deny default)\n";
} else {
sandboxProfile += "(deny default (with no-log))\n";
}
/* Violations will go to the syslog if you set this. Unfortunately the destination does not appear to be configurable */
if (settings.darwinLogSandboxViolations) {
sandboxProfile += "(deny default)\n";
} else {
sandboxProfile += "(deny default (with no-log))\n";
}
sandboxProfile +=
#include "sandbox-defaults.sb"
;
if (!derivationType->isSandboxed())
sandboxProfile +=
#include "sandbox-network.sb"
#include "sandbox-defaults.sb"
;
/* Add the output paths we'll use at build-time to the chroot */
sandboxProfile += "(allow file-read* file-write* process-exec\n";
for (auto & [_, path] : scratchOutputs)
sandboxProfile += fmt("\t(subpath \"%s\")\n", worker.store.printStorePath(path));
if (!derivationType->isSandboxed())
sandboxProfile +=
#include "sandbox-network.sb"
;
sandboxProfile += ")\n";
/* Add the output paths we'll use at build-time to the chroot */
sandboxProfile += "(allow file-read* file-write* process-exec\n";
for (auto & [_, path] : scratchOutputs)
sandboxProfile += fmt("\t(subpath \"%s\")\n", worker.store.printStorePath(path));
/* Our inputs (transitive dependencies and any impurities computed above)
sandboxProfile += ")\n";
without file-write* allowed, access() incorrectly returns EPERM
*/
sandboxProfile += "(allow file-read* file-write* process-exec\n";
for (auto & i : pathsInChroot) {
if (i.first != i.second.source)
throw Error(
"can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin",
i.first, i.second.source);
/* Our inputs (transitive dependencies and any impurities computed above)
std::string path = i.first;
struct stat st;
if (lstat(path.c_str(), &st)) {
if (i.second.optional && errno == ENOENT)
continue;
throw SysError("getting attributes of path '%s", path);
without file-write* allowed, access() incorrectly returns EPERM
*/
sandboxProfile += "(allow file-read* file-write* process-exec\n";
for (auto & i : pathsInChroot) {
if (i.first != i.second.source)
throw Error(
"can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin",
i.first, i.second.source);
std::string path = i.first;
struct stat st;
if (lstat(path.c_str(), &st)) {
if (i.second.optional && errno == ENOENT)
continue;
throw SysError("getting attributes of path '%s", path);
}
if (S_ISDIR(st.st_mode))
sandboxProfile += fmt("\t(subpath \"%s\")\n", path);
else
sandboxProfile += fmt("\t(literal \"%s\")\n", path);
}
if (S_ISDIR(st.st_mode))
sandboxProfile += fmt("\t(subpath \"%s\")\n", path);
else
sandboxProfile += fmt("\t(literal \"%s\")\n", path);
}
sandboxProfile += ")\n";
sandboxProfile += ")\n";
/* Allow file-read* on full directory hierarchy to self. Allows realpath() */
sandboxProfile += "(allow file-read*\n";
for (auto & i : ancestry) {
sandboxProfile += fmt("\t(literal \"%s\")\n", i);
}
sandboxProfile += ")\n";
/* Allow file-read* on full directory hierarchy to self. Allows realpath() */
sandboxProfile += "(allow file-read*\n";
for (auto & i : ancestry) {
sandboxProfile += fmt("\t(literal \"%s\")\n", i);
}
sandboxProfile += ")\n";
sandboxProfile += additionalSandboxProfile;
} else
sandboxProfile +=
#include "sandbox-minimal.sb"
;
sandboxProfile += additionalSandboxProfile;
} else
sandboxProfile +=
#include "sandbox-minimal.sb"
;
debug("Generated sandbox profile:");
debug(sandboxProfile);
debug("Generated sandbox profile:");
debug(sandboxProfile);
bool allowLocalNetworking = parsedDrv->getBoolAttr("__darwinAllowLocalNetworking");
Path sandboxFile = tmpDir + "/.sandbox.sb";
/* The tmpDir in scope points at the temporary build directory for our derivation. Some packages try different mechanisms
to find temporary directories, so we want to open up a broader place for them to dump their files, if needed. */
Path globalTmpDir = canonPath(getEnvNonEmpty("TMPDIR").value_or("/tmp"), true);
writeFile(sandboxFile, sandboxProfile);
/* They don't like trailing slashes on subpath directives */
if (globalTmpDir.back() == '/') globalTmpDir.pop_back();
bool allowLocalNetworking = parsedDrv->getBoolAttr("__darwinAllowLocalNetworking");
if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
Strings sandboxArgs;
sandboxArgs.push_back("_GLOBAL_TMP_DIR");
sandboxArgs.push_back(globalTmpDir);
if (allowLocalNetworking) {
sandboxArgs.push_back("_ALLOW_LOCAL_NETWORKING");
sandboxArgs.push_back("1");
}
char * sandbox_errbuf = nullptr;
if (sandbox_init_with_parameters(sandboxProfile.c_str(), 0, stringsToCharPtrs(sandboxArgs).data(), &sandbox_errbuf)) {
writeFull(STDERR_FILENO, fmt("failed to configure sandbox: %s\n", sandbox_errbuf ? sandbox_errbuf : "(null)"));
_exit(1);
/* The tmpDir in scope points at the temporary build directory for our derivation. Some packages try different mechanisms
to find temporary directories, so we want to open up a broader place for them to put their files, if needed. */
Path globalTmpDir = canonPath(defaultTempDir(), true);
/* They don't like trailing slashes on subpath directives */
while (!globalTmpDir.empty() && globalTmpDir.back() == '/')
globalTmpDir.pop_back();
if (getEnv("_NIX_TEST_NO_SANDBOX") != "1") {
builder = "/usr/bin/sandbox-exec";
args.push_back("sandbox-exec");
args.push_back("-f");
args.push_back(sandboxFile);
args.push_back("-D");
args.push_back("_GLOBAL_TMP_DIR=" + globalTmpDir);
if (allowLocalNetworking) {
args.push_back("-D");
args.push_back(std::string("_ALLOW_LOCAL_NETWORKING=1"));
}
args.push_back(drv->builder);
} else {
builder = drv->builder;
args.push_back(std::string(baseNameOf(drv->builder)));
}
}
#else
else {
builder = drv->builder;
args.push_back(std::string(baseNameOf(drv->builder)));
}
#endif
for (auto & i : drv->args)
args.push_back(rewriteStrings(i, inputRewrites));
/* Indicate that we managed to set up the build environment. */
writeFull(STDERR_FILENO, std::string("\2\n"));
@@ -2156,7 +2139,7 @@ void LocalDerivationGoal::runChild()
worker.store.printStorePath(scratchOutputs.at(e.first)));
if (drv->builder == "builtin:fetchurl")
builtinFetchurl(*drv, outputs, netrcData, caFileData);
builtinFetchurl(*drv, outputs, netrcData);
else if (drv->builder == "builtin:buildenv")
builtinBuildenv(*drv, outputs);
else if (drv->builder == "builtin:unpack-channel")
@@ -2170,14 +2153,6 @@ void LocalDerivationGoal::runChild()
}
}
// Now builder is not builtin
Strings args;
args.push_back(std::string(baseNameOf(drv->builder)));
for (auto & i : drv->args)
args.push_back(rewriteStrings(i, inputRewrites));
#if __APPLE__
posix_spawnattr_t attrp;
@@ -2199,9 +2174,9 @@ void LocalDerivationGoal::runChild()
posix_spawnattr_setbinpref_np(&attrp, 1, &cpu, NULL);
}
posix_spawn(NULL, drv->builder.c_str(), NULL, &attrp, stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
posix_spawn(NULL, builder.c_str(), NULL, &attrp, stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
#else
execve(drv->builder.c_str(), stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
execve(builder.c_str(), stringsToCharPtrs(args).data(), stringsToCharPtrs(envStrs).data());
#endif
throw SysError("executing '%1%'", drv->builder);
@@ -2952,17 +2927,15 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
void LocalDerivationGoal::deleteTmpDir(bool force)
{
if (topTmpDir != "") {
if (tmpDir != "") {
/* Don't keep temporary directories for builtins because they
might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv->isBuiltin()) {
printError("note: keeping build directory '%s'", tmpDir);
chmod(topTmpDir.c_str(), 0755);
chmod(tmpDir.c_str(), 0755);
}
else
deletePath(topTmpDir);
topTmpDir = "";
deletePath(tmpDir);
tmpDir = "";
}
}
+1 -17
View File
@@ -27,16 +27,10 @@ struct LocalDerivationGoal : public DerivationGoal
std::optional<Path> cgroup;
/**
* The temporary directory used for the build.
* The temporary directory.
*/
Path tmpDir;
/**
* The top-level temporary directory. `tmpDir` is either equal to
* or a child of this directory.
*/
Path topTmpDir;
/**
* The path of the temporary directory in the sandbox.
*/
@@ -71,16 +65,6 @@ struct LocalDerivationGoal : public DerivationGoal
*/
bool useChroot = false;
/**
* The parent directory of `chrootRootDir`. It has permission 700
* and is owned by root to ensure other users cannot mess with
* `chrootRootDir`.
*/
Path chrootParentDir;
/**
* The root of the chroot environment.
*/
Path chrootRootDir;
/**
-1
View File
@@ -46,7 +46,6 @@ R""(
(if (param "_ALLOW_LOCAL_NETWORKING")
(begin
(allow network* (remote ip "localhost:*"))
(allow network-inbound (local ip "*:*")) ; required to bind and listen
; Allow access to /etc/resolv.conf (which is a symlink to
; /private/var/run/resolv.conf).
+1 -2
View File
@@ -9,8 +9,7 @@ namespace nix {
void builtinFetchurl(
const BasicDerivation & drv,
const std::map<std::string, Path> & outputs,
const std::string & netrcData,
const std::string & caFileData);
const std::string & netrcData);
void builtinUnpackChannel(
const BasicDerivation & drv,
+4 -5
View File
@@ -9,8 +9,7 @@ namespace nix {
void builtinFetchurl(
const BasicDerivation & drv,
const std::map<std::string, Path> & outputs,
const std::string & netrcData,
const std::string & caFileData)
const std::string & netrcData)
{
/* Make the host's netrc data available. Too bad curl requires
this to be stored in a file. It would be nice if we could just
@@ -20,9 +19,6 @@ void builtinFetchurl(
writeFile(settings.netrcFile, netrcData, 0600);
}
settings.caFile = "ca-certificates.crt";
writeFile(settings.caFile, caFileData, 0600);
auto out = get(drv.outputs, "out");
if (!out)
throw Error("'builtin:fetchurl' requires an 'out' output");
@@ -42,7 +38,10 @@ void builtinFetchurl(
auto source = sinkToSource([&](Sink & sink) {
/* No need to do TLS verification, because we check the hash of
the result anyway. */
FileTransferRequest request(url);
request.verifyTLS = false;
request.decompress = false;
auto decompressor = makeDecompressionSink(
+1 -1
View File
@@ -254,7 +254,7 @@ struct ClientSettings
else if (setSubstituters(settings.substituters))
;
else
debug("ignoring the client-specified setting '%s', because it is a restricted setting and you are not a trusted user", name);
warn("ignoring the client-specified setting '%s', because it is a restricted setting and you are not a trusted user", name);
} catch (UsageError & e) {
warn(e.what());
}
+3 -12
View File
@@ -67,10 +67,7 @@ struct curlFileTransfer : public FileTransfer
curl_off_t writtenToSink = 0;
std::chrono::steady_clock::time_point startTime = std::chrono::steady_clock::now();
inline static const std::set<long> successfulStatuses {200, 201, 204, 206, 304, 0 /* other protocol */};
/* Get the HTTP status code, or 0 for other protocols. */
long getHTTPStatus()
{
@@ -372,14 +369,10 @@ struct curlFileTransfer : public FileTransfer
void finish(CURLcode code)
{
auto finishTime = std::chrono::steady_clock::now();
auto httpStatus = getHTTPStatus();
debug("finished %s of '%s'; curl status = %d, HTTP status = %d, body = %d bytes, duration = %.2f s",
request.verb(), request.uri, code, httpStatus, result.bodySize,
std::chrono::duration_cast<std::chrono::milliseconds>(finishTime - startTime).count() / 1000.0f
);
debug("finished %s of '%s'; curl status = %d, HTTP status = %d, body = %d bytes",
request.verb(), request.uri, code, httpStatus, result.bodySize);
appendCurrentUrl();
@@ -835,10 +828,8 @@ void FileTransfer::download(
buffer). We don't wait forever to prevent stalling the
download thread. (Hopefully sleeping will throttle the
sender.) */
if (state->data.size() > fileTransferSettings.downloadBufferSize) {
if (state->data.size() > 1024 * 1024) {
debug("download buffer is full; going to sleep");
static bool haveWarned = false;
warnOnce(haveWarned, "download buffer is full; consider increasing the 'download-buffer-size' setting");
state.wait_for(state->request, std::chrono::seconds(10));
}
-6
View File
@@ -45,12 +45,6 @@ struct FileTransferSettings : Config
Setting<unsigned int> tries{this, 5, "download-attempts",
"How often Nix will attempt to download a file before giving up."};
Setting<size_t> downloadBufferSize{this, 64 * 1024 * 1024, "download-buffer-size",
R"(
The size of Nix's internal download buffer during `curl` transfers. If data is
not processed quickly enough to exceed the size of this buffer, downloads may stall.
)"};
};
extern FileTransferSettings fileTransferSettings;
+1 -1
View File
@@ -415,7 +415,7 @@ void initLibStore() {
sshd). This breaks build users because they don't have access
to the TMPDIR, in particular in ‘nix-store --serve’. */
#if __APPLE__
if (hasPrefix(getEnv("TMPDIR").value_or("/tmp"), "/var/folders/"))
if (hasPrefix(defaultTempDir(), "/var/folders/"))
unsetenv("TMPDIR");
#endif
+25 -5
View File
@@ -687,16 +687,36 @@ public:
Setting<std::string> sandboxShmSize{
this, "50%", "sandbox-dev-shm-size",
R"(
This option determines the maximum size of the `tmpfs` filesystem
mounted on `/dev/shm` in Linux sandboxes. For the format, see the
description of the `size` option of `tmpfs` in mount(8). The default
is `50%`.
*Linux only*
This option determines the maximum size of the `tmpfs` filesystem
mounted on `/dev/shm` in Linux sandboxes. For the format, see the
description of the `size` option of `tmpfs` in mount(8). The default
is `50%`.
)"};
Setting<Path> sandboxBuildDir{this, "/build", "sandbox-build-dir",
"The build directory inside the sandbox."};
R"(
*Linux only*
The build directory inside the sandbox.
This directory is backed by [`build-dir`](#conf-build-dir) on the host.
)"};
#endif
Setting<std::optional<Path>> buildDir{this, std::nullopt, "build-dir",
R"(
The directory on the host, in which derivations' temporary build directories are created.
If not set, Nix will use the system temporary directory indicated by the `TMPDIR` environment variable.
Note that builds are often performed by the Nix daemon, so its `TMPDIR` is used, and not that of the Nix command line interface.
This is also the location where [`--keep-failed`](@docroot@/command-ref/opt-common.md#opt-keep-failed) leaves its files.
If Nix runs without sandbox, or if the platform does not support sandboxing with bind mounts (e.g. macOS), then the [`builder`](@docroot@/language/derivations.md#attr-builder)'s environment will contain this directory, instead of the virtual location [`sandbox-build-dir`](#conf-sandbox-build-dir).
)"};
Setting<PathSet> allowedImpureHostPrefixes{this, {}, "allowed-impure-host-deps",
"Which prefixes to allow derivations to ask for access to (primarily for Darwin)."};
+1 -1
View File
@@ -49,7 +49,7 @@ public:
, BinaryCacheStore(params)
, cacheUri(scheme + "://" + _cacheUri)
{
if (cacheUri.back() == '/')
while (!cacheUri.empty() && cacheUri.back() == '/')
cacheUri.pop_back();
diskCache = getNarInfoDiskCache();
-4
View File
@@ -33,10 +33,6 @@ struct LocalStoreAccessor : PosixSourceAccessor
std::optional<Stat> maybeLstat(const CanonPath & path) override
{
/* Handle the case where `path` is (a parent of) the store. */
if (isDirOrInDir(store->storeDir, path.abs()))
return Stat{ .type = tDirectory };
return PosixSourceAccessor::maybeLstat(toRealPath(path));
}
+6 -2
View File
@@ -10,7 +10,7 @@ libstore_LIBS = libutil
libstore_LDFLAGS += $(SQLITE3_LIBS) $(LIBCURL_LIBS) $(THREAD_LDFLAGS)
ifdef HOST_LINUX
libstore_LDFLAGS += -ldl
libstore_LDFLAGS += -ldl
endif
$(foreach file,$(libstore_FILES),$(eval $(call install-data-in,$(d)/$(file),$(datadir)/nix/sandbox)))
@@ -27,8 +27,12 @@ ifeq ($(HAVE_SECCOMP), 1)
libstore_LDFLAGS += $(LIBSECCOMP_LIBS)
endif
# Not just for this library itself, but also for downstream libraries using this library
INCLUDE_libstore := -I $(d) -I $(d)/build
libstore_CXXFLAGS += \
-I src/libutil -I src/libstore -I src/libstore/build \
$(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libstore) \
-DNIX_PREFIX=\"$(prefix)\" \
-DNIX_STORE_DIR=\"$(storedir)\" \
-DNIX_DATA_DIR=\"$(datadir)\" \
+1 -1
View File
@@ -186,7 +186,7 @@ std::optional<nlohmann::json> ParsedDerivation::prepareStructuredAttrs(Store & s
for (auto i = e->begin(); i != e->end(); ++i) {
StorePathSet storePaths;
for (auto & p : *i)
storePaths.insert(store.toStorePath(p.get<std::string>()).first);
storePaths.insert(store.parseStorePath(p.get<std::string>()));
json[i.key()] = pathInfoToJSON(store,
store.exportReferences(storePaths, inputPaths));
}
+1 -1
View File
@@ -285,7 +285,7 @@ void RootArgs::parseCmdline(const Strings & _cmdline, bool allowShebang)
std::string line;
std::getline(stream,line);
static const std::string commentChars("#/\\%@*-");
static const std::string commentChars("#/\\%@*-(");
std::string shebangContent;
while (std::getline(stream,line) && !line.empty() && commentChars.find(line[0]) != std::string::npos){
line = chomp(line);
+25 -15
View File
@@ -11,9 +11,9 @@
namespace nix {
void BaseError::addTrace(std::shared_ptr<Pos> && e, HintFmt hint)
void BaseError::addTrace(std::shared_ptr<Pos> && e, HintFmt hint, TracePrint print)
{
err.traces.push_front(Trace { .pos = std::move(e), .hint = hint });
err.traces.push_front(Trace { .pos = std::move(e), .hint = hint, .print = print });
}
void throwExceptionSelfCheck(){
@@ -163,7 +163,7 @@ static bool printPosMaybe(std::ostream & oss, std::string_view indent, const std
return hasPos;
}
void printTrace(
static void printTrace(
std::ostream & output,
const std::string_view & indent,
size_t & count,
@@ -379,29 +379,39 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
// A consecutive sequence of stack traces that are all in `tracesSeen`.
std::vector<Trace> skippedTraces;
size_t count = 0;
bool truncate = false;
for (const auto & trace : einfo.traces) {
if (trace.hint.str().empty()) continue;
if (!showTrace && count > 3) {
oss << "\n" << ANSI_WARNING "(stack trace truncated; use '--show-trace' to show the full trace)" ANSI_NORMAL << "\n";
break;
truncate = true;
}
if (tracesSeen.count(trace)) {
skippedTraces.push_back(trace);
continue;
if (!truncate || trace.print == TracePrint::Always) {
if (tracesSeen.count(trace)) {
skippedTraces.push_back(trace);
continue;
}
tracesSeen.insert(trace);
printSkippedTracesMaybe(oss, ellipsisIndent, count, skippedTraces, tracesSeen);
count++;
printTrace(oss, ellipsisIndent, count, trace);
}
tracesSeen.insert(trace);
printSkippedTracesMaybe(oss, ellipsisIndent, count, skippedTraces, tracesSeen);
count++;
printTrace(oss, ellipsisIndent, count, trace);
}
printSkippedTracesMaybe(oss, ellipsisIndent, count, skippedTraces, tracesSeen);
if (truncate) {
oss << "\n" << ANSI_WARNING "(stack trace truncated; use '--show-trace' to show the full, detailed trace)" ANSI_NORMAL << "\n";
}
oss << "\n" << prefix;
}
+14 -1
View File
@@ -61,9 +61,22 @@ void printCodeLines(std::ostream & out,
const Pos & errPos,
const LinesOfCode & loc);
/**
* When a stack frame is printed.
*/
enum struct TracePrint {
/**
* The default behavior; always printed when `--show-trace` is set.
*/
Default,
/** Always printed. Produced by `builtins.addErrorContext`. */
Always,
};
struct Trace {
std::shared_ptr<Pos> pos;
HintFmt hint;
TracePrint print = TracePrint::Default;
};
inline bool operator<(const Trace& lhs, const Trace& rhs);
@@ -161,7 +174,7 @@ public:
addTrace(std::move(e), HintFmt(std::string(fs), args...));
}
void addTrace(std::shared_ptr<Pos> && e, HintFmt hint);
void addTrace(std::shared_ptr<Pos> && e, HintFmt hint, TracePrint print = TracePrint::Default);
bool hasTrace() const { return !err.traces.empty(); }
-10
View File
@@ -203,16 +203,6 @@ constexpr std::array<ExperimentalFeatureDetails, numXpFeatures> xpFeatureDetails
)",
.trackingUrl = "https://github.com/NixOS/nix/milestone/40",
},
{
.tag = Xp::ReplFlake,
.name = "repl-flake",
.description = R"(
*Enabled with [`flakes`](#xp-feature-flakes) since 2.19*
Allow passing [installables](@docroot@/command-ref/new-cli/nix.md#installables) to `nix repl`, making its interface consistent with the other experimental commands.
)",
.trackingUrl = "https://github.com/NixOS/nix/milestone/32",
},
{
.tag = Xp::AutoAllocateUids,
.name = "auto-allocate-uids",
-1
View File
@@ -26,7 +26,6 @@ enum struct ExperimentalFeature
RecursiveNix,
NoUrlLiterals,
FetchClosure,
ReplFlake,
AutoAllocateUids,
Cgroups,
DaemonTrustOverride,
+7 -8
View File
@@ -128,7 +128,7 @@ std::string_view baseNameOf(std::string_view path)
return "";
auto last = path.size() - 1;
if (path[last] == '/' && last > 0)
while (last > 0 && path[last] == '/')
last -= 1;
auto pos = path.rfind('/', last);
@@ -421,11 +421,6 @@ void deletePath(const Path & path)
deletePath(path, dummy);
}
void createDir(const Path & path, mode_t mode)
{
if (mkdir(path.c_str(), mode) == -1)
throw SysError("creating directory '%1%'", path);
}
Paths createDirs(const Path & path)
{
@@ -499,10 +494,14 @@ void AutoDelete::reset(const Path & p, bool recursive) {
//////////////////////////////////////////////////////////////////////
std::string defaultTempDir() {
return getEnvNonEmpty("TMPDIR").value_or("/tmp");
}
static Path tempName(Path tmpRoot, const Path & prefix, bool includePid,
std::atomic<unsigned int> & counter)
{
tmpRoot = canonPath(tmpRoot.empty() ? getEnv("TMPDIR").value_or("/tmp") : tmpRoot, true);
tmpRoot = canonPath(tmpRoot.empty() ? defaultTempDir() : tmpRoot, true);
if (includePid)
return fmt("%1%/%2%-%3%-%4%", tmpRoot, prefix, getpid(), counter++);
else
@@ -542,7 +541,7 @@ Path createTempDir(const Path & tmpRoot, const Path & prefix,
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix)
{
Path tmpl(getEnv("TMPDIR").value_or("/tmp") + "/" + prefix + ".XXXXXX");
Path tmpl(defaultTempDir() + "/" + prefix + ".XXXXXX");
// Strictly speaking, this is UB, but who cares...
// FIXME: use O_TMPFILE.
AutoCloseFD fd(mkstemp((char *) tmpl.c_str()));
+4 -5
View File
@@ -165,11 +165,6 @@ inline Paths createDirs(PathView path)
return createDirs(Path(path));
}
/**
* Create a single directory.
*/
void createDir(const Path & path, mode_t mode = 0755);
/**
* Create a symlink.
*/
@@ -239,6 +234,10 @@ Path createTempDir(const Path & tmpRoot = "", const Path & prefix = "nix",
*/
std::pair<AutoCloseFD, Path> createTempFile(const Path & prefix = "nix");
/**
* Return `TMPDIR`, or the default temporary directory if unset or empty.
*/
Path defaultTempDir();
/**
* Used in various places.
+11 -2
View File
@@ -1,16 +1,25 @@
#pragma once
///@file
#include <utility>
/**
* A trivial class to run a function at the end of a scope.
*/
template<typename Fn>
class Finally
class [[nodiscard("Finally values must be used")]] Finally
{
private:
Fn fun;
bool movedFrom = false;
public:
Finally(Fn fun) : fun(std::move(fun)) { }
~Finally() { fun(); }
// Copying Finallys is definitely not a good idea and will cause them to be
// called twice.
Finally(Finally &other) = delete;
Finally(Finally &&other) : fun(std::move(other.fun)) {
other.movedFrom = true;
}
~Finally() { if (!movedFrom) fun(); }
};
+1
View File
@@ -251,6 +251,7 @@ void dumpTree(const Tree & entries, Sink & sink,
for (auto & [name, entry] : entries) {
auto name2 = name;
if (entry.mode == Mode::Directory) {
assert(!name2.empty());
assert(name2.back() == '/');
name2.pop_back();
}
+4 -1
View File
@@ -6,7 +6,10 @@ libutil_DIR := $(d)
libutil_SOURCES := $(wildcard $(d)/*.cc $(d)/signature/*.cc)
libutil_CXXFLAGS += -I src/libutil
# Not just for this library itself, but also for downstream libraries using this library
INCLUDE_libutil := -I $(d)
libutil_CXXFLAGS += $(INCLUDE_libutil)
libutil_LDFLAGS += $(THREAD_LDFLAGS) $(LIBCURL_LIBS) $(SODIUM_LIBS) $(OPENSSL_LIBS) $(LIBBROTLI_LIBS) $(LIBARCHIVE_LIBS) $(BOOST_LDFLAGS) -lboost_context
+1 -1
View File
@@ -52,7 +52,7 @@ public:
: printBuildLogs(printBuildLogs)
{
systemd = getEnv("IN_SYSTEMD") == "1";
tty = shouldANSI();
tty = isTTY();
}
bool isVerbose() override {
+1 -1
View File
@@ -22,7 +22,7 @@ Key::Key(std::string_view s)
key = ss.payload;
if (name == "" || key == "")
throw Error("key is corrupt");
throw Error("secret key is corrupt");
key = base64Decode(key);
}
+3 -41
View File
@@ -39,9 +39,9 @@ void SourceAccessor::readFile(
}
Hash SourceAccessor::hashPath(
const CanonPath & path,
PathFilter & filter,
HashAlgorithm ha)
const CanonPath & path,
PathFilter & filter,
HashAlgorithm ha)
{
HashSink sink(ha);
dumpPath(path, sink, filter);
@@ -67,42 +67,4 @@ std::string SourceAccessor::showPath(const CanonPath & path)
return displayPrefix + path.abs() + displaySuffix;
}
CanonPath SourceAccessor::resolveSymlinks(
const CanonPath & path,
SymlinkResolution mode)
{
auto res = CanonPath::root;
int linksAllowed = 1024;
std::list<std::string> todo;
for (auto & c : path)
todo.push_back(std::string(c));
while (!todo.empty()) {
auto c = *todo.begin();
todo.pop_front();
if (c == "" || c == ".")
;
else if (c == "..")
res.pop();
else {
res.push(c);
if (mode == SymlinkResolution::Full || !todo.empty()) {
if (auto st = maybeLstat(res); st && st->type == SourceAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", showPath(path));
auto target = readLink(res);
res.pop();
if (hasPrefix(target, "/"))
res = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
}
}
}
}
return res;
}
}
+3 -34
View File
@@ -9,26 +9,6 @@ namespace nix {
struct Sink;
/**
* Note there is a decent chance this type soon goes away because the problem is solved another way.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
enum class SymlinkResolution {
/**
* Resolve symlinks in the ancestors only.
*
* Only the last component of the result is possibly a symlink.
*/
Ancestors,
/**
* Resolve symlinks fully, realpath(3)-style.
*
* No component of the result will be a symlink.
*/
Full,
};
/**
* A read-only filesystem abstraction. This is used by the Nix
* evaluator and elsewhere for accessing sources in various
@@ -132,9 +112,9 @@ struct SourceAccessor
PathFilter & filter = defaultPathFilter);
Hash hashPath(
const CanonPath & path,
PathFilter & filter = defaultPathFilter,
HashAlgorithm ha = HashAlgorithm::SHA256);
const CanonPath & path,
PathFilter & filter = defaultPathFilter,
HashAlgorithm ha = HashAlgorithm::SHA256);
/**
* Return a corresponding path in the root filesystem, if
@@ -157,17 +137,6 @@ struct SourceAccessor
void setPathDisplay(std::string displayPrefix, std::string displaySuffix = "");
virtual std::string showPath(const CanonPath & path);
/**
* Resolve any symlinks in `path` according to the given
* resolution mode.
*
* @param mode might only be a temporary solution for this.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
CanonPath resolveSymlinks(
const CanonPath & path,
SymlinkResolution mode = SymlinkResolution::Full);
};
}
+38
View File
@@ -62,6 +62,44 @@ bool SourcePath::operator<(const SourcePath & x) const
return std::tie(*accessor, path) < std::tie(*x.accessor, x.path);
}
SourcePath SourcePath::resolveSymlinks(SymlinkResolution mode) const
{
auto res = SourcePath(accessor);
int linksAllowed = 1024;
std::list<std::string> todo;
for (auto & c : path)
todo.push_back(std::string(c));
bool resolve_last = mode == SymlinkResolution::Full;
while (!todo.empty()) {
auto c = *todo.begin();
todo.pop_front();
if (c == "" || c == ".")
;
else if (c == "..")
res.path.pop();
else {
res.path.push(c);
if (resolve_last || !todo.empty()) {
if (auto st = res.maybeLstat(); st && st->type == InputAccessor::tSymlink) {
if (!linksAllowed--)
throw Error("infinite symlink recursion in path '%s'", path);
auto target = res.readLink();
res.path.pop();
if (hasPrefix(target, "/"))
res.path = CanonPath::root;
todo.splice(todo.begin(), tokenizeString<std::list<std::string>>(target, "/"));
}
}
}
}
return res;
}
std::ostream & operator<<(std::ostream & str, const SourcePath & path)
{
str << path.to_string();
+26 -5
View File
@@ -11,6 +11,26 @@
namespace nix {
/**
* Note there is a decent chance this type soon goes away because the problem is solved another way.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
enum class SymlinkResolution {
/**
* Resolve symlinks in the ancestors only.
*
* Only the last component of the result is possibly a symlink.
*/
Ancestors,
/**
* Resolve symlinks fully, realpath(3)-style.
*
* No component of the result will be a symlink.
*/
Full,
};
/**
* An abstraction for accessing source files during
* evaluation. Currently, it's just a wrapper around `CanonPath` that
@@ -103,13 +123,14 @@ struct SourcePath
bool operator<(const SourcePath & x) const;
/**
* Convenience wrapper around `SourceAccessor::resolveSymlinks()`.
* Resolve any symlinks in this `SourcePath` according to the
* given resolution mode.
*
* @param mode might only be a temporary solution for this.
* See the discussion in https://github.com/NixOS/nix/pull/9985.
*/
SourcePath resolveSymlinks(
SymlinkResolution mode = SymlinkResolution::Full) const
{
return {accessor, accessor->resolveSymlinks(path, mode)};
}
SymlinkResolution mode = SymlinkResolution::Full) const;
};
std::ostream & operator << (std::ostream & str, const SourcePath & path);
+5 -2
View File
@@ -7,11 +7,14 @@
namespace nix {
bool shouldANSI()
bool isTTY()
{
return isatty(STDERR_FILENO)
static const bool tty =
isatty(STDERR_FILENO)
&& getEnv("TERM").value_or("dumb") != "dumb"
&& !(getEnv("NO_COLOR").has_value() || getEnv("NOCOLOR").has_value());
return tty;
}
std::string filterANSIEscapes(std::string_view s, bool filterAll, unsigned int width)
+1 -1
View File
@@ -8,7 +8,7 @@ namespace nix {
* Determine whether ANSI escape sequences are appropriate for the
* present output.
*/
bool shouldANSI();
bool isTTY();
/**
* Truncate a string to 'width' printable characters. If 'filterAll'
+9 -9
View File
@@ -171,16 +171,16 @@ std::string fixGitURL(const std::string & url)
std::regex scpRegex("([^/]*)@(.*):(.*)");
if (!hasPrefix(url, "/") && std::regex_match(url, scpRegex))
return std::regex_replace(url, scpRegex, "ssh://$1@$2/$3");
if (hasPrefix(url, "file:"))
return url;
if (url.find("://") == std::string::npos) {
return (ParsedURL {
.scheme = "file",
.authority = "",
.path = url
}).to_string();
else {
if (url.find("://") == std::string::npos) {
return (ParsedURL {
.scheme = "file",
.authority = "",
.path = url
}).to_string();
} else
return url;
}
return url;
}
// https://www.rfc-editor.org/rfc/rfc3986#section-3.1
+4 -3
View File
@@ -134,7 +134,7 @@ static void main_nix_build(int argc, char * * argv)
script = argv[1];
try {
auto lines = tokenizeString<Strings>(readFile(script), "\n");
if (!lines.empty() && std::regex_search(lines.front(), std::regex("^#!"))) {
if (std::regex_search(lines.front(), std::regex("^#!"))) {
lines.pop_front();
inShebang = true;
for (int i = 2; i < argc; ++i)
@@ -475,8 +475,9 @@ static void main_nix_build(int argc, char * * argv)
// Set the environment.
auto env = getEnv();
auto tmp = getEnv("TMPDIR");
if (!tmp) tmp = getEnv("XDG_RUNTIME_DIR").value_or("/tmp");
auto tmp = getEnvNonEmpty("TMPDIR");
if (!tmp)
tmp = getEnvNonEmpty("XDG_RUNTIME_DIR").value_or("/tmp");
if (pure) {
decltype(env) newEnv;
+4 -3
View File
@@ -16,6 +16,7 @@
#include "xml-writer.hh"
#include "legacy.hh"
#include "eval-settings.hh" // for defexpr
#include "terminal.hh"
#include <cerrno>
#include <ctime>
@@ -108,7 +109,7 @@ static void getAllExprs(EvalState & state,
const SourcePath & path, StringSet & seen, BindingsBuilder & attrs)
{
StringSet namesSorted;
for (auto & [name, _] : path.resolveSymlinks().readDirectory()) namesSorted.insert(name);
for (auto & [name, _] : path.readDirectory()) namesSorted.insert(name);
for (auto & i : namesSorted) {
/* Ignore the manifest.nix used by profiles. This is
@@ -172,7 +173,7 @@ static void loadSourceExpr(EvalState & state, const SourcePath & path, Value & v
directory). */
else if (st.type == InputAccessor::tDirectory) {
auto attrs = state.buildBindings(maxAttrs);
state.mkList(attrs.alloc("_combineChannels"), 0);
attrs.insert(state.symbols.create("_combineChannels"), &state.vEmptyList);
StringSet seen;
getAllExprs(state, path, seen, attrs);
v.mkAttrs(attrs);
@@ -1089,7 +1090,7 @@ static void opQuery(Globals & globals, Strings opFlags, Strings opArgs)
return;
}
bool tty = isatty(STDOUT_FILENO);
bool tty = isTTY();
RunPager pager;
Table table;
+9 -8
View File
@@ -49,10 +49,8 @@ bool createUserEnv(EvalState & state, PackageInfos & elems,
/* Construct the whole top level derivation. */
StorePathSet references;
Value manifest;
state.mkList(manifest, elems.size());
size_t n = 0;
for (auto & i : elems) {
auto list = state.buildList(elems.size());
for (const auto & [n, i] : enumerate(elems)) {
/* Create a pseudo-derivation containing the name, system,
output paths, and optionally the derivation path, as well
as the meta attributes. */
@@ -72,10 +70,9 @@ bool createUserEnv(EvalState & state, PackageInfos & elems,
attrs.alloc(state.sDrvPath).mkString(state.store->printStorePath(*drvPath));
// Copy each output meant for installation.
auto & vOutputs = attrs.alloc(state.sOutputs);
state.mkList(vOutputs, outputs.size());
auto outputsList = state.buildList(outputs.size());
for (const auto & [m, j] : enumerate(outputs)) {
(vOutputs.listElems()[m] = state.allocValue())->mkString(j.first);
(outputsList[m] = state.allocValue())->mkString(j.first);
auto outputAttrs = state.buildBindings(2);
outputAttrs.alloc(state.sOutPath).mkString(state.store->printStorePath(*j.second));
attrs.alloc(j.first).mkAttrs(outputAttrs);
@@ -87,6 +84,7 @@ bool createUserEnv(EvalState & state, PackageInfos & elems,
references.insert(*j.second);
}
attrs.alloc(state.sOutputs).mkList(outputsList);
// Copy the meta attributes.
auto meta = state.buildBindings(metaNames.size());
@@ -98,11 +96,14 @@ bool createUserEnv(EvalState & state, PackageInfos & elems,
attrs.alloc(state.sMeta).mkAttrs(meta);
(manifest.listElems()[n++] = state.allocValue())->mkAttrs(attrs);
(list[n] = state.allocValue())->mkAttrs(attrs);
if (drvPath) references.insert(*drvPath);
}
Value manifest;
manifest.mkList(list);
/* Also write a copy of the list of user environment elements to
the store; we need it for future modifications of the
environment. */
+12
View File
@@ -40,6 +40,18 @@ R""(
lrwxrwxrwx 1 … ./result-dev -> /nix/store/dkm3gwl0xrx0wrw6zi5x3px3lpgjhlw4-glibc-2.32-dev
```
* Build all outputs:
```console
# nix build "nixpkgs#openssl^*" --print-out-paths
/nix/store/gvad6v0cmq1qccmc4wphsazqbj0xzjsl-openssl-3.0.13-bin
/nix/store/a07jqdrc8afnk8r6f3lnhh4gvab7chk4-openssl-3.0.13-debug
/nix/store/yg75achq89wgqn2fi3gglgsd77kjpi03-openssl-3.0.13-dev
/nix/store/bvdcihi8c88fw31cg6gzzmpnwglpn1jv-openssl-3.0.13-doc
/nix/store/gjqcvq47cmxazxga0cirspm3jywkmvfv-openssl-3.0.13-man
/nix/store/7nmrrad8skxr47f9hfl3xc0pfqmwq51b-openssl-3.0.13
```
* Build attribute `build.x86_64-linux` from (non-flake) Nix expression
`release.nix`:
+6
View File
@@ -11,6 +11,12 @@ R""(
Note the `file://` - without this, the destination is a chroot
store, not a binary cache.
* Copy all store paths from a local binary cache in `/tmp/cache` to the local store:
```console
# nix copy --all --from file:///tmp/cache
```
* Copy the entire current NixOS system closure to another machine via
SSH:
+4 -8
View File
@@ -202,11 +202,7 @@ static PeerInfo getPeerInfo(int remote)
#if defined(SO_PEERCRED)
# if defined(__OpenBSD__)
struct sockpeercred cred;
# else
ucred cred;
# endif
ucred cred;
socklen_t credLen = sizeof(cred);
if (getsockopt(remote, SOL_SOCKET, SO_PEERCRED, &cred, &credLen) == -1)
throw SysError("getting peer credentials");
@@ -214,9 +210,9 @@ static PeerInfo getPeerInfo(int remote)
#elif defined(LOCAL_PEERCRED)
# if !defined(SOL_LOCAL)
# define SOL_LOCAL 0
# endif
#if !defined(SOL_LOCAL)
#define SOL_LOCAL 0
#endif
xucred cred;
socklen_t credLen = sizeof(cred);
+17
View File
@@ -54,6 +54,7 @@ struct BuildEnvironment
std::map<std::string, Value> vars;
std::map<std::string, std::string> bashFunctions;
std::optional<std::pair<std::string, std::string>> structuredAttrs;
std::optional<int> minimalBashVersion;
static BuildEnvironment fromJSON(std::string_view in)
{
@@ -81,6 +82,10 @@ struct BuildEnvironment
res.structuredAttrs = {json["structuredAttrs"][".attrs.json"], json["structuredAttrs"][".attrs.sh"]};
}
if (json.contains("meta") && json["meta"].contains("min-bash-version")) {
res.minimalBashVersion = std::optional<int>{ json["meta"]["min-bash-version"] };
}
return res;
}
@@ -142,6 +147,18 @@ struct BuildEnvironment
void toBash(std::ostream & out, const std::set<std::string> & ignoreVars) const
{
if (minimalBashVersion.has_value()) {
out << stripIndentation(fmt(R"__NIX_STR(
if [[ -n "${BASH_VERSINFO-}" && "${BASH_VERSINFO-}" -lt %d ]]; then
echo "Detected Bash version that isn't supported by this derivation (${BASH_VERSION})"
echo "Please install Bash %d or greater to continue."
exit 1
fi
)__NIX_STR",
minimalBashVersion.value(),
minimalBashVersion.value()));
}
for (auto & [name, value] : vars) {
if (!ignoreVars.count(name)) {
if (auto str = std::get_if<String>(&value)) {
+11 -2
View File
@@ -120,8 +120,17 @@ struct CmdEval : MixJSON, InstallableValueCommand, MixReadOnlyOption
}
else {
state->forceValueDeep(*v);
logger->cout("%s", ValuePrinter(*state, *v, PrintOptions { .force = true }));
logger->cout(
"%s",
ValuePrinter(
*state,
*v,
PrintOptions {
.force = true,
.derivationPaths = true
}
)
);
}
}
};
+10
View File
@@ -115,6 +115,16 @@ __dumpEnv() {
printf '\n }'
fi
printf ',\n "meta": {\n '
# min-bash-version is supposed to be an integer.
# If not, just ignore it.
if [[ -n "${NIX_ENV_MIN_BASH_VERSION-}" && "$NIX_ENV_MIN_BASH_VERSION" -eq "$NIX_ENV_MIN_BASH_VERSION" ]]; then
__escapeString "min-bash-version"
printf ': '
printf '%d' "${NIX_ENV_MIN_BASH_VERSION}"
fi
printf '\n }'
printf '\n}'
}
Loaded 100 of 145 files, more files were not shown because too many files have changed in this diff. Show more