Compare commits

...
Author SHA1 Message Date
Théophane Hufschmitt 16175e07a9 develop: Allow shells to declare the minimal bash version they require
Allow derivations to export a `NIX_ENV_MIN_BASH_VERSION` that sets the
minimal (major) bash version that is required to parse their setup
script, and use that to gracefully fail if the current bash is too old.

Fix #10263 (from the Nix side at least, needs https://github.com/NixOS/nixpkgs/pull/299490 from the Nixpkgs side to be useful in practice)
2024-03-29 18:13:42 +01:00
Théophane Hufschmitt b72e1c79da Merge pull request #10299 from detroyejr/remove-repl-flake
flakes: remove experimental repl-flake
2024-03-29 15:06:23 +01:00
Jonathan De TroyeandThéophane Hufschmitt a09d0e19c1 flakes: Reword doc/manual/rl-next/remove-repl-flake.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-29 08:57:41 -04:00
Théophane Hufschmitt 90f5189451 Merge pull request #10131 from intelfx/work/notty
`TERM=dumb` fixes
2024-03-29 13:38:37 +01:00
Théophane Hufschmitt de101417eb Merge pull request #10305 from hercules-ci/addErrorContext-is-great
Always print addErrorContext
2024-03-29 09:10:09 +01:00
Robert Hensing ef26133df3 Merge pull request #10337 from Cyclic4179/patch-1
`nix shell` shebang: support O'Caml comments
2024-03-28 19:54:03 +01:00
Eelco Dolstra 7829caab49 Merge pull request #10347 from Atry/patch-2
docs: clarify that `builtins.stringLength` is counting bytes
2024-03-28 15:56:22 +01:00
Yang, Bo c39afb28db Clarify stringLength is counting bytes 2024-03-27 16:46:50 -07:00
Robert Hensing c0b6907ccd doc/local.mk: Add manual-html-open phony target for auto-opening the browser (#10308) 2024-03-27 22:28:05 +01:00
vac (Brendan) 37f8edce99 docs: add cacert in macos multi-user upgrade (#10237) 2024-03-27 22:26:50 +01:00
Eelco Dolstra 1b71a4270a Merge pull request #10329 from obsidiansystems/mingw-devops
Build system changes for MinGW support
2024-03-27 22:17:11 +01:00
John Ericson 77205b2042 Allow for ergnomically putting Unix-only files in subdirs by creating INLCUDE_$(pkg) vars
Separate platform-specific files will allow avoiding a lot of CPP.
2024-03-27 12:01:59 -04:00
John Ericson 4a61827d2d Hack to make sure the DLL linking job works with the check output 2024-03-27 12:01:59 -04:00
John Ericson ae2b2849c9 Disable GC on windows
We can build the dep and the our GC code is totally portable, but for
some reason we get link errors saying `GC_throw_bad_alloc` is missing.
2024-03-27 12:01:59 -04:00
Robert HensingandRebecca Turner d4fa0a84a5 refact: TraceKind -> TracePrint
Co-authored-by: Rebecca Turner <rbt@sent.as>
2024-03-27 16:32:49 +01:00
Robert Hensing bed541b04e error.cc: Make printTrace static 2024-03-27 16:28:04 +01:00
Robert Hensing 981c309057 Remove trace item: while calling the 'addErrorContext' builtin 2024-03-27 16:28:04 +01:00
Robert Hensing bebacc475c Always print addErrorContext traces 2024-03-27 16:28:04 +01:00
Robert Hensing 189e40214e Merge pull request #10339 from edolstra/gha-vm-tests
Run some VM tests on GitHub
2024-03-27 16:13:41 +01:00
Cyclic4179 9f7b2b93ce fixup! add flakes.sh test for shebang supported types of comments 2024-03-27 15:03:54 +01:00
Cyclic4179 6227cd06bd add flakes.sh test for shebang supported types of comments 2024-03-27 14:55:02 +01:00
Eelco Dolstra 6f3972498b Run some VM tests on GitHub 2024-03-27 14:11:06 +01:00
Cyclic4179 a2c3333b97 fix #10336 2024-03-27 07:47:36 +01:00
Théophane Hufschmitt 6c10cc0eda Merge pull request #10312 from hercules-ci/add-build-dir
Add build-dir setting
2024-03-26 13:51:50 +01:00
Théophane Hufschmitt 0bf01917ef Merge pull request #10325 from K900/ooformatting
build-remote: fix format string shenanigans
2024-03-26 11:45:32 +01:00
Théophane Hufschmitt 1394d4e9c5 Merge pull request #10306 from hercules-ci/baseNameOf
Test and document `builtins.baseNameOf`, improve internal `baseNameOf()`
2024-03-26 09:59:27 +01:00
K900 2d4edb945b build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes https://github.com/NixOS/nix/issues/10238
2024-03-26 11:01:03 +03:00
Robert Hensing 8b16cced18 Add build-dir setting 2024-03-25 19:48:59 +01:00
Robert Hensing b1fe388d33 Remove uncalled for message 2024-03-25 19:48:57 +01:00
Robert Hensing 6221770c9d tests/functional: Add count() 2024-03-25 19:46:34 +01:00
Théophane Hufschmitt 290072ca1e Merge pull request #10321 from Ericson2314/libstore-codeowner
Add @Ericson2314 as libstore codeowner
2024-03-25 18:27:20 +01:00
John Ericson bead1a1cde Add @Ericson2314 as libstore codeowner 2024-03-25 11:26:13 -04:00
Eelco Dolstra 6d90287f5a Merge pull request #10303 from hercules-ci/fix-empty-TMPDIR
fix: Treat empty TMPDIR as unset
2024-03-25 12:49:10 +01:00
Robert Hensing 641b0bd746 Merge pull request #10310 from obsidiansystems/fix-simple-git-hashing-test
Improve `git-hashing/simple.sh` test
2024-03-24 17:31:49 +01:00
John Ericson 513ba1977c Merge pull request #10307 from hercules-ci/doc-language-nix-dev
doc/language: Link to nix.dev introduction
2024-03-24 09:17:32 -04:00
John Ericson 2a44b11f55 Fix git-hashing/simple.sh
I realized it was checking NAR hashes before of added objects, which
makes little sense --- we don't really care about ancillary NAR hashes.

Now, the bottom `nix store add` tests compare the CA field with a git
hash to hashes calculated by Git. This matches top `nix hash path` ones
in using git as a source of truth.
2024-03-23 22:08:52 -04:00
Robert Hensing f891673460 Merge pull request #10293 from NixOS/fix-missing-format-arg
EvalCache: Fix missing format string argument
2024-03-24 02:28:05 +01:00
Robert Hensing fbf493758c doc/language: Link to nix.dev introduction
Closes #10283
2024-03-24 02:08:28 +01:00
Robert Hensing 9884018dfa baseNameOf(): Remove all trailing slashes 2024-03-24 01:38:22 +01:00
Robert Hensing 754a15e2db builtins.baseNameOf: Fork 2024-03-24 01:37:58 +01:00
Robert Hensing 175afc7106 Test and document builtins.baseNameOf 2024-03-24 01:26:17 +01:00
Robert Hensing 3b7f2bf997 git/dumpTree: Assert name not empty before back() 2024-03-24 00:58:08 +01:00
Robert Hensing 850c9a6caf HttpBinaryCacheStore: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing dd26f41379 local-derivation-goal.cc: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing fd31945742 local-derivation-goal.cc: Reuse defaultTempDir() 2024-03-24 00:45:15 +01:00
Robert Hensing b9e7f5aa2d fix: Treat empty XDG_RUNTIME_DIR as unset
See preceding commit. Not observed in the wild, but is sensible
and consistent with TMPDIR behavior.
2024-03-24 00:42:31 +01:00
Robert Hensing c3fb2aa1f9 fix: Treat empty TMPDIR as unset
Fixes an instance of

    nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.

... which I've been getting in one of my shells for some reason.
I have yet to find out why TMPDIR was empty, but it's no reason for
Nix to break.
2024-03-24 00:42:31 +01:00
detroyejr 50885b81c9 fix: correct remove-repl-flake.md 2024-03-23 06:31:49 -04:00
Théophane Hufschmitt 38ba96d7b0 Merge pull request #10296 from picnoir/pic/doc-lsp
Doc: explicitly specify which shell to use to get clangd
2024-03-23 08:46:13 +01:00
detroyejr b11dd58fe4 flakes: test to ensure we get an error if --file isn't used 2024-03-22 21:58:39 -04:00
detroyejr c625b45357 flakes: add remove-repl-flake.md note 2024-03-22 21:39:48 -04:00
detroyejr 717391731c flakes: remove experimental repl-flake 2024-03-22 19:30:03 -04:00
Picnoir cc29d85d06 Doc: explicitly specify which shell to use to get clangd
I was using by mistake the .#nix-clangStdenv shell to retrieve clangd.
This clangd is unusable with the project and constantly segfaults.
Let's explicitly state which shell the user should use in the docs.

I don't really understand the source of this segfault. I assume it's
related to a clang version incompatibility. (16.0.6 for
.#nix-clangStdenv 14.0.6 for .#native-clangStdenvPackages)
2024-03-22 22:36:55 +01:00
Théophane Hufschmitt 4ec2a7d3bf Merge pull request #10292 from dottharun/docBuildOutput
Docs: Documentation manual build output path correction
2024-03-22 21:45:20 +01:00
Eelco Dolstra bfd36402ac EvalCache: Fix missing format string argument
Fixes

  terminate called after throwing an instance of 'boost::wrapexcept<boost::io::too_few_args>'
    what():  boost::too_few_args: format-string referred to more arguments than were passed
  Aborted (core dumped)

for type errors in AttrCursor.
2024-03-22 18:11:24 +01:00
Tharun T 4e2f11b692 doc build output correction
Signed-off-by: Tharun T <tharun242424@gmail.com>
2024-03-22 18:34:55 +05:30
Robert Hensing a223280664 Merge pull request #10286 from hercules-ci/various
Allocate a bit less
2024-03-22 11:19:58 +01:00
Théophane Hufschmitt 57d9d0d6e4 Merge pull request #10269 from andreabedini/make-compile-commands
Create compile-commands.json with Make
2024-03-21 17:22:35 +01:00
Théophane Hufschmitt d0824f661e Document the new compile_commands.json target 2024-03-21 16:54:28 +01:00
Théophane Hufschmitt f666eda94d Merge pull request #10274 from tweag/warn-on-untrusted-setting
Daemon: warn when an untrusted user cannot override a setting
2024-03-21 16:40:24 +01:00
Théophane Hufschmitt 039b3fbdfe Merge pull request #10271 from jonringer/document-flake-multiple-outputs
Document how to build many outputs of a flake package
2024-03-21 15:58:32 +01:00
Andrea Bedini 83fc988bec Create compile-commands.json with Make 2024-03-21 15:55:44 +08:00
Robert Hensing 1fcdd1640e functionArgs: Allocate bools only once 2024-03-20 23:25:28 +01:00
Robert Hensing 8c6e0df45f value.hh: Fix warning about {struct/class} Value 2024-03-20 23:25:28 +01:00
Robert Hensing a865049c4f tryEval: Allocate true and false once 2024-03-20 23:25:28 +01:00
Robert Hensing d71e74838a readDir: Allocate type strings only once 2024-03-20 23:25:28 +01:00
Jonathan RingerandThéophane Hufschmitt 4c8a33ce46 Update src/nix/build.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-20 14:42:44 -07:00
Robert Hensing 7d2ead50e5 Merge pull request #10083 from lf-/jade/refactor-repl-input
refactor: move the repl input code to its own file
2024-03-20 22:37:40 +01:00
Robert Hensing d16d7f5f31 Merge pull request #10251 from edolstra/list-builder
Add a ListBuilder helper for constructing list values
2024-03-20 22:30:38 +01:00
Robert Hensing d4b0b7f152 createBaseEnv: Use state.vNull 2024-03-20 21:34:23 +01:00
Robert Hensing 0b08dd45b0 prim_match: Use state.vNull 2024-03-20 21:28:38 +01:00
Félix Baylac-Jacqué 40a7929c8e Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.

This is a cherry-pick of 9e0f5f803f.
The above commit has been reverted by
a59e77d9e5 to prevent spamming warnings
with experimental features, but these are now totally ignored on the
daemon side, so there's no reason for the revert any more.
2024-03-20 17:55:35 +01:00
Théophane Hufschmitt 346f92aa4c Merge pull request #10241 from bouk/resolve-ref-packed-refs
git fetcher: use resolveRef for getting revision of reference
2024-03-20 17:37:15 +01:00
Théophane Hufschmitt 4cb0847743 Merge pull request #10273 from jesses-canva/patch-1
Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in
2024-03-20 12:24:17 +01:00
Jesse Schalken c448636f7c Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in 2024-03-20 15:23:31 +11:00
Jonathan Ringer a5262fb880 Document how to build many outputs of a flake package 2024-03-19 17:37:04 -07:00
Théophane Hufschmitt 7c506432ab Merge pull request #10264 from lelgenio/document-pure-eval-store-path
Document builtins.storePath being disabled by pure-eval
2024-03-19 09:59:19 +01:00
lelgenio ad217ddbbc Document builtins.storePath being disabled by pure-eval 2024-03-18 16:23:17 -03:00
Théophane Hufschmitt 5c8983b9e6 Merge pull request #10259 from ShamrockLee/doc-primops-fix
doc: builtins.addDrvOutputDependencies: fix link target
2024-03-17 21:10:52 +01:00
Yueh-Shun Li d2b512959c builtins.addDrvOutputDependencies: fix commentary 2024-03-18 02:38:31 +08:00
Yueh-Shun Li 39b0b8452f doc: builtins.addDrvOutputDependencies: fix link target 2024-03-17 06:13:14 +08:00
Eelco Dolstra 3e6730ee62 Mark Value pointers in Value::elems as const
This catches modification of finalized values (e.g. in prim_sort).
2024-03-15 18:26:37 +01:00
Eelco Dolstra fecff520d7 Add a ListBuilder helper for constructing list values
Previously, `state.mkList()` would set the type of the value to tList
and allocate the list vector, but it would not initialize the values
in the list. This has two problems:

* If an exception occurs, the list is left in an undefined state.

* More importantly, for multithreaded evaluation, if a value
  transitions from thunk to non-thunk, it should be final (i.e. other
  threads should be able to access the value safely).

To address this, there now is a `ListBuilder` class (analogous to
`BindingsBuilder`) to build the list vector prior to the call to
`Value::mkList()`. Typical usage:

   auto list = state.buildList(size);
   for (auto & v : list)
       v = ... set value ...;
   vRes.mkList(list);
2024-03-15 18:26:37 +01:00
Eelco Dolstra 0fb5024d8d Merge pull request #10243 from DimitarNestorov/patch-1
docs: update registry examples
2024-03-15 18:13:12 +01:00
Eelco Dolstra 650161b7ae Merge pull request #10242 from fricklerhandwerk/fix-index-redirects
adjust anchor redirects to point to new pages
2024-03-15 18:12:50 +01:00
Daniel Sidhion 49e9efeaaa doc: document SRI hash format for outputHash (#10230) 2024-03-15 07:09:47 +01:00
Dimitar Nestorov a50295425e docs: update registry examples
When you run `nix flake info` you get a deprecated message
2024-03-14 16:15:52 +02:00
Valentin Gagarin 3754614b9c adjust anchor redirects to point to new pages
a previous moving of files accounted for server-side redirects, but not
client-side redirects.
2024-03-14 15:01:47 +01:00
Bouke van der Bijl 60c2d15f5a git fetcher: use resolveRef for getting revision of reference
* Add regression test
* Fix 'no repo' test so it doesn't succeed if the data is still in cache
* Use git_revparse_single inside git-utils instead of reimplementing the same logic.
2024-03-14 14:04:51 +01:00
Robert Hensing c152c2767a Merge pull request #10239 from correabuscar/patch-1
typo consant->constant in context.cc
2024-03-13 22:45:36 +01:00
Emanuel Czirai e5840d5760 typo consant->constant in context.cc 2024-03-13 20:35:24 +01:00
Jade Lovelace ea31b8a117 refactor: repl prompts are now the job of the interacter 2024-03-12 18:25:35 -07:00
Jade Lovelace 70a6ce139b refactor: move readline stuff into its own file
This is in direct preparation for an automation mode of nix repl.
2024-03-12 18:25:35 -07:00
Jade Lovelace 76aced6915 finally.hh: delete copy constructor which is a bad idea 2024-03-12 18:25:35 -07:00
Eelco Dolstra bff5c94184 Merge pull request #10205 from 9999years/final-matchers
Make `Matcher` subclasses `final`
2024-03-12 07:23:15 +01:00
Eelco Dolstra 18b89ca58c Merge pull request #10227 from tmplt/doc/nix-copy
nix-copy: document --all --from local binary cache example
2024-03-12 07:22:28 +01:00
Eelco Dolstra e3cc701d86 Merge pull request #10231 from NixOS/dependabot/github_actions/cachix/install-nix-action-26
Bump cachix/install-nix-action from 25 to 26
2024-03-12 07:16:25 +01:00
dependabot[bot] aa121dc318 Bump cachix/install-nix-action from 25 to 26
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 25 to 26.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v25...v26)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-11 22:02:01 +00:00
Eelco Dolstra 90e86bf6d3 Merge pull request #10229 from NixOS/bump-2.22.0
Bump version
2024-03-11 21:51:52 +01:00
Eelco Dolstra 222c38370f Bump version 2024-03-11 21:16:10 +01:00
Théophane Hufschmitt 18b2ef8b20 Merge pull request #10209 from 9999years/rethrow-for-top-level-errors-in-repl
Print top-level errors normally in `nix repl`
2024-03-11 21:11:28 +01:00
Viktor Sonesten db36c9ca90 nix-copy: document --all --from local binary cache example 2024-03-11 19:17:45 +01:00
tomberek 25bf671015 Merge pull request #10200 from 9999years/fix-nix-eval-for-derivations
Fix `nix eval nixpkgs#bash` segfault
2024-03-11 13:56:49 -04:00
Eelco Dolstra 72e145e415 Merge pull request #10226 from edolstra/release-notes
Release notes
2024-03-11 18:32:51 +01:00
Rebecca Turner 7f45b1c8d8 Add release note 2024-03-11 10:03:01 -07:00
Eelco Dolstra 4c97a66b4c Tweak release notes 2024-03-11 18:00:10 +01:00
Eelco Dolstra b12dc76cfc release notes: 2.21.0 2024-03-11 16:22:29 +01:00
Eelco Dolstra 0ce58cd1c2 Merge pull request #10224 from tweag/fix-darwin-instaltests
Run preInstallCheck even when not building
2024-03-11 16:02:31 +01:00
Théophane Hufschmitt 74008d8215 Run preInstallCheck even when not building
Add `runHook preInstallCheck` to the overriden `installCheckPhase` used
for the non-build case.

In particular, this allow the fix from 2a34510776
to also apply there.
2024-03-11 15:34:23 +01:00
Robert Hensing 3539172fd2 Merge pull request #10208 from 9999years/print-strings-directly
`:print` strings directly in `nix repl`
2024-03-11 10:02:09 +01:00
Théophane Hufschmitt 83460fb64f Merge pull request #10216 from edolstra/fix-github-test
Fix GitHub test
2024-03-11 09:50:33 +01:00
Rebecca Turner db9bab2708 Matcher: Add virtual destructor 2024-03-10 12:56:07 -07:00
Eelco Dolstra 841fd78baa GitArchiveInputScheme: Support the narHash attribute
This is required to produce a locked flakeref.
2024-03-10 14:08:55 +01:00
Eelco Dolstra c1811c1eba Fix GitHub test
Cherry-picked from 03618bb85f.
2024-03-10 13:38:38 +01:00
Eelco Dolstra e65e9114d2 Merge pull request #10204 from 9999years/replace-foo-in-big-cxx-literal
Replace `foo` with `__NIX_STR` in `cxx-big-literal`
2024-03-10 13:11:36 +01:00
Rebecca Turner d13c63afa2 Print top-level errors normally in nix repl
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```

Now, errors will be displayed normally if they're emitted at the
top-level of an expression:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
       … in the condition of the assert statement
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
          267|     in commonAttrs // {
          268|       drvPath = assert condition; drv.drvPath;
             |                 ^
          269|       outPath = assert condition; drv.outPath;

       … in the left operand of the OR (||) operator
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
           27|         if lib.isDerivation value then
           28|           lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
             |                                             ^
           29|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```

Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.
2024-03-09 18:28:04 -08:00
Rebecca Turner d859d6c434 :print strings directly in nix repl
Strings are now printed directly when evaluated by `:print`, rather than
escaped. This makes it easier to debug multi-line strings or strings
containing quotes, like the results of `builtins.readFile`,
`lib.toShellArg`, and so on.

```
nix-repl> "cuppy\ndog\ncity"
"cuppy\ndog\ncity"

nix-repl> :p "cuppy\ndog\ncity"
cuppy
dog
city
```
2024-03-09 18:13:08 -08:00
Rebecca Turner 70e93c1e2b Make Matcher subclasses final
Fixes this very long warning, which I'll only include the first line of:

/nix/store/8wrjhrycpshhc3b41xmjwvgqr2m3yajq-libcxx-16.0.6-dev/include/c++/v1/__memory/construct_at.h:66:5: warning: destructor called on non-final 'RegexMatcher' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
    __loc->~_Tp();
2024-03-09 17:07:52 -08:00
Rebecca Turner ea8faf8e9a Replace foo with __NIX_STR in cxx-big-literal
Looks a little nicer when you check the generated sources.
2024-03-09 17:05:56 -08:00
Valentin Gagarin ac730622e8 document where the value of builtins.nixPath comes from (#9113)
* document default values for `nix-path`

also note how it's overridden and note the effect of `restrict-eval`
2024-03-09 17:57:57 +00:00
Rebecca Turner 4910d74086 Print derivation paths in nix eval
`nix eval` forces values and prints derivations as attribute sets, so
commands that print derivations (e.g. `nix eval nixpkgs#bash`) will
infinitely loop and segfault.

Printing derivations as `.drv` paths makes `nix eval` complete as
expected. Further work is needed, but this is better than a segfault.
2024-03-08 21:43:54 -08:00
tomberek 3120fef01b Merge pull request #10184 from iFreilicht/patch-2
docs: Explain how to add release notes
2024-03-08 19:49:44 -05:00
Felix Uhl 35f2b07668 docs: Fix link to release note documentation 2024-03-08 20:04:55 +01:00
tomberek a200ee6bf7 Merge pull request #9874 from pennae/error-reports
improve error reports somewhat
2024-03-08 10:51:14 -05:00
Eelco Dolstra ae2bd460a8 Merge pull request #10166 from bobvanderlinden/profile-regex-all
profile: introduce --regex and --all
2024-03-08 15:59:24 +01:00
Jonathan Dickinson 1ffcbddf62 docs: add inherit to language overview (#10194)
* docs: add inherit to language overview

Adds a short summary about `inherit` to the language overview.
2024-03-08 14:24:44 +00:00
Théophane Hufschmitt ef5cb44608 Merge pull request #10197 from edolstra/github-treehash
GitHub fetcher: Don't emit treeHash yet
2024-03-08 14:14:51 +01:00
Eelco Dolstra 6d245182e8 GitHub fetcher: Don't emit treeHash yet
But do accept it if it's there, so we don't choke on future lock files
that do have the treeHash attribute.
2024-03-08 12:40:14 +01:00
Bob van der Linden 3d628d1704 profile: convert Matcher to abstract class 2024-03-08 10:47:51 +01:00
Eelco Dolstra 61b006ddf6 Merge pull request #10174 from tweag/disable-failing-hydra-jobs
flake: Remove the cross-compilation to freebsd13
2024-03-08 09:46:38 +01:00
Eelco Dolstra 4fda08d867 Merge pull request #10195 from tweag/fix-nix-env-qap
Update the golden hash of the evalNixpkgs test
2024-03-08 09:44:16 +01:00
Eelco Dolstra a7b3249d68 Merge pull request #10179 from edolstra/upload-release
maintainers/upload-release.pl: Handle 2.3 and 2.18 branches
2024-03-08 09:38:15 +01:00
Eelco Dolstra cb7ee1af89 Merge pull request #10175 from tweag/disable-i686-perl-bindings
flake: Disable the perl bindings on i686-linux
2024-03-08 09:21:00 +01:00
Théophane Hufschmitt ff74c081e9 flake: Remove the cross-compilation to freebsd13
`libc` is broken there: https://hydra.nixos.org/build/252347598.

We can reintroduce it once the base system is working
2024-03-08 08:58:37 +01:00
Théophane Hufschmitt 520a1df208 flake: Disable the perl bindings on i686-linux
Some perl dependencies are failing: https://hydra.nixos.org/build/252347639/nixlog/1

Since the support is only best-effort there, disable the perl bindings
2024-03-08 08:58:10 +01:00
Théophane Hufschmitt 201369dceb tests.evalNixpkgs: Update the golden hash
`nix-env -qaP`'s output has changed a bit because of https://github.com/NixOS/nix/issues/10132.

Although that's a bit annoying, it isn't nearly as problematic as the
evaluation changes that this test is supposed to catch. So it's find to
just update the hash for the time being and fix the issue later
(properly fixing the issue will very likely change the hash any way).
2024-03-08 08:50:27 +01:00
Théophane Hufschmitt 4354b37fc4 Add more logs to the evalNixpkgs test
Make it possible to understand a mimimum what's going on in case of a
failure
2024-03-08 08:48:53 +01:00
Bob van der Linden 4741d3e308 add release note 2024-03-07 21:07:03 +01:00
Bob van der Linden 91f068c193 profile: make --all exclusive 2024-03-07 21:07:03 +01:00
Bob van der Linden 7a4d5e89d3 profile: add --all option to match any package 2024-03-07 21:07:03 +01:00
Bob van der Linden fb391ebc77 profile: add tests for not matching any packages 2024-03-07 21:07:03 +01:00
Bob van der Linden 9fac62435c tests/functional: add assertStderr function
Currently there isn't a convenient way to check for multiline output. In
addition, these outputs will easily change and having a diff between the
expected an the actual output upon failures is convenient.
2024-03-07 21:07:03 +01:00
Bob van der Linden 87741dbd21 profile: add --regex option to match packages 2024-03-07 21:07:03 +01:00
Bob van der Linden d6f5da51d3 profile: match on package name instead of regex 2024-03-07 21:07:03 +01:00
Bob van der Linden 741a6bfad5 profile: allow different types of matchers 2024-03-07 21:07:03 +01:00
Eelco Dolstra b3b312827c Merge pull request #10180 from 9999years/rebeccat/flake-check-log
Restore "checking Hydra job" message in `nix flake check`
2024-03-07 17:48:37 +01:00
Rebecca Turner 4b4c71e239 Restore "checking Hydra job" message in nix flake check
Mistakenly removed in #8893, thanks @lf- for catching this!

https://github.com/NixOS/nix/commit/9404ce36e4edd1df12892089bdab1ceb7d4d7a97#r139485316
2024-03-07 08:15:53 -08:00
Eelco Dolstra 091f232896 maintainers/upload-release.pl: Handle 2.3 and 2.18 branches 2024-03-07 16:53:34 +01:00
Eelco Dolstra 751cb7cf51 Merge pull request #10178 from tweag/copy-built-outputs
Fix the outputs moving on macOS
2024-03-07 15:40:54 +01:00
Théophane Hufschmitt a3163b9eab Fix the outputs moving on macOS 2024-03-07 14:52:40 +01:00
Eelco Dolstra fb37e184a5 Merge pull request #10176 from NixOS/relative-path-fingerprint
PathInputScheme::getFingerprint(): Don't barf on relative paths
2024-03-07 14:23:57 +01:00
Eelco Dolstra 0282499e18 PathInputScheme::getFingerprint(): Don't barf on relative paths
This wasn't caught by CI because #10149 and #10152 pass
individually... It doesn't happen on lazy-trees either because we
never try to fetch relative path flakes (#10089).
2024-03-07 13:28:52 +01:00
Eelco Dolstra da62528487 Merge pull request from GHSA-2ffj-w4mj-pg37
Copy built outputs
2024-03-07 11:56:24 +01:00
Eelco Dolstra 454456209f Merge pull request #10165 from link2xt/patch-1
Make search.nixos.org link in quick start clickable
2024-03-07 11:05:48 +01:00
Eelco Dolstra 7d76bc8a39 Merge pull request #10152 from edolstra/fetcher-cleanups
Fetcher cleanups
2024-03-07 10:21:11 +01:00
Eelco Dolstra bd828b1937 Merge pull request #10150 from edolstra/always-upgrade-unlocked-flakerefs
nix profile upgrade: Always upgrade unlocked flakerefs
2024-03-07 10:20:37 +01:00
Eelco Dolstra 38b79da100 Merge pull request #10149 from edolstra/use-flake-fingerprint
LockedFlake::getFingerprint(): Use Input::getFingerprint()
2024-03-07 10:18:19 +01:00
Théophane Hufschmitt f175b3a4b7 Revert "install-multi-user.sh: _sudo: add proxy variables to sudo"
This reverts commit 24fd7e2755.
2024-03-07 10:06:33 +01:00
Théophane Hufschmitt 739f53aca4 Revert "Fix sudo in the darwin installer (#10128)"
This reverts commit 686405ef41.
2024-03-07 10:06:33 +01:00
Robert Hensing 73cf23275e Merge pull request #10107 from tweag/repl-file-argument
Properly fail on flakerefs that don't point to a directory
2024-03-07 10:04:35 +01:00
tomberek 0a11080c58 Merge pull request #10172 from lf-/jade/fix-9725
fix: bounds check result in getMaxCPU
2024-03-07 00:58:44 -05:00
Théophane Hufschmitt c4cfa18a33 Merge pull request #10164 from hercules-ci/objc-workaround
package.nix: Apply OBJC_DISABLE_INITIALIZE_FORK_SAFETY workaround
2024-03-07 06:26:15 +01:00
Jade Lovelace 9c64a09c70 fix: bounds check result in getMaxCPU
Fixes https://github.com/NixOS/nix/issues/9725
2024-03-06 20:57:31 -08:00
Robert Hensing 16fa5aa305 Merge pull request #10168 from hercules-ci/doc-glossary-file-system-object-anchor
doc/glossary: Fix file system object anchor
2024-03-07 00:40:36 +01:00
Robert Hensing e4500e539e doc/glossary: Fix file system object anchor
It was stealing the store object id. Browsers pick the first one.
It was confusing.
2024-03-07 00:04:30 +01:00
pennae 5d9fdab3de use byte indexed locations for PosIdx
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).
2024-03-06 23:48:42 +01:00
pennae 855fd5a1bb diagnose "unexpected EOF" at EOF
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.

this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.
2024-03-06 23:11:12 +01:00
pennae 2be6b14328 match line endings used by parser and error reports
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).
2024-03-06 23:11:12 +01:00
pennae 1edd6fada5 report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
2024-03-06 23:11:12 +01:00
pennae 4147ecfb1c normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
2024-03-06 23:11:12 +01:00
pennae d384ecd553 keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.
2024-03-06 23:11:12 +01:00
link2xt fe13d4a6e0 Make search.nixos.org link in quick start clickable 2024-03-06 21:55:02 +00:00
Théophane Hufschmitt 686405ef41 Fix sudo in the darwin installer (#10128) 2024-03-06 22:36:37 +01:00
Robert Hensing 2a34510776 package.nix: Apply OBJC_DISABLE_INITIALIZE_FORK_SAFETY workaround
This was previously already used in the launchd configuration for
nix-daemon.

(cherry picked from commit 855741aea5)
2024-03-06 22:07:05 +01:00
Théophane Hufschmitt 67f95755d6 Merge pull request #10155 from NixOS/warn-against-changing-profile-version-number
Add a warning against carelessly changing the profile version number
2024-03-06 06:43:36 +01:00
Théophane Hufschmitt 0d26358bda Merge pull request #10151 from edolstra/fix-lstat-cache
PosixSourceAccessor::cachedLstat(): Use absolute path
2024-03-05 08:59:30 +01:00
Théophane Hufschmitt 7161ef14a2 Add a warning against carelessly changing the profile version number
Try and prevent the situation of https://github.com/NixOS/nix/issues/10109 to happen again in the future
2024-03-05 08:07:58 +01:00
Théophane Hufschmitt e164b39ee9 Merge pull request #9920 from 9999years/forbid-nested-debuggers
Forbid nested debuggers
2024-03-05 06:58:29 +01:00
Eelco Dolstra 0e07f81d2b Fetcher cleanups
* Convert all InputScheme::fetch() methods to getAccessor().

* Add checkLocks() method for checking lock attributes.

* Rename fetch() to fetchToStore().
2024-03-04 22:24:55 +01:00
Eelco Dolstra 4967c5ff6b Fix macOS build 2024-03-04 22:24:12 +01:00
Eelco Dolstra 9ee590e113 PosixSourceAccessor::cachedLstat(): Use absolute path
Using the relative path can cause collisions between cache entries for
PosixSourceAccessors with different roots.
2024-03-04 21:54:35 +01:00
Eelco Dolstra 6b976a1898 Merge pull request #10143 from fricklerhandwerk/test-help
add tests for showing help
2024-03-04 20:29:03 +01:00
Eelco Dolstra 32bf39c73a nix flake metadata: Don't show locked URL if it's not locked
This is the case for e.g. dirty Git workdirs, where we would get

  $ nix flake metadata
  Resolved URL:  git+file:///home/eelco/Dev/nix-master
  Locked URL:    git+file:///home/eelco/Dev/nix-master
2024-03-04 19:37:43 +01:00
Eelco Dolstra 2306e96767 nix profile upgrade: Always upgrade unlocked flakerefs
The "lockedRef" field is a misnomer, since it can be unlocked
(e.g. for a dirty Git workdir). In that case, `nix profile upgrade`
needs to assume that the package can have changed, and perform an
upgrade.
2024-03-04 19:30:38 +01:00
Valentin GagarinandThéophane Hufschmitt 8d23847571 fix indentation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:44 +01:00
Valentin GagarinandThéophane Hufschmitt 8a6ef3bae5 less scary jq
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:28 +01:00
Eelco Dolstra 6558da45f5 LockedFlake::getFingerprint(): Use Input::getFingerprint() 2024-03-04 19:22:23 +01:00
Eelco Dolstra 29049d2653 Implement getFingerprint() for store paths 2024-03-04 19:21:31 +01:00
Rebecca Turner 2e8f4faa10 Fix build
Not sure why that was giving a duplicate symbol error, or why marking it
inline fixes it. Here it is!
2024-03-04 09:32:06 -08:00
Rebecca Turner 14b0356dc5 Forbid nested debuggers 2024-03-04 09:24:57 -08:00
Rebecca Turner 4ee5433919 Add release note 2024-03-04 09:24:56 -08:00
Valentin Gagarin 4b15ca2ffb add tests for showing help 2024-03-04 16:14:00 +01:00
Théophane Hufschmitt eaa6c26189 Merge pull request #10088 from edolstra/flake-source-path
Use SourcePath for reading flake.{nix,lock}
2024-03-04 14:40:40 +01:00
Théophane Hufschmitt 7764edf0e4 Merge pull request #10078 from szlend/fix-macos-local-network-sandbox
Fix `__darwinAllowLocalNetworking` sandbox
2024-03-04 14:34:57 +01:00
Eelco Dolstra 438855952b Merge pull request #10122 from edolstra/arg-from-file
Add --arg-from-file and --arg-from-stdin
2024-03-04 13:29:58 +01:00
Eelco Dolstra cbfd211b39 Fix build 2024-03-04 12:49:32 +01:00
Eelco Dolstra e6b9432542 Add release note 2024-03-04 12:48:21 +01:00
Théophane Hufschmitt 8a9eec375c Merge pull request #10073 from olmokramer/nix-flake-update-multiple-inputs
Accept multiple inputs in `nix flake update`
2024-03-04 09:13:26 +01:00
Eelco Dolstra 75c5e1a5cc Merge pull request #10126 from lf-/jade/kill-ndebug
Ban building Nix with NDEBUG
2024-03-03 20:01:12 +01:00
Eelco Dolstra fa17ec6b3e Merge pull request #10130 from tweag/remove-autoconf-gen-files
Remove and gitignore the autoreconf generated files
2024-03-03 19:58:22 +01:00
Eelco Dolstra dfb05f3957 Merge pull request #10129 from NixOS/fix-docker-push
Fix the docker push job
2024-03-03 19:57:23 +01:00
Olmo Kramer b1ad729add Add test for nix flake update with multiple inputs 2024-03-03 13:59:39 +01:00
Jade Lovelace 2625e9fb0a Ban building Nix with NDEBUG
When reviewing old PRs, I found that #9997 adds some code to ensure one
particular assert is always present. But, removing asserts isn't
something we do in our own release builds either in the flake here or in
nixpkgs, and is plainly a bad idea that increases support burden,
especially if other distros make bad choices of build flags in their Nix
packaging.

For context, the assert macro in the C standard is defined to do nothing
if NDEBUG is set.

There is no way in our build system to set -DNDEBUG without manually
adding it to CFLAGS, so this is simply a configuration we do not use.
Let's ban it at compile time.

I put this preprocessor directive in src/libutil.cc because it is not
obvious where else to put it, and it seems like the most logical file
since you are not getting a usable nix without it.
2024-03-02 11:22:46 -08:00
Ivan Shapovalov c6f0407103 libutil/terminal: cache isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 8c1eeb4681 treewide: shouldANSI() -> isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov d9fc4bf5c5 treewide: replace usages of isatty(STDERR_FILENO) with shouldANSI() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 950b6401f9 libmain/progress-bar: try harder to avoid escape sequences if !isTTY 2024-03-02 16:31:51 +01:00
Théophane Hufschmitt 2f0bc6373c Don't fail if a flakeref directly points to the flake.nix
Just warn and redirect it to the parent directory
2024-03-02 10:34:20 +01:00
Théophane Hufschmitt 11a1dcc43b Properly fail on flakerefs that don't point to a directory
Directly fail if a flakeref points to something that isn't a directory
instead of falling back to the logic of trying to look up the hierarchy
to find a valid flake root.

Fix https://github.com/NixOS/nix/issues/9868
2024-03-02 10:24:20 +01:00
Théophane Hufschmitt f8dc9bc563 Remove and gitignore the autoreconf generated files
No need to have them checked-in since we require running `autoreconf`
when building, and these are regenerated by the `autoreconf` script.
2024-03-02 10:18:47 +01:00
Théophane Hufschmitt 089d91ed4c Fix the docker push job
After https://github.com/NixOS/nix/pull/10071, the CI was trying to push
ghcr.io/nixos/nix:master for backwards-compatibility, but the image was
not tagged as such, causing the job to fail.

Fix this.
2024-03-02 09:00:42 +01:00
Théophane Hufschmitt 75101396dc Merge pull request #10117 from med8bra/doc-xp-feature-issues
doc(xp-feature): add issue url in experimental feature details
2024-03-02 08:59:12 +01:00
med8bra 1bc89b588b doc(xp-feature): add issue url in experimental feature details 2024-03-02 03:51:12 +01:00
Eelco Dolstra ba9b6b29b7 Merge pull request #10111 from obsidiansystems/git-objects
Support symlinks properly with `git-hashing` experimental feature
2024-03-01 15:00:43 +01:00
Eelco Dolstra 8ce1f6800b Add --arg-from-stdin to read an argument from stdin 2024-03-01 14:39:42 +01:00
Eelco Dolstra 291b10c607 Add --arg-from-file for reading a string from a file 2024-03-01 14:35:27 +01:00
Eelco Dolstra d72ee91d07 Clean up --arg processing 2024-03-01 14:14:14 +01:00
Théophane Hufschmitt cd9baa1809 Add release notes 2024-03-01 09:39:50 +01:00
Théophane HufschmittandValentin Gagarin 65b79c52c6 Fix a typo in a test comment
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt c3878f510e Copy the output of fixed-output derivations before registering them
It is possible to exfiltrate a file descriptor out of the build sandbox
of FODs, and use it to modify the store path after it has been
registered.
To avoid that issue, don't register the output of the build, but a copy
of it (that will be free of any leaked file descriptor).
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt a55c6a0f47 Add a NixOS test for the sandbox escape
Test that we can't leverage abstract unix domain sockets to leak file
descriptors out of the sandbox and modify the path after it has been
registered.
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt bf48501194 Merge pull request #10112 from edolstra/fix-gcc12-warnings
Fix gcc 12 warnings
2024-02-29 15:55:59 +01:00
Eelco Dolstra b1586a6799 Merge pull request #10090 from bobvanderlinden/profile-error-install-twice
profile install: warn on installing package twice
2024-02-29 10:04:08 +01:00
Bob van der Linden 14adff1711 profile install: skip and warn on installing package twice 2024-02-29 08:29:38 +01:00
tomberek ffe67c86a8 Merge pull request #9915 from 9999years/evaluating-attribute-position
Add position information to `while evaluating the attribute` errors in the debugger
2024-02-28 18:11:07 -05:00
Eelco Dolstra 65bb12ba78 Fix gcc 12 warnings 2024-02-28 22:59:20 +01:00
Théophane Hufschmitt 90ec015d61 Merge pull request #10070 from ramboman/fix-proxy-installer
`install-multi-user.sh`: `_sudo`: add proxy variables to sudo
2024-02-28 22:00:15 +01:00
Jade Lovelace f6158ea53b finally.hh: include works by itself; mark as nodiscard 2024-02-28 11:40:32 -08:00
Robert Hensing bd21b4b134 Merge pull request #10110 from hercules-ci/bump-actions-docker
actions docker_push_image: Update nix 2.13.3 -> 2.20.3
2024-02-28 20:27:54 +01:00
Robert Hensing 4d769e7a76 actions docker_push_image: Update nix 2.13.3 -> 2.20.3 2024-02-28 19:54:17 +01:00
John Ericson bcb5f235f9 Support symlinks properly with git-hashing experimental feature
Before, they would not be written to a file `FileSystemObjectSink`
correctly.
2024-02-28 12:24:45 -05:00
Robert Hensing 587c7dcb2b Merge pull request #9546 from NixOS/nixos-23.11
Update to nixos-23.11
2024-02-28 17:51:16 +01:00
Théophane Hufschmitt 864fc85fc8 Merge pull request #10010 from 9999years/fix-9941
Fix "Failed tcsetattr(TCSADRAIN)" when `nix repl` is not a TTY
2024-02-28 14:25:58 +01:00
Théophane Hufschmitt f6142cd0d1 unset NIX_HARDENING_ENABLE in fast build mode
`NIX_HARDENING_ENABLE` causes `_FORTIFY_SOURCE` to be defined.
This isn't compatible with `-O0`, and the compiler will happily remind
us about it at every call, spamming the terminal with warnings and stack
traces.

We don't really care hardening in that case, so just disable it if we
pass `OPTIMIZE=0`.
2024-02-28 08:07:51 +01:00
Théophane Hufschmitt da90be789d Fix a too smart implicit cast
Apparently gcc is able to implicitly cast from `FileIngestionMethod` to
`ContentAddressMethod`, but clang isn't. So explicit the cast
2024-02-28 08:00:17 +01:00
Théophane Hufschmitt 6147d27afb Bump the required daemon version for the git hashing tests
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:11:22 +01:00
Robert Hensing 8dc4b41c7f flake.lock: Strip out treeHash. Too soon...
I hate this.
We should have it, but for now we can't.
2024-02-28 07:08:21 +01:00
Robert Hensing 945940f2ef nixpkgs: nixos-23.11-small -> release-23.11
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4dd376f7943c64b522224a548d9cab5627b4d9d6' (2024-02-26)
  → 'github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e' (2024-02-28)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt cf3ef060ff Disable the “static” darwin stdenvs
Don't evaluate, and probably not really useful (if at all)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt a0cb75d96f Disable bear on all the things with darwin as hostPlatform
Just `stdenv.isDarwin` isn't enough because it doesn't apply to the
build platform, which mean that cross packages building from darwin to
another platform will have `isDarwin` set to false.
Replace it by `stdenv.buildPlatform.isDarwin`.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt bbef03872b Bump the required daemon version for the impure-env test
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt 44f10f000a flake: Update to NixOS 23.11
About time :)

This required disabling `bear` on darwin as it's currently broken (fixed
on master, but not yet on 23.11).
2024-02-28 07:08:21 +01:00
John Ericson f489a6e42d Merge pull request #8918 from obsidiansystems/git-objects
Git object hashing in libstore
2024-02-27 19:02:22 -05:00
John EricsonandEelco Dolstra d4ad1fcf30 Avoid creating temporary store object for git over the wire
Instead, serialize as NAR and send that over, then rehash sever side.
This is alorithmically simpler, but comes at the cost of a newer
parameter to `Store::addToStoreFromDump`.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-02-27 11:27:34 -05:00
201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
John Ericson 04836c73e5 Merge nativeCheckInputs into nativeBuildInputs
They were getting skipped for the test-against checks.
2024-02-27 11:27:34 -05:00
Eelco Dolstra c3e9e3d0c3 Merge pull request #9767 from obsidiansystems/canon-path-split
Support Windows paths in `canonPath` and `absPath`
2024-02-27 17:12:29 +01:00
John Ericson 354ba27296 Merge pull request #10093 from NixOS/revert-10084-remove-dead-git-code
Revert "Remove dead Git code"
2024-02-27 09:54:25 -05:00
Eelco Dolstra 5b0d78ec73 Merge pull request #10091 from bobvanderlinden/default-package-name
Fix extraction of name for defaultPackage URLs
2024-02-27 15:14:20 +01:00
Bob van der Linden e5d9130a5b Fix extraction of name for defaultPackage URLs 2024-02-27 07:53:05 +01:00
Théophane Hufschmitt be0052b45f Revert "Remove dead Git code" 2024-02-27 06:39:30 +01:00
Bob van der Linden d28a240aa6 profile: extract getNameFromElement 2024-02-26 21:07:18 +01:00
Robert Hensing 4c7f0ef6ca Merge pull request #9847 from pennae/inherit-from-dedup
deduplicate inherit-from source expr work
2024-02-26 20:25:58 +01:00
Théophane Hufschmitt c10025d8ca Merge pull request #10084 from tweag/remove-dead-git-code
Remove dead Git code
2024-02-26 19:49:14 +01:00
pennaeandRobert Hensing f24e445bc0 add doc comment justifying ExprInheritFrom
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-26 19:07:08 +01:00
pennae 1cd87b7042 remove ExprAttrs::AttrDef::inherited
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.
2024-02-26 19:07:08 +01:00
pennae cefd0302b5 evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
2024-02-26 19:07:08 +01:00
Eelco Dolstra 598deb2b23 Use SourcePath for reading flake.{nix,lock}
Flakes still reside in the Nix store (so there shouldn't be any change
in behaviour), but they are now accessed via the rootFS
accessor. Since rootFS implements access checks, we no longer have to
worry about flake.{nix,lock} or their parents being symlinks that
escape from the flake.

Extracted from the lazy-trees branch.
2024-02-26 15:14:18 +01:00
John Ericson 62a13c8101 Merge pull request #10086 from ShamrockLee/hash-algorithm-rename
treewide: hash type -> hash algorithm
2024-02-26 08:37:09 -05:00
Théophane Hufschmitt cd2d8b6d4c Merge pull request #10085 from ShamrockLee/ignore-obsolete-testdir
.gitignore: ignore historical test binaries
2024-02-26 11:36:10 +01:00
Yueh-Shun Li fd47f76da9 treewide: hash type -> hash algorithm
"hash type" -> "hash algorithm" in all comments, documentation, and
messages.

ht -> ha, [Hh]ashType -> [HhashAlgo] for all local variables and
function arguments. No API change is made.

Continuation of 5334c9c792 and 837b889c41.
2024-02-26 18:09:06 +08:00
Théophane Hufschmitt 219705ff64 Remove dead code
Most of the code in `git.{cc,hh}` is dead, so get rid of it.
2024-02-26 11:07:47 +01:00
Yueh-Shun Li 8ac4542593 .gitignore: ignore historical test binaries
After commit 91b6833686 (" Move tests to separate directories, and
document"), previously-built test executables are now tracked by Git,
which is annoying for developers.

This patch add .gitignore rules to ignore the obsolete test directories
to solve such problem and enhance developer experience.
2024-02-26 18:05:07 +08:00
Théophane Hufschmitt 21282c3c20 Merge pull request #10074 from lf-/jade/ban-implicit-fallthrough
Warn on implicit switch case fallthrough
2024-02-26 10:48:07 +01:00
Eelco Dolstra 7d9ae5fd06 Merge pull request #10071 from zimbatm/ci-default-tag
ci default tag
2024-02-26 10:45:26 +01:00
Simon Žlender d60c3f7f7c Fix __darwinAllowLocalNetworking sandbox
The sandbox rule `(allow network* (local ip))` doesn't do what it
implies. Adding this rule permits all network traffic. We should be
matching on (remote ip "localhost:*")` instead.
2024-02-25 23:00:57 +01:00
Jade Lovelace a82aeedb5b Warn on implicit switch case fallthrough
This seems to have found one actual bug in fs-sink.cc: the symlink case
was falling into the regular file case, which can't possibly be
intentional, right?
2024-02-24 15:52:16 -08:00
Olmo Kramer 9f11b1b0c4 Accept multiple inputs in nix flake update 2024-02-24 20:58:44 +01:00
Johannes KirschbauerandValentin Gagarin d83008c3a7 documentation: clarify genericClosure (#10003)
* doc: clarify genericClosure documentation

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-02-24 12:34:53 +00:00
zimbatm 5598ce3e0f ci: fix docker default tag
Docker uses "latest" as the default label instead of "master".

This change will allow to docker run ghcr.io/nixos/nix without having to
specify the label.

It keeps the :master label on docker hub for back-compat.
2024-02-24 11:24:24 +01:00
ramboman 24fd7e2755 install-multi-user.sh: _sudo: add proxy variables to sudo 2024-02-24 01:00:13 +00:00
Robert Hensing 0b47783d0a Merge pull request #10066 from 9999years/print-all-frames
Do not skip any stack frames when `--show-trace` is given
2024-02-23 18:53:11 +01:00
Théophane Hufschmitt accae60e77 Merge pull request #10067 from ramboman/fix-proxy-nix
`nix`: Fix `haveInternet` to check for proxy
2024-02-23 11:06:36 +01:00
Théophane Hufschmitt bca737dcad c++-ize the proxy detection code
Just for consistency with the rest
2024-02-23 10:28:37 +01:00
ramboman d3bff699aa nix: Fix haveInternet to check for proxy 2024-02-23 01:05:25 -05:00
Rebecca Turner fe6408b5df Update snapshots 2024-02-22 17:58:55 -08:00
Rebecca Turner 91e89628fd Make addErrorTrace variadic 2024-02-22 17:18:27 -08:00
Rebecca Turner f05c13ecc2 Remove the concept of "skipped frames" 2024-02-22 17:14:55 -08:00
Rebecca Turner 040874e4db Print all stack frames 2024-02-22 17:14:33 -08:00
Théophane Hufschmitt 6a5210f48e Merge pull request #9815 from NixOS/nix-hash-path
`nix hash path`, text hashing for `nix store add`, and preparatory refactors
2024-02-22 17:15:34 +01:00
Eelco Dolstra f183eef4a0 Merge pull request #10060 from bobvanderlinden/pr-simple-profile-remove-suggestion
profile: suggest removal using profile entry name
2024-02-22 11:02:12 +01:00
Bob van der Linden 4ae5091716 nix profile: suggest removal using entry name
When a file conflict arises during a package install a suggestion is
made to remove the old entry. This was previously done using the
installable URLs of the old entry. These URLs are quite verbose and
often do not equal the URL of the existing entry.

This change uses the recently introduced profile entry name for the
suggestion, resulting in a simpler output.

The improvement is easily seen in the change to the functional test.
2024-02-21 21:58:36 +01:00
John Ericson 2080d89b87 Merge pull request #10038 from edolstra/tarball-git-cache
Use the Git cache for tarball flakes
2024-02-21 15:47:02 -05:00
Eelco Dolstra e391fc2101 Add comments 2024-02-21 21:15:28 +01:00
Théophane Hufschmitt 774e7ca584 Merge pull request #9914 from 9999years/debugger-on-trace
Enter debugger on `builtins.trace` with an option
2024-02-21 18:32:56 +01:00
Théophane Hufschmitt 36fa8b1bcc Merge pull request #9943 from 9999years/release-notes
Add release notes
2024-02-21 18:17:50 +01:00
John Ericson efd36b49e8 nix hash path, and preperatory refactors
- `nix store add` supports text hashing

  With functional test ensuring it matches `builtins.toFile`.

- Factored-out flags for both commands

- Move all common reusable flags to `libcmd`

  - They are not part of the *definition* of the CLI infra, just a usag
    of it.

  - The `libstore` flag couldn't go in `args.hh` in libutil anyways,
    would be awkward for it to live alone

- Shuffle around `Cmd*` hierarchy so flags for deprecated commands don't
  end up on the new ones
2024-02-21 12:11:25 -05:00
Rebecca Turner b111fba8cd Add documentation, rename to debugger-on-trace 2024-02-21 09:07:39 -08:00
Eelco Dolstra 3f5d7afe46 Merge pull request #10024 from edolstra/remove-locked-flag
Input: Replace 'locked' bool by isLocked() method
2024-02-21 16:19:15 +01:00
Eelco Dolstra 9ae665b9e1 Merge pull request #10054 from syvb/channel-unpack-count
Say how many channels were unpacked in nix-channel
2024-02-21 12:23:57 +01:00
Eelco Dolstra 09d76e512a GitArchiveInputScheme: Require a NAR hash 2024-02-21 12:08:18 +01:00
Eelco Dolstra ff4fa4dbd3 Merge pull request #10055 from GrahamDennis/gdennis/faster-flake-lock-parsing
Faster flake.lock parsing
2024-02-21 11:57:44 +01:00
Graham Dennis 7fd0de38c6 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.
2024-02-21 18:40:34 +11:00
syvb 60eeacc24a Say how many channels were unpacked 2024-02-20 19:17:18 -05:00
John Ericson a1b1070dbe Merge pull request #9916 from 9999years/quit-whole-program-from-debugger
`:quit` in the debugger should quit the whole program
2024-02-20 18:12:05 -05:00
Rebecca Turner 8e71883e3f Rename ProcessLineResult variants 2024-02-20 14:52:16 -08:00
Rebecca Turner 2a8fe9a938 :quit in the debugger should quit the whole program 2024-02-20 10:01:13 -08:00
Eelco Dolstra 071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
John Ericson 78e7c98b02 Merge pull request #10049 from edolstra/dont-send-experimental-features
Don't send settings that depend on disabled experimental features to the daemon
2024-02-20 09:53:08 -05:00
Eelco Dolstra 0acd783190 Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.
2024-02-20 15:28:42 +01:00
Théophane Hufschmitt d2c6a93bd5 Merge pull request #10044 from edolstra/empty-git-repos
Handle empty Git repositories / workdirs
2024-02-20 14:01:23 +01:00
Eelco Dolstra cabee98152 Tarball fetcher: Use the content-addressed Git cache
Backported from the lazy-trees branch.
2024-02-20 12:57:36 +01:00
Eelco Dolstra 930b9c8269 PosixSourceAccessor: Support roots that are not directories
We have to support this for `fetchTree { type = "file" }` (and
probably other types of trees that can have a non-directory at the
root, like NARs).
2024-02-20 12:51:15 +01:00
Eelco Dolstra b00f412f81 Remove bad.tar.xz check, since libarchive doesn't care 2024-02-20 12:51:15 +01:00
Eelco Dolstra db012d1e63 tests/functional/tarball.sh: Fix invalid file:// URLs 2024-02-20 12:51:15 +01:00
Théophane Hufschmitt 6f4bb1b584 Merge pull request #10043 from edolstra/fix-readonly-fetchToStore
fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:07:48 +01:00
Eelco Dolstra 7cb4d0c5b7 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:46:49 +01:00
Eelco Dolstra 9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
Eelco Dolstra d52d91fe7a AllowListInputAccessor: Clarify that the "allowed paths" are actually allowed prefixes
E.g. adding "/" will allow access to the root and *everything below it*.
2024-02-20 11:23:26 +01:00
Eelco Dolstra 6162105675 Don't say "copying X to the store" in read-only mode 2024-02-20 10:47:30 +01:00
John Ericson 319ec6f84a Support Windows paths in canonPath and absPath
`canonPath` and `absPath` work on native paths, and so should switch
between supporting Unix paths and Windows paths accordingly.

The templating is because `CanonPath`, which shares the implementation,
should always be Unix style. It is the pure "nix-native" path type for
virtual file operations --- it is part of Nix's "business logic", and
should not vary with the host OS accordingly.
2024-02-16 10:31:36 -05:00
John Ericson 4531585275 Factor out the Unix-specific parts of canonPathInner
This prepares the code to also support Windows paths in the next commit.
2024-02-16 10:12:07 -05:00
John Ericson 60936f28e5 Merge pull request #9881 from obsidiansystems/purify-canon-path
Purify `CanonPath`
2024-02-16 10:11:11 -05:00
d17e1d9737 Purify CanonPath
The core `CanonPath` constructors were using `absPath`, but `absPath` in
some situations does IO which is not appropriate. It turns out that
these constructors avoided those situations, and thus were pure, but it
was far from obvious this was the case.

To remedy the situation, abstract the core algorithm from `canonPath` to
use separately in `CanonPath` without any IO. No we know by-construction
that those constructors are pure.

That leaves `CanonPath::fromCWD` as the only operation which uses IO /
is impure. Add docs on it, and `CanonPath` as a whole, explaining the
situation.

This is also necessary to support Windows paths on windows without
messing up `CanonPath`. But, I think it is good even without that.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-16 09:47:25 -05:00
John Ericson d53c8901ef Merge pull request #9985 from alois31/symlink-resolution
Restore `builtins.pathExists` behavior on broken symlinks
2024-02-16 09:24:03 -05:00
John Ericson e27b7e04bf Add note about this being a temp solution 2024-02-16 08:45:15 -05:00
Eelco Dolstra 06be819b89 Merge pull request #9485 from edolstra/tarball-cache
Add a Git-based content-addressed tarball cache
2024-02-15 22:37:57 +01:00
Eelco Dolstra 78b8db72b5 Remove tarball-cache.{hh,cc}
TarballInfo is only used in github.cc, and getTarballCache() is a bit
too trivial to have its own file.
2024-02-15 21:58:08 +01:00
Eelco Dolstra 9b04ce16fc Merge branch 'tarball-cache' of github.com:Ericson2314/nix into tarball-cache 2024-02-15 21:48:10 +01:00
Eelco Dolstra 4b40fc5b46 Merge remote-tracking branch 'origin/master' into tarball-cache 2024-02-15 21:46:36 +01:00
Eelco Dolstra 27b8d5d920 Merge pull request #10013 from edolstra/libgit2-lookup
GitInputAccessor: Speed up lookup()
2024-02-15 17:30:22 +01:00
John Ericson ed24baaec4 Finish separating concerns with tarball cache
There is no longer an `importTarball` method. Instead, there is a
`unpackTarfileToSink` function (back in libutil). The caller can use
thisw with the `getParseSink` method we added in the last commit easily
enough.

In addition, tarball cache functionality is separated from `git-utils`
and moved into `tarball-cache`. This ensures we are separating mechanism
and policy.
2024-02-15 10:34:01 -05:00
John Ericson ba6a5f06ee Split GitRepoImpl::importTarball
There is now a separation of:

1. A `FileSystemObjectSink` for writing to git repos

2. Adapting libarchive to use that parse sink.

The prepares a proper separation of concerns.
2024-02-15 10:27:54 -05:00
Théophane Hufschmitt bc9471d032 Merge pull request #9966 from NixOS/mention-OPTIMIZE-in-hacking-guide
Mention `OPTIMIZE=0` more prominently in the hacking guide
2024-02-15 09:33:15 +01:00
Eelco Dolstra 67a6d34448 GitInputAccessor: Speed up lookup()
A command like

  rm -rf ~/.cache/nix/tarball-cache/ ~/.cache/nix/fetcher-cache-v1.sqlite*; nix flake metadata 'git+file:///home/eelco/Dev/nixpkgs?rev=9463103069725474698139ab10f17a9d125da859'

was spending about 84% of its runtime in lookup(), specifically in
git_tree_entry_bypath(). (The reading of blobs is less than 3%.)

It appears libgit2 doesn't do a lot of caching of trees, so we now
make sure that when we look up a path, we add all its parents, and all
the immediate children of the parents (since we have them in memory
anyway), to our own cache.

This speed up the command above from 17.2s to 7.8s on my machine.

Fixes (or at least should improve a lot) #9684.
2024-02-14 19:25:23 +01:00
Rebecca Turner 6d2b446e2b Add release notes for "Pretty print values in nix repl" 2024-02-14 08:58:54 -08:00
Eelco Dolstra 54354eaecf Merge remote-tracking branch 'origin/master' into tarball-cache 2024-02-14 14:45:19 +01:00
Théophane Hufschmitt d857914e1a Merge pull request #9931 from 9999years/pretty-printer
Pretty-print values in the REPL
2024-02-14 13:32:58 +01:00
John Ericson ee9f6d022f Merge pull request #10001 from abathur/fix_macos_daemon_perms
install-darwin: fix symbolic perms for install cmd
2024-02-13 17:39:11 -05:00
John Ericson ef0b82a267 Merge pull request #9986 from nbelakovski/patch-1
Add a note about lists values.md
2024-02-13 17:10:32 -05:00
Eelco Dolstra 2b946aad8e Merge pull request #10006 from edolstra/fix-impure-fetchurl
<nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 22:50:55 +01:00
Rebecca Turner a694cfb7bd Fix "Failed tcsetattr(TCSADRAIN)" when nix repl is not a TTY
Before:
```
$ echo builtins.nixVersion | nix repl
Welcome to Nix 2.18.1. Type :? for help.

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
"2.18.1"

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
```

After:
```
$ echo builtins.nixVersion | nix repl
Nix 2.21.0pre20240131_dirty
Type :? for help.
"2.21.0pre20240131_dirty"
```
2024-02-13 11:09:12 -08:00
John Ericson 5b26c66a8c Merge pull request #10009 from obsidiansystems/ca-type-names
Proper `parse` and `render` functions for `FileIngestionMethod` and `ContentAddressMethod`
2024-02-13 13:28:52 -05:00
John Ericson 9d64613dca Update src/libutil/file-content-address.cc 2024-02-13 12:50:10 -05:00
Alois Wohlschlager 89e21ab4bd Restore builtins.pathExists behavior on broken symlinks
Commit 83c067c0fa changed `builtins.pathExists`
to resolve symlinks before checking for existence. Consequently, if the path
refers to a symlink itself, existence of the target of the symlink (instead of
the symlink itself) was checked. Restore the previous behavior by skipping
symlink resolution in the last component.
2024-02-13 18:09:55 +01:00
John Ericson db41a0616a Use ContentAddressMethod::render in one more place
Good to deduplicate the code.
2024-02-13 11:14:49 -05:00
Eelco Dolstra 4254cdabf2 Merge pull request #10008 from edolstra/config-status-dep
*.in files: Depend on config.status
2024-02-13 16:49:41 +01:00
John Ericson f49efb32fb Merge pull request #10005 from fricklerhandwerk/local-store-glossary
update glossary entry on store types
2024-02-13 10:45:36 -05:00
John Ericson 41dd9857c7 Proper parse and render functions for FileIngestionMethod and ContentAddressMethod
No outward facing behavior is changed.

Older methods with same names that operate on on method + algo pair (for
old-style `<method>:algo`) are renamed to `*WithAlgo`.)

The functions are unit-tested in the same way the names for the hash
algorithms are tested.
2024-02-13 10:30:16 -05:00
Valentin Gagarin f27205f743 redirect local and chroot store to main page 2024-02-13 16:25:07 +01:00
John Ericson fb5a438dca Merge pull request #10007 from NixOS/split-test
Split `hash.sh` test in two
2024-02-13 10:22:12 -05:00
Eelco Dolstra bb21892351 *.in files: Depend on config.status 2024-02-13 16:10:56 +01:00
Valentin Gagarin 5b69409f6b only refer to the local store page 2024-02-13 16:08:44 +01:00
John Ericson 8bebf9607c Split hash.sh test in two
Converting hashes and hashing files are pretty separate tasks, and more
test parallelism is better.
2024-02-13 09:44:14 -05:00
Eelco Dolstra b439b1fc66 Merge pull request #9993 from edolstra/builtin-paths
builtin:{unpack-channel,buildenv}: Get output path from the derivation
2024-02-13 14:57:13 +01:00
Valentin Gagarin 39c353f6fa reword description of the cores setting (#9522)
* reword description of the `cores` setting

- be precise about the `builder` executable
- clearly distinguish between `builder` and job parallelism
- clarify the role of `mkDerivation` in the example
- remove prose for the default, it's shown programmatically
- mention relation to `max-jobs`
2024-02-13 13:52:51 +00:00
Eelco Dolstra bb63bd50e6 <nix/fetchurl.nix>: Restore support for "impure = true" 2024-02-13 14:14:20 +01:00
Valentin GagarinandJanik H fd82ba0985 extract reference documentation on remote builds (#9526)
- move all reference documentation to the `builders` configuration setting
- reword documentation on machine specification, add examples
- disable showing the default value, as it rendered as `@/dummy/machines`, which is wrong
- highlight the examples
- link to the configuration docs for distributed builds
- builder -> build machine

Co-authored-by: Janik H <janik@aq0.de>
2024-02-13 14:13:56 +01:00
Valentin Gagarin e37d502895 add instructions to wipe the substituter lookup cache (#9498)
* add instructions to wipe the substituter lookup cache
2024-02-13 13:34:51 +01:00
Valentin Gagarin ce19338f9f update glossary entry on store types
the interesting information is on the proper pages, and is now presented
a bit more prominently.

the paragraph was a bit confusing to read, also because an anchor link
to an inline definition was in the middle of the sentence. "local store"
now has its own glossary entry.
2024-02-13 13:30:17 +01:00
Théophane Hufschmitt 9d87f25673 Merge pull request #9997 from pg83/patch-1
Update nar-info-disk-cache.cc
2024-02-13 08:55:57 +01:00
Travis A. Everett 64a076fe06 install-darwin: fix symbolic perms for install cmd
The symbolic form in use here doesn't seem to have an effect
in either the BSD or coreutils install commands, leaving the
daemon plist with empty permissions. This seems to cause its
own problems.

I think I've got the right symbolic syntax now :)
2024-02-13 01:18:08 -06:00
Anton Samokhvalov 64cbd4c05a Update nar-info-disk-cache.cc
fix case when asserts are no-op, like in release build
2024-02-12 23:37:40 +03:00
Robert Hensing bdb6f56c90 Merge pull request #9295 from NixOS/store-path-complete-construction
Include store path exact spec in the docs
2024-02-12 19:02:03 +01:00
John Ericson f29d2a9d11 Small EBNF fix 2024-02-12 12:30:28 -05:00
John Ericson 898fd1e48d Update doc/manual/src/protocols/store-path.md 2024-02-12 12:04:37 -05:00
John EricsonandRobert Hensing 5169f5f4d9 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 12:01:54 -05:00
John EricsonandRobert Hensing c873a140d7 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 11:34:54 -05:00
John Ericson 4c3e4d6d71 Sections, EBNF tweaks 2024-02-12 11:33:06 -05:00
John Ericson 0862d7ce57 Move around non-terminals 2024-02-12 11:26:08 -05:00
John Ericson eb76b35efa Merge remote-tracking branch 'upstream/master' into store-path-complete-construction 2024-02-12 11:22:54 -05:00
John Ericson 30f6b0f9c5 pre -> fingerprint in store path grammar
As suggested by @roberth in
https://github.com/NixOS/nix/pull/9295#discussion_r1486402040.

Thanks!
2024-02-12 11:22:08 -05:00
John Ericson 95190e68ed Mention the use of Extended Backus–Naur form 2024-02-12 11:21:54 -05:00
John Ericson ac1301ddfd Convert store path "grammar" to EBNF 2024-02-12 11:16:53 -05:00
John EricsonandRobert Hensing 91557df4a7 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 11:16:12 -05:00
Eelco Dolstra a9b69b2fff builtin:{unpack-channel,buildenv}: Get output path from the derivation
Similar to 1ee42c5b88, get the "out"
path from the derivation (and complain if it doesn't exist), rather
than getting it from the environment.
2024-02-12 16:34:59 +01:00
Eelco Dolstra c4ebb82da4 Merge pull request #9992 from edolstra/fix-warning
Fix "may be used uninitialized" warning
2024-02-12 16:02:21 +01:00
Eelco Dolstra 619ca631d0 Fix "may be used uninitialized" warning 2024-02-12 15:29:48 +01:00
Eelco Dolstra d3c1997127 Merge pull request #9976 from alois31/restore-manual-pages
Restore manual pages
2024-02-12 15:21:47 +01:00
Théophane Hufschmitt e352c66161 Merge pull request #9983 from BOHverkill/fix-string-interpolation-link
Fix link to derivation in string interpolation doc
2024-02-12 14:26:03 +01:00
Eelco Dolstra ec6ca6e42c Merge pull request #9948 from obsidiansystems/no-canon-path-from-cwd
Get rid of `CanonPath::fromCwd`
2024-02-12 14:04:01 +01:00
pennae ecf8b12d60 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
2024-02-12 13:58:29 +01:00
pennae 6c08fba533 use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
2024-02-12 13:35:00 +01:00
pennae 1f542adb3e add ExprAttrs::AttrDef::chooseByKind
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.
2024-02-12 13:34:59 +01:00
pennae c66ee57edc preserve information about whether/how an attribute was inherited 2024-02-12 13:32:33 +01:00
pennae 73065a400d add test for inherit expr printing 2024-02-12 13:32:33 +01:00
pennae 8669c02468 add test for inherit-from semantics 2024-02-12 13:32:33 +01:00
nbelakovski 4496a4537b Update values.md
Link to elemAt
2024-02-11 22:52:49 -08:00
John Ericson 9d182e71fb Merge pull request #9984 from BOHverkill/fix-manual-link
Fix link to manual in CONTRIBUTING.md
2024-02-11 16:13:06 -05:00
nbelakovski f298159a2b Add a note about lists values.md
There's probably more that can be said, but I thought it might be helpful to put something here about how to access elements of a list for folks coming from more or less any other programming language. If this is rarely used, it might be nice to add to the documentation something about why it's rarely used.
2024-02-10 18:52:39 -08:00
BOHverkill fae8c15737 Fix link to manual in CONTRIBUTING.md 2024-02-10 17:44:33 +01:00
BOHverkill 53eecae525 Fix link to derivation in string interpolation doc
The reference link definition for it pointing to the glossary was
removed, so it is currently not displayed as a link.
2024-02-10 17:27:16 +01:00
Alois Wohlschlager 8f3253c6f4 Restore manual pages
Commit d536c57e87 inadvertedly broke build and
installation of all non-autogenerated manual pages (in particular, all the ones
documenting the stable CLI), by moving the definition of the man-pages variable
in doc/manual/local.mk after its usage in mk/lib.mk. Move including the former
earlier so that the correct order is restored.
2024-02-09 19:07:59 +01:00
Théophane Hufschmitt 28a988bdde Merge pull request #9974 from edolstra/less-chatty-post-build-hook
runPostBuildHook(): Be less chatty
2024-02-09 16:39:02 +01:00
Eelco Dolstra fb5a792280 runPostBuildHook(): Be less chatty
Don't spam the user with "running post-build-hook" messages. It's up
to the post-build hook if it has something interesting to say.
2024-02-09 15:55:24 +01:00
Eelco Dolstra 4487ebca66 Merge pull request #9973 from fricklerhandwerk/add-links-gc-confusion
add clickable anchor links
2024-02-09 13:41:18 +01:00
Eelco Dolstra 57ebcadb2f Merge pull request #9972 from NixOS/allow-unoptimized-builds
Don't hardcode the `-O2` compiler flag
2024-02-09 13:25:51 +01:00
Valentin Gagarin 60045f9c96 add clickable anchor links
how the different invocations relate to each other seems be
confusing, which is relatable because one has to wire it up in your head
while reading. an explicit reference should make it unambiguous and
easier to notice due to links being highlighted.
2024-02-09 10:41:03 +01:00
Théophane Hufschmitt 1fe7b01669 Don't hardcode the -O2 compiler flag
autoconf authors apparently decided that setting `-O2` by default was a good idea. I disagree, and Nix has its own way of deciding that (with `OPTIMIZE={0,1}`). Explicitly set `CFLAGS` and `CXXFLAGS` in the configure script to disable that behaviour.

Fix #9965
2024-02-09 06:27:24 +01:00
Théophane Hufschmitt 47a1dbb4b8 Merge pull request #9925 from 9999years/fmt-cleanup
Cleanup `fmt.hh`
2024-02-09 06:01:03 +01:00
Rebecca Turner 953eb0cba2 Fix tests 2024-02-08 15:55:20 -08:00
Rebecca Turner 4440eb54e7 Add release note 2024-02-08 12:07:38 -08:00
Rebecca Turner 8b7eb7400b Enter debugger on builtins.trace with an option 2024-02-08 12:07:37 -08:00
Rebecca Turner c0e7f50c1a Rename hintfmt to HintFmt 2024-02-08 11:58:25 -08:00
Rebecca Turner 149bd63afb Cleanup fmt.hh
When I started contributing to Nix, I found the mix of definitions and
names in `fmt.hh` to be rather confusing, especially the small
difference between `hintfmt` and `hintformat`. I've renamed many classes
and added documentation to most definitions.

- `formatHelper` is no longer exported.
- `fmt`'s documentation is now with `fmt` rather than (misleadingly)
  above `formatHelper`.
- `yellowtxt` is renamed to `Magenta`.

  `yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING`
  has been equal to `ANSI_MAGENTA` for a long time. Now the name is
  updated.
- `normaltxt` is renamed to `Uncolored`.
- `hintfmt` has been merged into `hintformat` as extra constructor
  functions.
- `hintformat` has been renamed to `hintfmt`.
- The single-argument `hintformat(std::string)` constructor has been
  renamed to a static member `hintformat::interpolate` to avoid pitfalls
  with using user-generated strings as format strings.
2024-02-08 11:51:03 -08:00
Théophane Hufschmitt 1ba9780cf5 Merge pull request #9834 from 9999years/structured-errors
Towards structured error classes
2024-02-08 20:00:25 +01:00
Rebecca Turner 403c90ddf5 Extract printSpace helper 2024-02-08 10:18:12 -08:00
Rebecca Turner 1c5f5d4291 prettyPrint -> shouldPrettyPrint 2024-02-08 10:18:10 -08:00
Rebecca TurnerandThéophane Hufschmitt a27651908f Add assertion for decreasing the indent
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-02-08 10:11:45 -08:00
Rebecca Turner 0f1269243b Revert "Add release notes for "Stack traces are more compact""
This reverts commit b2868acbdc.
2024-02-08 10:09:47 -08:00
Rebecca Turner b2868acbdc Add release notes for "Stack traces are more compact" 2024-02-08 10:08:48 -08:00
Rebecca Turner 7f8960d0f2 Add release notes for "Stack size is increased on macOS" 2024-02-08 10:08:48 -08:00
Rebecca Turner 837c350bcd Add release notes for "Cycle detection in nix repl is simpler and more reliable" 2024-02-08 10:08:48 -08:00
Rebecca Turner 4f0d43a397 Add release notes for "nix repl now respects Ctrl-C while printing values" 2024-02-08 10:08:48 -08:00
Rebecca Turner 24cdb81bb0 Add release notes for "Nix commands respect Ctrl-C" 2024-02-08 10:08:48 -08:00
Rebecca Turner abb5fef355 Add release notes for "Functions are printed with more detail" 2024-02-08 10:08:48 -08:00
Rebecca Turner 9a5d52262f Add release notes for "Nix no longer attempts to git add files that are .gitignored" 2024-02-08 10:08:48 -08:00
Rebecca Turner 754c35abfb Add release notes for "Debugger prints source position information" 2024-02-08 10:08:48 -08:00
John Ericson 4687beecef Get rid of CanonPath::fromCwd
As discussed in the last Nix team meeting (2024-02-95), this method
doesn't belong because `CanonPath` is a virtual/ideal absolute path
format, not used in file systems beyond the native OS format for which a
"current working directory" is defined.

Progress towards #9205
2024-02-08 11:01:41 -05:00
Eelco Dolstra 6563a58294 Merge pull request #9964 from fricklerhandwerk/merge-format-sections
move JSON section into Formats and Protocols
2024-02-08 16:53:59 +01:00
Théophane Hufschmitt a8050d9b83 Merge pull request #9928 from 9999years/error-messages-in-nix-repl
Improve error printing in `nix repl`
2024-02-08 16:21:13 +01:00
Théophane Hufschmitt 96686a2a29 Merge pull request #9963 from fricklerhandwerk/fold-sidebar
manual: fold sidebar sections
2024-02-08 16:18:27 +01:00
John Ericson f2f54cf087 Merge pull request #9863 from obsidiansystems/perl-open-other-store
Support arbitrary stores in Perl bindings
2024-02-08 09:30:18 -05:00
Théophane Hufschmitt 46a0625a40 Merge pull request #9929 from 9999years/dont-print-values-in-magenta
Don't print the first bracket in values in magenta in error messages
2024-02-08 10:56:18 +01:00
Théophane Hufschmitt 9b8b486091 Merge pull request #9933 from pennae/debugger-fix
fix debugger crashing while printing envs
2024-02-08 10:48:02 +01:00
Théophane Hufschmitt acef4f17a2 Merge pull request #9918 from 9999years/debugger-locals-for-let-expressions
Expose locals from `let` expressions to the debugger
2024-02-08 10:17:55 +01:00
Théophane Hufschmitt c4ed92fa6f Merge pull request #9917 from 9999years/enter-debugger-more-reliably
Enter debugger more reliably in `let` expressions and function calls
2024-02-08 10:09:54 +01:00
Théophane Hufschmitt fb78a99e04 Merge pull request #9924 from 9999years/rename-yellowtxt
Rename `yellowtxt` -> `magentatxt`
2024-02-08 10:01:30 +01:00
Théophane Hufschmitt 0dea16e686 Mention OPTIMIZE=0 more prominently in the hacking guide
This is a game-changer when developing, it shouldn't just be hidden amongst a list of more advanced variables.
2024-02-08 09:50:12 +01:00
Théophane Hufschmitt f388a6148d Merge pull request #9919 from 9999years/reduce-debugger-clutter
Reduce visual clutter in the debugger
2024-02-08 09:42:38 +01:00
Théophane Hufschmitt d24c8aa491 Simplify a conditional in the repl initialisation 2024-02-08 09:22:30 +01:00
Valentin Gagarin e486b76eef move JSON section into Formats and Protocols 2024-02-08 09:13:58 +01:00
Valentin Gagarin 140de3b278 manual: fold sidebar sections
the table of contents is very long now, and folded sections allow
for a better overview.
2024-02-08 09:00:00 +01:00
John Ericson bc08502249 Support arbitrary stores in Perl bindings
Fix #9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
2024-02-07 22:31:58 -05:00
Eelco Dolstra 09a6e8e703 Merge pull request #9949 from fricklerhandwerk/fix-redirects-file
fix location of `_redirects` file
2024-02-07 11:34:35 +01:00
Rebecca Turner 9723f533d8 Add comment 2024-02-06 16:50:47 -08:00
Rebecca Turner 474fc4078a Add comments 2024-02-06 16:49:28 -08:00
Valentin Gagarin 2d74b56aee fix location of _redirects file
the Netlify `_redirects` file must be in the root directory [0] of the
files to serve, and mdBook copies all the files in `src` that aren't
`.md` to the output directory [1].

[0]: https://docs.netlify.com/routing/redirects/
[1]: https://rust-lang.github.io/mdBook/guide/creating.html#source-files
2024-02-06 23:22:34 +01:00
Rebecca Turner c0a15fb7d0 Pretty-print values in the REPL
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.

Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```

After:
```
{
  attrs = {
    a = {
      b = {
        c = { };
      };
    };
  };
  list = [ 1 ];
  list' = [
    1
    2
    3
  ];
}
```
2024-02-05 13:23:38 -08:00
Rebecca Turner 657a607812 Add release note 2024-02-05 13:21:08 -08:00
Rebecca Turner 155bc761f6 Add release note 2024-02-05 13:16:39 -08:00
Rebecca Turner b63a8d7c46 Add release note 2024-02-05 13:15:29 -08:00
Rebecca Turner 601fc7d159 Add release note 2024-02-05 13:13:26 -08:00
Rebecca Turner 24205a8703 Add release note 2024-02-05 13:00:39 -08:00
Robert Hensing a31f2cb0cd Merge pull request #9939 from edolstra/slash-operator
CanonPath, SourcePath: Change operator + to /
2024-02-05 15:55:27 +01:00
Eelco Dolstra a6737b7e17 CanonPath, SourcePath: Change operator + to /
This is less confusing and makes it more similar to std::filesystem::path.
2024-02-05 15:17:39 +01:00
Eelco Dolstra c291d2d8dd Merge pull request #9927 from 9999years/catch-error-in-value-printer
Catch `Error`, not `BaseError` in `ValuePrinter`
2024-02-05 15:01:21 +01:00
Eelco Dolstra 6ec08b85f6 Merge pull request #9934 from nmeum/absPath-out-of-bounds
absPath: Explicitly check if path is empty before accessing it
2024-02-05 14:44:02 +01:00
Eelco Dolstra e190c20c33 Merge pull request #9926 from 9999years/fix-cycle-detection-in-nix-repl
Fix cycle detection in `nix repl`
2024-02-05 14:24:28 +01:00
Valentin Gagarin 8d4890c3f8 catch multiple use of link reference (#9937) 2024-02-04 21:45:10 +00:00
Valentin Gagarin 8b873edcca fix anchor link; less weird link texts (#9936) 2024-02-04 21:15:20 +00:00
Valentin Gagarin 721fddac2f use the right heading level (#9935) 2024-02-04 21:03:13 +00:00
Théophane Hufschmitt 25385a408e Merge pull request #9930 from rvl/print-dev-env-unbound-variables
print-dev-env: Avoid using unbound shellHook variable
2024-02-04 20:55:17 +01:00
pennae 5ccb06ee1b fix debugger crashing while printing envs
fixes #9932
2024-02-04 17:12:04 +01:00
Rodney Lorrimar e1131b5927 print-dev-env: Avoid using unbound shellHook variable
Some tools which consume the "nix print-dev-env" rc script (such as
"nix-direnv") are sensitive to the use of unbound variables. They use
"set -u".

The "nix print-dev-env" rc script initially unsets "shellHook", then
loads variables from the derivation, and then evaluates "shellHook".
However, most derivations don't have a "shellHook" attribute.

So users get the error "shellHook: unbound variable". This can be
demonstrated with the command:

    nix print-dev-env nixpkgs#hello | bash -u

This commit changes the rc script to provide an empty fallback value
for the "shellHook" variable.

Closes: #7951 #8253
2024-02-04 13:57:13 +08:00
Rebecca Turner 770d2bc779 Key repeated values on attribute binding pointers, not value pointers
Closes #8672
2024-02-03 21:23:06 -08:00
Rebecca Turner 9646d62b0c Don't print values in magenta
This fixes the opening bracket of lists/attrsets being printed in
magenta, unlike the closing bracket.

https://github.com/NixOS/nix/pull/9753#issuecomment-1904616088
2024-02-03 21:17:22 -08:00
Rebecca Turner c5d525cd84 Print error messages but not traces
This makes output of values that include errors much cleaner.

Before:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error:
       … while calling the 'throw' builtin
         at «string»:1:9:
            1| { err = builtins.throw "uh oh!"; }
             |         ^

       error: uh oh!»; }
```

After:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error: uh oh!»; }
```

But if the whole expression throws an error, source locations and (if
applicable) a stack trace are printed, like you'd expect:

```
nix-repl> builtins.throw "uh oh!"
error:
       … while calling the 'throw' builtin
         at «string»:1:1:
            1| builtins.throw "uh oh!"
             | ^

       error: uh oh!
```
2024-02-03 20:50:16 -08:00
Rebecca Turner a7927abdc1 Catch Error, not BaseError in ValuePrinter
`BaseError` includes `Interrupt`. We probably don't want the value
printer to tell you you pressed Ctrl-C while it was working.
2024-02-03 19:29:07 -08:00
Rebecca Turner a7939a6c2a Rename yellowtxt -> magentatxt
`yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING` has
been equal to `ANSI_MAGENTA` for a long time. Now the name is updated.
2024-02-03 19:28:11 -08:00
Sören Tempel ec5cc1026d absPath: Explicitly check if path is empty before accessing it
It is entirely possible for the path to be an empty string and many
unit tests actually pass it as an empty string (e.g. both_roundrip or
turnsEmptyPathIntoCWD). In this case, without this patch, absPath will
perform a one-byte out-of-bounds access.

This was discovered while enabling the nix test suite on Alpine where
we compile all software with `-D_GLIBCXX_ASSERTIONS=1`, thus resulting
in a test failure on Alpine.
2024-02-04 00:47:47 +01:00
Eelco Dolstra 49cf090cb2 Merge pull request #9913 from 9999years/debugger-positions
Print positions in `--debugger`, instead of pointers
2024-02-03 09:27:09 +01:00
Rebecca Turner 6414cd259e Reduce visual clutter in the debugger 2024-02-02 19:58:35 -08:00
Rebecca Turner 36dfac7560 Expose locals from let expressions to the debugger 2024-02-02 19:31:18 -08:00
Rebecca Turner 0127d54d5e Enter debugger more reliably in let expressions and calls 2024-02-02 19:14:22 -08:00
Rebecca Turner 016db2d10f Add position information to while evaluating the attribute 2024-02-02 17:49:54 -08:00
Rebecca Turner 7d7483cafc Print positions in --debugger, instead of pointers 2024-02-02 17:38:46 -08:00
Eelco Dolstra 081dc5daa1 Merge pull request #9902 from NixOS/require-fixed-output-fetchurl
builtin:fetchurl: Ensure a fixed-output derivation
2024-02-02 14:00:34 +01:00
Eelco Dolstra e67458e5b8 Better test fix 2024-02-02 13:34:56 +01:00
Eelco Dolstra 05535be03a Fix test 2024-02-02 13:07:08 +01:00
Rebecca Turner faaccecbc8 Remove EXCEPTION_NEEDS_THROW_SPEC
We're on C++ 20 now, we don't need this
2024-02-01 16:39:38 -08:00
Rebecca Turner 87dc4bc7d1 Attach positions to errors in derivationStrict 2024-02-01 16:39:38 -08:00
Rebecca Turner c6a89c1a16 libexpr: Support structured error classes
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of

    EvalError("expected 'boolean' but found '%1%'", showType(v))

we could write

    TypeError(v, "boolean")

or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.

This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).

The core design is in `eval-error.hh`. Generally, errors like this:

    state.error("'%s' is not a string", getAttrPathStr())
      .debugThrow<TypeError>()

are transformed like this:

    state.error<TypeError>("'%s' is not a string", getAttrPathStr())
      .debugThrow()

The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
2024-02-01 16:39:38 -08:00
Rebecca Turner c62c21e29a Move PodIdx to pos-idx.hh and PosTable to pos-table.hh 2024-02-01 13:12:59 -08:00
Eelco Dolstra b8b739e484 builtin:fetchurl: Get output hash info from the drv 2024-02-01 22:01:02 +01:00
Eelco Dolstra 1ee42c5b88 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 21:46:01 +01:00
Eelco Dolstra ef6d055ace Merge pull request #9896 from hercules-ci/tidy-tidy
Disable a slightly annoying clang-tidy check
2024-02-01 09:02:46 +01:00
Robert Hensing 58c26dd0f0 Add .clang-tidy 2024-02-01 01:01:39 +01:00
Robert Hensing 0f2e9e6bd2 Typo 2024-02-01 01:01:04 +01:00
Robert Hensing 4072a8fea0 Merge pull request #9867 from hercules-ci/issue-912
#912 allow leading period
2024-01-31 19:10:59 +01:00
Robert Hensing 7c9ed1b1a3 Merge pull request #9890 from obsidiansystems/remove-unneeded-toRealPath
Change an `allowPath` call to take a store path again
2024-01-31 18:45:19 +01:00
Robert Hensing f1b4663805 Disallow store path names that are . or .. (plus opt. -)
As discussed in the maintainer meeting on 2024-01-29.

Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.

What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
2024-01-31 18:35:19 +01:00
Robert Hensing 8406da2877 test: Generate distinct hashes
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing 69bbd5852a test: Generate distinct path names
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing b13e6a76b4 parseStorePath: Support leading period 2024-01-31 18:35:19 +01:00
John Ericson caea7dcb7e Change an allowPath call to take a store path again
This looks like a revert of #5844, but is not.

That one was needed because
https://github.com/NixOS/nix/commit/d90f9d4b9994dc1f15b9d664ae313f06261d6058#diff-0f59bb6f197822ef9f19ceae9624989499d170c84dfdc1f486a8959bb4588cafR85
changed the type of the argument to `allowPath` from a `StorePath` to a
`Path`. But since
https://github.com/NixOS/nix/commit/caabc4f64889d5a4c47d6102b3aa1d3c80bbc107#diff-0f59bb6f197822ef9f19ceae9624989499d170c84dfdc1f486a8959bb4588cafL100-R92,
it is a `StorePath` again.

I think this is worth changing because we want to be very careful about
`toRealPath` and the evaluator --- ideally the choice of real path does
not affect evaluation at all. So using it fewer times is better.
2024-01-31 11:44:06 -05:00
John Ericson 75ebb90a70 Merge pull request #9884 from edolstra/resolve-symlinks
Resolve symlinks in a few more places
2024-01-30 10:42:54 -05:00
Eelco Dolstra d661a89398 Merge pull request #9880 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.4.1
build(deps): bump zeebe-io/backport-action from 2.4.0 to 2.4.1
2024-01-30 15:37:25 +01:00
Eelco Dolstra b36ff47e7c Resolve symlinks in a few more places
Fixes #9882.
2024-01-30 15:35:31 +01:00
Eelco Dolstra a7db7b6e44 Merge pull request #9879 from NixOS/bump-2.21.0
Bump version
2024-01-30 10:13:42 +01:00
dependabot[bot] a3aae7beef build(deps): bump zeebe-io/backport-action from 2.4.0 to 2.4.1
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-29 22:14:10 +00:00
Eelco Dolstra 2f3fb6c12e Bump version 2024-01-29 22:57:25 +01:00
Robert Hensing 9ddd0f2af8 Revert "StorePath: reject names starting with '.'"
This reverts commit 24bda0c7b3.
2024-01-27 11:18:03 +01:00
John Ericson a34ec0bd12 Include store path exact spec in the docs
This is niche, but deserves to be in the manual because it is describing
behavior visible to the outside world, not mere implementation details.
2024-01-18 22:19:14 -05:00
Eelco Dolstra cf02b3335c Merge remote-tracking branch 'origin/master' into tarball-cache 2023-12-22 16:45:01 +01:00
John Ericson e7d95a1d80 Merge remote-tracking branch 'upstream/master' into tarball-cache 2023-12-15 09:37:13 -05:00
Eelco Dolstra 06e106beff Disable GitHub tree hash mismatch warning 2023-12-14 13:38:10 +01:00
Eelco Dolstra 1fce12ec32 Merge remote-tracking branch 'origin/master' into tarball-cache 2023-12-14 13:31:29 +01:00
Eelco Dolstra 043413bb59 boost::span -> std::span 2023-11-29 12:38:46 +01:00
Eelco Dolstra b36857ac8d Add a Git-based content-addressed tarball cache
GitArchiveInputScheme now streams tarballs into a Git repository. This
deduplicates data a lot, e.g. when you're fetching different revisions
of the Nixpkgs repo. It also warns if the tree hash returned by GitHub
doesn't match the tree hash of the imported tarball.
2023-11-29 12:37:03 +01:00
349 changed files with 8376 additions and 7427 deletions

No files matched your search

+3
View File
@@ -0,0 +1,3 @@
# We use pointers to aggregates in a couple of places, intentionally.
# void * would look weird.
Checks: '-bugprone-sizeof-expression'
+1 -1
View File
@@ -14,4 +14,4 @@
src/libexpr/primops.cc @roberth
# Libstore layer
/src/libstore @thufschmitt
/src/libstore @thufschmitt @ericson2314
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v2.4.0
uses: zeebe-io/backport-action@v2.4.1
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
+18 -7
View File
@@ -20,7 +20,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
@@ -62,9 +62,9 @@ jobs:
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v14
with:
name: '${{ env.CACHIX_NAME }}'
@@ -84,7 +84,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
@@ -114,9 +114,9 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v25
- uses: cachix/install-nix-action@v26
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v14
@@ -153,6 +153,17 @@ jobs:
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag nix:$NIX_VERSION $IMAGE_ID:$NIX_VERSION
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker tag nix:$NIX_VERSION $IMAGE_ID:latest
docker push $IMAGE_ID:$NIX_VERSION
docker push $IMAGE_ID:latest
# deprecated 2024-02-24
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes
+6
View File
@@ -10,6 +10,7 @@ perl/Makefile.config
/stamp-h1
/svn-revision
/libtool
/config/config.*
# /doc/manual/
/doc/manual/*.1
@@ -45,13 +46,16 @@ perl/Makefile.config
/src/libexpr/parser-tab.hh
/src/libexpr/parser-tab.output
/src/libexpr/nix.tbl
/src/libexpr/tests
/tests/unit/libexpr/libnixexpr-tests
# /src/libstore/
*.gen.*
/src/libstore/tests
/tests/unit/libstore/libnixstore-tests
# /src/libutil/
/src/libutil/tests
/tests/unit/libutil/libnixutil-tests
/src/nix/nix
@@ -94,6 +98,7 @@ perl/Makefile.config
/tests/functional/ca/config.nix
/tests/functional/dyn-drv/config.nix
/tests/functional/repl-result-out
/tests/functional/debugger-test-out
/tests/functional/test-libstoreconsumer/test-libstoreconsumer
# /tests/functional/lang/
@@ -137,6 +142,7 @@ GTAGS
# auto-generated compilation database
compile_commands.json
*.compile_commands.json
nix-rust/target
+1 -1
View File
@@ -1 +1 @@
2.20.0
2.22.0
+2 -2
View File
@@ -63,11 +63,11 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
- Functional tests – [`tests/functional/**.sh`](./tests/functional)
- Unit tests – [`src/*/tests`](./src/)
- Integration tests – [`tests/nixos/*`](./tests/nixos)
- [ ] User documentation in the [manual](..doc/manual/src)
- [ ] User documentation in the [manual](./doc/manual/src)
- [ ] API documentation in header files
- [ ] Code and comments are self-explanatory
- [ ] Commit message explains **why** the change was made
- [ ] New feature or incompatible change: updated [release notes](./doc/manual/src/release-notes/rl-next.md)
- [ ] New feature or incompatible change: [add a release note](https://nixos.org/manual/nix/stable/contributing/hacking#add-a-release-note)
7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams).
+17 -12
View File
@@ -42,11 +42,23 @@ ifeq ($(ENABLE_FUNCTIONAL_TESTS), yes)
makefiles += \
tests/functional/local.mk \
tests/functional/ca/local.mk \
tests/functional/git-hashing/local.mk \
tests/functional/dyn-drv/local.mk \
tests/functional/test-libstoreconsumer/local.mk \
tests/functional/plugins/local.mk
endif
# Some makefiles require access to built programs and must be included late.
makefiles-late =
ifeq ($(ENABLE_DOC_GEN), yes)
makefiles-late += doc/manual/local.mk
endif
ifeq ($(ENABLE_INTERNAL_API_DOCS), yes)
makefiles-late += doc/internal-api/local.mk
endif
# Miscellaneous global Flags
OPTIMIZE = 1
@@ -56,6 +68,7 @@ ifeq ($(OPTIMIZE), 1)
GLOBAL_LDFLAGS += $(CXXLTO)
else
GLOBAL_CXXFLAGS += -O0 -U_FORTIFY_SOURCE
unexport NIX_HARDENING_ENABLE
endif
include mk/platform.mk
@@ -70,7 +83,7 @@ ifdef HOST_WINDOWS
GLOBAL_LDFLAGS += -Wl,--export-all-symbols
endif
GLOBAL_CXXFLAGS += -g -Wall -include $(buildprefix)config.h -std=c++2a -I src
GLOBAL_CXXFLAGS += -g -Wall -Wimplicit-fallthrough -include $(buildprefix)config.h -std=c++2a -I src
# Include the main lib, causing rules to be defined
@@ -95,24 +108,16 @@ installcheck:
@exit 1
endif
# Documentation or else fallback stub rules.
#
# The documentation makefiles be included after `mk/lib.mk` so rules
# refer to variables defined by `mk/lib.mk`. Rules are not "lazy" like
# variables, unfortunately.
# Documentation fallback stub rules.
ifeq ($(ENABLE_DOC_GEN), yes)
$(eval $(call include-sub-makefile, doc/manual/local.mk))
else
ifneq ($(ENABLE_DOC_GEN), yes)
.PHONY: manual-html manpages
manual-html manpages:
@echo "Generated docs are disabled. Configure without '--disable-doc-gen', or avoid calling 'make manpages' and 'make manual-html'."
@exit 1
endif
ifeq ($(ENABLE_INTERNAL_API_DOCS), yes)
$(eval $(call include-sub-makefile, doc/internal-api/local.mk))
else
ifneq ($(ENABLE_INTERNAL_API_DOCS), yes)
.PHONY: internal-api-html
internal-api-html:
@echo "Internal API docs are disabled. Configure with '--enable-internal-api-docs', or avoid calling 'make internal-api-html'."
-1700
View File
File diff suppressed because it is too large. Load diff
-1860
View File
File diff suppressed because it is too large. Load diff
+4
View File
@@ -47,6 +47,10 @@ AC_DEFINE_UNQUOTED(SYSTEM, ["$system"], [platform identifier ('cpu-os')])
# State should be stored in /nix/var, unless the user overrides it explicitly.
test "$localstatedir" = '${prefix}/var' && localstatedir=/nix/var
# Assign a default value to C{,XX}FLAGS as the default configure script sets them
# to -O2 otherwise, which we don't want to have hardcoded
CFLAGS=${CFLAGS-""}
CXXFLAGS=${CXXFLAGS-""}
AC_PROG_CC
AC_PROG_CXX
+2
View File
@@ -6,6 +6,8 @@ additional-css = ["custom.css"]
additional-js = ["redirects.js"]
edit-url-template = "https://github.com/NixOS/nix/tree/master/doc/manual/{path}"
git-repository-url = "https://github.com/NixOS/nix"
fold.enable = true
fold.level = 1
[preprocessor.anchors]
renderers = ["html"]
+10
View File
@@ -175,6 +175,16 @@ $(d)/src/SUMMARY-rl-next.md: $(d)/src/release-notes/rl-next.md
# Generate the HTML manual.
.PHONY: manual-html
manual-html: $(docdir)/manual/index.html
# Open the built HTML manual in the default browser.
manual-html-open: $(docdir)/manual/index.html
@echo " OPEN " $<; \
xdg-open $< \
|| open $< \
|| { \
echo "Could not open the manual in a browser. Please open '$<'" >&2; \
false; \
}
install: $(docdir)/manual/index.html
# Generate 'nix' manpages.
+11 -7
View File
@@ -14,15 +14,15 @@
const redirects = {
"index.html": {
"part-advanced-topics": "advanced-topics/advanced-topics.html",
"part-advanced-topics": "advanced-topics/index.html",
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"chap-diff-hook": "advanced-topics/diff-hook.html",
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"chap-distributed-builds": "advanced-topics/distributed-builds.html",
"chap-distributed-builds": "command-ref/conf-file.html#conf-builders",
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"chap-writing-nix-expressions": "language/index.html",
"part-command-ref": "command-ref/command-ref.html",
"part-command-ref": "command-ref/index.html",
"conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
@@ -261,7 +261,7 @@ const redirects = {
"sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"chap-installation": "installation/installation.html",
"chap-installation": "installation/index.html",
"ch-installing-binary": "installation/installing-binary.html",
"sect-macos-installation": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
@@ -288,7 +288,7 @@ const redirects = {
"ssec-copy-closure": "package-management/copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"chap-package-management": "package-management/index.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
@@ -297,7 +297,7 @@ const redirects = {
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",
"sec-relnotes": "release-notes/release-notes.html",
"sec-relnotes": "release-notes/index.html",
"ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"ch-relnotes-0.10": "release-notes/rl-0.10.html",
"ssec-relnotes-0.11": "release-notes/rl-0.11.html",
@@ -358,7 +358,11 @@ const redirects = {
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
"characterization-testing": "#characterisation-testing-unit",
}
},
"glossary.html": {
"gloss-local-store": "store/types/local-store.html",
"gloss-chroot-store": "store/types/local-store.html",
},
};
// the following code matches the current page's URL against the set of redirects.
@@ -0,0 +1,13 @@
---
synopsis: "`nix eval` prints derivations as `.drv` paths"
prs: 10200
---
`nix eval` will now print derivations as their `.drv` paths, rather than as
attribute sets. This makes commands like `nix eval nixpkgs#bash` terminate
instead of infinitely looping into recursive self-referential attributes:
```ShellSession
$ nix eval nixpkgs#bash
«derivation /nix/store/m32cbgbd598f4w299g0hwyv7gbw6rqcg-bash-5.2p26.drv»
```
+8
View File
@@ -0,0 +1,8 @@
---
synopsis: Remove experimental repl-flake
significance: significant
issues: 10103
prs: 10299
---
The `repl-flake` experimental feature has been removed. The `nix repl` command now works like the rest of the new CLI in that `nix repl {path}` now tries to load a flake at `{path}` (or fails if the `flakes` experimental feature isn't enabled).*
+6 -4
View File
@@ -104,11 +104,12 @@
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [JSON Formats](json/index.md)
- [Store Object Info](json/store-object-info.md)
- [Derivation](json/derivation.md)
- [Protocols](protocols/index.md)
- [Formats and Protocols](protocols/index.md)
- [JSON Formats](protocols/json/index.md)
- [Store Object Info](protocols/json/store-object-info.md)
- [Derivation](protocols/json/derivation.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Store Path Specification](protocols/store-path.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
- [Glossary](glossary.md)
- [Contributing](contributing/index.md)
@@ -120,6 +121,7 @@
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/index.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.21 (2024-03-11)](release-notes/rl-2.21.md)
- [Release 2.20 (2024-01-29)](release-notes/rl-2.20.md)
- [Release 2.19 (2023-11-17)](release-notes/rl-2.19.md)
- [Release 2.18 (2023-09-20)](release-notes/rl-2.18.md)
@@ -36,5 +36,6 @@
/package-management/s3-substituter /store/types/s3-binary-cache-store 301!
/protocols/protocols /protocols 301!
/json/* /protocols/json/:splat 301!
/release-notes/release-notes /release-notes 301!
@@ -36,16 +36,8 @@ error: cannot connect to 'mac'
then you need to ensure that the `PATH` of non-interactive login shells
contains Nix.
> **Warning**
>
> If you are building via the Nix daemon, it is the Nix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
The list of remote machines can be specified on the command line or in
the Nix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file.
For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine:
```console
$ uname
@@ -60,97 +52,20 @@ $ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
It is possible to specify multiple build machines separated by a semicolon or a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname`, e.g. `ssh://nix@mac` or `ssh://mac`.
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Nix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Nix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Nix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machine’s mandatory
features appear in the derivation’s `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g.
Remote build machines can also be configured in [`nix.conf`](@docroot@/command-ref/conf-file.md), e.g.
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
Finally, remote builders can be configured in a separate configuration
file included in `builders` via the syntax `@file`. For example,
Finally, remote build machines can be configured in a separate configuration
file included in `builders` via the syntax `@/path/to/file`. For example,
builders = @/etc/nix/machines
causes the list of machines in `/etc/nix/machines` to be included. (This
is the default.)
If you want the builders to use caches, you likely want to set the
option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`.
causes the list of machines in `/etc/nix/machines` to be included.
(This is the default.)
@@ -51,7 +51,7 @@ These options are for deleting old [profiles] prior to deleting unreachable [sto
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`\
Delete all old generations of profiles.
This is the equivalent of invoking `nix-env --delete-generations old` on each found profile.
This is the equivalent of invoking [`nix-env --delete-generations old`](@docroot@/command-ref/nix-env/delete-generations.md#generations-old) on each found profile.
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*\
Delete all generations of profiles older than the specified amount (except for the generations that were active at that point in time).
@@ -12,13 +12,13 @@ This operation deletes the specified generations of the current profile.
*generations* can be a one of the following:
- <span id="generations-list">`<number>...`</span>:\
- <span id="generations-list">[`<number>...`](#generations-list)</span>:\
A list of generation numbers, each one a separate command-line argument.
Delete exactly the profile generations given by their generation number.
Deleting the current generation is not allowed.
- The special value <span id="generations-old">`old`</span>
- <span id="generations-old">[The special value `old`](#generations-old)</span>
Delete all generations except the current one.
@@ -30,7 +30,7 @@ This operation deletes the specified generations of the current profile.
> Because one can roll back to a previous generation, it is possible to have generations newer than the current one.
> They will also be deleted.
- <span id="generations-time">`<number>d`</span>:\
- <span id="generations-time">[`<number>d`](#generations-time)</span>:\
The last *number* days
*Example*: `30d`
@@ -38,7 +38,7 @@ This operation deletes the specified generations of the current profile.
Delete all generations created more than *number* days ago, except the most recent one of them.
This allows rolling back to generations that were available within the specified period.
- <span id="generations-count">`+<number>`</span>:\
- <span id="generations-count">[`+<number>`](#generations-count)</span>:\
The last *number* generations up to the present
*Example*: `+5`
+1 -3
View File
@@ -27,11 +27,9 @@ and open `./result-doc/share/doc/nix/manual/index.html`.
To build the manual incrementally, [enter the development shell](./hacking.md) and run:
```console
make manual-html -j $NIX_BUILD_CORES
make manual-html-open -j $NIX_BUILD_CORES
```
and open `./outputs/out/share/doc/nix/manual/language/index.html`.
In order to reflect changes to the [Makefile for the manual], clear all generated files before re-building:
[Makefile for the manual]: https://github.com/NixOS/nix/blob/master/doc/manual/local.mk
+6 -6
View File
@@ -44,13 +44,13 @@ To build Nix itself in this shell:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make -j $NIX_BUILD_CORES
[nix-shell]$ make -j $NIX_BUILD_CORES OPTIMIZE=0
```
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ make install
[nix-shell]$ make install OPTIMIZE=0
[nix-shell]$ make installcheck check -j $NIX_BUILD_CORES
[nix-shell]$ nix --version
nix (Nix) 2.12
@@ -147,10 +147,10 @@ Nix can be built for various platforms, as specified in [`flake.nix`]:
In order to build Nix for a different platform than the one you're currently
on, you need a way for your current Nix installation to build code for that
platform. Common solutions include [remote builders] and [binary format emulation]
platform. Common solutions include [remote build machines] and [binary format emulation]
(only supported on NixOS).
[remote builders]: ../advanced-topics/distributed-builds.md
[remote builders]: @docroot@/language/derivations.md#attr-builder
[binary format emulation]: https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems
Given such a setup, executing the build only requires selecting the respective attribute.
@@ -258,10 +258,10 @@ See [supported compilation environments](#compilation-environments) and instruct
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make clean && bear -- make -j$NIX_BUILD_CORES default check install
make compile_commands.json
```
Configure your editor to use the `clangd` from the shell, either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
Configure your editor to use the `clangd` from the `.#native-clangStdenvPackages` shell. You can do that either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
> **Note**
>
+7 -17
View File
@@ -37,7 +37,7 @@
This can be achieved by:
- Fetching a pre-built [store object] from a [substituter]
- Running the [`builder`](@docroot@/language/derivations.md#attr-builder) executable as specified in the corresponding [derivation]
- Delegating to a [remote builder](@docroot@/advanced-topics/distributed-builds.html) and retrieving the outputs
- Delegating to a [remote machine](@docroot@/command-ref/conf-file.md#conf-builders) and retrieving the outputs
<!-- TODO: link [running] to build process page, #8888 -->
See [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md) for a detailed description of the algorithm.
@@ -59,23 +59,12 @@
- [store]{#gloss-store}
A collection of store objects, with operations to manipulate that collection.
See [Nix store](./store/index.md) for details.
A collection of [store objects][store object], with operations to manipulate that collection.
See [Nix Store](./store/index.md) for details.
There are many types of stores.
See [`nix help-stores`](@docroot@/command-ref/new-cli/nix3-help-stores.md) for a complete list.
From the perspective of the location where Nix is invoked, the Nix store can be referred to _local_ or _remote_.
Only a [local store]{#gloss-local-store} exposes a location in the file system of the machine where Nix is invoked that allows access to store objects, typically `/nix/store`.
Local stores can be used for building [derivations](#gloss-derivation).
See [Local Store](@docroot@/command-ref/new-cli/nix3-help-stores.md#local-store) for details.
There are many types of stores, see [Store Types](./store/types/index.md) for details.
[store]: #gloss-store
[local store]: #gloss-local-store
- [chroot store]{#gloss-chroot-store}
A [local store] whose canonical path is anything other than `/nix/store`.
- [binary cache]{#gloss-binary-cache}
@@ -87,7 +76,7 @@
- [store path]{#gloss-store-path}
The location of a [store object](@docroot@/store/index.md#store-object) in the file system, i.e., an immediate child of the Nix store directory.
The location of a [store object] in the file system, i.e., an immediate child of the Nix store directory.
> **Example**
>
@@ -97,7 +86,7 @@
[store path]: #gloss-store-path
- [file system object]{#gloss-store-object}
- [file system object]{#gloss-file-system-object}
The Nix data model for representing simplified file system data.
@@ -243,6 +232,7 @@
- All paths in the store path's [closure] are valid.
[validity]: #gloss-validity
[local store]: @docroot@/store/types/local-store.md
- [user environment]{#gloss-user-env}
+2 -2
View File
@@ -16,7 +16,7 @@ nix (Nix) 2.18.1
> Writing to the [local store](@docroot@/store/types/local-store.md) with a newer version of Nix, for example by building derivations with [`nix-build`](@docroot@/command-ref/nix-build.md) or [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md), may change the database schema!
> Reverting to an older version of Nix may therefore require purging the store database before it can be used.
### Linux multi-user
## Linux multi-user
```console
$ sudo su
@@ -28,7 +28,7 @@ $ sudo su
## macOS multi-user
```console
$ sudo nix-env --install --file '<nixpkgs>' --attr nix -I nixpkgs=channel:nixpkgs-unstable
$ sudo nix-env --install --file '<nixpkgs>' --attr nix cacert -I nixpkgs=channel:nixpkgs-unstable
$ sudo launchctl remove org.nixos.nix-daemon
$ sudo launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist
```
@@ -188,9 +188,13 @@ Derivations can declare some infrequently used optional attributes.
}
```
The `outputHashAlgo` attribute specifies the hash algorithm used to
compute the hash. It can currently be `"sha1"`, `"sha256"` or
`"sha512"`.
The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
The "nix32" encoding is an adaptation of base-32 encoding.
The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings, and the [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
It can currently be `"sha1"`, `"sha256"`, `"sha512"`, or `null`.
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
The `outputHashMode` attribute determines how the hash is computed.
It must be one of the following two values:
@@ -209,11 +213,6 @@ Derivations can declare some infrequently used optional attributes.
this case, the output can be anything, including a directory
tree.
The `outputHash` attribute, finally, must be a string containing
the hash in either hexadecimal or base-32 notation. (See the
[`nix-hash` command](../command-ref/nix-hash.md) for information
about converting to and from base-32 notation.)
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
> **Warning**
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
@@ -257,7 +256,7 @@ Derivations can declare some infrequently used optional attributes.
of the environment (typically, a few hundred kilobyte).
- [`preferLocalBuild`]{#adv-attr-preferLocalBuild}\
If this attribute is set to `true` and [distributed building is enabled](../advanced-topics/distributed-builds.md), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
If this attribute is set to `true` and [distributed building is enabled](@docroot@/command-ref/conf-file.md#conf-builders), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
This is useful for derivations that are cheapest to build locally.
- [`allowSubstitutes`]{#adv-attr-allowSubstitutes}\
+1 -1
View File
@@ -36,7 +36,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
A necessary condition for Nix to build derivations locally is that the `system` attribute matches the current [`system` configuration option].
It can automatically [build on other platforms](../advanced-topics/distributed-builds.md) by forwarding build requests to other machines.
It can automatically [build on other platforms](@docroot@/language/derivations.md#attr-builder) by forwarding build requests to other machines.
[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
@@ -1,6 +1,8 @@
# Import From Derivation
The value of a Nix expression can depend on the contents of a [store object](@docroot@/glossary.md#gloss-store-object).
The value of a Nix expression can depend on the contents of a [store object].
[store object]: @docroot@/glossary.md#gloss-store-object
Passing an expression `expr` that evaluates to a [store path](@docroot@/glossary.md#gloss-store-path) to any built-in function which reads from the filesystem constitutes Import From Derivation (IFD):
+33 -1
View File
@@ -1,7 +1,13 @@
# Nix Language
The Nix language is designed for conveniently creating and composing *derivations* – precise descriptions of how contents of existing files are used to derive new files.
It is:
> **Tip**
>
> These pages are written as a reference.
> If you are learning Nix, nix.dev has a good [introduction to the Nix language](https://nix.dev/tutorials/nix-language).
The language is:
- *domain-specific*
@@ -432,6 +438,32 @@ This is an incomplete overview of language features, by example.
</td>
</tr>
<tr>
<td>
`inherit pkgs src;`
</td>
<td>
Adds the variables to the current scope (attribute set or `let` binding).
Desugars to `pkgs = pkgs; src = src;`
</td>
</tr>
<tr>
<td>
`inherit (pkgs) lib stdenv;`
</td>
<td>
Adds the attributes, from the attribute set in parentheses, to the current scope (attribute set or `let` binding).
Desugars to `lib = pkgs.lib; stdenv = pkgs.stdenv;`
</td>
</tr>
<tr>
<td>
+8 -8
View File
@@ -84,7 +84,7 @@ The `+` operator is overloaded to also work on strings and paths.
>
> *string* `+` *string*
Concatenate two [string]s and merge their string contexts.
Concatenate two [strings][string] and merge their string contexts.
[String concatenation]: #string-concatenation
@@ -94,7 +94,7 @@ Concatenate two [string]s and merge their string contexts.
>
> *path* `+` *path*
Concatenate two [path]s.
Concatenate two [paths][path].
The result is a path.
[Path concatenation]: #path-concatenation
@@ -150,9 +150,9 @@ If an attribute name is present in both, the attribute value from the latter is
Comparison is
- [arithmetic] for [number]s
- lexicographic for [string]s and [path]s
- item-wise lexicographic for [list]s:
- [arithmetic] for [numbers][number]
- lexicographic for [strings][string] and [paths][path]
- item-wise lexicographic for [lists][list]:
elements at the same index in both lists are compared according to their type and skipped if they are equal.
All comparison operators are implemented in terms of `<`, and the following equivalencies hold:
@@ -163,12 +163,12 @@ All comparison operators are implemented in terms of `<`, and the following equi
| *a* `>` *b* | *b* `<` *a* |
| *a* `>=` *b* | `! (` *a* `<` *b* `)` |
[Comparison]: #comparison-operators
[Comparison]: #comparison
## Equality
- [Attribute sets][attribute set] and [list]s are compared recursively, and therefore are fully evaluated.
- Comparison of [function]s always returns `false`.
- [Attribute sets][attribute set] and [lists][list] are compared recursively, and therefore are fully evaluated.
- Comparison of [functions][function] always returns `false`.
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
@@ -20,6 +20,8 @@ Rather than writing
(where `freetype` is a [derivation]), you can instead write
[derivation]: ../glossary.md#gloss-derivation
```nix
"--with-freetype2-library=${freetype}/lib"
```
+2
View File
@@ -156,6 +156,8 @@ function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
Elements in a list can be accessed using [`builtins.elemAt`](./builtins.md#builtins-elemAt).
## Attribute Set
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
@@ -14,11 +14,11 @@ Info about a [store object].
* `narHash`:
Hash of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).
Hash of the [file system object] part of the store object when serialized as a [Nix Archive].
* `narSize`:
Size of the [file system object] part of the store object when serialized as a [Nix Archive](#gloss-nar).
Size of the [file system object] part of the store object when serialized as a [Nix Archive].
* `references`:
@@ -30,6 +30,7 @@ Info about a [store object].
[store path]: @docroot@/glossary.md#gloss-store-path
[file system object]: @docroot@/store/file-system-object.md
[Nix Archive]: @docroot@/glossary.md#gloss-nar
## Impure fields
+131
View File
@@ -0,0 +1,131 @@
# Complete Store Path Calculation
This is the complete specification for how store paths are calculated.
The format of this specification is close to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), but must deviate for a few things such as hash functions which we treat as bidirectional for specification purposes.
Regular users do *not* need to know this information --- store paths can be treated as black boxes computed from the properties of the store objects they refer to.
But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful.
## Store path proper
```ebnf
store-path = store-dir "/" digest "-" name
```
where
- `name` = the name of the store object.
- `store-dir` = the [store directory](@docroot@/store/store-path.md#store-directory)
- `digest` = base-32 representation of the first 160 bits of a [SHA-256] hash of `fingerprint`
This the hash part of the store name
## Fingerprint
- ```ebnf
fingerprint = type ":" sha256 ":" inner-digest ":" store ":" name
```
Note that it includes the location of the store as well as the name to make sure that changes to either of those are reflected in the hash
(e.g. you won't get `/nix/store/<digest>-name1` and `/nix/store/<digest>-name2`, or `/gnu/store/<digest>-name1`, with equal hash parts).
- `type` = one of:
- ```ebnf
| "text" ( ":" store-path )*
```
for encoded derivations written to the store.
The optional trailing store paths are the references of the store object.
- ```ebnf
| "source" ( ":" store-path )*
```
For paths copied to the store and hashed via a [Nix Archive (NAR)] and [SHA-256][sha-256].
Just like in the text case, we can have the store objects referenced by their paths.
Additionally, we can have an optional `:self` label to denote self reference.
- ```ebnf
| "output:" id
```
For either the outputs built from derivations,
paths copied to the store hashed that area single file hashed directly, or the via a hash algorithm other than [SHA-256][sha-256].
(in that case "source" is used; this is only necessary for compatibility).
`id` is the name of the output (usually, "out").
For content-addressed store objects, `id`, is always "out".
- `inner-digest` = base-16 representation of a SHA-256 hash of `inner-fingerprint`
## Inner fingerprint
- `inner-fingerprint` = one of the following based on `type`:
- if `type` = `"text:" ...`:
the string written to the resulting store path.
- if `type` = `"source:" ...`:
the the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
- if `type` = `"output:" id`:
- For input-addressed derivation outputs:
the [ATerm](@docroot@/protocols/derivation-aterm.md) serialization of the derivation modulo fixed output derivations.
- For content-addressed store paths:
```ebnf
"fixed:out:" rec algo ":" hash ":"
```
where
- `rec` = one of:
- ```ebnf
| ""
```
(empty string) for hashes of the flat (single file) serialization
- ```ebnf
| "r:"
```
hashes of the for [Nix Archive (NAR)] (arbitrary file system object) serialization
- ```ebnf
| "git:"
```
hashes of the [Git blob/tree](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) [Merkel tree](https://en.wikipedia.org/wiki/Merkle_tree) format
- ```ebnf
algo = "md5" | "sha1" | "sha256"
```
- `hash` = base-16 representation of the path or flat hash of the contents of the path (or expected contents of the path for fixed-output derivations).
Note that `id` = `"out"`, regardless of the name part of the store path.
Also note that NAR + SHA-256 must not use this case, and instead must use the `type` = `"source:" ...` case.
[Nix Archive (NAR)]: @docroot@/glossary.md#gloss-NAR
[sha-256]: https://en.m.wikipedia.org/wiki/SHA-256
### Historical Note
The `type` = `"source:" ...` and `type` = `"output:out"` grammars technically overlap in purpose,
in that both can represent data hashed by its SHA-256 NAR serialization.
The original reason for this way of computing names was to prevent name collisions (for security).
For instance, the thinking was that it shouldn't be feasible to come up with a derivation whose output path collides with the path for a copied source.
The former would have an `inner-fingerprint` starting with `output:out:`, while the latter would have an `inner-fingerprint` starting with `source:`.
Since `64519cfd657d024ae6e2bb74cb21ad21b886fd2a` (2008), however, it was decided that separating derivation-produced vs manually-hashed content-addressed data like this was not useful.
Now, data that is content-addressed with SHA-256 + NAR-serialization always uses the `source:...` construction, regardless of how it was produced (manually or by derivation).
This allows freely switching between using [fixed-output derivations](@docroot@/glossary.md#gloss-fixed-output-derivation) for fetching, and fetching out-of-band and then manually adding.
It also removes the ambiguity from the grammar.
+1 -1
View File
@@ -34,7 +34,7 @@ For more in-depth information you are kindly referred to subsequent chapters.
lolcat: command not found
```
1. Search for more packages on <search.nixos.org> to try them out.
1. Search for more packages on [search.nixos.org](https://search.nixos.org/) to try them out.
1. Free up storage space:
+33
View File
@@ -167,3 +167,36 @@
error: expected a set but found an integer
```
- Functions are printed with more detail [#7145](https://github.com/NixOS/nix/issues/7145) [#9606](https://github.com/NixOS/nix/pull/9606)
`nix repl`, `nix eval`, `builtins.trace`, and most other places values are
printed will now include function names and source location information:
```
$ nix repl nixpkgs
nix-repl> builtins.map
«primop map»
nix-repl> builtins.map lib.id
«partially applied primop map»
nix-repl> builtins.trace lib.id "my-value"
trace: «lambda id @ /nix/store/8rrzq23h2zq7sv5l2vhw44kls5w0f654-source/lib/trivial.nix:26:5»
"my-value"
```
- Flake operations like `nix develop` will no longer fail when run in a Git
repository where the `flake.lock` file is `.gitignore`d
[#8854](https://github.com/NixOS/nix/issues/8854)
[#9324](https://github.com/NixOS/nix/pull/9324)
- Nix commands will now respect Ctrl-C
[#7145](https://github.com/NixOS/nix/issues/7145)
[#6995](https://github.com/NixOS/nix/pull/6995)
[#9687](https://github.com/NixOS/nix/pull/9687)
Previously, many Nix commands would hang indefinitely if Ctrl-C was pressed
while performing various operations (including `nix develop`, `nix flake
update`, and so on). With several fixes to Nix's signal handlers, Nix
commands will now exit quickly after Ctrl-C is pressed.
+302
View File
@@ -0,0 +1,302 @@
# Release 2.21.0 (2024-03-11)
- Fix a fixed-output derivation sandbox escape (CVE-2024-27297)
Cooperating Nix derivations could send file descriptors to files in the Nix
store to each other via Unix domain sockets in the abstract namespace. This
allowed one derivation to modify the output of the other derivation, after Nix
has registered the path as "valid" and immutable in the Nix database.
In particular, this allowed the output of fixed-output derivations to be
modified from their expected content.
This isn't the case any more.
- CLI options `--arg-from-file` and `--arg-from-stdin` [#10122](https://github.com/NixOS/nix/pull/10122)
The new CLI option `--arg-from-file` *name* *path* passes the contents
of file *path* as a string value via the function argument *name* to a
Nix expression. Similarly, the new option `--arg-from-stdin` *name*
reads the contents of the string from standard input.
- Concise error printing in `nix repl` [#9928](https://github.com/NixOS/nix/pull/9928)
Previously, if an element of a list or attribute set threw an error while
evaluating, `nix repl` would print the entire error (including source location
information) inline. This output was clumsy and difficult to parse:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error:
… while calling the 'throw' builtin
at «string»:1:9:
1| { err = builtins.throw "uh oh!"; }
| ^
error: uh oh!»; }
```
Now, only the error message is displayed, making the output much more readable.
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error: uh oh!»; }
```
However, if the whole expression being evaluated throws an error, source
locations and (if applicable) a stack trace are printed, just like you'd expect:
```
nix-repl> builtins.throw "uh oh!"
error:
… while calling the 'throw' builtin
at «string»:1:1:
1| builtins.throw "uh oh!"
| ^
error: uh oh!
```
- `--debugger` can now access bindings from `let` expressions [#8827](https://github.com/NixOS/nix/issues/8827) [#9918](https://github.com/NixOS/nix/pull/9918)
Breakpoints and errors in the bindings of a `let` expression can now access
those bindings in the debugger. Previously, only the body of `let` expressions
could access those bindings.
- Enter the `--debugger` when `builtins.trace` is called if `debugger-on-trace` is set [#9914](https://github.com/NixOS/nix/pull/9914)
If the `debugger-on-trace` option is set and `--debugger` is given,
`builtins.trace` calls will behave similarly to `builtins.break` and will enter
the debug REPL. This is useful for determining where warnings are being emitted
from.
- Debugger prints source position information [#9913](https://github.com/NixOS/nix/pull/9913)
The `--debugger` now prints source location information, instead of the
pointers of source location information. Before:
```
nix-repl> :bt
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
0x600001522598
```
After:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
/nix/store/hg65h51xnp74ikahns9hyf3py5mlbbqq-source/overrides/default.nix:132:27
131|
132| bootstrappingBase = pkgs.${self.python.pythonAttr}.pythonForBuild.pkgs;
| ^
133| in
```
- The `--debugger` will start more reliably in `let` expressions and function calls [#6649](https://github.com/NixOS/nix/issues/6649) [#9917](https://github.com/NixOS/nix/pull/9917)
Previously, if you attempted to evaluate this file with the debugger:
```nix
let
a = builtins.trace "before inner break" (
builtins.break "hello"
);
b = builtins.trace "before outer break" (
builtins.break a
);
in
b
```
Nix would correctly enter the debugger at `builtins.break a`, but if you asked
it to `:continue`, it would skip over the `builtins.break "hello"` expression
entirely.
Now, Nix will correctly enter the debugger at both breakpoints.
- Nested debuggers are no longer supported [#9920](https://github.com/NixOS/nix/pull/9920)
Previously, evaluating an expression that throws an error in the debugger would
enter a second, nested debugger:
```
nix-repl> builtins.throw "what"
error: what
Starting REPL to allow you to inspect the current state of the evaluator.
Welcome to Nix 2.18.1. Type :? for help.
nix-repl>
```
Now, it just prints the error message like `nix repl`:
```
nix-repl> builtins.throw "what"
error:
… while calling the 'throw' builtin
at «string»:1:1:
1| builtins.throw "what"
| ^
error: what
```
- Consistent order of function arguments in printed expressions [#9874](https://github.com/NixOS/nix/pull/9874)
Function arguments are now printed in lexicographic order rather than the internal, creation-time based symbol order.
- Fix duplicate attribute error positions for `inherit` [#9874](https://github.com/NixOS/nix/pull/9874)
When an `inherit` caused a duplicate attribute error the position of the error was not reported correctly, placing the error with the inherit itself or at the start of the bindings block instead of the offending attribute name.
- `inherit (x) ...` evaluates `x` only once [#9847](https://github.com/NixOS/nix/pull/9847)
`inherit (x) a b ...` now evaluates the expression `x` only once for all inherited attributes rather than once for each inherited attribute.
This does not usually have a measurable impact, but side-effects (such as `builtins.trace`) would be duplicated and expensive expressions (such as derivations) could cause a measurable slowdown.
- Store paths are allowed to start with `.` [#912](https://github.com/NixOS/nix/issues/912) [#9091](https://github.com/NixOS/nix/pull/9091) [#9095](https://github.com/NixOS/nix/pull/9095) [#9120](https://github.com/NixOS/nix/pull/9120) [#9121](https://github.com/NixOS/nix/pull/9121) [#9122](https://github.com/NixOS/nix/pull/9122) [#9130](https://github.com/NixOS/nix/pull/9130) [#9219](https://github.com/NixOS/nix/pull/9219) [#9224](https://github.com/NixOS/nix/pull/9224) [#9867](https://github.com/NixOS/nix/pull/9867)
Leading periods were allowed by accident in Nix 2.4. The Nix team has considered this to be a bug, but this behavior has since been relied on by users, leading to unnecessary difficulties.
From now on, leading periods are supported. The names `.` and `..` are disallowed, as well as those starting with `.-` or `..-`.
Nix versions that denied leading periods are documented [in the issue](https://github.com/NixOS/nix/issues/912#issuecomment-1919583286).
- `nix repl` pretty-prints values [#9931](https://github.com/NixOS/nix/pull/9931)
`nix repl` will now pretty-print values:
```
{
attrs = {
a = {
b = {
c = { };
};
};
};
list = [ 1 ];
list' = [
1
2
3
];
}
```
- Introduction of `--regex` and `--all` in `nix profile remove` and `nix profile upgrade` [#10166](https://github.com/NixOS/nix/pull/10166)
Previously the command-line arguments for `nix profile remove` and `nix profile upgrade` matched the package entries using regular expression.
For instance:
```
nix profile remove '.*vim.*'
```
This would remove all packages that contain `vim` in their name.
In most cases, only singular package names were used to remove and upgrade packages. Mixing this with regular expressions sometimes lead to unintended behavior. For instance, `python3.1` could match `python311`.
To avoid unintended behavior, the arguments are now only matching exact names.
Matching using regular expressions is still possible by using the new `--regex` flag:
```
nix profile remove --regex '.*vim.*'
```
One of the most useful cases for using regular expressions was to upgrade all packages. This was previously accomplished by:
```
nix profile upgrade '.*'
```
With the introduction of the `--all` flag, this now becomes more straightforward:
```
nix profile upgrade --all
```
- Visual clutter in `--debugger` is reduced [#9919](https://github.com/NixOS/nix/pull/9919)
Before:
```
info: breakpoint reached
Starting REPL to allow you to inspect the current state of the evaluator.
Welcome to Nix 2.20.0pre20231222_dirty. Type :? for help.
nix-repl> :continue
error: uh oh
Starting REPL to allow you to inspect the current state of the evaluator.
Welcome to Nix 2.20.0pre20231222_dirty. Type :? for help.
nix-repl>
```
After:
```
info: breakpoint reached
Nix 2.20.0pre20231222_dirty debugger
Type :? for help.
nix-repl> :continue
error: uh oh
nix-repl>
```
- Cycle detection in `nix repl` is simpler and more reliable [#8672](https://github.com/NixOS/nix/issues/8672) [#9926](https://github.com/NixOS/nix/pull/9926)
The cycle detection in `nix repl`, `nix eval`, `builtins.trace`, and everywhere
else values are printed is now simpler and matches the cycle detection in
`nix-instantiate --eval` output.
Before:
```
nix eval --expr 'let self = { inherit self; }; in self'
{ self = { self = «repeated»; }; }
```
After:
```
{ self = «repeated»; }
```
- In the debugger, `while evaluating the attribute` errors now include position information [#9915](https://github.com/NixOS/nix/pull/9915)
Before:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
0x600001522598
```
After:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
/nix/store/hg65h51xnp74ikahns9hyf3py5mlbbqq-source/overrides/default.nix:132:27
131|
132| bootstrappingBase = pkgs.${self.python.pythonAttr}.pythonForBuild.pkgs;
| ^
133| in
```
- Stack size is increased on macOS [#9860](https://github.com/NixOS/nix/pull/9860)
Previously, Nix would set the stack size to 64MiB on Linux, but would leave the
stack size set to the default (approximately 8KiB) on macOS. Now, the stack
size is correctly set to 64MiB on macOS as well, which should reduce stack
overflow segfaults in deeply-recursive Nix expressions.
Generated
+4 -4
View File
@@ -34,16 +34,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1705033721,
"narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
"lastModified": 1709083642,
"narHash": "sha256-7kkJQd4rZ+vFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
"rev": "b550fe4b4776908ac2a861124307045f8e717c8e",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05-small",
"ref": "release-23.11",
"repo": "nixpkgs",
"type": "github"
}
+21 -19
View File
@@ -1,7 +1,9 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05-small";
# TODO switch to nixos-23.11-small
# https://nixpk.gs/pr-tracker.html?pr=291954
inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-23.11";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2"; flake = false; };
@@ -10,20 +12,10 @@
let
inherit (nixpkgs) lib;
# Experimental fileset library: https://github.com/NixOS/nixpkgs/pull/222981
# Not an "idiomatic" flake input because:
# - Propagation to dependent locks: https://github.com/NixOS/nix/issues/7730
# - Subflake would download redundant and huge parent flake
# - No git tree hash support: https://github.com/NixOS/nix/issues/6044
inherit (import (builtins.fetchTarball { url = "https://github.com/NixOS/nix/archive/1bdcd7fc8a6a40b2e805bad759b36e64e911036b.tar.gz"; sha256 = "sha256:14ljlpdsp4x7h1fkhbmc4bd3vsqnx8zdql4h3037wh09ad6a0893"; }))
fileset;
inherit (lib) fileset;
officialRelease = false;
# Set to true to build the release notes for the next release.
buildUnreleasedNotes = false;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
if officialRelease
@@ -39,7 +31,6 @@
crossSystems = [
"armv6l-unknown-linux-gnueabihf"
"armv7l-unknown-linux-gnueabihf"
"x86_64-unknown-freebsd13"
"x86_64-unknown-netbsd"
];
@@ -307,8 +298,11 @@
''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = ff451c521e61e4fe72bdbe2d0ca5d1809affa733 ]]
(
set -x
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = e01b031fc9785a572a38be6bc473957e3b6faad7 ]]
)
mkdir $out
'';
@@ -349,7 +343,6 @@
checks = forAllSystems (system: {
binaryTarball = self.hydraJobs.binaryTarball.${system};
perlBindings = self.hydraJobs.perlBindings.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
@@ -359,6 +352,11 @@
'';
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
} // (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
perlBindings = self.hydraJobs.perlBindings.${system};
});
packages = forAllSystems (system: rec {
@@ -404,8 +402,11 @@
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
nativeBuildInputs = attrs.nativeBuildInputs or []
++ lib.optional stdenv.cc.isClang pkgs.buildPackages.bear
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) pkgs.buildPackages.clang-tools;
});
in
@@ -417,8 +418,9 @@
(forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName}));
in
(makeShells "native" nixpkgsFor.${system}.native) //
(makeShells "static" nixpkgsFor.${system}.static) //
(lib.genAttrs shellCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv)) //
(lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin)
(makeShells "static" nixpkgsFor.${system}.static)) //
(lib.genAttrs shellCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv)) //
{
default = self.devShells.${system}.native-stdenvPackages;
}
+1 -1
View File
@@ -7,4 +7,4 @@ $(foreach i, config.h $(wildcard src/lib*/*.hh), \
$(GCH): src/libutil/util.hh config.h
GCH_CXXFLAGS = -I src/libutil
GCH_CXXFLAGS = $(INCLUDE_libutil)
+47 -22
View File
@@ -11,6 +11,8 @@ use JSON::PP;
use LWP::UserAgent;
use Net::Amazon::S3;
delete $ENV{'shell'}; # shut up a LWP::UserAgent.pm warning
my $evalId = $ARGV[0] or die "Usage: $0 EVAL-ID\n";
my $releasesBucketName = "nix-releases";
@@ -36,9 +38,9 @@ sub fetch {
my $evalUrl = "https://hydra.nixos.org/eval/$evalId";
my $evalInfo = decode_json(fetch($evalUrl, 'application/json'));
#print Dumper($evalInfo);
my $flakeUrl = $evalInfo->{flake} or die;
my $flakeInfo = decode_json(`nix flake metadata --json "$flakeUrl"` or die);
my $nixRev = $flakeInfo->{revision} or die;
my $flakeUrl = $evalInfo->{flake};
my $flakeInfo = decode_json(`nix flake metadata --json "$flakeUrl"` or die) if $flakeUrl;
my $nixRev = ($flakeInfo ? $flakeInfo->{revision} : $evalInfo->{jobsetevalinputs}->{nix}->{revision}) or die;
my $buildInfo = decode_json(fetch("$evalUrl/job/build.x86_64-linux", 'application/json'));
#print Dumper($buildInfo);
@@ -83,12 +85,19 @@ my $channelsBucket = $s3_us->bucket($channelsBucketName) or die;
sub getStorePath {
my ($jobName, $output) = @_;
my $buildInfo = decode_json(fetch("$evalUrl/job/$jobName", 'application/json'));
return $buildInfo->{buildoutputs}->{$output or "out"}->{path} or die "cannot get store path for '$jobName'";
return $buildInfo->{buildoutputs}->{$output or "out"}->{path} // die "cannot get store path for '$jobName'";
}
sub copyManual {
my $manual = getStorePath("build.x86_64-linux", "doc");
print "$manual\n";
my $manual;
eval {
$manual = getStorePath("build.x86_64-linux", "doc");
};
if ($@) {
warn "$@";
return;
}
print "Manual: $manual\n";
my $manualNar = "$tmpDir/$releaseName-manual.nar.xz";
print "$manualNar\n";
@@ -154,19 +163,33 @@ downloadFile("binaryTarball.x86_64-linux", "1");
downloadFile("binaryTarball.aarch64-linux", "1");
downloadFile("binaryTarball.x86_64-darwin", "1");
downloadFile("binaryTarball.aarch64-darwin", "1");
downloadFile("binaryTarballCross.x86_64-linux.armv6l-unknown-linux-gnueabihf", "1");
downloadFile("binaryTarballCross.x86_64-linux.armv7l-unknown-linux-gnueabihf", "1");
eval {
downloadFile("binaryTarballCross.x86_64-linux.armv6l-unknown-linux-gnueabihf", "1");
};
warn "$@" if $@;
eval {
downloadFile("binaryTarballCross.x86_64-linux.armv7l-unknown-linux-gnueabihf", "1");
};
warn "$@" if $@;
downloadFile("installerScript", "1");
# Upload docker images to dockerhub.
my $dockerManifest = "";
my $dockerManifestLatest = "";
my $haveDocker = 0;
for my $platforms (["x86_64-linux", "amd64"], ["aarch64-linux", "arm64"]) {
my $system = $platforms->[0];
my $dockerPlatform = $platforms->[1];
my $fn = "nix-$version-docker-image-$dockerPlatform.tar.gz";
downloadFile("dockerImage.$system", "1", $fn);
eval {
downloadFile("dockerImage.$system", "1", $fn);
};
if ($@) {
warn "$@" if $@;
next;
}
$haveDocker = 1;
print STDERR "loading docker image for $dockerPlatform...\n";
system("docker load -i $tmpDir/$fn") == 0 or die;
@@ -194,21 +217,23 @@ for my $platforms (["x86_64-linux", "amd64"], ["aarch64-linux", "arm64"]) {
$dockerManifestLatest .= " --amend $latestTag"
}
print STDERR "creating multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:$version");
system("docker manifest create nixos/nix:$version $dockerManifest") == 0 or die;
if ($isLatest) {
print STDERR "creating latest multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:latest");
system("docker manifest create nixos/nix:latest $dockerManifestLatest") == 0 or die;
}
if ($haveDocker) {
print STDERR "creating multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:$version");
system("docker manifest create nixos/nix:$version $dockerManifest") == 0 or die;
if ($isLatest) {
print STDERR "creating latest multi-platform docker manifest...\n";
system("docker manifest rm nixos/nix:latest");
system("docker manifest create nixos/nix:latest $dockerManifestLatest") == 0 or die;
}
print STDERR "pushing multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:$version") == 0 or die;
print STDERR "pushing multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:$version") == 0 or die;
if ($isLatest) {
print STDERR "pushing latest multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:latest") == 0 or die;
if ($isLatest) {
print STDERR "pushing latest multi-platform docker manifest...\n";
system("docker manifest push nixos/nix:latest") == 0 or die;
}
}
# Upload nix-fallback-paths.nix.
+11
View File
@@ -0,0 +1,11 @@
compile-commands-json-files :=
define write-compile-commands
_srcs := $$(sort $$(foreach src, $$($(1)_SOURCES), $$(src)))
$(1)_COMPILE_COMMANDS_JSON := $$(addprefix $(buildprefix), $$(addsuffix .compile_commands.json, $$(basename $$(_srcs))))
compile-commands-json-files += $$($(1)_COMPILE_COMMANDS_JSON)
clean-files += $$($(1)_COMPILE_COMMANDS_JSON)
endef
+2 -2
View File
@@ -1,5 +1,5 @@
%.gen.hh: %
@echo 'R"foo(' >> $@.tmp
@echo 'R"__NIX_STR(' >> $@.tmp
$(trace-gen) cat $< >> $@.tmp
@echo ')foo"' >> $@.tmp
@echo ')__NIX_STR"' >> $@.tmp
@mv $@.tmp $@
+12
View File
@@ -68,6 +68,7 @@ include mk/patterns.mk
include mk/templates.mk
include mk/cxx-big-literal.mk
include mk/tests.mk
include mk/compilation-database.mk
# Include all sub-Makefiles.
@@ -97,6 +98,17 @@ $(foreach test-group, $(install-tests-groups), \
$(eval $(call run-test,$(test),$(install_test_init))) \
$(eval $(test-group).test-group: $(test).test)))
# Compilation database.
$(foreach lib, $(libraries), $(eval $(call write-compile-commands,$(lib))))
$(foreach prog, $(programs), $(eval $(call write-compile-commands,$(prog))))
compile_commands.json: $(compile-commands-json-files)
@jq --slurp '.' $^ >$@
# Include makefiles requiring built programs.
$(foreach mf, $(makefiles-late), $(eval $(call include-sub-makefile,$(mf))))
$(foreach file, $(man-pages), $(eval $(call install-data-in, $(file), $(mandir)/man$(patsubst .%,%,$(suffix $(file))))))
+33 -3
View File
@@ -1,11 +1,41 @@
# These are the complete command lines we use to compile C and C++ files.
# - $< is the source file.
# - $1 is the object file to create.
CC_CMD=$(CC) -o $1 -c $< $(CPPFLAGS) $(GLOBAL_CFLAGS) $(CFLAGS) $($1_CFLAGS) -MMD -MF $(call filename-to-dep,$1) -MP
CXX_CMD=$(CXX) -o $1 -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($1_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep,$1) -MP
# We use COMPILE_COMMANDS_JSON_CMD to turn a compilation command (like CC_CMD
# or CXX_CMD above) into a comple_commands.json file. We rely on bash native
# word splitting to define the positional arguments.
# - $< is the source file being compiled.
COMPILE_COMMANDS_JSON_CMD=jq --null-input '{ directory: $$ENV.PWD, file: "$<", arguments: $$ARGS.positional }' --args --
$(buildprefix)%.o: %.cc
@mkdir -p "$(dir $@)"
$(trace-cxx) $(CXX) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($@_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cxx) $(call CXX_CMD,$@)
$(buildprefix)%.o: %.cpp
@mkdir -p "$(dir $@)"
$(trace-cxx) $(CXX) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CXXFLAGS_PCH) $(GLOBAL_CXXFLAGS) $(CXXFLAGS) $($@_CXXFLAGS) $(ERROR_SWITCH_ENUM) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cxx) $(call CXX_CMD,$@)
$(buildprefix)%.o: %.c
@mkdir -p "$(dir $@)"
$(trace-cc) $(CC) -o $@ -c $< $(CPPFLAGS) $(GLOBAL_CFLAGS) $(CFLAGS) $($@_CFLAGS) -MMD -MF $(call filename-to-dep, $@) -MP
$(trace-cc) $(call CC_CMD,$@)
# In the following we need to replace the .compile_commands.json extension in $@ with .o
# to make the object file. This is needed because CC_CMD and CXX_CMD do further expansions
# based on the object file name (i.e. *_CXXFLAGS and filename-to-dep).
$(buildprefix)%.compile_commands.json: %.cc
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CXX_CMD,$(@:.compile_commands.json=.o)) > $@
$(buildprefix)%.compile_commands.json: %.cpp
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CXX_CMD,$(@:.compile_commands.json=.o)) > $@
$(buildprefix)%.compile_commands.json: %.c
@mkdir -p "$(dir $@)"
$(trace-jq) $(COMPILE_COMMANDS_JSON_CMD) $(call CC_CMD,$(@:.compile_commands.json=.o)) > $@
+2 -2
View File
@@ -10,10 +10,10 @@ endef
ifneq ($(MAKECMDGOALS), clean)
$(buildprefix)%.h: %.h.in
$(buildprefix)%.h: %.h.in $(buildprefix)config.status
$(trace-gen) rm -f $@ && cd $(buildprefixrel) && ./config.status --quiet --header=$(@:$(buildprefix)%=%)
$(buildprefix)%: %.in
$(buildprefix)%: %.in $(buildprefix)config.status
$(trace-gen) rm -f $@ && cd $(buildprefixrel) && ./config.status --quiet --file=$(@:$(buildprefix)%=%)
endif
+2
View File
@@ -10,6 +10,8 @@ ifeq ($(V), 0)
trace-install = @echo " INST " $@;
trace-mkdir = @echo " MKDIR " $@;
trace-test = @echo " TEST " $@;
trace-sh = @echo " SH " $@;
trace-jq = @echo " JQ " $@;
suppress = @
+27 -11
View File
@@ -24,6 +24,7 @@
, libgit2
, libseccomp
, libsodium
, man
, lowdown
, mdbook
, mdbook-linkcheck
@@ -74,7 +75,10 @@
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
, enableGC ? true
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows
# Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows
@@ -154,7 +158,7 @@ in {
in
fileset.toSource {
root = ./.;
fileset = fileset.intersect baseFiles (fileset.unions ([
fileset = fileset.intersection baseFiles (fileset.unions ([
# For configure
./.version
./configure.ac
@@ -209,6 +213,11 @@ in {
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
] ++ lib.optionals doInstallCheck [
git
mercurial
openssh
man # for testing `nix-* --help`
] ++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
] ++ lib.optional stdenv.hostPlatform.isLinux util-linux
@@ -249,12 +258,6 @@ in {
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
nativeCheckInputs = [
git
mercurial
openssh
];
disallowedReferences = [ boost ];
preConfigure = lib.optionalString (doBuild && ! stdenv.hostPlatform.isStatic) (
@@ -335,6 +338,12 @@ in {
echo "doc internal-api-docs $out/share/doc/nix/internal-api/html" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
# So the check output gets links for DLLs in the out output.
preFixup = lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs) ''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
doInstallCheck = attrs.doInstallCheck;
installCheckFlags = "sysconfdir=$(out)/etc";
@@ -343,15 +352,22 @@ in {
# Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase = if (!doBuild && doInstallCheck) then ''
runHook preInstallCheck
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
'' else null;
# Needed for tests if we are not doing a build, but testing existing
# built Nix.
preInstallCheck = lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel
'';
preInstallCheck =
lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel
''
# See https://github.com/NixOS/nix/issues/2523
# Occurs often in tests since https://github.com/NixOS/nix/pull/9900
+ lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
separateDebugInfo = !stdenv.hostPlatform.isStatic;
+2
View File
@@ -0,0 +1,2 @@
[test]
-I=rel(lib/Nix)
+14 -4
View File
@@ -5,12 +5,12 @@
, nix, curl, bzip2, xz, boost, libsodium, darwin
}:
perl.pkgs.toPerlModule (stdenv.mkDerivation {
perl.pkgs.toPerlModule (stdenv.mkDerivation (finalAttrs: {
name = "nix-perl-${nix.version}";
src = fileset.toSource {
root = ../.;
fileset = fileset.unions [
fileset = fileset.unions ([
../.version
../m4
../mk
@@ -20,7 +20,10 @@ perl.pkgs.toPerlModule (stdenv.mkDerivation {
./configure.ac
./lib
./local.mk
];
] ++ lib.optionals finalAttrs.doCheck [
./.yath.rc
./t
]);
};
nativeBuildInputs =
@@ -40,6 +43,13 @@ perl.pkgs.toPerlModule (stdenv.mkDerivation {
++ lib.optional (stdenv.isLinux || stdenv.isDarwin) libsodium
++ lib.optional stdenv.isDarwin darwin.apple_sdk.frameworks.Security;
# `perlPackages.Test2Harness` is marked broken for Darwin
doCheck = !stdenv.isDarwin;
nativeCheckInputs = [
perlPackages.Test2Harness
];
configureFlags = [
"--with-dbi=${perlPackages.DBI}/${perl.libPrefix}"
"--with-dbd-sqlite=${perlPackages.DBDSQLite}/${perl.libPrefix}"
@@ -48,4 +58,4 @@ perl.pkgs.toPerlModule (stdenv.mkDerivation {
enableParallelBuilding = true;
postUnpack = "sourceRoot=$sourceRoot/perl";
})
}))
+11 -8
View File
@@ -12,17 +12,20 @@ our %EXPORT_TAGS = ( 'all' => [ qw( ) ] );
our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
our @EXPORT = qw(
setVerbosity
isValidPath queryReferences queryPathInfo queryDeriver queryPathHash
queryPathFromHashPart
topoSortPaths computeFSClosure followLinksToStorePath exportPaths importPaths
StoreWrapper
StoreWrapper::new
StoreWrapper::isValidPath StoreWrapper::queryReferences StoreWrapper::queryPathInfo StoreWrapper::queryDeriver StoreWrapper::queryPathHash
StoreWrapper::queryPathFromHashPart
StoreWrapper::topoSortPaths StoreWrapper::computeFSClosure followLinksToStorePath StoreWrapper::exportPaths StoreWrapper::importPaths
StoreWrapper::addToStore StoreWrapper::makeFixedOutputPath
StoreWrapper::derivationFromPath
StoreWrapper::addTempRoot
StoreWrapper::queryRawRealisation
hashPath hashFile hashString convertHash
signString checkSignature
addToStore makeFixedOutputPath
derivationFromPath
addTempRoot
getBinDir getStoreDir
queryRawRealisation
setVerbosity
);
our $VERSION = '0.15';
+133 -78
View File
@@ -17,47 +17,61 @@
#include <sodium.h>
#include <nlohmann/json.hpp>
using namespace nix;
static bool libStoreInitialized = false;
static ref<Store> store()
{
static std::shared_ptr<Store> _store;
if (!_store) {
try {
initLibStore();
_store = openStore();
} catch (Error & e) {
croak("%s", e.what());
}
}
return ref<Store>(_store);
}
struct StoreWrapper {
ref<Store> store;
};
MODULE = Nix::Store PACKAGE = Nix::Store
PROTOTYPES: ENABLE
TYPEMAP: <<HERE
StoreWrapper * O_OBJECT
OUTPUT
O_OBJECT
sv_setref_pv( $arg, CLASS, (void*)$var );
INPUT
O_OBJECT
if ( sv_isobject($arg) && (SvTYPE(SvRV($arg)) == SVt_PVMG) ) {
$var = ($type)SvIV((SV*)SvRV( $arg ));
}
else {
warn( \"${Package}::$func_name() -- \"
\"$var not a blessed SV reference\");
XSRETURN_UNDEF;
}
HERE
#undef dNOOP // Hack to work around "error: declaration of 'Perl___notused' has a different language linkage" error message on clang.
#define dNOOP
void
StoreWrapper::DESTROY()
void init()
CODE:
store();
void setVerbosity(int level)
CODE:
verbosity = (Verbosity) level;
int isValidPath(char * path)
StoreWrapper *
StoreWrapper::new(char * s = nullptr)
CODE:
static std::shared_ptr<Store> _store;
try {
RETVAL = store()->isValidPath(store()->parseStorePath(path));
if (!libStoreInitialized) {
initLibStore();
libStoreInitialized = true;
}
if (items == 1) {
_store = openStore();
RETVAL = new StoreWrapper {
.store = ref<Store>{_store}
};
} else {
RETVAL = new StoreWrapper {
.store = openStore(s)
};
}
} catch (Error & e) {
croak("%s", e.what());
}
@@ -65,52 +79,81 @@ int isValidPath(char * path)
RETVAL
SV * queryReferences(char * path)
void init()
CODE:
if (!libStoreInitialized) {
initLibStore();
libStoreInitialized = true;
}
void setVerbosity(int level)
CODE:
verbosity = (Verbosity) level;
int
StoreWrapper::isValidPath(char * path)
CODE:
try {
RETVAL = THIS->store->isValidPath(THIS->store->parseStorePath(path));
} catch (Error & e) {
croak("%s", e.what());
}
OUTPUT:
RETVAL
SV *
StoreWrapper::queryReferences(char * path)
PPCODE:
try {
for (auto & i : store()->queryPathInfo(store()->parseStorePath(path))->references)
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(i).c_str(), 0)));
for (auto & i : THIS->store->queryPathInfo(THIS->store->parseStorePath(path))->references)
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(i).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * queryPathHash(char * path)
SV *
StoreWrapper::queryPathHash(char * path)
PPCODE:
try {
auto s = store()->queryPathInfo(store()->parseStorePath(path))->narHash.to_string(HashFormat::Nix32, true);
auto s = THIS->store->queryPathInfo(THIS->store->parseStorePath(path))->narHash.to_string(HashFormat::Nix32, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * queryDeriver(char * path)
SV *
StoreWrapper::queryDeriver(char * path)
PPCODE:
try {
auto info = store()->queryPathInfo(store()->parseStorePath(path));
auto info = THIS->store->queryPathInfo(THIS->store->parseStorePath(path));
if (!info->deriver) XSRETURN_UNDEF;
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(*info->deriver).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(*info->deriver).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * queryPathInfo(char * path, int base32)
SV *
StoreWrapper::queryPathInfo(char * path, int base32)
PPCODE:
try {
auto info = store()->queryPathInfo(store()->parseStorePath(path));
auto info = THIS->store->queryPathInfo(THIS->store->parseStorePath(path));
if (!info->deriver)
XPUSHs(&PL_sv_undef);
else
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(*info->deriver).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(*info->deriver).c_str(), 0)));
auto s = info->narHash.to_string(base32 ? HashFormat::Nix32 : HashFormat::Base16, true);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
mXPUSHi(info->registrationTime);
mXPUSHi(info->narSize);
AV * refs = newAV();
for (auto & i : info->references)
av_push(refs, newSVpv(store()->printStorePath(i).c_str(), 0));
av_push(refs, newSVpv(THIS->store->printStorePath(i).c_str(), 0));
XPUSHs(sv_2mortal(newRV((SV *) refs)));
AV * sigs = newAV();
for (auto & i : info->sigs)
@@ -120,10 +163,11 @@ SV * queryPathInfo(char * path, int base32)
croak("%s", e.what());
}
SV * queryRawRealisation(char * outputId)
SV *
StoreWrapper::queryRawRealisation(char * outputId)
PPCODE:
try {
auto realisation = store()->queryRealisation(DrvOutput::parse(outputId));
auto realisation = THIS->store->queryRealisation(DrvOutput::parse(outputId));
if (realisation)
XPUSHs(sv_2mortal(newSVpv(realisation->toJSON().dump().c_str(), 0)));
else
@@ -133,46 +177,50 @@ SV * queryRawRealisation(char * outputId)
}
SV * queryPathFromHashPart(char * hashPart)
SV *
StoreWrapper::queryPathFromHashPart(char * hashPart)
PPCODE:
try {
auto path = store()->queryPathFromHashPart(hashPart);
XPUSHs(sv_2mortal(newSVpv(path ? store()->printStorePath(*path).c_str() : "", 0)));
auto path = THIS->store->queryPathFromHashPart(hashPart);
XPUSHs(sv_2mortal(newSVpv(path ? THIS->store->printStorePath(*path).c_str() : "", 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * computeFSClosure(int flipDirection, int includeOutputs, ...)
SV *
StoreWrapper::computeFSClosure(int flipDirection, int includeOutputs, ...)
PPCODE:
try {
StorePathSet paths;
for (int n = 2; n < items; ++n)
store()->computeFSClosure(store()->parseStorePath(SvPV_nolen(ST(n))), paths, flipDirection, includeOutputs);
THIS->store->computeFSClosure(THIS->store->parseStorePath(SvPV_nolen(ST(n))), paths, flipDirection, includeOutputs);
for (auto & i : paths)
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(i).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(i).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * topoSortPaths(...)
SV *
StoreWrapper::topoSortPaths(...)
PPCODE:
try {
StorePathSet paths;
for (int n = 0; n < items; ++n) paths.insert(store()->parseStorePath(SvPV_nolen(ST(n))));
auto sorted = store()->topoSortPaths(paths);
for (int n = 0; n < items; ++n) paths.insert(THIS->store->parseStorePath(SvPV_nolen(ST(n))));
auto sorted = THIS->store->topoSortPaths(paths);
for (auto & i : sorted)
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(i).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(i).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * followLinksToStorePath(char * path)
SV *
StoreWrapper::followLinksToStorePath(char * path)
CODE:
try {
RETVAL = newSVpv(store()->printStorePath(store()->followLinksToStorePath(path)).c_str(), 0);
RETVAL = newSVpv(THIS->store->printStorePath(THIS->store->followLinksToStorePath(path)).c_str(), 0);
} catch (Error & e) {
croak("%s", e.what());
}
@@ -180,35 +228,38 @@ SV * followLinksToStorePath(char * path)
RETVAL
void exportPaths(int fd, ...)
void
StoreWrapper::exportPaths(int fd, ...)
PPCODE:
try {
StorePathSet paths;
for (int n = 1; n < items; ++n) paths.insert(store()->parseStorePath(SvPV_nolen(ST(n))));
for (int n = 1; n < items; ++n) paths.insert(THIS->store->parseStorePath(SvPV_nolen(ST(n))));
FdSink sink(fd);
store()->exportPaths(paths, sink);
THIS->store->exportPaths(paths, sink);
} catch (Error & e) {
croak("%s", e.what());
}
void importPaths(int fd, int dontCheckSigs)
void
StoreWrapper::importPaths(int fd, int dontCheckSigs)
PPCODE:
try {
FdSource source(fd);
store()->importPaths(source, dontCheckSigs ? NoCheckSigs : CheckSigs);
THIS->store->importPaths(source, dontCheckSigs ? NoCheckSigs : CheckSigs);
} catch (Error & e) {
croak("%s", e.what());
}
SV * hashPath(char * algo, int base32, char * path)
SV *
hashPath(char * algo, int base32, char * path)
PPCODE:
try {
PosixSourceAccessor accessor;
auto [accessor, canonPath] = PosixSourceAccessor::createAtRoot(path);
Hash h = hashPath(
accessor, CanonPath::fromCwd(path),
FileIngestionMethod::Recursive, parseHashAlgo(algo)).first;
accessor, canonPath,
FileIngestionMethod::Recursive, parseHashAlgo(algo));
auto s = h.to_string(base32 ? HashFormat::Nix32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
@@ -280,64 +331,67 @@ int checkSignature(SV * publicKey_, SV * sig_, char * msg)
RETVAL
SV * addToStore(char * srcPath, int recursive, char * algo)
SV *
StoreWrapper::addToStore(char * srcPath, int recursive, char * algo)
PPCODE:
try {
auto method = recursive ? FileIngestionMethod::Recursive : FileIngestionMethod::Flat;
PosixSourceAccessor accessor;
auto path = store()->addToStore(
auto [accessor, canonPath] = PosixSourceAccessor::createAtRoot(srcPath);
auto path = THIS->store->addToStore(
std::string(baseNameOf(srcPath)),
accessor, CanonPath::fromCwd(srcPath),
accessor, canonPath,
method, parseHashAlgo(algo));
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(path).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(path).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * makeFixedOutputPath(int recursive, char * algo, char * hash, char * name)
SV *
StoreWrapper::makeFixedOutputPath(int recursive, char * algo, char * hash, char * name)
PPCODE:
try {
auto h = Hash::parseAny(hash, parseHashAlgo(algo));
auto method = recursive ? FileIngestionMethod::Recursive : FileIngestionMethod::Flat;
auto path = store()->makeFixedOutputPath(name, FixedOutputInfo {
auto path = THIS->store->makeFixedOutputPath(name, FixedOutputInfo {
.method = method,
.hash = h,
.references = {},
});
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(path).c_str(), 0)));
XPUSHs(sv_2mortal(newSVpv(THIS->store->printStorePath(path).c_str(), 0)));
} catch (Error & e) {
croak("%s", e.what());
}
SV * derivationFromPath(char * drvPath)
SV *
StoreWrapper::derivationFromPath(char * drvPath)
PREINIT:
HV *hash;
CODE:
try {
Derivation drv = store()->derivationFromPath(store()->parseStorePath(drvPath));
Derivation drv = THIS->store->derivationFromPath(THIS->store->parseStorePath(drvPath));
hash = newHV();
HV * outputs = newHV();
for (auto & i : drv.outputsAndOptPaths(*store())) {
for (auto & i : drv.outputsAndOptPaths(*THIS->store)) {
hv_store(
outputs, i.first.c_str(), i.first.size(),
!i.second.second
? newSV(0) /* null value */
: newSVpv(store()->printStorePath(*i.second.second).c_str(), 0),
: newSVpv(THIS->store->printStorePath(*i.second.second).c_str(), 0),
0);
}
hv_stores(hash, "outputs", newRV((SV *) outputs));
AV * inputDrvs = newAV();
for (auto & i : drv.inputDrvs.map)
av_push(inputDrvs, newSVpv(store()->printStorePath(i.first).c_str(), 0)); // !!! ignores i->second
av_push(inputDrvs, newSVpv(THIS->store->printStorePath(i.first).c_str(), 0)); // !!! ignores i->second
hv_stores(hash, "inputDrvs", newRV((SV *) inputDrvs));
AV * inputSrcs = newAV();
for (auto & i : drv.inputSrcs)
av_push(inputSrcs, newSVpv(store()->printStorePath(i).c_str(), 0));
av_push(inputSrcs, newSVpv(THIS->store->printStorePath(i).c_str(), 0));
hv_stores(hash, "inputSrcs", newRV((SV *) inputSrcs));
hv_stores(hash, "platform", newSVpv(drv.platform.c_str(), 0));
@@ -361,10 +415,11 @@ SV * derivationFromPath(char * drvPath)
RETVAL
void addTempRoot(char * storePath)
void
StoreWrapper::addTempRoot(char * storePath)
PPCODE:
try {
store()->addTempRoot(store()->parseStorePath(storePath));
THIS->store->addTempRoot(THIS->store->parseStorePath(storePath));
} catch (Error & e) {
croak("%s", e.what());
}
+3
View File
@@ -41,3 +41,6 @@ Store_FORCE_INSTALL = 1
Store_INSTALL_DIR = $(perllibdir)/auto/Nix/Store
clean-files += lib/Nix/Config.pm lib/Nix/Store.cc Makefile.config
check: all
yath test
+13
View File
@@ -0,0 +1,13 @@
use strict;
use warnings;
use Test2::V0;
use Nix::Store;
my $s = new Nix::Store("dummy://");
my $res = $s->isValidPath("/nix/store/g1w7hy3qg1w7hy3qg1w7hy3qg1w7hy3q-bar");
ok(!$res, "should not have path");
done_testing;
+1 -1
View File
@@ -102,7 +102,7 @@ poly_extra_try_me_commands() {
poly_configure_nix_daemon_service() {
task "Setting up the nix-daemon LaunchDaemon"
_sudo "to set up the nix-daemon as a LaunchDaemon" \
/usr/bin/install -m -rw-r--r-- "/nix/var/nix/profiles/default$NIX_DAEMON_DEST" "$NIX_DAEMON_DEST"
/usr/bin/install -m "u=rw,go=r" "/nix/var/nix/profiles/default$NIX_DAEMON_DEST" "$NIX_DAEMON_DEST"
_sudo "to load the LaunchDaemon plist for nix-daemon" \
launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist
+2 -2
View File
@@ -28,7 +28,7 @@ else
end
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE"
if test -n "$NIX_SSL_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE
else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt
@@ -44,7 +44,7 @@ else if test -e "$NIX_LINK/etc/ca-bundle.crt" # old cacert in Nix profile
set --export NIX_SSL_CERT_FILE "$NIX_LINK/etc/ca-bundle.crt"
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
for i in $NIX_PROFILES
for i in (string split ' ' $NIX_PROFILES)
if test -e "$i/etc/ssl/certs/ca-bundle.crt"
set --export NIX_SSL_CERT_FILE "$i/etc/ssl/certs/ca-bundle.crt"
end
+1 -1
View File
@@ -202,7 +202,7 @@ static int main_build_remote(int argc, char * * argv)
else
drvstr = "<unknown>";
auto error = hintformat(errorText);
auto error = HintFmt::fromFormatString(errorText);
error
% drvstr
% neededSystem
+41 -12
View File
@@ -9,6 +9,7 @@
#include "store-api.hh"
#include "command.hh"
#include "tarball.hh"
#include "fetch-to-store.hh"
namespace nix {
@@ -19,7 +20,7 @@ MixEvalArgs::MixEvalArgs()
.description = "Pass the value *expr* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "expr"},
.handler = {[&](std::string name, std::string expr) { autoArgs[name] = 'E' + expr; }}
.handler = {[&](std::string name, std::string expr) { autoArgs.insert_or_assign(name, AutoArg{AutoArgExpr(expr)}); }}
});
addFlag({
@@ -27,7 +28,24 @@ MixEvalArgs::MixEvalArgs()
.description = "Pass the string *string* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "string"},
.handler = {[&](std::string name, std::string s) { autoArgs[name] = 'S' + s; }},
.handler = {[&](std::string name, std::string s) { autoArgs.insert_or_assign(name, AutoArg{AutoArgString(s)}); }},
});
addFlag({
.longName = "arg-from-file",
.description = "Pass the contents of file *path* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "path"},
.handler = {[&](std::string name, std::string path) { autoArgs.insert_or_assign(name, AutoArg{AutoArgFile(path)}); }},
.completer = completePath
});
addFlag({
.longName = "arg-from-stdin",
.description = "Pass the contents of stdin as the argument *name* to Nix functions.",
.category = category,
.labels = {"name"},
.handler = {[&](std::string name) { autoArgs.insert_or_assign(name, AutoArg{AutoArgStdin{}}); }},
});
addFlag({
@@ -153,22 +171,33 @@ MixEvalArgs::MixEvalArgs()
Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
{
auto res = state.buildBindings(autoArgs.size());
for (auto & i : autoArgs) {
for (auto & [name, arg] : autoArgs) {
auto v = state.allocValue();
if (i.second[0] == 'E')
state.mkThunk_(*v, state.parseExprFromString(i.second.substr(1), state.rootPath(CanonPath::fromCwd())));
else
v->mkString(((std::string_view) i.second).substr(1));
res.insert(state.symbols.create(i.first), v);
std::visit(overloaded {
[&](const AutoArgExpr & arg) {
state.mkThunk_(*v, state.parseExprFromString(arg.expr, state.rootPath(".")));
},
[&](const AutoArgString & arg) {
v->mkString(arg.s);
},
[&](const AutoArgFile & arg) {
v->mkString(readFile(arg.path));
},
[&](const AutoArgStdin & arg) {
v->mkString(readFile(STDIN_FILENO));
}
}, arg);
res.insert(state.symbols.create(name), v);
}
return res.finish();
}
SourcePath lookupFileArg(EvalState & state, std::string_view s, CanonPath baseDir)
SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * baseDir)
{
if (EvalSettings::isPseudoUrl(s)) {
auto storePath = fetchers::downloadTarball(
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).storePath;
auto accessor = fetchers::downloadTarball(
EvalSettings::resolvePseudoUrl(s)).accessor;
auto storePath = fetchToStore(*state.store, SourcePath(accessor), FetchMode::Copy);
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
@@ -185,7 +214,7 @@ SourcePath lookupFileArg(EvalState & state, std::string_view s, CanonPath baseDi
}
else
return state.rootPath(CanonPath(s, baseDir));
return state.rootPath(baseDir ? absPath(s, *baseDir) : absPath(s));
}
}
+11 -2
View File
@@ -6,6 +6,8 @@
#include "common-args.hh"
#include "search-path.hh"
#include <filesystem>
namespace nix {
class Store;
@@ -26,9 +28,16 @@ struct MixEvalArgs : virtual Args, virtual MixRepair
std::optional<std::string> evalStoreUrl;
private:
std::map<std::string, std::string> autoArgs;
struct AutoArgExpr { std::string expr; };
struct AutoArgString { std::string s; };
struct AutoArgFile { std::filesystem::path path; };
struct AutoArgStdin { };
using AutoArg = std::variant<AutoArgExpr, AutoArgString, AutoArgFile, AutoArgStdin>;
std::map<std::string, AutoArg> autoArgs;
};
SourcePath lookupFileArg(EvalState & state, std::string_view s, CanonPath baseDir = CanonPath::fromCwd());
SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * baseDir = nullptr);
}
+1 -1
View File
@@ -17,7 +17,7 @@ Strings editorFor(const SourcePath & file, uint32_t line)
editor.find("vim") != std::string::npos ||
editor.find("kak") != std::string::npos))
args.push_back(fmt("+%d", line));
args.push_back(path->abs());
args.push_back(path->string());
return args;
}
+1 -1
View File
@@ -45,7 +45,7 @@ ref<InstallableValue> InstallableValue::require(ref<Installable> installable)
std::optional<DerivedPathWithInfo> InstallableValue::trySinglePathToDerivedPaths(Value & v, const PosIdx pos, std::string_view errorCtx)
{
if (v.type() == nPath) {
auto storePath = fetchToStore(*state->store, v.path());
auto storePath = fetchToStore(*state->store, v.path(), FetchMode::Copy);
return {{
.path = DerivedPath::Opaque {
.path = std::move(storePath),
+7 -5
View File
@@ -21,6 +21,7 @@
#include "url.hh"
#include "registry.hh"
#include "build-result.hh"
#include "fs-input-accessor.hh"
#include <regex>
#include <queue>
@@ -146,7 +147,7 @@ MixFlakeOptions::MixFlakeOptions()
.category = category,
.labels = {"flake-lock-path"},
.handler = {[&](std::string lockFilePath) {
lockFlags.referenceLockFilePath = lockFilePath;
lockFlags.referenceLockFilePath = getUnfilteredRootPath(CanonPath(absPath(lockFilePath)));
}},
.completer = completePath
});
@@ -442,10 +443,10 @@ ref<eval_cache::EvalCache> openEvalCache(
EvalState & state,
std::shared_ptr<flake::LockedFlake> lockedFlake)
{
auto fingerprint = lockedFlake->getFingerprint();
auto fingerprint = lockedFlake->getFingerprint(state.store);
return make_ref<nix::eval_cache::EvalCache>(
evalSettings.useEvalCache && evalSettings.pureEval
? std::optional { std::cref(fingerprint) }
? fingerprint
: std::nullopt,
state,
[&state, lockedFlake]()
@@ -487,10 +488,11 @@ Installables SourceExprCommand::parseInstallables(
state->eval(e, *vFile);
}
else if (file) {
state->evalFile(lookupFileArg(*state, *file, CanonPath::fromCwd(getCommandBaseDir())), *vFile);
auto dir = absPath(getCommandBaseDir());
state->evalFile(lookupFileArg(*state, *file, &dir), *vFile);
}
else {
CanonPath dir(CanonPath::fromCwd(getCommandBaseDir()));
Path dir = absPath(getCommandBaseDir());
auto e = state->parseExprFromString(*expr, state->rootPath(dir));
state->eval(e, *vFile);
}
+1 -1
View File
@@ -6,7 +6,7 @@ libcmd_DIR := $(d)
libcmd_SOURCES := $(wildcard $(d)/*.cc)
libcmd_CXXFLAGS += -I src/libutil -I src/libstore -I src/libexpr -I src/libmain -I src/libfetchers
libcmd_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libfetchers) $(INCLUDE_libexpr) -I src/libmain
libcmd_LDFLAGS = $(EDITLINE_LIBS) $(LOWDOWN_LIBS) $(THREAD_LDFLAGS)
+1 -1
View File
@@ -50,7 +50,7 @@ std::string renderMarkdownToTerminal(std::string_view markdown)
if (!rndr_res)
throw Error("allocation error while rendering Markdown");
return filterANSIEscapes(std::string(buf->data, buf->size), !shouldANSI());
return filterANSIEscapes(std::string(buf->data, buf->size), !isTTY());
#else
return std::string(markdown);
#endif
+121
View File
@@ -0,0 +1,121 @@
#include "misc-store-flags.hh"
namespace nix::flag
{
static void hashFormatCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & format : hashFormats) {
if (hasPrefix(format, prefix)) {
completions.add(format);
}
}
}
Args::Flag hashFormatWithDefault(std::string && longName, HashFormat * hf)
{
assert(*hf == nix::HashFormat::SRI);
return Args::Flag {
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`). Default: `sri`.",
.labels = {"hash-format"},
.handler = {[hf](std::string s) {
*hf = parseHashFormat(s);
}},
.completer = hashFormatCompleter,
};
}
Args::Flag hashFormatOpt(std::string && longName, std::optional<HashFormat> * ohf)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`).",
.labels = {"hash-format"},
.handler = {[ohf](std::string s) {
*ohf = std::optional<HashFormat>{parseHashFormat(s)};
}},
.completer = hashFormatCompleter,
};
}
static void hashAlgoCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & algo : hashAlgorithms)
if (hasPrefix(algo, prefix))
completions.add(algo);
}
Args::Flag hashAlgo(std::string && longName, HashAlgorithm * ha)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`).",
.labels = {"hash-algo"},
.handler = {[ha](std::string s) {
*ha = parseHashAlgo(s);
}},
.completer = hashAlgoCompleter,
};
}
Args::Flag hashAlgoOpt(std::string && longName, std::optional<HashAlgorithm> * oha)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`). Can be omitted for SRI hashes.",
.labels = {"hash-algo"},
.handler = {[oha](std::string s) {
*oha = std::optional<HashAlgorithm>{parseHashAlgo(s)};
}},
.completer = hashAlgoCompleter,
};
}
Args::Flag fileIngestionMethod(FileIngestionMethod * method)
{
return Args::Flag {
.longName = "mode",
// FIXME indentation carefully made for context, this is messed up.
.description = R"(
How to compute the hash of the input.
One of:
- `nar` (the default): Serialises the input as an archive (following the [_Nix Archive Format_](https://edolstra.github.io/pubs/phd-thesis.pdf#page=101)) and passes that to the hash function.
- `flat`: Assumes that the input is a single file and directly passes it to the hash function;
)",
.labels = {"file-ingestion-method"},
.handler = {[method](std::string s) {
*method = parseFileIngestionMethod(s);
}},
};
}
Args::Flag contentAddressMethod(ContentAddressMethod * method)
{
return Args::Flag {
.longName = "mode",
// FIXME indentation carefully made for context, this is messed up.
.description = R"(
How to compute the content-address of the store object.
One of:
- `nar` (the default): Serialises the input as an archive (following the [_Nix Archive Format_](https://edolstra.github.io/pubs/phd-thesis.pdf#page=101)) and passes that to the hash function.
- `flat`: Assumes that the input is a single file and directly passes it to the hash function;
- `text`: Like `flat`, but used for
[derivations](@docroot@/glossary.md#store-derivation) serialized in store object and
[`builtins.toFile`](@docroot@/language/builtins.html#builtins-toFile).
For advanced use-cases only;
for regular usage prefer `nar` and `flat.
)",
.labels = {"content-address-method"},
.handler = {[method](std::string s) {
*method = ContentAddressMethod::parse(s);
}},
};
}
}
+21
View File
@@ -0,0 +1,21 @@
#include "args.hh"
#include "content-address.hh"
namespace nix::flag {
Args::Flag hashAlgo(std::string && longName, HashAlgorithm * ha);
static inline Args::Flag hashAlgo(HashAlgorithm * ha)
{
return hashAlgo("hash-algo", ha);
}
Args::Flag hashAlgoOpt(std::string && longName, std::optional<HashAlgorithm> * oha);
Args::Flag hashFormatWithDefault(std::string && longName, HashFormat * hf);
Args::Flag hashFormatOpt(std::string && longName, std::optional<HashFormat> * ohf);
static inline Args::Flag hashAlgoOpt(std::optional<HashAlgorithm> * oha)
{
return hashAlgoOpt("hash-algo", oha);
}
Args::Flag fileIngestionMethod(FileIngestionMethod * method);
Args::Flag contentAddressMethod(ContentAddressMethod * method);
}
+186
View File
@@ -0,0 +1,186 @@
#include <cstdio>
#ifdef USE_READLINE
#include <readline/history.h>
#include <readline/readline.h>
#else
// editline < 1.15.2 don't wrap their API for C++ usage
// (added in https://github.com/troglobit/editline/commit/91398ceb3427b730995357e9d120539fb9bb7461).
// This results in linker errors due to to name-mangling of editline C symbols.
// For compatibility with these versions, we wrap the API here
// (wrapping multiple times on newer versions is no problem).
extern "C" {
#include <editline.h>
}
#endif
#include "signals.hh"
#include "finally.hh"
#include "repl-interacter.hh"
#include "file-system.hh"
#include "libcmd/repl.hh"
namespace nix {
namespace {
// Used to communicate to NixRepl::getLine whether a signal occurred in ::readline.
volatile sig_atomic_t g_signal_received = 0;
void sigintHandler(int signo)
{
g_signal_received = signo;
}
};
static detail::ReplCompleterMixin * curRepl; // ugly
static char * completionCallback(char * s, int * match)
{
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
auto * res = strdup(possible.begin()->c_str() + strlen(s));
if (!res)
throw Error("allocation failure");
return res;
} else if (possible.size() > 1) {
auto checkAllHaveSameAt = [&](size_t pos) {
auto & first = *possible.begin();
for (auto & p : possible) {
if (p.size() <= pos || p[pos] != first[pos])
return false;
}
return true;
};
size_t start = strlen(s);
size_t len = 0;
while (checkAllHaveSameAt(start + len))
++len;
if (len > 0) {
*match = 1;
auto * res = strdup(std::string(*possible.begin(), start, len).c_str());
if (!res)
throw Error("allocation failure");
return res;
}
}
*match = 0;
return nullptr;
}
static int listPossibleCallback(char * s, char *** avp)
{
auto possible = curRepl->completePrefix(s);
if (possible.size() > (INT_MAX / sizeof(char *)))
throw Error("too many completions");
int ac = 0;
char ** vp = nullptr;
auto check = [&](auto * p) {
if (!p) {
if (vp) {
while (--ac >= 0)
free(vp[ac]);
free(vp);
}
throw Error("allocation failure");
}
return p;
};
vp = check((char **) malloc(possible.size() * sizeof(char *)));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
*avp = vp;
return ac;
}
ReadlineLikeInteracter::Guard ReadlineLikeInteracter::init(detail::ReplCompleterMixin * repl)
{
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
try {
createDirs(dirOf(historyFile));
} catch (SystemError & e) {
logWarning(e.info());
}
#ifndef USE_READLINE
el_hist_size = 1000;
#endif
read_history(historyFile.c_str());
auto oldRepl = curRepl;
curRepl = repl;
Guard restoreRepl([oldRepl] { curRepl = oldRepl; });
#ifndef USE_READLINE
rl_set_complete_func(completionCallback);
rl_set_list_possib_func(listPossibleCallback);
#endif
return restoreRepl;
}
static constexpr const char * promptForType(ReplPromptType promptType)
{
switch (promptType) {
case ReplPromptType::ReplPrompt:
return "nix-repl> ";
case ReplPromptType::ContinuationPrompt:
return " ";
}
assert(false);
}
bool ReadlineLikeInteracter::getLine(std::string & input, ReplPromptType promptType)
{
struct sigaction act, old;
sigset_t savedSignalMask, set;
auto setupSignals = [&]() {
act.sa_handler = sigintHandler;
sigfillset(&act.sa_mask);
act.sa_flags = 0;
if (sigaction(SIGINT, &act, &old))
throw SysError("installing handler for SIGINT");
sigemptyset(&set);
sigaddset(&set, SIGINT);
if (sigprocmask(SIG_UNBLOCK, &set, &savedSignalMask))
throw SysError("unblocking SIGINT");
};
auto restoreSignals = [&]() {
if (sigprocmask(SIG_SETMASK, &savedSignalMask, nullptr))
throw SysError("restoring signals");
if (sigaction(SIGINT, &old, 0))
throw SysError("restoring handler for SIGINT");
};
setupSignals();
char * s = readline(promptForType(promptType));
Finally doFree([&]() { free(s); });
restoreSignals();
if (g_signal_received) {
g_signal_received = 0;
input.clear();
return true;
}
if (!s)
return false;
input += s;
input += '\n';
return true;
}
ReadlineLikeInteracter::~ReadlineLikeInteracter()
{
write_history(historyFile.c_str());
}
};
+48
View File
@@ -0,0 +1,48 @@
#pragma once
/// @file
#include "finally.hh"
#include "types.hh"
#include <functional>
#include <string>
namespace nix {
namespace detail {
/** Provides the completion hooks for the repl, without exposing its complete
* internals. */
struct ReplCompleterMixin {
virtual StringSet completePrefix(const std::string & prefix) = 0;
};
};
enum class ReplPromptType {
ReplPrompt,
ContinuationPrompt,
};
class ReplInteracter
{
public:
using Guard = Finally<std::function<void()>>;
virtual Guard init(detail::ReplCompleterMixin * repl) = 0;
/** Returns a boolean of whether the interacter got EOF */
virtual bool getLine(std::string & input, ReplPromptType promptType) = 0;
virtual ~ReplInteracter(){};
};
class ReadlineLikeInteracter : public virtual ReplInteracter
{
std::string historyFile;
public:
ReadlineLikeInteracter(std::string historyFile)
: historyFile(historyFile)
{
}
virtual Guard init(detail::ReplCompleterMixin * repl) override;
virtual bool getLine(std::string & input, ReplPromptType promptType) override;
virtual ~ReadlineLikeInteracter() override;
};
};
+79 -198
View File
@@ -3,32 +3,17 @@
#include <cstring>
#include <climits>
#include <setjmp.h>
#ifdef USE_READLINE
#include <readline/history.h>
#include <readline/readline.h>
#else
// editline < 1.15.2 don't wrap their API for C++ usage
// (added in https://github.com/troglobit/editline/commit/91398ceb3427b730995357e9d120539fb9bb7461).
// This results in linker errors due to to name-mangling of editline C symbols.
// For compatibility with these versions, we wrap the API here
// (wrapping multiple times on newer versions is no problem).
extern "C" {
#include <editline.h>
}
#endif
#include "libcmd/repl-interacter.hh"
#include "repl.hh"
#include "ansicolor.hh"
#include "signals.hh"
#include "shared.hh"
#include "eval.hh"
#include "eval-cache.hh"
#include "eval-inline.hh"
#include "eval-settings.hh"
#include "attr-path.hh"
#include "signals.hh"
#include "store-api.hh"
#include "log-store.hh"
#include "common-eval-args.hh"
@@ -38,7 +23,6 @@ extern "C" {
#include "flake/flake.hh"
#include "flake/lockfile.hh"
#include "users.hh"
#include "terminal.hh"
#include "editor-for.hh"
#include "finally.hh"
#include "markdown.hh"
@@ -52,8 +36,30 @@ extern "C" {
namespace nix {
/**
* Returned by `NixRepl::processLine`.
*/
enum class ProcessLineResult {
/**
* The user exited with `:quit`. The REPL should exit. The surrounding
* program or evaluation (e.g., if the REPL was acting as the debugger)
* should also exit.
*/
Quit,
/**
* The user exited with `:continue`. The REPL should exit, but the program
* should continue running.
*/
Continue,
/**
* The user did not exit. The REPL should request another line of input.
*/
PromptAgain,
};
struct NixRepl
: AbstractNixRepl
, detail::ReplCompleterMixin
#if HAVE_BOEHMGC
, gc
#endif
@@ -69,19 +75,18 @@ struct NixRepl
int displ;
StringSet varNames;
const Path historyFile;
std::unique_ptr<ReplInteracter> interacter;
NixRepl(const SearchPath & searchPath, nix::ref<Store> store,ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
virtual ~NixRepl();
virtual ~NixRepl() = default;
void mainLoop() override;
ReplExitStatus mainLoop() override;
void initEnv() override;
StringSet completePrefix(const std::string & prefix);
bool getLine(std::string & input, const std::string & prompt);
virtual StringSet completePrefix(const std::string & prefix) override;
StorePath getDerivationPath(Value & v);
bool processLine(std::string line);
ProcessLineResult processLine(std::string line);
void loadFile(const Path & path);
void loadFlake(const std::string & flakeRef);
@@ -101,7 +106,9 @@ struct NixRepl
.ansiColors = true,
.force = true,
.derivationPaths = true,
.maxDepth = maxDepth
.maxDepth = maxDepth,
.prettyIndent = 2,
.errors = ErrorPrintBehavior::ThrowTopLevel,
});
}
};
@@ -121,16 +128,10 @@ NixRepl::NixRepl(const SearchPath & searchPath, nix::ref<Store> store, ref<EvalS
, debugTraceIndex(0)
, getValues(getValues)
, staticEnv(new StaticEnv(nullptr, state->staticBaseEnv.get()))
, historyFile(getDataDir() + "/nix/repl-history")
, interacter(make_unique<ReadlineLikeInteracter>(getDataDir() + "/nix/repl-history"))
{
}
NixRepl::~NixRepl()
{
write_history(historyFile.c_str());
}
void runNix(Path program, const Strings & args,
const std::optional<std::string> & input = {})
{
@@ -147,79 +148,6 @@ void runNix(Path program, const Strings & args,
return;
}
static NixRepl * curRepl; // ugly
static char * completionCallback(char * s, int *match) {
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
auto *res = strdup(possible.begin()->c_str() + strlen(s));
if (!res) throw Error("allocation failure");
return res;
} else if (possible.size() > 1) {
auto checkAllHaveSameAt = [&](size_t pos) {
auto &first = *possible.begin();
for (auto &p : possible) {
if (p.size() <= pos || p[pos] != first[pos])
return false;
}
return true;
};
size_t start = strlen(s);
size_t len = 0;
while (checkAllHaveSameAt(start + len)) ++len;
if (len > 0) {
*match = 1;
auto *res = strdup(std::string(*possible.begin(), start, len).c_str());
if (!res) throw Error("allocation failure");
return res;
}
}
*match = 0;
return nullptr;
}
static int listPossibleCallback(char *s, char ***avp) {
auto possible = curRepl->completePrefix(s);
if (possible.size() > (INT_MAX / sizeof(char*)))
throw Error("too many completions");
int ac = 0;
char **vp = nullptr;
auto check = [&](auto *p) {
if (!p) {
if (vp) {
while (--ac >= 0)
free(vp[ac]);
free(vp);
}
throw Error("allocation failure");
}
return p;
};
vp = check((char **)malloc(possible.size() * sizeof(char*)));
for (auto & p : possible)
vp[ac++] = check(strdup(p.c_str()));
*avp = vp;
return ac;
}
namespace {
// Used to communicate to NixRepl::getLine whether a signal occurred in ::readline.
volatile sig_atomic_t g_signal_received = 0;
void sigintHandler(int signo) {
g_signal_received = signo;
}
}
static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positions, const DebugTrace & dt)
{
if (dt.isError)
@@ -232,7 +160,7 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
if (pos) {
out << pos;
out << *pos;
if (auto loc = pos->getCodeLines()) {
out << "\n";
printCodeLines(out, "", *pos, *loc);
@@ -243,31 +171,23 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
return out;
}
void NixRepl::mainLoop()
static bool isFirstRepl = true;
ReplExitStatus NixRepl::mainLoop()
{
std::string error = ANSI_RED "error:" ANSI_NORMAL " ";
notice("Welcome to Nix " + nixVersion + ". Type :? for help.\n");
if (isFirstRepl) {
std::string_view debuggerNotice = "";
if (state->debugRepl) {
debuggerNotice = " debugger";
}
notice("Nix %1%%2%\nType :? for help.", nixVersion, debuggerNotice);
}
isFirstRepl = false;
loadFiles();
// Allow nix-repl specific settings in .inputrc
rl_readline_name = "nix-repl";
try {
createDirs(dirOf(historyFile));
} catch (SystemError & e) {
logWarning(e.info());
}
#ifndef USE_READLINE
el_hist_size = 1000;
#endif
read_history(historyFile.c_str());
auto oldRepl = curRepl;
curRepl = this;
Finally restoreRepl([&] { curRepl = oldRepl; });
#ifndef USE_READLINE
rl_set_complete_func(completionCallback);
rl_set_list_possib_func(listPossibleCallback);
#endif
auto _guard = interacter->init(static_cast<detail::ReplCompleterMixin *>(this));
std::string input;
@@ -276,16 +196,26 @@ void NixRepl::mainLoop()
logger->pause();
// When continuing input from previous lines, don't print a prompt, just align to the same
// number of chars as the prompt.
if (!getLine(input, input.empty() ? "nix-repl> " : " ")) {
// ctrl-D should exit the debugger.
if (!interacter->getLine(input, input.empty() ? ReplPromptType::ReplPrompt : ReplPromptType::ContinuationPrompt)) {
// Ctrl-D should exit the debugger.
state->debugStop = false;
state->debugQuit = true;
logger->cout("");
break;
// TODO: Should Ctrl-D exit just the current debugger session or
// the entire program?
return ReplExitStatus::QuitAll;
}
logger->resume();
try {
if (!removeWhitespace(input).empty() && !processLine(input)) return;
switch (processLine(input)) {
case ProcessLineResult::Quit:
return ReplExitStatus::QuitAll;
case ProcessLineResult::Continue:
return ReplExitStatus::Continue;
case ProcessLineResult::PromptAgain:
break;
default:
abort();
}
} catch (ParseError & e) {
if (e.msg().find("unexpected end of file") != std::string::npos) {
// For parse errors on incomplete input, we continue waiting for the next line of
@@ -295,13 +225,7 @@ void NixRepl::mainLoop()
printMsg(lvlError, e.msg());
}
} catch (EvalError & e) {
// in debugger mode, an EvalError should trigger another repl session.
// when that session returns the exception will land here. No need to show it again;
// show the error for this repl session instead.
if (state->debugRepl && !state->debugTraces.empty())
showDebugTrace(std::cout, state->positions, state->debugTraces.front());
else
printMsg(lvlError, e.msg());
printMsg(lvlError, e.msg());
} catch (Error & e) {
printMsg(lvlError, e.msg());
} catch (Interrupted & e) {
@@ -315,52 +239,6 @@ void NixRepl::mainLoop()
}
}
bool NixRepl::getLine(std::string & input, const std::string & prompt)
{
struct sigaction act, old;
sigset_t savedSignalMask, set;
auto setupSignals = [&]() {
act.sa_handler = sigintHandler;
sigfillset(&act.sa_mask);
act.sa_flags = 0;
if (sigaction(SIGINT, &act, &old))
throw SysError("installing handler for SIGINT");
sigemptyset(&set);
sigaddset(&set, SIGINT);
if (sigprocmask(SIG_UNBLOCK, &set, &savedSignalMask))
throw SysError("unblocking SIGINT");
};
auto restoreSignals = [&]() {
if (sigprocmask(SIG_SETMASK, &savedSignalMask, nullptr))
throw SysError("restoring signals");
if (sigaction(SIGINT, &old, 0))
throw SysError("restoring handler for SIGINT");
};
setupSignals();
Finally resetTerminal([&]() { rl_deprep_terminal(); });
char * s = readline(prompt.c_str());
Finally doFree([&]() { free(s); });
restoreSignals();
if (g_signal_received) {
g_signal_received = 0;
input.clear();
return true;
}
if (!s)
return false;
input += s;
input += '\n';
return true;
}
StringSet NixRepl::completePrefix(const std::string & prefix)
{
StringSet completions;
@@ -422,8 +300,6 @@ StringSet NixRepl::completePrefix(const std::string & prefix)
// Quietly ignore parse errors.
} catch (EvalError & e) {
// Quietly ignore evaluation errors.
} catch (UndefinedVarError & e) {
// Quietly ignore undefined variable errors.
} catch (BadURL & e) {
// Quietly ignore BadURL flake-related errors.
}
@@ -475,10 +351,11 @@ void NixRepl::loadDebugTraceEnv(DebugTrace & dt)
}
}
bool NixRepl::processLine(std::string line)
ProcessLineResult NixRepl::processLine(std::string line)
{
line = trim(line);
if (line == "") return true;
if (line.empty())
return ProcessLineResult::PromptAgain;
_isInterrupted = false;
@@ -509,6 +386,7 @@ bool NixRepl::processLine(std::string line)
<< " :l, :load <path> Load Nix expression and add it to scope\n"
<< " :lf, :load-flake <ref> Load Nix flake and add it to scope\n"
<< " :p, :print <expr> Evaluate and print expression recursively\n"
<< " Strings are printed directly, without escaping.\n"
<< " :q, :quit Exit nix-repl\n"
<< " :r, :reload Reload all files\n"
<< " :sh <expr> Build dependencies of derivation, then start\n"
@@ -573,13 +451,13 @@ bool NixRepl::processLine(std::string line)
else if (state->debugRepl && (command == ":s" || command == ":step")) {
// set flag to stop at next DebugTrace; exit repl.
state->debugStop = true;
return false;
return ProcessLineResult::Continue;
}
else if (state->debugRepl && (command == ":c" || command == ":continue")) {
// set flag to run to next breakpoint or end of program; exit repl.
state->debugStop = false;
return false;
return ProcessLineResult::Continue;
}
else if (command == ":a" || command == ":add") {
@@ -716,14 +594,17 @@ bool NixRepl::processLine(std::string line)
else if (command == ":p" || command == ":print") {
Value v;
evalString(arg, v);
printValue(std::cout, v);
if (v.type() == nString) {
std::cout << v.string_view();
} else {
printValue(std::cout, v);
}
std::cout << std::endl;
}
else if (command == ":q" || command == ":quit") {
state->debugStop = false;
state->debugQuit = true;
return false;
return ProcessLineResult::Quit;
}
else if (command == ":doc") {
@@ -784,7 +665,7 @@ bool NixRepl::processLine(std::string line)
}
}
return true;
return ProcessLineResult::PromptAgain;
}
void NixRepl::loadFile(const Path & path)
@@ -890,7 +771,7 @@ void NixRepl::addVarToScope(const Symbol name, Value & v)
Expr * NixRepl::parseString(std::string s)
{
return state->parseExprFromString(std::move(s), state->rootPath(CanonPath::fromCwd()), staticEnv);
return state->parseExprFromString(std::move(s), state->rootPath("."), staticEnv);
}
@@ -915,7 +796,7 @@ std::unique_ptr<AbstractNixRepl> AbstractNixRepl::create(
}
void AbstractNixRepl::runSimple(
ReplExitStatus AbstractNixRepl::runSimple(
ref<EvalState> evalState,
const ValMap & extraEnv)
{
@@ -937,7 +818,7 @@ void AbstractNixRepl::runSimple(
for (auto & [name, value] : extraEnv)
repl->addVarToScope(repl->state->symbols.create(name), *value);
repl->mainLoop();
return repl->mainLoop();
}
}
+2 -7
View File
@@ -3,11 +3,6 @@
#include "eval.hh"
#if HAVE_BOEHMGC
#define GC_INCLUDE_NEW
#include <gc/gc_cpp.h>
#endif
namespace nix {
struct AbstractNixRepl
@@ -28,13 +23,13 @@ struct AbstractNixRepl
const SearchPath & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
static void runSimple(
static ReplExitStatus runSimple(
ref<EvalState> evalState,
const ValMap & extraEnv);
virtual void initEnv() = 0;
virtual void mainLoop() = 0;
virtual ReplExitStatus mainLoop() = 0;
};
}
+4 -4
View File
@@ -65,10 +65,10 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
if (!attrIndex) {
if (v->type() != nAttrs)
throw TypeError(
state.error<TypeError>(
"the expression selected by the selection path '%1%' should be a set but is %2%",
attrPath,
showType(*v));
showType(*v)).debugThrow();
if (attr.empty())
throw Error("empty attribute name in selection path '%1%'", attrPath);
@@ -88,10 +88,10 @@ std::pair<Value *, PosIdx> findAlongAttrPath(EvalState & state, const std::strin
else {
if (!v->isList())
throw TypeError(
state.error<TypeError>(
"the expression selected by the selection path '%1%' should be a list but is %2%",
attrPath,
showType(*v));
showType(*v)).debugThrow();
if (*attrIndex >= v->listSize())
throw AttrPathNotFound("list index %1% in selection path '%2%' is out of range", *attrIndex, attrPath);
+15 -15
View File
@@ -491,7 +491,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
if (forceErrors)
debug("reevaluating failed cached attribute '%s'", getAttrPathStr(name));
else
throw CachedEvalError("cached failure of attribute '%s'", getAttrPathStr(name));
throw CachedEvalError(root->state, "cached failure of attribute '%s'", getAttrPathStr(name));
} else
return std::make_shared<AttrCursor>(root,
std::make_pair(shared_from_this(), name), nullptr, std::move(attr));
@@ -500,7 +500,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
// evaluate to see whether 'name' exists
} else
return nullptr;
//throw TypeError("'%s' is not an attribute set", getAttrPathStr());
//error<TypeError>("'%s' is not an attribute set", getAttrPathStr()).debugThrow();
}
}
@@ -508,7 +508,7 @@ std::shared_ptr<AttrCursor> AttrCursor::maybeGetAttr(Symbol name, bool forceErro
if (v.type() != nAttrs)
return nullptr;
//throw TypeError("'%s' is not an attribute set", getAttrPathStr());
//error<TypeError>("'%s' is not an attribute set", getAttrPathStr()).debugThrow();
auto attr = v.attrs->get(name);
@@ -574,14 +574,14 @@ std::string AttrCursor::getString()
debug("using cached string attribute '%s'", getAttrPathStr());
return s->first;
} else
root->state.error("'%s' is not a string", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a string", getAttrPathStr()).debugThrow();
}
}
auto & v = forceValue();
if (v.type() != nString && v.type() != nPath)
root->state.error("'%s' is not a string but %s", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr(), showType(v)).debugThrow();
return v.type() == nString ? v.c_str() : v.path().to_string();
}
@@ -616,7 +616,7 @@ string_t AttrCursor::getStringWithContext()
return *s;
}
} else
root->state.error("'%s' is not a string", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a string", getAttrPathStr()).debugThrow();
}
}
@@ -630,7 +630,7 @@ string_t AttrCursor::getStringWithContext()
else if (v.type() == nPath)
return {v.path().to_string(), {}};
else
root->state.error("'%s' is not a string but %s", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a string but %s", getAttrPathStr(), showType(v)).debugThrow();
}
bool AttrCursor::getBool()
@@ -643,14 +643,14 @@ bool AttrCursor::getBool()
debug("using cached Boolean attribute '%s'", getAttrPathStr());
return *b;
} else
root->state.error("'%s' is not a Boolean", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a Boolean", getAttrPathStr()).debugThrow();
}
}
auto & v = forceValue();
if (v.type() != nBool)
root->state.error("'%s' is not a Boolean", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not a Boolean", getAttrPathStr()).debugThrow();
return v.boolean;
}
@@ -665,14 +665,14 @@ NixInt AttrCursor::getInt()
debug("using cached integer attribute '%s'", getAttrPathStr());
return i->x;
} else
throw TypeError("'%s' is not an integer", getAttrPathStr());
root->state.error<TypeError>("'%s' is not an integer", getAttrPathStr()).debugThrow();
}
}
auto & v = forceValue();
if (v.type() != nInt)
throw TypeError("'%s' is not an integer", getAttrPathStr());
root->state.error<TypeError>("'%s' is not an integer", getAttrPathStr()).debugThrow();
return v.integer;
}
@@ -687,7 +687,7 @@ std::vector<std::string> AttrCursor::getListOfStrings()
debug("using cached list of strings attribute '%s'", getAttrPathStr());
return *l;
} else
throw TypeError("'%s' is not a list of strings", getAttrPathStr());
root->state.error<TypeError>("'%s' is not a list of strings", getAttrPathStr()).debugThrow();
}
}
@@ -697,7 +697,7 @@ std::vector<std::string> AttrCursor::getListOfStrings()
root->state.forceValue(v, noPos);
if (v.type() != nList)
throw TypeError("'%s' is not a list", getAttrPathStr());
root->state.error<TypeError>("'%s' is not a list", getAttrPathStr()).debugThrow();
std::vector<std::string> res;
@@ -720,14 +720,14 @@ std::vector<Symbol> AttrCursor::getAttrs()
debug("using cached attrset attribute '%s'", getAttrPathStr());
return *attrs;
} else
root->state.error("'%s' is not an attribute set", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not an attribute set", getAttrPathStr()).debugThrow();
}
}
auto & v = forceValue();
if (v.type() != nAttrs)
root->state.error("'%s' is not an attribute set", getAttrPathStr()).debugThrow<TypeError>();
root->state.error<TypeError>("'%s' is not an attribute set", getAttrPathStr()).debugThrow();
std::vector<Symbol> attrs;
for (auto & attr : *getValue().attrs)
+105
View File
@@ -0,0 +1,105 @@
#include "eval-error.hh"
#include "eval.hh"
#include "value.hh"
namespace nix {
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withExitStatus(unsigned int exitStatus)
{
error.withExitStatus(exitStatus);
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::atPos(PosIdx pos)
{
error.err.pos = error.state.positions[pos];
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::atPos(Value & value, PosIdx fallback)
{
return atPos(value.determinePos(fallback));
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withTrace(PosIdx pos, const std::string_view text)
{
error.err.traces.push_front(
Trace{.pos = error.state.positions[pos], .hint = HintFmt(std::string(text))});
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withSuggestions(Suggestions & s)
{
error.err.suggestions = s;
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr & expr)
{
// NOTE: This is abusing side-effects.
// TODO: check compatibility with nested debugger calls.
// TODO: What side-effects??
error.state.debugTraces.push_front(DebugTrace{
.pos = error.state.positions[expr.getPos()],
.expr = expr,
.env = env,
.hint = HintFmt("Fake frame for debugging purposes"),
.isError = true});
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::addTrace(PosIdx pos, HintFmt hint)
{
error.addTrace(error.state.positions[pos], hint);
return *this;
}
template<class T>
template<typename... Args>
EvalErrorBuilder<T> &
EvalErrorBuilder<T>::addTrace(PosIdx pos, std::string_view formatString, const Args &... formatArgs)
{
addTrace(error.state.positions[pos], HintFmt(std::string(formatString), formatArgs...));
return *this;
}
template<class T>
void EvalErrorBuilder<T>::debugThrow()
{
if (error.state.debugRepl && !error.state.debugTraces.empty()) {
const DebugTrace & last = error.state.debugTraces.front();
const Env * env = &last.env;
const Expr * expr = &last.expr;
error.state.runDebugRepl(&error, *env, *expr);
}
// `EvalState` is the only class that can construct an `EvalErrorBuilder`,
// and it does so in dynamic storage. This is the final method called on
// any such instance and must delete itself before throwing the underlying
// error.
auto error = std::move(this->error);
delete this;
throw error;
}
template class EvalErrorBuilder<EvalError>;
template class EvalErrorBuilder<AssertionError>;
template class EvalErrorBuilder<ThrownError>;
template class EvalErrorBuilder<Abort>;
template class EvalErrorBuilder<TypeError>;
template class EvalErrorBuilder<UndefinedVarError>;
template class EvalErrorBuilder<MissingArgumentError>;
template class EvalErrorBuilder<InfiniteRecursionError>;
template class EvalErrorBuilder<CachedEvalError>;
template class EvalErrorBuilder<InvalidPathError>;
}
+104
View File
@@ -0,0 +1,104 @@
#pragma once
#include <algorithm>
#include "error.hh"
#include "pos-idx.hh"
namespace nix {
struct Env;
struct Expr;
struct Value;
class EvalState;
template<class T>
class EvalErrorBuilder;
class EvalError : public Error
{
template<class T>
friend class EvalErrorBuilder;
public:
EvalState & state;
EvalError(EvalState & state, ErrorInfo && errorInfo)
: Error(errorInfo)
, state(state)
{
}
template<typename... Args>
explicit EvalError(EvalState & state, const std::string & formatString, const Args &... formatArgs)
: Error(formatString, formatArgs...)
, state(state)
{
}
};
MakeError(ParseError, Error);
MakeError(AssertionError, EvalError);
MakeError(ThrownError, AssertionError);
MakeError(Abort, EvalError);
MakeError(TypeError, EvalError);
MakeError(UndefinedVarError, EvalError);
MakeError(MissingArgumentError, EvalError);
MakeError(CachedEvalError, EvalError);
MakeError(InfiniteRecursionError, EvalError);
struct InvalidPathError : public EvalError
{
public:
Path path;
InvalidPathError(EvalState & state, const Path & path)
: EvalError(state, "path '%s' is not valid", path)
{
}
};
/**
* `EvalErrorBuilder`s may only be constructed by `EvalState`. The `debugThrow`
* method must be the final method in any such `EvalErrorBuilder` usage, and it
* handles deleting the object.
*/
template<class T>
class EvalErrorBuilder final
{
friend class EvalState;
template<typename... Args>
explicit EvalErrorBuilder(EvalState & state, const Args &... args)
: error(T(state, args...))
{
}
public:
T error;
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withExitStatus(unsigned int exitStatus);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & atPos(PosIdx pos);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & atPos(Value & value, PosIdx fallback = noPos);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withTrace(PosIdx pos, const std::string_view text);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withFrameTrace(PosIdx pos, const std::string_view text);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withSuggestions(Suggestions & s);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withFrame(const Env & e, const Expr & ex);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & addTrace(PosIdx pos, HintFmt hint);
template<typename... Args>
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> &
addTrace(PosIdx pos, std::string_view formatString, const Args &... formatArgs);
/**
* Delete the `EvalErrorBuilder` and throw the underlying exception.
*/
[[gnu::noinline, gnu::noreturn]] void debugThrow();
};
}
+11 -8
View File
@@ -3,6 +3,7 @@
#include "print.hh"
#include "eval.hh"
#include "eval-error.hh"
namespace nix {
@@ -115,10 +116,11 @@ inline void EvalState::forceAttrs(Value & v, Callable getPos, std::string_view e
PosIdx pos = getPos();
forceValue(v, pos);
if (v.type() != nAttrs) {
error("expected a set but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx).debugThrow<TypeError>();
error<TypeError>(
"expected a set but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).withTrace(pos, errorCtx).debugThrow();
}
}
@@ -128,10 +130,11 @@ inline void EvalState::forceList(Value & v, const PosIdx pos, std::string_view e
{
forceValue(v, pos);
if (!v.isList()) {
error("expected a list but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx).debugThrow<TypeError>();
error<TypeError>(
"expected a list but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).withTrace(pos, errorCtx).debugThrow();
}
}
+29 -7
View File
@@ -21,11 +21,24 @@ struct EvalSettings : Config
Setting<Strings> nixPath{
this, getDefaultNixPath(), "nix-path",
R"(
List of directories to be searched for `<...>` file references
List of search paths to use for [lookup path](@docroot@/language/constructs/lookup-path.md) resolution.
This setting determines the value of
[`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath) and can be used with [`builtins.findFile`](@docroot@/language/builtin-constants.md#builtins-findFile).
In particular, outside of [pure evaluation mode](#conf-pure-eval), this determines the value of
[`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath).
)"};
The default value is
```
$HOME/.nix-defexpr/channels
nixpkgs=$NIX_STATE_DIR/profiles/per-user/root/channels/nixpkgs
$NIX_STATE_DIR/profiles/per-user/root/channels
```
It can be overridden with the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH) or the [`-I` command line option](@docroot@/command-ref/opt-common.md#opt-I).
> **Note**
>
> If [pure evaluation](#conf-pure-eval) is enabled, `nixPath` evaluates to the empty list `[ ]`.
)", {}, false};
Setting<std::string> currentSystem{
this, "", "eval-system",
@@ -55,8 +68,6 @@ struct EvalSettings : Config
[`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath),
or to URIs outside of
[`allowed-uris`](@docroot@/command-ref/conf-file.md#conf-allowed-uris).
Also the default value for [`nix-path`](#conf-nix-path) is ignored, such that only explicitly set search path entries are taken into account.
)"};
Setting<bool> pureEval{this, false, "pure-eval",
@@ -65,9 +76,10 @@ struct EvalSettings : Config
- Restrict file system and network access to files specified by cryptographic hash
- Disable impure constants:
- [`bultins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
- [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem)
- [`builtins.currentTime`](@docroot@/language/builtin-constants.md#builtins-currentTime)
- [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath)
- [`builtins.storePath`](@docroot@/language/builtin-constants.md#builtins-storePath)
)"
};
@@ -127,6 +139,16 @@ struct EvalSettings : Config
Setting<unsigned int> maxCallDepth{this, 10000, "max-call-depth",
"The maximum function call depth to allow before erroring."};
Setting<bool> builtinsTraceDebugger{this, false, "debugger-on-trace",
R"(
If set to true and the `--debugger` flag is given,
[`builtins.trace`](@docroot@/language/builtins.md#builtins-trace) will
enter the debugger like
[`builtins.break`](@docroot@/language/builtins.md#builtins-break).
This is useful for debugging warnings in third-party Nix code.
)"};
};
extern EvalSettings evalSettings;
+261 -185
View File
@@ -3,6 +3,7 @@
#include "hash.hh"
#include "primops.hh"
#include "print-options.hh"
#include "shared.hh"
#include "types.hh"
#include "util.hh"
#include "store-api.hh"
@@ -339,46 +340,6 @@ void initGC()
gcInitialised = true;
}
ErrorBuilder & ErrorBuilder::atPos(PosIdx pos)
{
info.errPos = state.positions[pos];
return *this;
}
ErrorBuilder & ErrorBuilder::withTrace(PosIdx pos, const std::string_view text)
{
info.traces.push_front(Trace{ .pos = state.positions[pos], .hint = hintformat(std::string(text)), .frame = false });
return *this;
}
ErrorBuilder & ErrorBuilder::withFrameTrace(PosIdx pos, const std::string_view text)
{
info.traces.push_front(Trace{ .pos = state.positions[pos], .hint = hintformat(std::string(text)), .frame = true });
return *this;
}
ErrorBuilder & ErrorBuilder::withSuggestions(Suggestions & s)
{
info.suggestions = s;
return *this;
}
ErrorBuilder & ErrorBuilder::withFrame(const Env & env, const Expr & expr)
{
// NOTE: This is abusing side-effects.
// TODO: check compatibility with nested debugger calls.
state.debugTraces.push_front(DebugTrace {
.pos = nullptr,
.expr = expr,
.env = env,
.hint = hintformat("Fake frame for debugging purposes"),
.isError = true
});
return *this;
}
EvalState::EvalState(
const SearchPath & _searchPath,
ref<Store> store,
@@ -434,14 +395,14 @@ EvalState::EvalState(
, emptyBindings(0)
, rootFS(
evalSettings.restrictEval || evalSettings.pureEval
? ref<InputAccessor>(AllowListInputAccessor::create(makeFSInputAccessor(CanonPath::root), {},
? ref<InputAccessor>(AllowListInputAccessor::create(makeFSInputAccessor(), {},
[](const CanonPath & path) -> RestrictedPathError {
auto modeInformation = evalSettings.pureEval
? "in pure evaluation mode (use '--impure' to override)"
: "in restricted mode";
throw RestrictedPathError("access to absolute path '%1%' is forbidden %2%", path, modeInformation);
}))
: makeFSInputAccessor(CanonPath::root))
: makeFSInputAccessor())
, corepkgsFS(makeMemoryInputAccessor())
, internalFS(makeMemoryInputAccessor())
, derivationInternal{corepkgsFS->addFile(
@@ -456,7 +417,6 @@ EvalState::EvalState(
, buildStore(buildStore ? buildStore : store)
, debugRepl(nullptr)
, debugStop(false)
, debugQuit(false)
, trylevel(0)
, regexCache(makeRegexCache())
#if HAVE_BOEHMGC
@@ -475,7 +435,14 @@ EvalState::EvalState(
static_assert(sizeof(Env) <= 16, "environment must be <= 16 bytes");
vEmptyList.mkList(0);
vEmptyList.mkList(buildList(0));
vNull.mkNull();
vTrue.mkBool(true);
vFalse.mkBool(false);
vStringRegular.mkString("regular");
vStringDirectory.mkString("directory");
vStringSymlink.mkString("symlink");
vStringUnknown.mkString("unknown");
/* Initialise the Nix expression search path. */
if (!evalSettings.pureEval) {
@@ -507,13 +474,13 @@ EvalState::~EvalState()
void EvalState::allowPath(const Path & path)
{
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListInputAccessor>())
rootFS2->allowPath(CanonPath(path));
rootFS2->allowPrefix(CanonPath(path));
}
void EvalState::allowPath(const StorePath & storePath)
{
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListInputAccessor>())
rootFS2->allowPath(CanonPath(store->toRealPath(storePath)));
rootFS2->allowPrefix(CanonPath(store->toRealPath(storePath)));
}
void EvalState::allowAndSetStorePathString(const StorePath & storePath, Value & v)
@@ -744,7 +711,8 @@ void printEnvBindings(const SymbolTable & st, const StaticEnv & se, const Env &
if (se.up && env.up) {
std::cout << "static: ";
printStaticEnvBindings(st, se);
printWithBindings(st, env);
if (se.isWith)
printWithBindings(st, env);
std::cout << std::endl;
printEnvBindings(st, *se.up, *env.up, ++lvl);
} else {
@@ -756,7 +724,8 @@ void printEnvBindings(const SymbolTable & st, const StaticEnv & se, const Env &
std::cout << st[i.first] << " ";
std::cout << ANSI_NORMAL;
std::cout << std::endl;
printWithBindings(st, env); // probably nothing there for the top level.
if (se.isWith)
printWithBindings(st, env); // probably nothing there for the top level.
std::cout << std::endl;
}
@@ -778,7 +747,7 @@ void mapStaticEnvBindings(const SymbolTable & st, const StaticEnv & se, const En
if (env.up && se.up) {
mapStaticEnvBindings(st, *se.up, *env.up, vm);
if (!env.values[0]->isThunk()) {
if (se.isWith && !env.values[0]->isThunk()) {
// add 'with' bindings.
Bindings::iterator j = env.values[0]->attrs->begin();
while (j != env.values[0]->attrs->end()) {
@@ -800,10 +769,24 @@ std::unique_ptr<ValMap> mapStaticEnvBindings(const SymbolTable & st, const Stati
return vm;
}
/**
* Sets `inDebugger` to true on construction and false on destruction.
*/
class DebuggerGuard {
bool & inDebugger;
public:
DebuggerGuard(bool & inDebugger) : inDebugger(inDebugger) {
inDebugger = true;
}
~DebuggerGuard() {
inDebugger = false;
}
};
void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr & expr)
{
// double check we've got the debugRepl function pointer.
if (!debugRepl)
// Make sure we have a debugger to run and we're not already in a debugger.
if (!debugRepl || inDebugger)
return;
auto dts =
@@ -811,7 +794,7 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
? std::make_unique<DebugTraceStacker>(
*this,
DebugTrace {
.pos = error->info().errPos ? error->info().errPos : positions[expr.getPos()],
.pos = error->info().pos ? error->info().pos : positions[expr.getPos()],
.expr = expr,
.env = env,
.hint = error->info().msg,
@@ -821,45 +804,56 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
if (error)
{
printError("%s\n\n", error->what());
printError("%s\n", error->what());
if (trylevel > 0 && error->info().level != lvlInfo)
printError("This exception occurred in a 'tryEval' call. Use " ANSI_GREEN "--ignore-try" ANSI_NORMAL " to skip these.\n");
printError(ANSI_BOLD "Starting REPL to allow you to inspect the current state of the evaluator.\n" ANSI_NORMAL);
}
auto se = getStaticEnv(expr);
if (se) {
auto vm = mapStaticEnvBindings(symbols, *se.get(), env);
(debugRepl)(ref<EvalState>(shared_from_this()), *vm);
DebuggerGuard _guard(inDebugger);
auto exitStatus = (debugRepl)(ref<EvalState>(shared_from_this()), *vm);
switch (exitStatus) {
case ReplExitStatus::QuitAll:
if (error)
throw *error;
throw Exit(0);
case ReplExitStatus::Continue:
break;
default:
abort();
}
}
}
void EvalState::addErrorTrace(Error & e, const char * s, const std::string & s2) const
template<typename... Args>
void EvalState::addErrorTrace(Error & e, const Args & ... formatArgs) const
{
e.addTrace(nullptr, s, s2);
e.addTrace(nullptr, HintFmt(formatArgs...));
}
void EvalState::addErrorTrace(Error & e, const PosIdx pos, const char * s, const std::string & s2, bool frame) const
template<typename... Args>
void EvalState::addErrorTrace(Error & e, const PosIdx pos, const Args & ... formatArgs) const
{
e.addTrace(positions[pos], hintfmt(s, s2), frame);
e.addTrace(positions[pos], HintFmt(formatArgs...));
}
template<typename... Args>
static std::unique_ptr<DebugTraceStacker> makeDebugTraceStacker(
EvalState & state,
Expr & expr,
Env & env,
std::shared_ptr<Pos> && pos,
const char * s,
const std::string & s2)
const Args & ... formatArgs)
{
return std::make_unique<DebugTraceStacker>(state,
DebugTrace {
.pos = std::move(pos),
.expr = expr,
.env = env,
.hint = hintfmt(s, s2),
.hint = HintFmt(formatArgs...),
.isError = false
});
}
@@ -930,20 +924,22 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
return j->value;
}
if (!fromWith->parentWith)
error("undefined variable '%1%'", symbols[var.name]).atPos(var.pos).withFrame(*env, var).debugThrow<UndefinedVarError>();
error<UndefinedVarError>("undefined variable '%1%'", symbols[var.name]).atPos(var.pos).withFrame(*env, var).debugThrow();
for (size_t l = fromWith->prevWith; l; --l, env = env->up) ;
fromWith = fromWith->parentWith;
}
}
void EvalState::mkList(Value & v, size_t size)
ListBuilder::ListBuilder(EvalState & state, size_t size)
: size(size)
, elems(size <= 2 ? inlineElems : (Value * *) allocBytes(size * sizeof(Value *)))
{
v.mkList(size);
if (size > 2)
v.bigList.elems = (Value * *) allocBytes(size * sizeof(Value *));
nrListElems += size;
state.nrListElems += size;
}
Value * EvalState::getBool(bool b) {
return b ? &vTrue : &vFalse;
}
unsigned long nrThunks = 0;
@@ -962,12 +958,11 @@ void EvalState::mkThunk_(Value & v, Expr * expr)
void EvalState::mkPos(Value & v, PosIdx p)
{
auto pos = positions[p];
if (auto path = std::get_if<SourcePath>(&pos.origin)) {
auto origin = positions.originOf(p);
if (auto path = std::get_if<SourcePath>(&origin)) {
auto attrs = buildBindings(3);
attrs.alloc(sFile).mkString(path->path.abs());
attrs.alloc(sLine).mkInt(pos.line);
attrs.alloc(sColumn).mkInt(pos.column);
makePositionThunks(*this, p, attrs.alloc(sLine), attrs.alloc(sColumn));
v.mkAttrs(attrs);
} else
v.mkNull();
@@ -1136,7 +1131,7 @@ void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
// computation.
if (mustBeTrivial &&
!(dynamic_cast<ExprAttrs *>(e)))
error("file '%s' must be an attribute set", path).debugThrow<EvalError>();
error<EvalError>("file '%s' must be an attribute set", path).debugThrow();
eval(e, v);
} catch (Error & e) {
addErrorTrace(e, "while evaluating the file '%1%':", resolvedPath.to_string());
@@ -1167,10 +1162,11 @@ inline bool EvalState::evalBool(Env & env, Expr * e, const PosIdx pos, std::stri
Value v;
e->eval(*this, env, v);
if (v.type() != nBool)
error("expected a Boolean but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withFrame(env, *e).debugThrow<TypeError>();
error<TypeError>(
"expected a Boolean but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).withFrame(env, *e).debugThrow();
return v.boolean;
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
@@ -1184,10 +1180,11 @@ inline void EvalState::evalAttrs(Env & env, Expr * e, Value & v, const PosIdx po
try {
e->eval(*this, env, v);
if (v.type() != nAttrs)
error("expected a set but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withFrame(env, *e).debugThrow<TypeError>();
error<TypeError>(
"expected a set but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).withFrame(env, *e).debugThrow();
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
@@ -1224,6 +1221,18 @@ void ExprPath::eval(EvalState & state, Env & env, Value & v)
}
Env * ExprAttrs::buildInheritFromEnv(EvalState & state, Env & up)
{
Env & inheritEnv = state.allocEnv(inheritFromExprs->size());
inheritEnv.up = &up;
Displacement displ = 0;
for (auto from : *inheritFromExprs)
inheritEnv.values[displ++] = from->maybeThunk(state, up);
return &inheritEnv;
}
void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
{
v.mkAttrs(state.buildBindings(attrs.size() + dynamicAttrs.size()).finish());
@@ -1235,6 +1244,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Env & env2(state.allocEnv(attrs.size()));
env2.up = &env;
dynamicEnv = &env2;
Env * inheritEnv = inheritFromExprs ? buildInheritFromEnv(state, env2) : nullptr;
AttrDefs::iterator overrides = attrs.find(state.sOverrides);
bool hasOverrides = overrides != attrs.end();
@@ -1245,11 +1255,11 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Displacement displ = 0;
for (auto & i : attrs) {
Value * vAttr;
if (hasOverrides && !i.second.inherited) {
if (hasOverrides && i.second.kind != AttrDef::Kind::Inherited) {
vAttr = state.allocValue();
mkThunk(*vAttr, env2, i.second.e);
mkThunk(*vAttr, *i.second.chooseByKind(&env2, &env, inheritEnv), i.second.e);
} else
vAttr = i.second.e->maybeThunk(state, i.second.inherited ? env : env2);
vAttr = i.second.e->maybeThunk(state, *i.second.chooseByKind(&env2, &env, inheritEnv));
env2.values[displ++] = vAttr;
v.attrs->push_back(Attr(i.first, vAttr, i.second.pos));
}
@@ -1281,9 +1291,15 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
}
}
else
for (auto & i : attrs)
v.attrs->push_back(Attr(i.first, i.second.e->maybeThunk(state, env), i.second.pos));
else {
Env * inheritEnv = inheritFromExprs ? buildInheritFromEnv(state, env) : nullptr;
for (auto & i : attrs) {
v.attrs->push_back(Attr(
i.first,
i.second.e->maybeThunk(state, *i.second.chooseByKind(&env, &env, inheritEnv)),
i.second.pos));
}
}
/* Dynamic attrs apply *after* rec and __overrides. */
for (auto & i : dynamicAttrs) {
@@ -1296,7 +1312,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
auto nameSym = state.symbols.create(nameVal.string_view());
Bindings::iterator j = v.attrs->find(nameSym);
if (j != v.attrs->end())
state.error("dynamic attribute '%1%' already defined at %2%", state.symbols[nameSym], state.positions[j->pos]).atPos(i.pos).withFrame(env, *this).debugThrow<EvalError>();
state.error<EvalError>("dynamic attribute '%1%' already defined at %2%", state.symbols[nameSym], state.positions[j->pos]).atPos(i.pos).withFrame(env, *this).debugThrow();
i.valueExpr->setName(nameSym);
/* Keep sorted order so find can catch duplicates */
@@ -1315,12 +1331,30 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
Env & env2(state.allocEnv(attrs->attrs.size()));
env2.up = &env;
Env * inheritEnv = attrs->inheritFromExprs ? attrs->buildInheritFromEnv(state, env2) : nullptr;
/* The recursive attributes are evaluated in the new environment,
while the inherited attributes are evaluated in the original
environment. */
Displacement displ = 0;
for (auto & i : attrs->attrs)
env2.values[displ++] = i.second.e->maybeThunk(state, i.second.inherited ? env : env2);
for (auto & i : attrs->attrs) {
env2.values[displ++] = i.second.e->maybeThunk(
state,
*i.second.chooseByKind(&env2, &env, inheritEnv));
}
auto dts = state.debugRepl
? makeDebugTraceStacker(
state,
*this,
env2,
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while evaluating a '%1%' expression",
"let"
)
: nullptr;
body->eval(state, env2, v);
}
@@ -1328,9 +1362,10 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
void ExprList::eval(EvalState & state, Env & env, Value & v)
{
state.mkList(v, elems.size());
for (auto [n, v2] : enumerate(v.listItems()))
const_cast<Value * &>(v2) = elems[n]->maybeThunk(state, env);
auto list = state.buildList(elems.size());
for (const auto & [n, v2] : enumerate(list))
v2 = elems[n]->maybeThunk(state, env);
v.mkList(list);
}
@@ -1384,7 +1419,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
state.positions[pos2],
state.positions[getPos()],
"while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath))
: nullptr;
@@ -1408,8 +1443,8 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
for (auto & attr : *vAttrs->attrs)
allAttrNames.insert(state.symbols[attr.name]);
auto suggestions = Suggestions::bestMatches(allAttrNames, state.symbols[name]);
state.error("attribute '%1%' missing", state.symbols[name])
.atPos(pos).withSuggestions(suggestions).withFrame(env, *this).debugThrow<EvalError>();
state.error<EvalError>("attribute '%1%' missing", state.symbols[name])
.atPos(pos).withSuggestions(suggestions).withFrame(env, *this).debugThrow();
}
}
vAttrs = j->value;
@@ -1482,7 +1517,7 @@ public:
void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value & vRes, const PosIdx pos)
{
if (callDepth > evalSettings.maxCallDepth)
error("stack overflow; max-call-depth exceeded").atPos(pos).template debugThrow<EvalError>();
error<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
CallDepth _level(callDepth);
auto trace = evalSettings.traceFunctionCalls
@@ -1540,13 +1575,13 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
auto j = args[0]->attrs->get(i.name);
if (!j) {
if (!i.def) {
error("function '%1%' called without required argument '%2%'",
error<TypeError>("function '%1%' called without required argument '%2%'",
(lambda.name ? std::string(symbols[lambda.name]) : "anonymous lambda"),
symbols[i.name])
.atPos(lambda.pos)
.withTrace(pos, "from call site")
.withFrame(*fun.lambda.env, lambda)
.debugThrow<TypeError>();
.debugThrow();
}
env2.values[displ++] = i.def->maybeThunk(*this, env2);
} else {
@@ -1566,14 +1601,14 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
for (auto & formal : lambda.formals->formals)
formalNames.insert(symbols[formal.name]);
auto suggestions = Suggestions::bestMatches(formalNames, symbols[i.name]);
error("function '%1%' called with unexpected argument '%2%'",
error<TypeError>("function '%1%' called with unexpected argument '%2%'",
(lambda.name ? std::string(symbols[lambda.name]) : "anonymous lambda"),
symbols[i.name])
.atPos(lambda.pos)
.withTrace(pos, "from call site")
.withSuggestions(suggestions)
.withFrame(*fun.lambda.env, lambda)
.debugThrow<TypeError>();
.debugThrow();
}
abort(); // can't happen
}
@@ -1602,9 +1637,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
"while calling %s",
lambda.name
? concatStrings("'", symbols[lambda.name], "'")
: "anonymous lambda",
true);
if (pos) addErrorTrace(e, pos, "from call site%s", "", true);
: "anonymous lambda");
if (pos) addErrorTrace(e, pos, "from call site");
}
throw;
}
@@ -1631,7 +1665,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
try {
fn->fun(*this, vCur.determinePos(noPos), args, vCur);
} catch (Error & e) {
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
if (fn->addTrace)
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
throw;
}
@@ -1679,7 +1714,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
// so the debugger allows to inspect the wrong parameters passed to the builtin.
fn->fun(*this, vCur.determinePos(noPos), vArgs, vCur);
} catch (Error & e) {
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
if (fn->addTrace)
addErrorTrace(e, pos, "while calling the '%1%' builtin", fn->name);
throw;
}
@@ -1705,11 +1741,12 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
}
else
error("attempt to call something which is not a function but %1%: %2%",
error<TypeError>(
"attempt to call something which is not a function but %1%: %2%",
showType(vCur),
ValuePrinter(*this, vCur, errorPrintOptions))
.atPos(pos)
.debugThrow<TypeError>();
.debugThrow();
}
vRes = vCur;
@@ -1718,6 +1755,18 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
void ExprCall::eval(EvalState & state, Env & env, Value & v)
{
auto dts = state.debugRepl
? makeDebugTraceStacker(
state,
*this,
env,
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while calling a function"
)
: nullptr;
Value vFun;
fun->eval(state, env, vFun);
@@ -1779,12 +1828,12 @@ void EvalState::autoCallFunction(Bindings & args, Value & fun, Value & res)
if (j != args.end()) {
attrs.insert(*j);
} else if (!i.def) {
error(R"(cannot evaluate a function that has an argument without a value ('%1%')
error<MissingArgumentError>(R"(cannot evaluate a function that has an argument without a value ('%1%')
Nix attempted to evaluate a function as a top level expression; in
this case it must have its arguments supplied either by default
values, or passed explicitly with '--arg' or '--argstr'. See
https://nixos.org/manual/nix/stable/language/constructs.html#functions.)", symbols[i.name])
.atPos(i.pos).withFrame(*fun.lambda.env, *fun.lambda.fun).debugThrow<MissingArgumentError>();
.atPos(i.pos).withFrame(*fun.lambda.env, *fun.lambda.fun).debugThrow();
}
}
}
@@ -1815,7 +1864,7 @@ void ExprAssert::eval(EvalState & state, Env & env, Value & v)
if (!state.evalBool(env, cond, pos, "in the condition of the assert statement")) {
std::ostringstream out;
cond->show(state.symbols, out);
state.error("assertion '%1%' failed", out.str()).atPos(pos).withFrame(env, *this).debugThrow<AssertionError>();
state.error<AssertionError>("assertion '%1%' failed", out.str()).atPos(pos).withFrame(env, *this).debugThrow();
}
body->eval(state, env, v);
}
@@ -1908,7 +1957,7 @@ void ExprOpConcatLists::eval(EvalState & state, Env & env, Value & v)
}
void EvalState::concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx)
void EvalState::concatLists(Value & v, size_t nrLists, Value * const * lists, const PosIdx pos, std::string_view errorCtx)
{
nrListConcats++;
@@ -1926,14 +1975,15 @@ void EvalState::concatLists(Value & v, size_t nrLists, Value * * lists, const Po
return;
}
mkList(v, len);
auto out = v.listElems();
auto list = buildList(len);
auto out = list.elems;
for (size_t n = 0, pos = 0; n < nrLists; ++n) {
auto l = lists[n]->listSize();
if (l)
memcpy(out + pos, lists[n]->listElems(), l * sizeof(Value *));
pos += l;
}
v.mkList(list);
}
@@ -1993,14 +2043,14 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
nf = n;
nf += vTmp.fpoint;
} else
state.error("cannot add %1% to an integer", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow<EvalError>();
state.error<EvalError>("cannot add %1% to an integer", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow();
} else if (firstType == nFloat) {
if (vTmp.type() == nInt) {
nf += vTmp.integer;
} else if (vTmp.type() == nFloat) {
nf += vTmp.fpoint;
} else
state.error("cannot add %1% to a float", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow<EvalError>();
state.error<EvalError>("cannot add %1% to a float", showType(vTmp)).atPos(i_pos).withFrame(env, *this).debugThrow();
} else {
if (s.empty()) s.reserve(es->size());
/* skip canonization of first path, which would only be not
@@ -2022,7 +2072,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
v.mkFloat(nf);
else if (firstType == nPath) {
if (!context.empty())
state.error("a string that refers to a store path cannot be appended to a path").atPos(pos).withFrame(env, *this).debugThrow<EvalError>();
state.error<EvalError>("a string that refers to a store path cannot be appended to a path").atPos(pos).withFrame(env, *this).debugThrow();
v.mkPath(state.rootPath(CanonPath(canonPath(str()))));
} else
v.mkStringMove(c_str(), context);
@@ -2037,8 +2087,9 @@ void ExprPos::eval(EvalState & state, Env & env, Value & v)
void ExprBlackHole::eval(EvalState & state, Env & env, Value & v)
{
state.error("infinite recursion encountered")
.debugThrow<InfiniteRecursionError>();
state.error<InfiniteRecursionError>("infinite recursion encountered")
.atPos(v.determinePos(noPos))
.debugThrow();
}
// always force this to be separate, otherwise forceValue may inline it and take
@@ -2052,7 +2103,7 @@ void EvalState::tryFixupBlackHolePos(Value & v, PosIdx pos)
try {
std::rethrow_exception(e);
} catch (InfiniteRecursionError & e) {
e.err.errPos = positions[pos];
e.atPos(positions[pos]);
} catch (...) {
}
}
@@ -2100,15 +2151,18 @@ NixInt EvalState::forceInt(Value & v, const PosIdx pos, std::string_view errorCt
try {
forceValue(v, pos);
if (v.type() != nInt)
error("expected an integer but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.debugThrow<TypeError>();
error<TypeError>(
"expected an integer but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).debugThrow();
return v.integer;
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
}
return v.integer;
}
@@ -2119,10 +2173,11 @@ NixFloat EvalState::forceFloat(Value & v, const PosIdx pos, std::string_view err
if (v.type() == nInt)
return v.integer;
else if (v.type() != nFloat)
error("expected a float but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.debugThrow<TypeError>();
error<TypeError>(
"expected a float but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).debugThrow();
return v.fpoint;
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
@@ -2136,15 +2191,18 @@ bool EvalState::forceBool(Value & v, const PosIdx pos, std::string_view errorCtx
try {
forceValue(v, pos);
if (v.type() != nBool)
error("expected a Boolean but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.debugThrow<TypeError>();
error<TypeError>(
"expected a Boolean but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).debugThrow();
return v.boolean;
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
}
return v.boolean;
}
@@ -2159,10 +2217,11 @@ void EvalState::forceFunction(Value & v, const PosIdx pos, std::string_view erro
try {
forceValue(v, pos);
if (v.type() != nFunction && !isFunctor(v))
error("expected a function but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.debugThrow<TypeError>();
error<TypeError>(
"expected a function but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).debugThrow();
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
@@ -2175,10 +2234,11 @@ std::string_view EvalState::forceString(Value & v, const PosIdx pos, std::string
try {
forceValue(v, pos);
if (v.type() != nString)
error("expected a string but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.debugThrow<TypeError>();
error<TypeError>(
"expected a string but found %1%: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
).atPos(pos).debugThrow();
return v.string_view();
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
@@ -2207,7 +2267,7 @@ std::string_view EvalState::forceStringNoCtx(Value & v, const PosIdx pos, std::s
{
auto s = forceString(v, pos, errorCtx);
if (v.context()) {
error("the string '%1%' is not allowed to refer to a store path (such as '%2%')", v.string_view(), v.context()[0]).withTrace(pos, errorCtx).debugThrow<EvalError>();
error<EvalError>("the string '%1%' is not allowed to refer to a store path (such as '%2%')", v.string_view(), v.context()[0]).withTrace(pos, errorCtx).debugThrow();
}
return s;
}
@@ -2272,11 +2332,13 @@ BackedStringView EvalState::coerceToString(
return std::move(*maybeString);
auto i = v.attrs->find(sOutPath);
if (i == v.attrs->end()) {
error("cannot coerce %1% to a string: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
error<TypeError>(
"cannot coerce %1% to a string: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
)
.withTrace(pos, errorCtx)
.debugThrow<TypeError>();
.debugThrow();
}
return coerceToString(pos, *i->value, context, errorCtx,
coerceMore, copyToStore, canonicalizePath);
@@ -2284,7 +2346,7 @@ BackedStringView EvalState::coerceToString(
if (v.type() == nExternal) {
try {
return v.external->coerceToString(positions[pos], context, coerceMore, copyToStore);
return v.external->coerceToString(*this, pos, context, coerceMore, copyToStore);
} catch (Error & e) {
e.addTrace(nullptr, errorCtx);
throw;
@@ -2320,25 +2382,33 @@ BackedStringView EvalState::coerceToString(
}
}
error("cannot coerce %1% to a string: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
error<TypeError>("cannot coerce %1% to a string: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions)
)
.withTrace(pos, errorCtx)
.debugThrow<TypeError>();
.debugThrow();
}
StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePath & path)
{
if (nix::isDerivation(path.path.abs()))
error("file names are not allowed to end in '%1%'", drvExtension).debugThrow<EvalError>();
error<EvalError>("file names are not allowed to end in '%1%'", drvExtension).debugThrow();
auto i = srcToStore.find(path);
auto dstPath = i != srcToStore.end()
? i->second
: [&]() {
auto dstPath = fetchToStore(*store, path, path.baseName(), FileIngestionMethod::Recursive, nullptr, repair);
auto dstPath = fetchToStore(
*store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(),
FileIngestionMethod::Recursive,
nullptr,
repair);
allowPath(dstPath);
srcToStore.insert_or_assign(path, dstPath);
printMsg(lvlChatty, "copied source '%1%' -> '%2%'", path, store->printStorePath(dstPath));
@@ -2380,7 +2450,7 @@ SourcePath EvalState::coerceToPath(const PosIdx pos, Value & v, NixStringContext
relative to the root filesystem. */
auto path = coerceToString(pos, v, context, errorCtx, false, false, true).toOwned();
if (path == "" || path[0] != '/')
error("string '%1%' doesn't represent an absolute path", path).withTrace(pos, errorCtx).debugThrow<EvalError>();
error<EvalError>("string '%1%' doesn't represent an absolute path", path).withTrace(pos, errorCtx).debugThrow();
return rootPath(CanonPath(path));
}
@@ -2390,7 +2460,7 @@ StorePath EvalState::coerceToStorePath(const PosIdx pos, Value & v, NixStringCon
auto path = coerceToString(pos, v, context, errorCtx, false, false, true).toOwned();
if (auto storePath = store->maybeParseStorePath(path))
return *storePath;
error("path '%1%' is not in the Nix store", path).withTrace(pos, errorCtx).debugThrow<EvalError>();
error<EvalError>("path '%1%' is not in the Nix store", path).withTrace(pos, errorCtx).debugThrow();
}
@@ -2400,18 +2470,18 @@ std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedP
auto s = forceString(v, context, pos, errorCtx);
auto csize = context.size();
if (csize != 1)
error(
error<EvalError>(
"string '%s' has %d entries in its context. It should only have exactly one entry",
s, csize)
.withTrace(pos, errorCtx).debugThrow<EvalError>();
.withTrace(pos, errorCtx).debugThrow();
auto derivedPath = std::visit(overloaded {
[&](NixStringContextElem::Opaque && o) -> SingleDerivedPath {
return std::move(o);
},
[&](NixStringContextElem::DrvDeep &&) -> SingleDerivedPath {
error(
error<EvalError>(
"string '%s' has a context which refers to a complete source and binary closure. This is not supported at this time",
s).withTrace(pos, errorCtx).debugThrow<EvalError>();
s).withTrace(pos, errorCtx).debugThrow();
},
[&](NixStringContextElem::Built && b) -> SingleDerivedPath {
return std::move(b);
@@ -2434,16 +2504,16 @@ SingleDerivedPath EvalState::coerceToSingleDerivedPath(const PosIdx pos, Value &
error message. */
std::visit(overloaded {
[&](const SingleDerivedPath::Opaque & o) {
error(
error<EvalError>(
"path string '%s' has context with the different path '%s'",
s, sExpected)
.withTrace(pos, errorCtx).debugThrow<EvalError>();
.withTrace(pos, errorCtx).debugThrow();
},
[&](const SingleDerivedPath::Built & b) {
error(
error<EvalError>(
"string '%s' has context with the output '%s' from derivation '%s', but the string is not the right placeholder for this derivation output. It should be '%s'",
s, b.output, b.drvPath->to_string(*store), sExpected)
.withTrace(pos, errorCtx).debugThrow<EvalError>();
.withTrace(pos, errorCtx).debugThrow();
}
}, derivedPath.raw());
}
@@ -2528,7 +2598,7 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
case nThunk: // Must not be left by forceValue
default:
error("cannot compare %1% with %2%", showType(v1), showType(v2)).withTrace(pos, errorCtx).debugThrow<EvalError>();
error<EvalError>("cannot compare %1% with %2%", showType(v1), showType(v2)).withTrace(pos, errorCtx).debugThrow();
}
}
@@ -2689,14 +2759,14 @@ SourcePath resolveExprPath(SourcePath path)
// Basic cycle/depth limit to avoid infinite loops.
if (++followCount >= maxFollow)
throw Error("too many symbolic links encountered while traversing the path '%s'", path);
auto p = path.parent().resolveSymlinks() + path.baseName();
auto p = path.parent().resolveSymlinks() / path.baseName();
if (p.lstat().type != InputAccessor::tSymlink) break;
path = {path.accessor, CanonPath(p.readLink(), path.path.parent().value_or(CanonPath::root))};
}
/* If `path' refers to a directory, append `/default.nix'. */
if (path.resolveSymlinks().lstat().type == InputAccessor::tDirectory)
return path + "default.nix";
return path / "default.nix";
return path;
}
@@ -2719,9 +2789,12 @@ Expr * EvalState::parseExprFromFile(const SourcePath & path, std::shared_ptr<Sta
Expr * EvalState::parseExprFromString(std::string s_, const SourcePath & basePath, std::shared_ptr<StaticEnv> & staticEnv)
{
auto s = make_ref<std::string>(std::move(s_));
s->append("\0\0", 2);
return parse(s->data(), s->size(), Pos::String{.source = s}, basePath, staticEnv);
// NOTE this method (and parseStdin) must take care to *fully copy* their input
// into their respective Pos::Origin until the parser stops overwriting its input
// data.
auto s = make_ref<std::string>(s_);
s_.append("\0\0", 2);
return parse(s_.data(), s_.size(), Pos::String{.source = s}, basePath, staticEnv);
}
@@ -2733,12 +2806,15 @@ Expr * EvalState::parseExprFromString(std::string s, const SourcePath & basePath
Expr * EvalState::parseStdin()
{
// NOTE this method (and parseExprFromString) must take care to *fully copy* their
// input into their respective Pos::Origin until the parser stops overwriting its
// input data.
//Activity act(*logger, lvlTalkative, "parsing standard input");
auto buffer = drainFD(0);
// drainFD should have left some extra space for terminators
buffer.append("\0\0", 2);
auto s = make_ref<std::string>(std::move(buffer));
return parse(s->data(), s->size(), Pos::Stdin{.source = s}, rootPath(CanonPath::fromCwd()), staticBaseEnv);
auto s = make_ref<std::string>(buffer);
return parse(buffer.data(), buffer.size(), Pos::Stdin{.source = s}, rootPath("."), staticBaseEnv);
}
@@ -2767,13 +2843,12 @@ SourcePath EvalState::findFile(const SearchPath & searchPath, const std::string_
if (hasPrefix(path, "nix/"))
return {corepkgsFS, CanonPath(path.substr(3))};
debugThrow(ThrownError({
.msg = hintfmt(evalSettings.pureEval
error<ThrownError>(
evalSettings.pureEval
? "cannot look up '<%s>' in pure evaluation mode (use '--impure' to override)"
: "file '%s' was not found in the Nix search path (add it using $NIX_PATH or -I)",
path),
.errPos = positions[pos]
}), 0, 0);
path
).atPos(pos).debugThrow();
}
@@ -2787,12 +2862,13 @@ std::optional<std::string> EvalState::resolveSearchPathPath(const SearchPath::Pa
if (EvalSettings::isPseudoUrl(value)) {
try {
auto storePath = fetchers::downloadTarball(
store, EvalSettings::resolvePseudoUrl(value), "source", false).storePath;
auto accessor = fetchers::downloadTarball(
EvalSettings::resolvePseudoUrl(value)).accessor;
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy);
res = { store->toRealPath(storePath) };
} catch (FileTransferError & e) {
} catch (Error & e) {
logWarning({
.msg = hintfmt("Nix search path entry '%1%' cannot be downloaded, ignoring", value)
.msg = HintFmt("Nix search path entry '%1%' cannot be downloaded, ignoring", value)
});
}
}
@@ -2825,7 +2901,7 @@ std::optional<std::string> EvalState::resolveSearchPathPath(const SearchPath::Pa
res = { path };
else {
logWarning({
.msg = hintfmt("Nix search path entry '%1%' does not exist, ignoring", value)
.msg = HintFmt("Nix search path entry '%1%' does not exist, ignoring", value)
});
res = std::nullopt;
}
@@ -2856,11 +2932,11 @@ Expr * EvalState::parse(
}
std::string ExternalValueBase::coerceToString(const Pos & pos, NixStringContext & context, bool copyMore, bool copyToStore) const
std::string ExternalValueBase::coerceToString(EvalState & state, const PosIdx & pos, NixStringContext & context, bool copyMore, bool copyToStore) const
{
throw TypeError({
.msg = hintfmt("cannot coerce %1% to a string: %2%", showType(), *this)
});
state.error<TypeError>(
"cannot coerce %1% to a string: %2%", showType(), *this
).atPos(pos).debugThrow();
}
+67 -94
View File
@@ -2,6 +2,7 @@
///@file
#include "attr-set.hh"
#include "eval-error.hh"
#include "types.hh"
#include "value.hh"
#include "nixexpr.hh"
@@ -10,6 +11,7 @@
#include "experimental-features.hh"
#include "input-accessor.hh"
#include "search-path.hh"
#include "repl-exit-status.hh"
#include <map>
#include <optional>
@@ -67,6 +69,13 @@ struct PrimOp
*/
const char * doc = nullptr;
/**
* Add a trace item, `while calling the '<name>' builtin`
*
* This is used to remove the redundant item for `builtins.addErrorContext`.
*/
bool addTrace = true;
/**
* Implementation of the primop.
*/
@@ -147,48 +156,10 @@ struct DebugTrace {
std::shared_ptr<Pos> pos;
const Expr & expr;
const Env & env;
hintformat hint;
HintFmt hint;
bool isError;
};
void debugError(Error * e, Env & env, Expr & expr);
class ErrorBuilder
{
private:
EvalState & state;
ErrorInfo info;
ErrorBuilder(EvalState & s, ErrorInfo && i): state(s), info(i) { }
public:
template<typename... Args>
[[nodiscard, gnu::noinline]]
static ErrorBuilder * create(EvalState & s, const Args & ... args)
{
return new ErrorBuilder(s, ErrorInfo { .msg = hintfmt(args...) });
}
[[nodiscard, gnu::noinline]]
ErrorBuilder & atPos(PosIdx pos);
[[nodiscard, gnu::noinline]]
ErrorBuilder & withTrace(PosIdx pos, const std::string_view text);
[[nodiscard, gnu::noinline]]
ErrorBuilder & withFrameTrace(PosIdx pos, const std::string_view text);
[[nodiscard, gnu::noinline]]
ErrorBuilder & withSuggestions(Suggestions & s);
[[nodiscard, gnu::noinline]]
ErrorBuilder & withFrame(const Env & e, const Expr & ex);
template<class ErrorType>
[[gnu::noinline, gnu::noreturn]]
void debugThrow();
};
class EvalState : public std::enable_shared_from_this<EvalState>
{
@@ -222,6 +193,36 @@ public:
*/
Value vEmptyList;
/**
* `null` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vNull;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vTrue;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vFalse;
/** `"regular"` */
Value vStringRegular;
/** `"directory"` */
Value vStringDirectory;
/** `"symlink"` */
Value vStringSymlink;
/** `"unknown"` */
Value vStringUnknown;
/**
* The accessor for the root filesystem.
*/
@@ -257,9 +258,9 @@ public:
/**
* Debugger
*/
void (* debugRepl)(ref<EvalState> es, const ValMap & extraEnv);
ReplExitStatus (* debugRepl)(ref<EvalState> es, const ValMap & extraEnv);
bool debugStop;
bool debugQuit;
bool inDebugger = false;
int trylevel;
std::list<DebugTrace> debugTraces;
std::map<const Expr*, const std::shared_ptr<const StaticEnv>> exprEnvs;
@@ -274,39 +275,11 @@ public:
void runDebugRepl(const Error * error, const Env & env, const Expr & expr);
template<class E>
[[gnu::noinline, gnu::noreturn]]
void debugThrowLastTrace(E && error)
{
debugThrow(error, nullptr, nullptr);
}
template<class E>
[[gnu::noinline, gnu::noreturn]]
void debugThrow(E && error, const Env * env, const Expr * expr)
{
if (debugRepl && ((env && expr) || !debugTraces.empty())) {
if (!env || !expr) {
const DebugTrace & last = debugTraces.front();
env = &last.env;
expr = &last.expr;
}
runDebugRepl(&error, *env, *expr);
}
throw std::move(error);
}
// This is dangerous, but gets in line with the idea that error creation and
// throwing should not allocate on the stack of hot functions.
// as long as errors are immediately thrown, it works.
ErrorBuilder * errorBuilder;
template<typename... Args>
template<class T, typename... Args>
[[nodiscard, gnu::noinline]]
ErrorBuilder & error(const Args & ... args) {
errorBuilder = ErrorBuilder::create(*this, args...);
return *errorBuilder;
EvalErrorBuilder<T> & error(const Args & ... args) {
// `EvalErrorBuilder::debugThrow` performs the corresponding `delete`.
return *new EvalErrorBuilder<T>(*this, args...);
}
private:
@@ -372,6 +345,11 @@ public:
*/
SourcePath rootPath(CanonPath path);
/**
* Variant which accepts relative paths too.
*/
SourcePath rootPath(PathView path);
/**
* Allow access to a path.
*/
@@ -493,10 +471,12 @@ public:
std::string_view forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx);
std::string_view forceStringNoCtx(Value & v, const PosIdx pos, std::string_view errorCtx);
template<typename... Args>
[[gnu::noinline]]
void addErrorTrace(Error & e, const char * s, const std::string & s2) const;
void addErrorTrace(Error & e, const Args & ... formatArgs) const;
template<typename... Args>
[[gnu::noinline]]
void addErrorTrace(Error & e, const PosIdx pos, const char * s, const std::string & s2, bool frame = false) const;
void addErrorTrace(Error & e, const PosIdx pos, const Args & ... formatArgs) const;
public:
/**
@@ -672,7 +652,16 @@ public:
return BindingsBuilder(*this, allocBindings(capacity));
}
void mkList(Value & v, size_t length);
ListBuilder buildList(size_t size)
{
return ListBuilder(*this, size);
}
/**
* Return a boolean `Value *` without allocating.
*/
Value *getBool(bool b);
void mkThunk_(Value & v, Expr * expr);
void mkPos(Value & v, PosIdx pos);
@@ -719,7 +708,7 @@ public:
const SingleDerivedPath & p,
Value & v);
void concatLists(Value & v, size_t nrLists, Value * * lists, const PosIdx pos, std::string_view errorCtx);
void concatLists(Value & v, size_t nrLists, Value * const * lists, const PosIdx pos, std::string_view errorCtx);
/**
* Print statistics, if enabled.
@@ -813,13 +802,13 @@ private:
friend void prim_split(EvalState & state, const PosIdx pos, Value * * args, Value & v);
friend struct Value;
friend class ListBuilder;
};
struct DebugTraceStacker {
DebugTraceStacker(EvalState & evalState, DebugTrace t);
~DebugTraceStacker()
{
// assert(evalState.debugTraces.front() == trace);
evalState.debugTraces.pop_front();
}
EvalState & evalState;
@@ -845,22 +834,6 @@ SourcePath resolveExprPath(SourcePath path);
*/
bool isAllowedURI(std::string_view uri, const Strings & allowedPaths);
struct InvalidPathError : EvalError
{
Path path;
InvalidPathError(const Path & path);
#ifdef EXCEPTION_NEEDS_THROW_SPEC
~InvalidPathError() throw () { };
#endif
};
template<class ErrorType>
void ErrorBuilder::debugThrow()
{
// NOTE: We always use the -LastTrace version as we push the new trace in withFrame()
state.debugThrowLastTrace(ErrorType(info));
}
}
#include "eval-inline.hh"
+37 -24
View File
@@ -1,20 +1,52 @@
lockFileStr: rootSrc: rootSubdir:
# This is a helper to callFlake() to lazily fetch flake inputs.
# The contents of the lock file, in JSON format.
lockFileStr:
# A mapping of lock file node IDs to { sourceInfo, subdir } attrsets,
# with sourceInfo.outPath providing an InputAccessor to a previously
# fetched tree. This is necessary for possibly unlocked inputs, in
# particular the root input, but also --override-inputs pointing to
# unlocked trees.
overrides:
let
lockFile = builtins.fromJSON lockFileStr;
# Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root
# node.
resolveInput = inputSpec:
if builtins.isList inputSpec
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node.
getInputByPath = nodeName: path:
if path == []
then nodeName
else
getInputByPath
# Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path);
allNodes =
builtins.mapAttrs
(key: node:
let
sourceInfo =
if key == lockFile.root
then rootSrc
else fetchTree (node.info or {} // removeAttrs node.locked ["dir"]);
if overrides ? ${key}
then
overrides.${key}.sourceInfo
else
# FIXME: remove obsolete node.info.
fetchTree (node.info or {} // removeAttrs node.locked ["dir"]);
subdir = if key == lockFile.root then rootSubdir else node.locked.dir or "";
subdir = overrides.${key}.dir or node.locked.dir or "";
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
@@ -24,25 +56,6 @@ let
(inputName: inputSpec: allNodes.${resolveInput inputSpec})
(node.inputs or {});
# Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root
# node.
resolveInput = inputSpec:
if builtins.isList inputSpec
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node.
getInputByPath = nodeName: path:
if path == []
then nodeName
else
getInputByPath
# Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path);
outputs = flake.outputs (inputs // { self = result; });
result =
+111 -86
View File
@@ -139,7 +139,7 @@ static FlakeInput parseFlakeInput(EvalState & state,
attrs.emplace(state.symbols[attr.name], Explicit<bool> { attr.value->boolean });
break;
case nInt:
attrs.emplace(state.symbols[attr.name], (long unsigned int)attr.value->integer);
attrs.emplace(state.symbols[attr.name], (long unsigned int) attr.value->integer);
break;
default:
if (attr.name == state.symbols.create("publicKeys")) {
@@ -147,15 +147,15 @@ static FlakeInput parseFlakeInput(EvalState & state,
NixStringContext emptyContext = {};
attrs.emplace(state.symbols[attr.name], printValueAsJSON(state, true, *attr.value, pos, emptyContext).dump());
} else
throw TypeError("flake input attribute '%s' is %s while a string, Boolean, or integer is expected",
state.symbols[attr.name], showType(*attr.value));
state.error<TypeError>("flake input attribute '%s' is %s while a string, Boolean, or integer is expected",
state.symbols[attr.name], showType(*attr.value)).debugThrow();
}
#pragma GCC diagnostic pop
}
} catch (Error & e) {
e.addTrace(
state.positions[attr.pos],
hintfmt("while evaluating flake attribute '%s'", state.symbols[attr.name]));
HintFmt("while evaluating flake attribute '%s'", state.symbols[attr.name]));
throw;
}
}
@@ -164,7 +164,7 @@ static FlakeInput parseFlakeInput(EvalState & state,
try {
input.ref = FlakeRef::fromAttrs(attrs);
} catch (Error & e) {
e.addTrace(state.positions[pos], hintfmt("while evaluating flake input"));
e.addTrace(state.positions[pos], HintFmt("while evaluating flake input"));
throw;
}
else {
@@ -202,43 +202,27 @@ static std::map<FlakeId, FlakeInput> parseFlakeInputs(
return inputs;
}
static Flake getFlake(
static Flake readFlake(
EvalState & state,
const FlakeRef & originalRef,
bool allowLookup,
FlakeCache & flakeCache,
InputPath lockRootPath)
const FlakeRef & resolvedRef,
const FlakeRef & lockedRef,
const SourcePath & rootDir,
const InputPath & lockRootPath)
{
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, originalRef, allowLookup, flakeCache);
auto flakePath = rootDir / CanonPath(resolvedRef.subdir) / "flake.nix";
// We need to guard against symlink attacks, but before we start doing
// filesystem operations we should make sure there's a flake.nix in the
// first place.
auto unsafeFlakeDir = state.store->toRealPath(storePath) + "/" + lockedRef.subdir;
auto unsafeFlakeFile = unsafeFlakeDir + "/flake.nix";
if (!pathExists(unsafeFlakeFile))
throw Error("source tree referenced by '%s' does not contain a '%s/flake.nix' file", lockedRef, lockedRef.subdir);
// Guard against symlink attacks.
auto flakeDir = canonPath(unsafeFlakeDir, true);
auto flakeFile = canonPath(flakeDir + "/flake.nix", true);
if (!isInDir(flakeFile, state.store->toRealPath(storePath)))
throw Error("'flake.nix' file of flake '%s' escapes from '%s'",
lockedRef, state.store->printStorePath(storePath));
// NOTE evalFile forces vInfo to be an attrset because mustBeTrivial is true.
Value vInfo;
state.evalFile(flakePath, vInfo, true);
Flake flake {
.originalRef = originalRef,
.resolvedRef = resolvedRef,
.lockedRef = lockedRef,
.storePath = storePath,
.path = flakePath,
};
Value vInfo;
state.evalFile(state.rootPath(CanonPath(flakeFile)), vInfo, true); // FIXME: symlink attack
expectType(state, nAttrs, vInfo, state.positions.add({state.rootPath(CanonPath(flakeFile))}, 1, 1));
if (auto description = vInfo.attrs->get(state.sDescription)) {
expectType(state, nString, *description->value, description->pos);
flake.description = description->value->c_str();
@@ -247,7 +231,7 @@ static Flake getFlake(
auto sInputs = state.symbols.create("inputs");
if (auto inputs = vInfo.attrs->get(sInputs))
flake.inputs = parseFlakeInputs(state, inputs->value, inputs->pos, flakeDir, lockRootPath);
flake.inputs = parseFlakeInputs(state, inputs->value, inputs->pos, flakePath.parent().path.abs(), lockRootPath); // FIXME
auto sOutputs = state.symbols.create("outputs");
@@ -264,7 +248,7 @@ static Flake getFlake(
}
} else
throw Error("flake '%s' lacks attribute 'outputs'", lockedRef);
throw Error("flake '%s' lacks attribute 'outputs'", resolvedRef);
auto sNixConfig = state.symbols.create("nixConfig");
@@ -281,7 +265,7 @@ static Flake getFlake(
NixStringContext emptyContext = {};
flake.config.settings.emplace(
state.symbols[setting.name],
state.coerceToString(setting.pos, *setting.value, emptyContext, "", false, true, true) .toOwned());
state.coerceToString(setting.pos, *setting.value, emptyContext, "", false, true, true).toOwned());
}
else if (setting.value->type() == nInt)
flake.config.settings.emplace(
@@ -295,15 +279,15 @@ static Flake getFlake(
std::vector<std::string> ss;
for (auto elem : setting.value->listItems()) {
if (elem->type() != nString)
throw TypeError("list element in flake configuration setting '%s' is %s while a string is expected",
state.symbols[setting.name], showType(*setting.value));
state.error<TypeError>("list element in flake configuration setting '%s' is %s while a string is expected",
state.symbols[setting.name], showType(*setting.value)).debugThrow();
ss.emplace_back(state.forceStringNoCtx(*elem, setting.pos, ""));
}
flake.config.settings.emplace(state.symbols[setting.name], ss);
}
else
throw TypeError("flake configuration setting '%s' is %s",
state.symbols[setting.name], showType(*setting.value));
state.error<TypeError>("flake configuration setting '%s' is %s",
state.symbols[setting.name], showType(*setting.value)).debugThrow();
}
}
@@ -313,12 +297,25 @@ static Flake getFlake(
attr.name != sOutputs &&
attr.name != sNixConfig)
throw Error("flake '%s' has an unsupported attribute '%s', at %s",
lockedRef, state.symbols[attr.name], state.positions[attr.pos]);
resolvedRef, state.symbols[attr.name], state.positions[attr.pos]);
}
return flake;
}
static Flake getFlake(
EvalState & state,
const FlakeRef & originalRef,
bool allowLookup,
FlakeCache & flakeCache,
InputPath lockRootPath)
{
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, originalRef, allowLookup, flakeCache);
return readFlake(state, originalRef, resolvedRef, lockedRef, state.rootPath(state.store->toRealPath(storePath)), lockRootPath);
}
Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup, FlakeCache & flakeCache)
{
return getFlake(state, originalRef, allowLookup, flakeCache, {});
@@ -330,6 +327,13 @@ Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup
return getFlake(state, originalRef, allowLookup, flakeCache);
}
static LockFile readLockFile(const SourcePath & lockFilePath)
{
return lockFilePath.pathExists()
? LockFile(lockFilePath.readFile(), fmt("%s", lockFilePath))
: LockFile();
}
/* Compute an in-memory lock file for the specified top-level flake,
and optionally write it to file, if the flake is writable. */
LockedFlake lockFlake(
@@ -355,16 +359,16 @@ LockedFlake lockFlake(
throw Error("reference lock file was provided, but the `allow-dirty` setting is set to false");
}
// FIXME: symlink attack
auto oldLockFile = LockFile::read(
auto oldLockFile = readLockFile(
lockFlags.referenceLockFilePath.value_or(
state.store->toRealPath(flake.storePath) + "/" + flake.lockedRef.subdir + "/flake.lock"));
flake.lockFilePath()));
debug("old lock file: %s", oldLockFile);
std::map<InputPath, FlakeInput> overrides;
std::set<InputPath> explicitCliOverrides;
std::set<InputPath> overridesUsed, updatesUsed;
std::map<ref<Node>, SourcePath> nodePaths;
for (auto & i : lockFlags.inputOverrides) {
overrides.insert_or_assign(i.first, FlakeInput { .ref = i.second });
@@ -535,11 +539,13 @@ LockedFlake lockFlake(
}
}
computeLocks(
mustRefetch
? getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath).inputs
: fakeInputs,
childNode, inputPath, oldLock, lockRootPath, parentPath, !mustRefetch);
if (mustRefetch) {
auto inputFlake = getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath);
nodePaths.emplace(childNode, inputFlake.path.parent());
computeLocks(inputFlake.inputs, childNode, inputPath, oldLock, lockRootPath, parentPath, false);
} else {
computeLocks(fakeInputs, childNode, inputPath, oldLock, lockRootPath, parentPath, true);
}
} else {
/* We need to create a new lock file entry. So fetch
@@ -584,23 +590,25 @@ LockedFlake lockFlake(
flake. Also, unless we already have this flake
in the top-level lock file, use this flake's
own lock file. */
nodePaths.emplace(childNode, inputFlake.path.parent());
computeLocks(
inputFlake.inputs, childNode, inputPath,
oldLock
? std::dynamic_pointer_cast<const Node>(oldLock)
: LockFile::read(
state.store->toRealPath(inputFlake.storePath) + "/" + inputFlake.lockedRef.subdir + "/flake.lock").root.get_ptr(),
: readLockFile(inputFlake.lockFilePath()).root.get_ptr(),
oldLock ? lockRootPath : inputPath,
localPath,
false);
}
else {
auto [sourceInfo, resolvedRef, lockedRef] = fetchOrSubstituteTree(
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, *input.ref, useRegistries, flakeCache);
auto childNode = make_ref<LockedNode>(lockedRef, ref, false);
nodePaths.emplace(childNode, state.rootPath(state.store->toRealPath(storePath)));
node->inputs.insert_or_assign(id, childNode);
}
}
@@ -613,7 +621,9 @@ LockedFlake lockFlake(
};
// Bring in the current ref for relative path resolution if we have it
auto parentPath = canonPath(state.store->toRealPath(flake.storePath) + "/" + flake.lockedRef.subdir, true);
auto parentPath = flake.path.parent().path.abs();
nodePaths.emplace(newLockFile.root, flake.path.parent());
computeLocks(
flake.inputs,
@@ -707,14 +717,6 @@ LockedFlake lockFlake(
flake.lockedRef.input.getRev() &&
prevLockedRef.input.getRev() != flake.lockedRef.input.getRev())
warn("committed new revision '%s'", flake.lockedRef.input.getRev()->gitRev());
/* Make sure that we picked up the change,
i.e. the tree should usually be dirty
now. Corner case: we could have reverted from a
dirty to a clean tree! */
if (flake.lockedRef.input == prevLockedRef.input
&& !flake.lockedRef.input.isLocked())
throw Error("'%s' did not change after I updated its 'flake.lock' file; is 'flake.lock' under version control?", flake.originalRef);
}
} else
throw Error("cannot write modified lock file of flake '%s' (use '--no-write-lock-file' to ignore)", topRef);
@@ -724,7 +726,11 @@ LockedFlake lockFlake(
}
}
return LockedFlake { .flake = std::move(flake), .lockFile = std::move(newLockFile) };
return LockedFlake {
.flake = std::move(flake),
.lockFile = std::move(newLockFile),
.nodePaths = std::move(nodePaths)
};
} catch (Error & e) {
e.addTrace({}, "while updating the lock file of flake '%s'", flake.lockedRef.to_string());
@@ -736,30 +742,50 @@ void callFlake(EvalState & state,
const LockedFlake & lockedFlake,
Value & vRes)
{
auto vLocks = state.allocValue();
auto vRootSrc = state.allocValue();
auto vRootSubdir = state.allocValue();
auto vTmp1 = state.allocValue();
auto vTmp2 = state.allocValue();
experimentalFeatureSettings.require(Xp::Flakes);
vLocks->mkString(lockedFlake.lockFile.to_string());
auto [lockFileStr, keyMap] = lockedFlake.lockFile.to_string();
emitTreeAttrs(
state,
lockedFlake.flake.storePath,
lockedFlake.flake.lockedRef.input,
*vRootSrc,
false,
lockedFlake.flake.forceDirty);
auto overrides = state.buildBindings(lockedFlake.nodePaths.size());
vRootSubdir->mkString(lockedFlake.flake.lockedRef.subdir);
for (auto & [node, sourcePath] : lockedFlake.nodePaths) {
auto override = state.buildBindings(2);
auto & vSourceInfo = override.alloc(state.symbols.create("sourceInfo"));
auto lockedNode = node.dynamic_pointer_cast<const LockedNode>();
auto [storePath, subdir] = state.store->toStorePath(sourcePath.path.abs());
emitTreeAttrs(
state,
storePath,
lockedNode ? lockedNode->lockedRef.input : lockedFlake.flake.lockedRef.input,
vSourceInfo,
false,
!lockedNode && lockedFlake.flake.forceDirty);
auto key = keyMap.find(node);
assert(key != keyMap.end());
override
.alloc(state.symbols.create("dir"))
.mkString(CanonPath(subdir).rel());
overrides.alloc(state.symbols.create(key->second)).mkAttrs(override);
}
auto & vOverrides = state.allocValue()->mkAttrs(overrides);
auto vCallFlake = state.allocValue();
state.evalFile(state.callFlakeInternal, *vCallFlake);
auto vTmp1 = state.allocValue();
auto vLocks = state.allocValue();
vLocks->mkString(lockFileStr);
state.callFunction(*vCallFlake, *vLocks, *vTmp1, noPos);
state.callFunction(*vTmp1, *vRootSrc, *vTmp2, noPos);
state.callFunction(*vTmp2, *vRootSubdir, vRes, noPos);
state.callFunction(*vTmp1, vOverrides, vRes, noPos);
}
static void prim_getFlake(EvalState & state, const PosIdx pos, Value * * args, Value & v)
@@ -865,11 +891,11 @@ static void prim_flakeRefToString(
attrs.emplace(state.symbols[attr.name],
std::string(attr.value->string_view()));
} else {
state.error(
state.error<EvalError>(
"flake reference attribute sets may only contain integers, Booleans, "
"and strings, but attribute '%s' is %s",
state.symbols[attr.name],
showType(*attr.value)).debugThrow<EvalError>();
showType(*attr.value)).debugThrow();
}
}
auto flakeRef = FlakeRef::fromAttrs(attrs);
@@ -899,18 +925,17 @@ static RegisterPrimOp r4({
}
Fingerprint LockedFlake::getFingerprint() const
std::optional<Fingerprint> LockedFlake::getFingerprint(ref<Store> store) const
{
if (lockFile.isUnlocked()) return std::nullopt;
auto fingerprint = flake.lockedRef.input.getFingerprint(store);
if (!fingerprint) return std::nullopt;
// FIXME: as an optimization, if the flake contains a lock file
// and we haven't changed it, then it's sufficient to use
// flake.sourceInfo.storePath for the fingerprint.
return hashString(HashAlgorithm::SHA256,
fmt("%s;%s;%d;%d;%s",
flake.storePath.to_string(),
flake.lockedRef.subdir,
flake.lockedRef.input.getRevCount().value_or(0),
flake.lockedRef.input.getLastModified().value_or(0),
lockFile));
return hashString(HashAlgorithm::SHA256, fmt("%s;%s;%s", *fingerprint, flake.lockedRef.subdir, lockFile));
}
Flake::~Flake() { }
+19 -3
View File
@@ -77,18 +77,27 @@ struct Flake
* the specific local store result of invoking the fetcher
*/
FlakeRef lockedRef;
/**
* The path of `flake.nix`.
*/
SourcePath path;
/**
* pretend that 'lockedRef' is dirty
*/
bool forceDirty = false;
std::optional<std::string> description;
StorePath storePath;
FlakeInputs inputs;
/**
* 'nixConfig' attribute
*/
ConfigFile config;
~Flake();
SourcePath lockFilePath()
{
return path.parent() / "flake.lock";
}
};
Flake getFlake(EvalState & state, const FlakeRef & flakeRef, bool allowLookup);
@@ -103,7 +112,14 @@ struct LockedFlake
Flake flake;
LockFile lockFile;
Fingerprint getFingerprint() const;
/**
* Source tree accessors for nodes that have been fetched in
* lockFlake(); in particular, the root node and the overriden
* inputs.
*/
std::map<ref<Node>, SourcePath> nodePaths;
std::optional<Fingerprint> getFingerprint(ref<Store> store) const;
};
struct LockFlags
@@ -158,7 +174,7 @@ struct LockFlags
/**
* The path to a lock file to read instead of the `flake.lock` file in the top-level flake
*/
std::optional<std::string> referenceLockFilePath;
std::optional<SourcePath> referenceLockFilePath;
/**
* The path to a lock file to write to instead of the `flake.lock` file in the top-level flake
+14 -4
View File
@@ -102,6 +102,19 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
if (isFlake) {
if (!S_ISDIR(lstat(path).st_mode)) {
if (baseNameOf(path) == "flake.nix") {
// Be gentle with people who accidentally write `/foo/bar/flake.nix` instead of `/foo/bar`
warn(
"Path '%s' should point at the directory containing the 'flake.nix' file, not the file itself. "
"Pretending that you meant '%s'"
, path, dirOf(path));
path = dirOf(path);
} else {
throw BadURL("path '%s' is not a flake (because it's not a directory)", path);
}
}
if (!allowMissing && !pathExists(path + "/flake.nix")){
notice("path '%s' does not contain a 'flake.nix', searching up",path);
@@ -124,9 +137,6 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
throw BadURL("could not find a flake.nix file");
}
if (!S_ISDIR(lstat(path).st_mode))
throw BadURL("path '%s' is not a flake (because it's not a directory)", path);
if (!allowMissing && !pathExists(path + "/flake.nix"))
throw BadURL("path '%s' is not a flake (because it doesn't contain a 'flake.nix' file)", path);
@@ -274,7 +284,7 @@ FlakeRef FlakeRef::fromAttrs(const fetchers::Attrs & attrs)
std::pair<StorePath, FlakeRef> FlakeRef::fetchTree(ref<Store> store) const
{
auto [storePath, lockedInput] = input.fetch(store);
auto [storePath, lockedInput] = input.fetchToStore(store);
return {std::move(storePath), FlakeRef(std::move(lockedInput), subdir)};
}
+13 -16
View File
@@ -38,7 +38,7 @@ LockedNode::LockedNode(const nlohmann::json & json)
, isFlake(json.find("flake") != json.end() ? (bool) json["flake"] : true)
{
if (!lockedRef.input.isLocked())
throw Error("lock file contains mutable lock '%s'",
throw Error("lock file contains unlocked input '%s'",
fetchers::attrsToJSON(lockedRef.input.toAttrs()));
}
@@ -84,8 +84,10 @@ std::shared_ptr<Node> LockFile::findInput(const InputPath & path)
return doFind(root, path, visited);
}
LockFile::LockFile(const nlohmann::json & json, const Path & path)
LockFile::LockFile(std::string_view contents, std::string_view path)
{
auto json = nlohmann::json::parse(contents);
auto version = json.value("version", 0);
if (version < 5 || version > 7)
throw Error("lock file '%s' has unsupported version %d", path, version);
@@ -107,7 +109,7 @@ LockFile::LockFile(const nlohmann::json & json, const Path & path)
std::string inputKey = i.value();
auto k = nodeMap.find(inputKey);
if (k == nodeMap.end()) {
auto nodes = json["nodes"];
auto & nodes = json["nodes"];
auto jsonNode2 = nodes.find(inputKey);
if (jsonNode2 == nodes.end())
throw Error("lock file references missing node '%s'", inputKey);
@@ -134,10 +136,10 @@ LockFile::LockFile(const nlohmann::json & json, const Path & path)
// a bit since we don't need to worry about cycles.
}
nlohmann::json LockFile::toJSON() const
std::pair<nlohmann::json, LockFile::KeyMap> LockFile::toJSON() const
{
nlohmann::json nodes;
std::unordered_map<std::shared_ptr<const Node>, std::string> nodeKeys;
KeyMap nodeKeys;
std::unordered_set<std::string> keys;
std::function<std::string(const std::string & key, ref<const Node> node)> dumpNode;
@@ -194,23 +196,18 @@ nlohmann::json LockFile::toJSON() const
json["root"] = dumpNode("root", root);
json["nodes"] = std::move(nodes);
return json;
return {json, std::move(nodeKeys)};
}
std::string LockFile::to_string() const
std::pair<std::string, LockFile::KeyMap> LockFile::to_string() const
{
return toJSON().dump(2);
}
LockFile LockFile::read(const Path & path)
{
if (!pathExists(path)) return LockFile();
return LockFile(nlohmann::json::parse(readFile(path)), path);
auto [json, nodeKeys] = toJSON();
return {json.dump(2), std::move(nodeKeys)};
}
std::ostream & operator <<(std::ostream & stream, const LockFile & lockFile)
{
stream << lockFile.toJSON().dump(2);
stream << lockFile.toJSON().first.dump(2);
return stream;
}
@@ -243,7 +240,7 @@ std::optional<FlakeRef> LockFile::isUnlocked() const
bool LockFile::operator ==(const LockFile & other) const
{
// FIXME: slow
return toJSON() == other.toJSON();
return toJSON().first == other.toJSON().first;
}
bool LockFile::operator !=(const LockFile & other) const
+5 -6
View File
@@ -55,18 +55,17 @@ struct LockFile
ref<Node> root = make_ref<Node>();
LockFile() {};
LockFile(const nlohmann::json & json, const Path & path);
LockFile(std::string_view contents, std::string_view path);
typedef std::map<ref<const Node>, std::string> KeyMap;
nlohmann::json toJSON() const;
std::pair<nlohmann::json, KeyMap> toJSON() const;
std::string to_string() const;
static LockFile read(const Path & path);
std::pair<std::string, KeyMap> to_string() const;
/**
* Check whether this lock file has any unlocked inputs.
* Check whether this lock file has any unlocked inputs. If so,
* return one.
*/
std::optional<FlakeRef> isUnlocked() const;
+6 -8
View File
@@ -5,13 +5,12 @@
namespace nix {
static const std::string attributeNamePattern("[a-zA-Z0-9_-]+");
static const std::regex lastAttributeRegex("(?:" + attributeNamePattern + "\\.)*(?!default)(" + attributeNamePattern +")(\\^.*)?");
static const std::regex lastAttributeRegex("^((?:" + attributeNamePattern + "\\.)*)(" + attributeNamePattern +")(\\^.*)?$");
static const std::string pathSegmentPattern("[a-zA-Z0-9_-]+");
static const std::regex lastPathSegmentRegex(".*/(" + pathSegmentPattern +")");
static const std::regex secondPathSegmentRegex("(?:" + pathSegmentPattern + ")/(" + pathSegmentPattern +")(?:/.*)?");
static const std::regex gitProviderRegex("github|gitlab|sourcehut");
static const std::regex gitSchemeRegex("git($|\\+.*)");
static const std::regex defaultOutputRegex(".*\\.default($|\\^.*)");
std::optional<std::string> getNameFromURL(const ParsedURL & url)
{
@@ -22,8 +21,11 @@ std::optional<std::string> getNameFromURL(const ParsedURL & url)
return url.query.at("dir");
/* If the fragment isn't a "default" and contains two attribute elements, use the last one */
if (std::regex_match(url.fragment, match, lastAttributeRegex))
return match.str(1);
if (std::regex_match(url.fragment, match, lastAttributeRegex)
&& match.str(1) != "defaultPackage."
&& match.str(2) != "default") {
return match.str(2);
}
/* If this is a github/gitlab/sourcehut flake, use the repo name */
if (std::regex_match(url.scheme, gitProviderRegex) && std::regex_match(url.path, match, secondPathSegmentRegex))
@@ -33,10 +35,6 @@ std::optional<std::string> getNameFromURL(const ParsedURL & url)
if (std::regex_match(url.scheme, gitSchemeRegex) && std::regex_match(url.path, match, lastPathSegmentRegex))
return match.str(1);
/* If everything failed but there is a non-default fragment, use it in full */
if (!url.fragment.empty() && !std::regex_match(url.fragment, defaultOutputRegex))
return url.fragment;
/* If there is no fragment, take the last element of the path */
if (std::regex_match(url.path, match, lastPathSegmentRegex))
return match.str(1);
+3 -2
View File
@@ -49,7 +49,7 @@ std::string PackageInfo::queryName() const
{
if (name == "" && attrs) {
auto i = attrs->find(state->sName);
if (i == attrs->end()) throw TypeError("derivation name missing");
if (i == attrs->end()) state->error<TypeError>("derivation name missing").debugThrow();
name = state->forceStringNoCtx(*i->value, noPos, "while evaluating the 'name' attribute of a derivation");
}
return name;
@@ -396,7 +396,8 @@ static void getDerivations(EvalState & state, Value & vIn,
}
}
else throw TypeError("expression does not evaluate to a derivation (or a set or list of those)");
else
state.error<TypeError>("expression does not evaluate to a derivation (or a set or list of those)").debugThrow();
}
+7 -6
View File
@@ -1,4 +1,6 @@
#include "json-to-value.hh"
#include "value.hh"
#include "eval.hh"
#include <variant>
#include <nlohmann/json.hpp>
@@ -55,11 +57,10 @@ class JSONSax : nlohmann::json_sax<json> {
ValueVector values;
std::unique_ptr<JSONState> resolve(EvalState & state) override
{
Value & v = parent->value(state);
state.mkList(v, values.size());
for (size_t n = 0; n < values.size(); ++n) {
v.listElems()[n] = values[n];
}
auto list = state.buildList(values.size());
for (const auto & [n, v2] : enumerate(list))
v2 = values[n];
parent->value(state).mkList(list);
return std::move(parent);
}
void add() override {
@@ -159,7 +160,7 @@ public:
}
bool parse_error(std::size_t, const std::string&, const nlohmann::detail::exception& ex) {
throw JSONParseError(ex.what());
throw JSONParseError("%s", ex.what());
}
};
+5 -2
View File
@@ -1,13 +1,16 @@
#pragma once
///@file
#include "eval.hh"
#include "error.hh"
#include <string>
namespace nix {
MakeError(JSONParseError, EvalError);
class EvalState;
struct Value;
MakeError(JSONParseError, Error);
void parseJSON(EvalState & state, const std::string_view & s, Value & v);
+15 -29
View File
@@ -33,33 +33,16 @@ namespace nix {
static void initLoc(YYLTYPE * loc)
{
loc->first_line = loc->last_line = 1;
loc->first_column = loc->last_column = 1;
loc->first_line = loc->last_line = 0;
loc->first_column = loc->last_column = 0;
}
static void adjustLoc(YYLTYPE * loc, const char * s, size_t len)
{
loc->stash();
loc->first_line = loc->last_line;
loc->first_column = loc->last_column;
for (size_t i = 0; i < len; i++) {
switch (*s++) {
case '\r':
if (*s == '\n') { /* cr/lf */
i++;
s++;
}
/* fall through */
case '\n':
++loc->last_line;
loc->last_column = 1;
break;
default:
++loc->last_column;
}
}
loc->last_column += len;
}
@@ -94,6 +77,9 @@ static StringToken unescapeStr(SymbolTable & symbols, char * s, size_t length)
}
// yacc generates code that uses unannotated fallthrough.
#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
#define YY_USER_INIT initLoc(yylloc)
#define YY_USER_ACTION adjustLoc(yylloc, yytext, yyleng);
@@ -146,9 +132,9 @@ or { return OR_KW; }
try {
yylval->n = boost::lexical_cast<int64_t>(yytext);
} catch (const boost::bad_lexical_cast &) {
throw ParseError({
.msg = hintfmt("invalid integer '%1%'", yytext),
.errPos = state->positions[CUR_POS],
throw ParseError(ErrorInfo{
.msg = HintFmt("invalid integer '%1%'", yytext),
.pos = state->positions[CUR_POS],
});
}
return INT_LIT;
@@ -156,9 +142,9 @@ or { return OR_KW; }
{FLOAT} { errno = 0;
yylval->nf = strtod(yytext, 0);
if (errno != 0)
throw ParseError({
.msg = hintfmt("invalid float '%1%'", yytext),
.errPos = state->positions[CUR_POS],
throw ParseError(ErrorInfo{
.msg = HintFmt("invalid float '%1%'", yytext),
.pos = state->positions[CUR_POS],
});
return FLOAT_LIT;
}
@@ -285,9 +271,9 @@ or { return OR_KW; }
<INPATH_SLASH>{ANY} |
<INPATH_SLASH><<EOF>> {
throw ParseError({
.msg = hintfmt("path has a trailing slash"),
.errPos = state->positions[CUR_POS],
throw ParseError(ErrorInfo{
.msg = HintFmt("path has a trailing slash"),
.pos = state->positions[CUR_POS],
});
}
+4 -1
View File
@@ -11,8 +11,11 @@ libexpr_SOURCES := \
$(wildcard $(d)/flake/*.cc) \
$(d)/lexer-tab.cc \
$(d)/parser-tab.cc
# Not just for this library itself, but also for downstream libraries using this library
libexpr_CXXFLAGS += -I src/libutil -I src/libstore -I src/libfetchers -I src/libmain -I src/libexpr
INCLUDE_libexpr := -I $(d)
libexpr_CXXFLAGS += $(INCLUDE_libutil) $(INCLUDE_libstore) $(INCLUDE_libfetchers) -I src/libmain $(INCLUDE_libexpr)
libexpr_LIBS = libutil libstore libfetchers
+131 -37
View File
@@ -70,10 +70,8 @@ void ExprOpHasAttr::show(const SymbolTable & symbols, std::ostream & str) const
str << ") ? " << showAttrPath(symbols, attrPath) << ")";
}
void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
void ExprAttrs::showBindings(const SymbolTable & symbols, std::ostream & str) const
{
if (recursive) str << "rec ";
str << "{ ";
typedef const decltype(attrs)::value_type * Attr;
std::vector<Attr> sorted;
for (auto & i : attrs) sorted.push_back(&i);
@@ -81,10 +79,37 @@ void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
std::string_view sa = symbols[a->first], sb = symbols[b->first];
return sa < sb;
});
std::vector<Symbol> inherits;
std::map<ExprInheritFrom *, std::vector<Symbol>> inheritsFrom;
for (auto & i : sorted) {
if (i->second.inherited)
str << "inherit " << symbols[i->first] << " " << "; ";
else {
switch (i->second.kind) {
case AttrDef::Kind::Plain:
break;
case AttrDef::Kind::Inherited:
inherits.push_back(i->first);
break;
case AttrDef::Kind::InheritedFrom: {
auto & select = dynamic_cast<ExprSelect &>(*i->second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*select.e);
inheritsFrom[&from].push_back(i->first);
break;
}
}
}
if (!inherits.empty()) {
str << "inherit";
for (auto sym : inherits) str << " " << symbols[sym];
str << "; ";
}
for (const auto & [from, syms] : inheritsFrom) {
str << "inherit (";
(*inheritFromExprs)[from->displ]->show(symbols, str);
str << ")";
for (auto sym : syms) str << " " << symbols[sym];
str << "; ";
}
for (auto & i : sorted) {
if (i->second.kind == AttrDef::Kind::Plain) {
str << symbols[i->first] << " = ";
i->second.e->show(symbols, str);
str << "; ";
@@ -97,6 +122,13 @@ void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
i.valueExpr->show(symbols, str);
str << "; ";
}
}
void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
{
if (recursive) str << "rec ";
str << "{ ";
showBindings(symbols, str);
str << "}";
}
@@ -117,7 +149,10 @@ void ExprLambda::show(const SymbolTable & symbols, std::ostream & str) const
if (hasFormals()) {
str << "{ ";
bool first = true;
for (auto & i : formals->formals) {
// the natural Symbol ordering is by creation time, which can lead to the
// same expression being printed in two different ways depending on its
// context. always use lexicographic ordering to avoid this.
for (auto & i : formals->lexicographicOrder(symbols)) {
if (first) first = false; else str << ", ";
str << symbols[i.name];
if (i.def) {
@@ -152,15 +187,7 @@ void ExprCall::show(const SymbolTable & symbols, std::ostream & str) const
void ExprLet::show(const SymbolTable & symbols, std::ostream & str) const
{
str << "(let ";
for (auto & i : attrs->attrs)
if (i.second.inherited) {
str << "inherit " << symbols[i.first] << "; ";
}
else {
str << symbols[i.first] << " = ";
i.second.e->show(symbols, str);
str << "; ";
}
attrs->showBindings(symbols, str);
str << "in ";
body->show(symbols, str);
str << ")";
@@ -296,15 +323,21 @@ void ExprVar::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> &
enclosing `with'. If there is no `with', then we can issue an
"undefined variable" error now. */
if (withLevel == -1)
throw UndefinedVarError({
.msg = hintfmt("undefined variable '%1%'", es.symbols[name]),
.errPos = es.positions[pos]
});
es.error<UndefinedVarError>(
"undefined variable '%1%'",
es.symbols[name]
).atPos(pos).debugThrow();
for (auto * e = env.get(); e && !fromWith; e = e->up)
fromWith = e->isWith;
this->level = withLevel;
}
void ExprInheritFrom::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
}
void ExprSelect::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
@@ -328,22 +361,47 @@ void ExprOpHasAttr::bindVars(EvalState & es, const std::shared_ptr<const StaticE
i.expr->bindVars(es, env);
}
std::shared_ptr<const StaticEnv> ExprAttrs::bindInheritSources(
EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (!inheritFromExprs)
return nullptr;
// the inherit (from) source values are inserted into an env of its own, which
// does not introduce any variable names.
// analysis must see an empty env, or an env that contains only entries with
// otherwise unused names to not interfere with regular names. the parser
// has already filled all exprs that access this env with appropriate level
// and displacement, and nothing else is allowed to access it. ideally we'd
// not even *have* an expr that grabs anything from this env since it's fully
// invisible, but the evaluator does not allow for this yet.
auto inner = std::make_shared<StaticEnv>(nullptr, env.get(), 0);
for (auto from : *inheritFromExprs)
from->bindVars(es, env);
return inner;
}
void ExprAttrs::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
if (recursive) {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), recursive ? attrs.size() : 0);
auto newEnv = [&] () -> std::shared_ptr<const StaticEnv> {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs.size());
Displacement displ = 0;
for (auto & i : attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
Displacement displ = 0;
for (auto & i : attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
return newEnv;
}();
// No need to sort newEnv since attrs is in sorted order.
auto inheritFromEnv = bindInheritSources(es, newEnv);
for (auto & i : attrs)
i.second.e->bindVars(es, i.second.inherited ? env : newEnv);
i.second.e->bindVars(es, i.second.chooseByKind(newEnv, env, inheritFromEnv));
for (auto & i : dynamicAttrs) {
i.nameExpr->bindVars(es, newEnv);
@@ -351,8 +409,10 @@ void ExprAttrs::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv>
}
}
else {
auto inheritFromEnv = bindInheritSources(es, env);
for (auto & i : attrs)
i.second.e->bindVars(es, env);
i.second.e->bindVars(es, i.second.chooseByKind(env, env, inheritFromEnv));
for (auto & i : dynamicAttrs) {
i.nameExpr->bindVars(es, env);
@@ -409,19 +469,23 @@ void ExprCall::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> &
void ExprLet::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
auto newEnv = [&] () -> std::shared_ptr<const StaticEnv> {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs->attrs.size());
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs->attrs.size());
Displacement displ = 0;
for (auto & i : attrs->attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
Displacement displ = 0;
for (auto & i : attrs->attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
return newEnv;
}();
// No need to sort newEnv since attrs->attrs is in sorted order.
auto inheritFromEnv = attrs->bindInheritSources(es, newEnv);
for (auto & i : attrs->attrs)
i.second.e->bindVars(es, i.second.inherited ? env : newEnv);
i.second.e->bindVars(es, i.second.chooseByKind(newEnv, env, inheritFromEnv));
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, newEnv));
body->bindVars(es, newEnv);
}
@@ -447,9 +511,6 @@ void ExprWith::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> &
break;
}
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
attrs->bindVars(es, env);
auto newEnv = std::make_shared<StaticEnv>(this, env.get());
body->bindVars(es, newEnv);
@@ -522,6 +583,39 @@ std::string ExprLambda::showNamePos(const EvalState & state) const
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(
std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
/* Symbol table. */
size_t SymbolTable::totalSize() const
+52 -104
View File
@@ -7,112 +7,14 @@
#include "value.hh"
#include "symbol-table.hh"
#include "error.hh"
#include "chunked-vector.hh"
#include "position.hh"
#include "eval-error.hh"
#include "pos-idx.hh"
#include "pos-table.hh"
namespace nix {
MakeError(EvalError, Error);
MakeError(ParseError, Error);
MakeError(AssertionError, EvalError);
MakeError(ThrownError, AssertionError);
MakeError(Abort, EvalError);
MakeError(TypeError, EvalError);
MakeError(UndefinedVarError, Error);
MakeError(MissingArgumentError, EvalError);
class InfiniteRecursionError : public EvalError
{
friend class EvalState;
public:
using EvalError::EvalError;
};
class PosIdx {
friend class PosTable;
private:
uint32_t id;
explicit PosIdx(uint32_t id): id(id) {}
public:
PosIdx() : id(0) {}
explicit operator bool() const { return id > 0; }
bool operator <(const PosIdx other) const { return id < other.id; }
bool operator ==(const PosIdx other) const { return id == other.id; }
bool operator !=(const PosIdx other) const { return id != other.id; }
};
class PosTable
{
public:
class Origin {
friend PosTable;
private:
// must always be invalid by default, add() replaces this with the actual value.
// subsequent add() calls use this index as a token to quickly check whether the
// current origins.back() can be reused or not.
mutable uint32_t idx = std::numeric_limits<uint32_t>::max();
// Used for searching in PosTable::[].
explicit Origin(uint32_t idx): idx(idx), origin{std::monostate()} {}
public:
const Pos::Origin origin;
Origin(Pos::Origin origin): origin(origin) {}
};
struct Offset {
uint32_t line, column;
};
private:
std::vector<Origin> origins;
ChunkedVector<Offset, 8192> offsets;
public:
PosTable(): offsets(1024)
{
origins.reserve(1024);
}
PosIdx add(const Origin & origin, uint32_t line, uint32_t column)
{
const auto idx = offsets.add({line, column}).second;
if (origins.empty() || origins.back().idx != origin.idx) {
origin.idx = idx;
origins.push_back(origin);
}
return PosIdx(idx + 1);
}
Pos operator[](PosIdx p) const
{
if (p.id == 0 || p.id > offsets.size())
return {};
const auto idx = p.id - 1;
/* we want the last key <= idx, so we'll take prev(first key > idx).
this is guaranteed to never rewind origin.begin because the first
key is always 0. */
const auto pastOrigin = std::upper_bound(
origins.begin(), origins.end(), Origin(idx),
[] (const auto & a, const auto & b) { return a.idx < b.idx; });
const auto origin = *std::prev(pastOrigin);
const auto offset = offsets[idx];
return {offset.line, offset.column, origin.origin};
}
};
inline PosIdx noPos = {};
struct Env;
struct Value;
class EvalState;
@@ -232,6 +134,23 @@ struct ExprVar : Expr
COMMON_METHODS
};
/**
* A pseudo-expression for the purpose of evaluating the `from` expression in `inherit (from)` syntax.
* Unlike normal variable references, the displacement is set during parsing, and always refers to
* `ExprAttrs::inheritFromExprs` (by itself or in `ExprLet`), whose values are put into their own `Env`.
*/
struct ExprInheritFrom : ExprVar
{
ExprInheritFrom(PosIdx pos, Displacement displ): ExprVar(pos, {})
{
this->level = 0;
this->displ = displ;
this->fromWith = nullptr;
}
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env);
};
struct ExprSelect : Expr
{
PosIdx pos;
@@ -257,16 +176,40 @@ struct ExprAttrs : Expr
bool recursive;
PosIdx pos;
struct AttrDef {
bool inherited;
enum class Kind {
/** `attr = expr;` */
Plain,
/** `inherit attr1 attrn;` */
Inherited,
/** `inherit (expr) attr1 attrn;` */
InheritedFrom,
};
Kind kind;
Expr * e;
PosIdx pos;
Displacement displ; // displacement
AttrDef(Expr * e, const PosIdx & pos, bool inherited=false)
: inherited(inherited), e(e), pos(pos) { };
AttrDef(Expr * e, const PosIdx & pos, Kind kind = Kind::Plain)
: kind(kind), e(e), pos(pos) { };
AttrDef() { };
template<typename T>
const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
{
switch (kind) {
case Kind::Plain:
return plain;
case Kind::Inherited:
return inherited;
default:
case Kind::InheritedFrom:
return inheritedFrom;
}
}
};
typedef std::map<Symbol, AttrDef> AttrDefs;
AttrDefs attrs;
std::unique_ptr<std::vector<Expr *>> inheritFromExprs;
struct DynamicAttrDef {
Expr * nameExpr, * valueExpr;
PosIdx pos;
@@ -279,6 +222,11 @@ struct ExprAttrs : Expr
ExprAttrs() : recursive(false) { };
PosIdx getPos() const override { return pos; }
COMMON_METHODS
std::shared_ptr<const StaticEnv> bindInheritSources(
EvalState & es, const std::shared_ptr<const StaticEnv> & env);
Env * buildInheritFromEnv(EvalState & state, Env & up);
void showBindings(const SymbolTable & symbols, std::ostream & str) const;
};
struct ExprList : Expr
+22 -16
View File
@@ -24,20 +24,15 @@ struct ParserLocation
int last_line, last_column;
// backup to recover from yyless(0)
int stashed_first_line, stashed_first_column;
int stashed_last_line, stashed_last_column;
int stashed_first_column, stashed_last_column;
void stash() {
stashed_first_line = first_line;
stashed_first_column = first_column;
stashed_last_line = last_line;
stashed_last_column = last_column;
}
void unstash() {
first_line = stashed_first_line;
first_column = stashed_first_column;
last_line = stashed_last_line;
last_column = stashed_last_column;
}
};
@@ -64,17 +59,17 @@ struct ParserState
inline void ParserState::dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos)
{
throw ParseError({
.msg = hintfmt("attribute '%1%' already defined at %2%",
.msg = HintFmt("attribute '%1%' already defined at %2%",
showAttrPath(symbols, attrPath), positions[prevPos]),
.errPos = positions[pos]
.pos = positions[pos]
});
}
inline void ParserState::dupAttr(Symbol attr, const PosIdx pos, const PosIdx prevPos)
{
throw ParseError({
.msg = hintfmt("attribute '%1%' already defined at %2%", symbols[attr], positions[prevPos]),
.errPos = positions[pos]
.msg = HintFmt("attribute '%1%' already defined at %2%", symbols[attr], positions[prevPos]),
.pos = positions[pos]
});
}
@@ -89,7 +84,7 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, Expr *
if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
if (!j->second.inherited) {
if (j->second.kind != ExprAttrs::AttrDef::Kind::Inherited) {
ExprAttrs * attrs2 = dynamic_cast<ExprAttrs *>(j->second.e);
if (!attrs2) dupAttr(attrPath, pos, j->second.pos);
attrs = attrs2;
@@ -118,13 +113,24 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, Expr *
auto ae = dynamic_cast<ExprAttrs *>(e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
dupAttr(ad.first, j2->second.pos, ad.second.pos);
jAttrs->attrs.emplace(ad.first, ad.second);
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
}
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(), ae->dynamicAttrs.begin(), ae->dynamicAttrs.end());
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
ae->inheritFromExprs->begin(), ae->inheritFromExprs->end());
}
} else {
dupAttr(attrPath, pos, j->second.pos);
}
@@ -154,14 +160,14 @@ inline Formals * ParserState::validateFormals(Formals * formals, PosIdx pos, Sym
}
if (duplicate)
throw ParseError({
.msg = hintfmt("duplicate formal function argument '%1%'", symbols[duplicate->first]),
.errPos = positions[duplicate->second]
.msg = HintFmt("duplicate formal function argument '%1%'", symbols[duplicate->first]),
.pos = positions[duplicate->second]
});
if (arg && formals->has(arg))
throw ParseError({
.msg = hintfmt("duplicate formal function argument '%1%'", symbols[arg]),
.errPos = positions[pos]
.msg = HintFmt("duplicate formal function argument '%1%'", symbols[arg]),
.pos = positions[pos]
});
return formals;
@@ -265,7 +271,7 @@ inline Expr * ParserState::stripIndentation(const PosIdx pos,
inline PosIdx ParserState::at(const ParserLocation & loc)
{
return positions.add(origin, loc.first_line, loc.first_column);
return positions.add(origin, loc.first_column);
}
}
+36 -21
View File
@@ -64,9 +64,13 @@ using namespace nix;
void yyerror(YYLTYPE * loc, yyscan_t scanner, ParserState * state, const char * error)
{
if (std::string_view(error).starts_with("syntax error, unexpected end of file")) {
loc->first_column = loc->last_column;
loc->first_line = loc->last_line;
}
throw ParseError({
.msg = hintfmt(error),
.errPos = state->positions[state->at(*loc)]
.msg = HintFmt(error),
.pos = state->positions[state->at(*loc)]
});
}
@@ -87,6 +91,7 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParserState * state, const char *
nix::StringToken uri;
nix::StringToken str;
std::vector<nix::AttrName> * attrNames;
std::vector<std::pair<nix::AttrName, nix::PosIdx>> * inheritAttrs;
std::vector<std::pair<nix::PosIdx, nix::Expr *>> * string_parts;
std::vector<std::pair<nix::PosIdx, std::variant<nix::Expr *, nix::StringToken>>> * ind_string_parts;
}
@@ -97,7 +102,8 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParserState * state, const char *
%type <attrs> binds
%type <formals> formals
%type <formal> formal
%type <attrNames> attrs attrpath
%type <attrNames> attrpath
%type <inheritAttrs> attrs
%type <string_parts> string_parts_interpolated
%type <ind_string_parts> ind_string_parts
%type <e> path_start string_parts string_attr
@@ -154,8 +160,8 @@ expr_function
| LET binds IN_KW expr_function
{ if (!$2->dynamicAttrs.empty())
throw ParseError({
.msg = hintfmt("dynamic attributes not allowed in let"),
.errPos = state->positions[CUR_POS]
.msg = HintFmt("dynamic attributes not allowed in let"),
.pos = state->positions[CUR_POS]
});
$$ = new ExprLet($2, $4);
}
@@ -244,8 +250,8 @@ expr_simple
static bool noURLLiterals = experimentalFeatureSettings.isEnabled(Xp::NoUrlLiterals);
if (noURLLiterals)
throw ParseError({
.msg = hintfmt("URL literals are disabled"),
.errPos = state->positions[CUR_POS]
.msg = HintFmt("URL literals are disabled"),
.pos = state->positions[CUR_POS]
});
$$ = new ExprString(std::string($1));
}
@@ -309,21 +315,30 @@ binds
: binds attrpath '=' expr ';' { $$ = $1; state->addAttr($$, std::move(*$2), $4, state->at(@2)); delete $2; }
| binds INHERIT attrs ';'
{ $$ = $1;
for (auto & i : *$3) {
for (auto & [i, iPos] : *$3) {
if ($$->attrs.find(i.symbol) != $$->attrs.end())
state->dupAttr(i.symbol, state->at(@3), $$->attrs[i.symbol].pos);
auto pos = state->at(@3);
$$->attrs.emplace(i.symbol, ExprAttrs::AttrDef(new ExprVar(CUR_POS, i.symbol), pos, true));
state->dupAttr(i.symbol, iPos, $$->attrs[i.symbol].pos);
$$->attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(new ExprVar(iPos, i.symbol), iPos, ExprAttrs::AttrDef::Kind::Inherited));
}
delete $3;
}
| binds INHERIT '(' expr ')' attrs ';'
{ $$ = $1;
/* !!! Should ensure sharing of the expression in $4. */
for (auto & i : *$6) {
if (!$$->inheritFromExprs)
$$->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
$$->inheritFromExprs->push_back($4);
auto from = new nix::ExprInheritFrom(state->at(@4), $$->inheritFromExprs->size() - 1);
for (auto & [i, iPos] : *$6) {
if ($$->attrs.find(i.symbol) != $$->attrs.end())
state->dupAttr(i.symbol, state->at(@6), $$->attrs[i.symbol].pos);
$$->attrs.emplace(i.symbol, ExprAttrs::AttrDef(new ExprSelect(CUR_POS, $4, i.symbol), state->at(@6)));
state->dupAttr(i.symbol, iPos, $$->attrs[i.symbol].pos);
$$->attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(
new ExprSelect(iPos, from, i.symbol),
iPos,
ExprAttrs::AttrDef::Kind::InheritedFrom));
}
delete $6;
}
@@ -331,20 +346,20 @@ binds
;
attrs
: attrs attr { $$ = $1; $1->push_back(AttrName(state->symbols.create($2))); }
: attrs attr { $$ = $1; $1->emplace_back(AttrName(state->symbols.create($2)), state->at(@2)); }
| attrs string_attr
{ $$ = $1;
ExprString * str = dynamic_cast<ExprString *>($2);
if (str) {
$$->push_back(AttrName(state->symbols.create(str->s)));
$$->emplace_back(AttrName(state->symbols.create(str->s)), state->at(@2));
delete str;
} else
throw ParseError({
.msg = hintfmt("dynamic attributes not allowed in inherit"),
.errPos = state->positions[state->at(@2)]
.msg = HintFmt("dynamic attributes not allowed in inherit"),
.pos = state->positions[state->at(@2)]
});
}
| { $$ = new AttrPath; }
| { $$ = new std::vector<std::pair<AttrName, PosIdx>>; }
;
attrpath
@@ -423,7 +438,7 @@ Expr * parseExprFromBuf(
.symbols = symbols,
.positions = positions,
.basePath = basePath,
.origin = {origin},
.origin = positions.addOrigin(origin, length),
.rootFS = rootFS,
.s = astSymbols,
};
+5 -1
View File
@@ -1,5 +1,4 @@
#include "eval.hh"
#include "fs-input-accessor.hh"
namespace nix {
@@ -8,4 +7,9 @@ SourcePath EvalState::rootPath(CanonPath path)
return {rootFS, std::move(path)};
}
SourcePath EvalState::rootPath(PathView path)
{
return {rootFS, CanonPath(absPath(path))};
}
}
+49
View File
@@ -0,0 +1,49 @@
#pragma once
#include <cinttypes>
namespace nix {
class PosIdx
{
friend struct LazyPosAcessors;
friend class PosTable;
private:
uint32_t id;
explicit PosIdx(uint32_t id)
: id(id)
{
}
public:
PosIdx()
: id(0)
{
}
explicit operator bool() const
{
return id > 0;
}
bool operator<(const PosIdx other) const
{
return id < other.id;
}
bool operator==(const PosIdx other) const
{
return id == other.id;
}
bool operator!=(const PosIdx other) const
{
return id != other.id;
}
};
inline PosIdx noPos = {};
}
+87
View File
@@ -0,0 +1,87 @@
#pragma once
#include <cinttypes>
#include <numeric>
#include <vector>
#include "chunked-vector.hh"
#include "pos-idx.hh"
#include "position.hh"
#include "sync.hh"
namespace nix {
class PosTable
{
public:
class Origin
{
friend PosTable;
private:
uint32_t offset;
Origin(Pos::Origin origin, uint32_t offset, size_t size):
offset(offset), origin(origin), size(size)
{}
public:
const Pos::Origin origin;
const size_t size;
uint32_t offsetOf(PosIdx p) const
{
return p.id - 1 - offset;
}
};
private:
using Lines = std::vector<uint32_t>;
std::map<uint32_t, Origin> origins;
mutable Sync<std::map<uint32_t, Lines>> lines;
const Origin * resolve(PosIdx p) const
{
if (p.id == 0)
return nullptr;
const auto idx = p.id - 1;
/* we want the last key <= idx, so we'll take prev(first key > idx).
this is guaranteed to never rewind origin.begin because the first
key is always 0. */
const auto pastOrigin = origins.upper_bound(idx);
return &std::prev(pastOrigin)->second;
}
public:
Origin addOrigin(Pos::Origin origin, size_t size)
{
uint32_t offset = 0;
if (auto it = origins.rbegin(); it != origins.rend())
offset = it->first + it->second.size;
// +1 because all PosIdx are offset by 1 to begin with, and
// another +1 to ensure that all origins can point to EOF, eg
// on (invalid) empty inputs.
if (2 + offset + size < offset)
return Origin{origin, offset, 0};
return origins.emplace(offset, Origin{origin, offset, size}).first->second;
}
PosIdx add(const Origin & origin, size_t offset)
{
if (offset > origin.size)
return PosIdx();
return PosIdx(1 + origin.offset + offset);
}
Pos operator[](PosIdx p) const;
Pos::Origin originOf(PosIdx p) const
{
if (auto o = resolve(p))
return o->origin;
return std::monostate{};
}
};
}
+414 -327
View File
File diff suppressed because it is too large. Load diff
+2
View File
@@ -51,4 +51,6 @@ void prim_importNative(EvalState & state, const PosIdx pos, Value * * args, Valu
*/
void prim_exec(EvalState & state, const PosIdx pos, Value * * args, Value & v);
void makePositionThunks(EvalState & state, const PosIdx pos, Value & line, Value & column);
}
Loaded 100 of 349 files, more files were not shown because too many files have changed in this diff. Show more