Compare commits

..
Author SHA1 Message Date
Théophane Hufschmitt 3740cdb654 Fix the parsing of path flake refs with a query
`/path?foo=bar` was parsed as `/path`, omitting the query altogether
(`/path?foo=bar#` was parsed correctly though).

Rewrite the path flake ref parser to fix this
2024-03-02 10:12:12 +01:00
Théophane Hufschmitt fbf3f9398a Take the query into account for plain path flakerefs
The query parameters for path flakerefs (`/foo/bar?query=blah` or
`.?foo=bar`) was ignored when the path wasn't a git repo.
Fix that
2024-03-02 10:12:12 +01:00
Théophane Hufschmitt f97d6a3178 Test the path flakeref parsing
Make sure that various combinations of query strings, fragments, relative
and absolute paths behave properly.
2024-03-02 10:11:22 +01:00
Théophane Hufschmitt bf48501194 Merge pull request #10112 from edolstra/fix-gcc12-warnings
Fix gcc 12 warnings
2024-02-29 15:55:59 +01:00
Eelco Dolstra b1586a6799 Merge pull request #10090 from bobvanderlinden/profile-error-install-twice
profile install: warn on installing package twice
2024-02-29 10:04:08 +01:00
Bob van der Linden 14adff1711 profile install: skip and warn on installing package twice 2024-02-29 08:29:38 +01:00
tomberek ffe67c86a8 Merge pull request #9915 from 9999years/evaluating-attribute-position
Add position information to `while evaluating the attribute` errors in the debugger
2024-02-28 18:11:07 -05:00
Eelco Dolstra 65bb12ba78 Fix gcc 12 warnings 2024-02-28 22:59:20 +01:00
Théophane Hufschmitt 90ec015d61 Merge pull request #10070 from ramboman/fix-proxy-installer
`install-multi-user.sh`: `_sudo`: add proxy variables to sudo
2024-02-28 22:00:15 +01:00
Robert Hensing bd21b4b134 Merge pull request #10110 from hercules-ci/bump-actions-docker
actions docker_push_image: Update nix 2.13.3 -> 2.20.3
2024-02-28 20:27:54 +01:00
Robert Hensing 4d769e7a76 actions docker_push_image: Update nix 2.13.3 -> 2.20.3 2024-02-28 19:54:17 +01:00
Robert Hensing 587c7dcb2b Merge pull request #9546 from NixOS/nixos-23.11
Update to nixos-23.11
2024-02-28 17:51:16 +01:00
Théophane Hufschmitt 864fc85fc8 Merge pull request #10010 from 9999years/fix-9941
Fix "Failed tcsetattr(TCSADRAIN)" when `nix repl` is not a TTY
2024-02-28 14:25:58 +01:00
Théophane Hufschmitt f6142cd0d1 unset NIX_HARDENING_ENABLE in fast build mode
`NIX_HARDENING_ENABLE` causes `_FORTIFY_SOURCE` to be defined.
This isn't compatible with `-O0`, and the compiler will happily remind
us about it at every call, spamming the terminal with warnings and stack
traces.

We don't really care hardening in that case, so just disable it if we
pass `OPTIMIZE=0`.
2024-02-28 08:07:51 +01:00
Théophane Hufschmitt da90be789d Fix a too smart implicit cast
Apparently gcc is able to implicitly cast from `FileIngestionMethod` to
`ContentAddressMethod`, but clang isn't. So explicit the cast
2024-02-28 08:00:17 +01:00
Théophane Hufschmitt 6147d27afb Bump the required daemon version for the git hashing tests
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:11:22 +01:00
Robert Hensing 8dc4b41c7f flake.lock: Strip out treeHash. Too soon...
I hate this.
We should have it, but for now we can't.
2024-02-28 07:08:21 +01:00
Robert Hensing 945940f2ef nixpkgs: nixos-23.11-small -> release-23.11
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4dd376f7943c64b522224a548d9cab5627b4d9d6' (2024-02-26)
  → 'github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e' (2024-02-28)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt cf3ef060ff Disable the “static” darwin stdenvs
Don't evaluate, and probably not really useful (if at all)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt a0cb75d96f Disable bear on all the things with darwin as hostPlatform
Just `stdenv.isDarwin` isn't enough because it doesn't apply to the
build platform, which mean that cross packages building from darwin to
another platform will have `isDarwin` set to false.
Replace it by `stdenv.buildPlatform.isDarwin`.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt bbef03872b Bump the required daemon version for the impure-env test
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt 44f10f000a flake: Update to NixOS 23.11
About time :)

This required disabling `bear` on darwin as it's currently broken (fixed
on master, but not yet on 23.11).
2024-02-28 07:08:21 +01:00
John Ericson f489a6e42d Merge pull request #8918 from obsidiansystems/git-objects
Git object hashing in libstore
2024-02-27 19:02:22 -05:00
John EricsonandEelco Dolstra d4ad1fcf30 Avoid creating temporary store object for git over the wire
Instead, serialize as NAR and send that over, then rehash sever side.
This is alorithmically simpler, but comes at the cost of a newer
parameter to `Store::addToStoreFromDump`.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-02-27 11:27:34 -05:00
201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
John Ericson 04836c73e5 Merge nativeCheckInputs into nativeBuildInputs
They were getting skipped for the test-against checks.
2024-02-27 11:27:34 -05:00
Eelco Dolstra c3e9e3d0c3 Merge pull request #9767 from obsidiansystems/canon-path-split
Support Windows paths in `canonPath` and `absPath`
2024-02-27 17:12:29 +01:00
John Ericson 354ba27296 Merge pull request #10093 from NixOS/revert-10084-remove-dead-git-code
Revert "Remove dead Git code"
2024-02-27 09:54:25 -05:00
Eelco Dolstra 5b0d78ec73 Merge pull request #10091 from bobvanderlinden/default-package-name
Fix extraction of name for defaultPackage URLs
2024-02-27 15:14:20 +01:00
Bob van der Linden e5d9130a5b Fix extraction of name for defaultPackage URLs 2024-02-27 07:53:05 +01:00
Théophane Hufschmitt be0052b45f Revert "Remove dead Git code" 2024-02-27 06:39:30 +01:00
Bob van der Linden d28a240aa6 profile: extract getNameFromElement 2024-02-26 21:07:18 +01:00
Robert Hensing 4c7f0ef6ca Merge pull request #9847 from pennae/inherit-from-dedup
deduplicate inherit-from source expr work
2024-02-26 20:25:58 +01:00
Théophane Hufschmitt c10025d8ca Merge pull request #10084 from tweag/remove-dead-git-code
Remove dead Git code
2024-02-26 19:49:14 +01:00
pennaeandRobert Hensing f24e445bc0 add doc comment justifying ExprInheritFrom
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-26 19:07:08 +01:00
pennae 1cd87b7042 remove ExprAttrs::AttrDef::inherited
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.
2024-02-26 19:07:08 +01:00
pennae cefd0302b5 evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
2024-02-26 19:07:08 +01:00
John Ericson 62a13c8101 Merge pull request #10086 from ShamrockLee/hash-algorithm-rename
treewide: hash type -> hash algorithm
2024-02-26 08:37:09 -05:00
Théophane Hufschmitt cd2d8b6d4c Merge pull request #10085 from ShamrockLee/ignore-obsolete-testdir
.gitignore: ignore historical test binaries
2024-02-26 11:36:10 +01:00
Yueh-Shun Li fd47f76da9 treewide: hash type -> hash algorithm
"hash type" -> "hash algorithm" in all comments, documentation, and
messages.

ht -> ha, [Hh]ashType -> [HhashAlgo] for all local variables and
function arguments. No API change is made.

Continuation of 5334c9c792 and 837b889c41.
2024-02-26 18:09:06 +08:00
Théophane Hufschmitt 219705ff64 Remove dead code
Most of the code in `git.{cc,hh}` is dead, so get rid of it.
2024-02-26 11:07:47 +01:00
Yueh-Shun Li 8ac4542593 .gitignore: ignore historical test binaries
After commit 91b6833686 (" Move tests to separate directories, and
document"), previously-built test executables are now tracked by Git,
which is annoying for developers.

This patch add .gitignore rules to ignore the obsolete test directories
to solve such problem and enhance developer experience.
2024-02-26 18:05:07 +08:00
Théophane Hufschmitt 21282c3c20 Merge pull request #10074 from lf-/jade/ban-implicit-fallthrough
Warn on implicit switch case fallthrough
2024-02-26 10:48:07 +01:00
Eelco Dolstra 7d9ae5fd06 Merge pull request #10071 from zimbatm/ci-default-tag
ci default tag
2024-02-26 10:45:26 +01:00
Jade Lovelace a82aeedb5b Warn on implicit switch case fallthrough
This seems to have found one actual bug in fs-sink.cc: the symlink case
was falling into the regular file case, which can't possibly be
intentional, right?
2024-02-24 15:52:16 -08:00
Johannes KirschbauerandValentin Gagarin d83008c3a7 documentation: clarify genericClosure (#10003)
* doc: clarify genericClosure documentation

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-02-24 12:34:53 +00:00
zimbatm 5598ce3e0f ci: fix docker default tag
Docker uses "latest" as the default label instead of "master".

This change will allow to docker run ghcr.io/nixos/nix without having to
specify the label.

It keeps the :master label on docker hub for back-compat.
2024-02-24 11:24:24 +01:00
ramboman 24fd7e2755 install-multi-user.sh: _sudo: add proxy variables to sudo 2024-02-24 01:00:13 +00:00
Robert Hensing 0b47783d0a Merge pull request #10066 from 9999years/print-all-frames
Do not skip any stack frames when `--show-trace` is given
2024-02-23 18:53:11 +01:00
Théophane Hufschmitt accae60e77 Merge pull request #10067 from ramboman/fix-proxy-nix
`nix`: Fix `haveInternet` to check for proxy
2024-02-23 11:06:36 +01:00
Théophane Hufschmitt bca737dcad c++-ize the proxy detection code
Just for consistency with the rest
2024-02-23 10:28:37 +01:00
ramboman d3bff699aa nix: Fix haveInternet to check for proxy 2024-02-23 01:05:25 -05:00
Rebecca Turner fe6408b5df Update snapshots 2024-02-22 17:58:55 -08:00
Rebecca Turner 91e89628fd Make addErrorTrace variadic 2024-02-22 17:18:27 -08:00
Rebecca Turner f05c13ecc2 Remove the concept of "skipped frames" 2024-02-22 17:14:55 -08:00
Rebecca Turner 040874e4db Print all stack frames 2024-02-22 17:14:33 -08:00
Théophane Hufschmitt 6a5210f48e Merge pull request #9815 from NixOS/nix-hash-path
`nix hash path`, text hashing for `nix store add`, and preparatory refactors
2024-02-22 17:15:34 +01:00
Eelco Dolstra f183eef4a0 Merge pull request #10060 from bobvanderlinden/pr-simple-profile-remove-suggestion
profile: suggest removal using profile entry name
2024-02-22 11:02:12 +01:00
Bob van der Linden 4ae5091716 nix profile: suggest removal using entry name
When a file conflict arises during a package install a suggestion is
made to remove the old entry. This was previously done using the
installable URLs of the old entry. These URLs are quite verbose and
often do not equal the URL of the existing entry.

This change uses the recently introduced profile entry name for the
suggestion, resulting in a simpler output.

The improvement is easily seen in the change to the functional test.
2024-02-21 21:58:36 +01:00
John Ericson 2080d89b87 Merge pull request #10038 from edolstra/tarball-git-cache
Use the Git cache for tarball flakes
2024-02-21 15:47:02 -05:00
Eelco Dolstra e391fc2101 Add comments 2024-02-21 21:15:28 +01:00
Théophane Hufschmitt 774e7ca584 Merge pull request #9914 from 9999years/debugger-on-trace
Enter debugger on `builtins.trace` with an option
2024-02-21 18:32:56 +01:00
Théophane Hufschmitt 36fa8b1bcc Merge pull request #9943 from 9999years/release-notes
Add release notes
2024-02-21 18:17:50 +01:00
John Ericson efd36b49e8 nix hash path, and preperatory refactors
- `nix store add` supports text hashing

  With functional test ensuring it matches `builtins.toFile`.

- Factored-out flags for both commands

- Move all common reusable flags to `libcmd`

  - They are not part of the *definition* of the CLI infra, just a usag
    of it.

  - The `libstore` flag couldn't go in `args.hh` in libutil anyways,
    would be awkward for it to live alone

- Shuffle around `Cmd*` hierarchy so flags for deprecated commands don't
  end up on the new ones
2024-02-21 12:11:25 -05:00
Rebecca Turner b111fba8cd Add documentation, rename to debugger-on-trace 2024-02-21 09:07:39 -08:00
Eelco Dolstra 3f5d7afe46 Merge pull request #10024 from edolstra/remove-locked-flag
Input: Replace 'locked' bool by isLocked() method
2024-02-21 16:19:15 +01:00
Eelco Dolstra 9ae665b9e1 Merge pull request #10054 from syvb/channel-unpack-count
Say how many channels were unpacked in nix-channel
2024-02-21 12:23:57 +01:00
Eelco Dolstra 09d76e512a GitArchiveInputScheme: Require a NAR hash 2024-02-21 12:08:18 +01:00
Eelco Dolstra ff4fa4dbd3 Merge pull request #10055 from GrahamDennis/gdennis/faster-flake-lock-parsing
Faster flake.lock parsing
2024-02-21 11:57:44 +01:00
Graham Dennis 7fd0de38c6 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.
2024-02-21 18:40:34 +11:00
syvb 60eeacc24a Say how many channels were unpacked 2024-02-20 19:17:18 -05:00
John Ericson a1b1070dbe Merge pull request #9916 from 9999years/quit-whole-program-from-debugger
`:quit` in the debugger should quit the whole program
2024-02-20 18:12:05 -05:00
Rebecca Turner 8e71883e3f Rename ProcessLineResult variants 2024-02-20 14:52:16 -08:00
Rebecca Turner 2a8fe9a938 :quit in the debugger should quit the whole program 2024-02-20 10:01:13 -08:00
Eelco Dolstra 071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
John Ericson 78e7c98b02 Merge pull request #10049 from edolstra/dont-send-experimental-features
Don't send settings that depend on disabled experimental features to the daemon
2024-02-20 09:53:08 -05:00
Eelco Dolstra 0acd783190 Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.
2024-02-20 15:28:42 +01:00
Théophane Hufschmitt d2c6a93bd5 Merge pull request #10044 from edolstra/empty-git-repos
Handle empty Git repositories / workdirs
2024-02-20 14:01:23 +01:00
Eelco Dolstra cabee98152 Tarball fetcher: Use the content-addressed Git cache
Backported from the lazy-trees branch.
2024-02-20 12:57:36 +01:00
Eelco Dolstra 930b9c8269 PosixSourceAccessor: Support roots that are not directories
We have to support this for `fetchTree { type = "file" }` (and
probably other types of trees that can have a non-directory at the
root, like NARs).
2024-02-20 12:51:15 +01:00
Eelco Dolstra b00f412f81 Remove bad.tar.xz check, since libarchive doesn't care 2024-02-20 12:51:15 +01:00
Eelco Dolstra db012d1e63 tests/functional/tarball.sh: Fix invalid file:// URLs 2024-02-20 12:51:15 +01:00
Théophane Hufschmitt 6f4bb1b584 Merge pull request #10043 from edolstra/fix-readonly-fetchToStore
fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:07:48 +01:00
Eelco Dolstra 7cb4d0c5b7 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:46:49 +01:00
Eelco Dolstra 9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
Eelco Dolstra d52d91fe7a AllowListInputAccessor: Clarify that the "allowed paths" are actually allowed prefixes
E.g. adding "/" will allow access to the root and *everything below it*.
2024-02-20 11:23:26 +01:00
Eelco Dolstra 6162105675 Don't say "copying X to the store" in read-only mode 2024-02-20 10:47:30 +01:00
John Ericson 319ec6f84a Support Windows paths in canonPath and absPath
`canonPath` and `absPath` work on native paths, and so should switch
between supporting Unix paths and Windows paths accordingly.

The templating is because `CanonPath`, which shares the implementation,
should always be Unix style. It is the pure "nix-native" path type for
virtual file operations --- it is part of Nix's "business logic", and
should not vary with the host OS accordingly.
2024-02-16 10:31:36 -05:00
John Ericson 4531585275 Factor out the Unix-specific parts of canonPathInner
This prepares the code to also support Windows paths in the next commit.
2024-02-16 10:12:07 -05:00
Rebecca Turner 6d2b446e2b Add release notes for "Pretty print values in nix repl" 2024-02-14 08:58:54 -08:00
Rebecca Turner a694cfb7bd Fix "Failed tcsetattr(TCSADRAIN)" when nix repl is not a TTY
Before:
```
$ echo builtins.nixVersion | nix repl
Welcome to Nix 2.18.1. Type :? for help.

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
"2.18.1"

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
```

After:
```
$ echo builtins.nixVersion | nix repl
Nix 2.21.0pre20240131_dirty
Type :? for help.
"2.21.0pre20240131_dirty"
```
2024-02-13 11:09:12 -08:00
pennae ecf8b12d60 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
2024-02-12 13:58:29 +01:00
pennae 6c08fba533 use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
2024-02-12 13:35:00 +01:00
pennae 1f542adb3e add ExprAttrs::AttrDef::chooseByKind
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.
2024-02-12 13:34:59 +01:00
pennae c66ee57edc preserve information about whether/how an attribute was inherited 2024-02-12 13:32:33 +01:00
pennae 73065a400d add test for inherit expr printing 2024-02-12 13:32:33 +01:00
pennae 8669c02468 add test for inherit-from semantics 2024-02-12 13:32:33 +01:00
Rebecca Turner 4440eb54e7 Add release note 2024-02-08 12:07:38 -08:00
Rebecca Turner 8b7eb7400b Enter debugger on builtins.trace with an option 2024-02-08 12:07:37 -08:00
Rebecca Turner 0f1269243b Revert "Add release notes for "Stack traces are more compact""
This reverts commit b2868acbdc.
2024-02-08 10:09:47 -08:00
Rebecca Turner b2868acbdc Add release notes for "Stack traces are more compact" 2024-02-08 10:08:48 -08:00
Rebecca Turner 7f8960d0f2 Add release notes for "Stack size is increased on macOS" 2024-02-08 10:08:48 -08:00
Rebecca Turner 837c350bcd Add release notes for "Cycle detection in nix repl is simpler and more reliable" 2024-02-08 10:08:48 -08:00
Rebecca Turner 4f0d43a397 Add release notes for "nix repl now respects Ctrl-C while printing values" 2024-02-08 10:08:48 -08:00
Rebecca Turner 24cdb81bb0 Add release notes for "Nix commands respect Ctrl-C" 2024-02-08 10:08:48 -08:00
Rebecca Turner abb5fef355 Add release notes for "Functions are printed with more detail" 2024-02-08 10:08:48 -08:00
Rebecca Turner 9a5d52262f Add release notes for "Nix no longer attempts to git add files that are .gitignored" 2024-02-08 10:08:48 -08:00
Rebecca Turner 754c35abfb Add release notes for "Debugger prints source position information" 2024-02-08 10:08:48 -08:00
Rebecca Turner 24205a8703 Add release note 2024-02-05 13:00:39 -08:00
Rebecca Turner 016db2d10f Add position information to while evaluating the attribute 2024-02-02 17:49:54 -08:00
149 changed files with 2326 additions and 826 deletions

No files matched your search

+5 -3
View File
@@ -64,7 +64,7 @@ jobs:
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v25
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v14
with:
name: '${{ env.CACHIX_NAME }}'
@@ -116,7 +116,7 @@ jobs:
fetch-depth: 0
- uses: cachix/install-nix-action@v25
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v14
@@ -153,6 +153,8 @@ jobs:
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag nix:$NIX_VERSION $IMAGE_ID:$NIX_VERSION
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker tag nix:$NIX_VERSION $IMAGE_ID:latest
docker push $IMAGE_ID:$NIX_VERSION
docker push $IMAGE_ID:latest
# deprecated 2024-02-24
docker push $IMAGE_ID:master
+3
View File
@@ -45,13 +45,16 @@ perl/Makefile.config
/src/libexpr/parser-tab.hh
/src/libexpr/parser-tab.output
/src/libexpr/nix.tbl
/src/libexpr/tests
/tests/unit/libexpr/libnixexpr-tests
# /src/libstore/
*.gen.*
/src/libstore/tests
/tests/unit/libstore/libnixstore-tests
# /src/libutil/
/src/libutil/tests
/tests/unit/libutil/libnixutil-tests
/src/nix/nix
+3 -1
View File
@@ -42,6 +42,7 @@ ifeq ($(ENABLE_FUNCTIONAL_TESTS), yes)
makefiles += \
tests/functional/local.mk \
tests/functional/ca/local.mk \
tests/functional/git-hashing/local.mk \
tests/functional/dyn-drv/local.mk \
tests/functional/test-libstoreconsumer/local.mk \
tests/functional/plugins/local.mk
@@ -67,6 +68,7 @@ ifeq ($(OPTIMIZE), 1)
GLOBAL_LDFLAGS += $(CXXLTO)
else
GLOBAL_CXXFLAGS += -O0 -U_FORTIFY_SOURCE
unexport NIX_HARDENING_ENABLE
endif
include mk/platform.mk
@@ -81,7 +83,7 @@ ifdef HOST_WINDOWS
GLOBAL_LDFLAGS += -Wl,--export-all-symbols
endif
GLOBAL_CXXFLAGS += -g -Wall -include $(buildprefix)config.h -std=c++2a -I src
GLOBAL_CXXFLAGS += -g -Wall -Wimplicit-fallthrough -include $(buildprefix)config.h -std=c++2a -I src
# Include the main lib, causing rules to be defined
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: Enter the `--debugger` when `builtins.trace` is called if `debugger-on-trace` is set
prs: 9914
---
If the `debugger-on-trace` option is set and `--debugger` is given,
`builtins.trace` calls will behave similarly to `builtins.break` and will enter
the debug REPL. This is useful for determining where warnings are being emitted
from.
+25
View File
@@ -0,0 +1,25 @@
---
synopsis: Debugger prints source position information
prs: 9913
---
The `--debugger` now prints source location information, instead of the
pointers of source location information. Before:
```
nix-repl> :bt
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
0x600001522598
```
After:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
/nix/store/hg65h51xnp74ikahns9hyf3py5mlbbqq-source/overrides/default.nix:132:27
131|
132| bootstrappingBase = pkgs.${self.python.pythonAttr}.pythonForBuild.pkgs;
| ^
133| in
```
@@ -0,0 +1,7 @@
---
synopsis: "`inherit (x) ...` evaluates `x` only once"
prs: 9847
---
`inherit (x) a b ...` now evaluates the expression `x` only once for all inherited attributes rather than once for each inherited attribute.
This does not usually have a measurable impact, but side-effects (such as `builtins.trace`) would be duplicated and expensive expressions (such as derivations) could cause a measurable slowdown.
+50
View File
@@ -0,0 +1,50 @@
---
synopsis: Functions are printed with more detail
prs: 9606
issues: 7145
---
Functions and `builtins` are printed with more detail in `nix repl`, `nix
eval`, `builtins.trace`, and most other places values are printed.
Before:
```
$ nix repl nixpkgs
nix-repl> builtins.map
«primop»
nix-repl> builtins.map lib.id
«primop-app»
nix-repl> builtins.trace lib.id "my-value"
trace: <LAMBDA>
"my-value"
$ nix eval --file functions.nix
{ id = <LAMBDA>; primop = <PRIMOP>; primop-app = <PRIMOP-APP>; }
```
After:
```
$ nix repl nixpkgs
nix-repl> builtins.map
«primop map»
nix-repl> builtins.map lib.id
«partially applied primop map»
nix-repl> builtins.trace lib.id "my-value"
trace: «lambda id @ /nix/store/8rrzq23h2zq7sv5l2vhw44kls5w0f654-source/lib/trivial.nix:26:5»
"my-value"
$ nix eval --file functions.nix
{ id = «lambda id @ /Users/wiggles/nix/functions.nix:2:8»; primop = «primop map»; primop-app = «partially applied primop map»; }
```
This was actually released in Nix 2.20, but wasn't added to the release notes
so we're announcing it here. The historical release notes have been updated as well.
[type-error]: https://github.com/NixOS/nix/pull/9753
[coercion-error]: https://github.com/NixOS/nix/pull/9754
@@ -0,0 +1,13 @@
---
synopsis: Nix commands respect Ctrl-C
prs: 9687 6995
issues: 7245
---
Previously, many Nix commands would hang indefinitely if Ctrl-C was pressed
while performing various operations (including `nix develop`, `nix flake
update`, and so on). With several fixes to Nix's signal handlers, Nix commands
will now exit quickly after Ctrl-C is pressed.
This was actually released in Nix 2.20, but wasn't added to the release notes
so we're announcing it here. The historical release notes have been updated as well.
@@ -0,0 +1,24 @@
---
synopsis: "`nix repl` pretty-prints values"
prs: 9931
---
`nix repl` will now pretty-print values:
```
{
attrs = {
a = {
b = {
c = { };
};
};
};
list = [ 1 ];
list' = [
1
2
3
];
}
```
@@ -0,0 +1,8 @@
---
synopsis: "`nix repl` now respects Ctrl-C while printing values"
prs: 9927
---
`nix repl` will now halt immediately when Ctrl-C is pressed while it's printing
a value. This is useful if you got curious about what would happen if you
printed all of Nixpkgs.
@@ -0,0 +1,22 @@
---
synopsis: Cycle detection in `nix repl` is simpler and more reliable
prs: 9926
issues: 8672
---
The cycle detection in `nix repl`, `nix eval`, `builtins.trace`, and everywhere
else values are printed is now simpler and matches the cycle detection in
`nix-instantiate --eval` output.
Before:
```
nix eval --expr 'let self = { inherit self; }; in self'
{ self = { self = «repeated»; }; }
```
After:
```
{ self = «repeated»; }
```
@@ -0,0 +1,23 @@
---
synopsis: "In the debugger, `while evaluating the attribute` errors now include position information"
prs: 9915
---
Before:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
0x600001522598
```
After:
```
0: while evaluating the attribute 'python311.pythonForBuild.pkgs'
/nix/store/hg65h51xnp74ikahns9hyf3py5mlbbqq-source/overrides/default.nix:132:27
131|
132| bootstrappingBase = pkgs.${self.python.pythonAttr}.pythonForBuild.pkgs;
| ^
133| in
```
@@ -1,9 +0,0 @@
---
synopsis: Stabilize fetchTree
prs: 10068
---
The core of `fetchTree` is now stable.
This includes
- the `fetchTree` function itself
- all the existing fetchers, except `git` (still unstable because of some reproducibility concerns)
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: Stack size is increased on macOS
prs: 9860
---
Previously, Nix would set the stack size to 64MiB on Linux, but would leave the
stack size set to the default (approximately 8KiB) on macOS. Now, the stack
size is correctly set to 64MiB on macOS as well, which should reduce stack
overflow segfaults in deeply-recursive Nix expressions.
+7 -2
View File
@@ -89,15 +89,20 @@ where
- `rec` = one of:
- ```ebnf
| ""
```
(empty string) for hashes of the flat (single file) serialization
- ```ebnf
| "r:"
```
hashes of the for [Nix Archive (NAR)] (arbitrary file system object) serialization
- ```ebnf
| ""
| "git:"
```
(empty string) for hashes of the flat (single file) serialization
hashes of the [Git blob/tree](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) [Merkel tree](https://en.wikipedia.org/wiki/Merkle_tree) format
- ```ebnf
algo = "md5" | "sha1" | "sha256"
+33
View File
@@ -167,3 +167,36 @@
error: expected a set but found an integer
```
- Functions are printed with more detail [#7145](https://github.com/NixOS/nix/issues/7145) [#9606](https://github.com/NixOS/nix/pull/9606)
`nix repl`, `nix eval`, `builtins.trace`, and most other places values are
printed will now include function names and source location information:
```
$ nix repl nixpkgs
nix-repl> builtins.map
«primop map»
nix-repl> builtins.map lib.id
«partially applied primop map»
nix-repl> builtins.trace lib.id "my-value"
trace: «lambda id @ /nix/store/8rrzq23h2zq7sv5l2vhw44kls5w0f654-source/lib/trivial.nix:26:5»
"my-value"
```
- Flake operations like `nix develop` will no longer fail when run in a Git
repository where the `flake.lock` file is `.gitignore`d
[#8854](https://github.com/NixOS/nix/issues/8854)
[#9324](https://github.com/NixOS/nix/pull/9324)
- Nix commands will now respect Ctrl-C
[#7145](https://github.com/NixOS/nix/issues/7145)
[#6995](https://github.com/NixOS/nix/pull/6995)
[#9687](https://github.com/NixOS/nix/pull/9687)
Previously, many Nix commands would hang indefinitely if Ctrl-C was pressed
while performing various operations (including `nix develop`, `nix flake
update`, and so on). With several fixes to Nix's signal handlers, Nix
commands will now exit quickly after Ctrl-C is pressed.
Generated
+4 -4
View File
@@ -34,16 +34,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1705033721,
"narHash": "sha256-K5eJHmL1/kev6WuqyqqbS1cdNnSidIZ3jeqJ7GbrYnQ=",
"lastModified": 1709083642,
"narHash": "sha256-7kkJQd4rZ+vFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "a1982c92d8980a0114372973cbdfe0a307f1bdea",
"rev": "b550fe4b4776908ac2a861124307045f8e717c8e",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-23.05-small",
"ref": "release-23.11",
"repo": "nixpkgs",
"type": "github"
}
+11 -15
View File
@@ -1,7 +1,9 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05-small";
# TODO switch to nixos-23.11-small
# https://nixpk.gs/pr-tracker.html?pr=291954
inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-23.11";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2"; flake = false; };
@@ -10,20 +12,10 @@
let
inherit (nixpkgs) lib;
# Experimental fileset library: https://github.com/NixOS/nixpkgs/pull/222981
# Not an "idiomatic" flake input because:
# - Propagation to dependent locks: https://github.com/NixOS/nix/issues/7730
# - Subflake would download redundant and huge parent flake
# - No git tree hash support: https://github.com/NixOS/nix/issues/6044
inherit (import (builtins.fetchTarball { url = "https://github.com/NixOS/nix/archive/1bdcd7fc8a6a40b2e805bad759b36e64e911036b.tar.gz"; sha256 = "sha256:14ljlpdsp4x7h1fkhbmc4bd3vsqnx8zdql4h3037wh09ad6a0893"; }))
fileset;
inherit (lib) fileset;
officialRelease = false;
# Set to true to build the release notes for the next release.
buildUnreleasedNotes = false;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
if officialRelease
@@ -404,8 +396,11 @@
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
nativeBuildInputs = attrs.nativeBuildInputs or []
++ lib.optional stdenv.cc.isClang pkgs.buildPackages.bear
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) pkgs.buildPackages.clang-tools;
});
in
@@ -417,8 +412,9 @@
(forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName}));
in
(makeShells "native" nixpkgsFor.${system}.native) //
(makeShells "static" nixpkgsFor.${system}.static) //
(lib.genAttrs shellCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv)) //
(lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin)
(makeShells "static" nixpkgsFor.${system}.static)) //
(lib.genAttrs shellCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv)) //
{
default = self.devShells.${system}.native-stdenvPackages;
}
+5 -7
View File
@@ -154,7 +154,7 @@ in {
in
fileset.toSource {
root = ./.;
fileset = fileset.intersect baseFiles (fileset.unions ([
fileset = fileset.intersection baseFiles (fileset.unions ([
# For configure
./.version
./configure.ac
@@ -209,6 +209,10 @@ in {
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
] ++ lib.optionals doInstallCheck [
git
mercurial
openssh
] ++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
] ++ lib.optional stdenv.hostPlatform.isLinux util-linux
@@ -249,12 +253,6 @@ in {
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
nativeCheckInputs = [
git
mercurial
openssh
];
disallowedReferences = [ boost ];
preConfigure = lib.optionalString (doBuild && ! stdenv.hostPlatform.isStatic) (
+1 -1
View File
@@ -259,7 +259,7 @@ hashPath(char * algo, int base32, char * path)
auto [accessor, canonPath] = PosixSourceAccessor::createAtRoot(path);
Hash h = hashPath(
accessor, canonPath,
FileIngestionMethod::Recursive, parseHashAlgo(algo)).first;
FileIngestionMethod::Recursive, parseHashAlgo(algo));
auto s = h.to_string(base32 ? HashFormat::Nix32 : HashFormat::Base16, false);
XPUSHs(sv_2mortal(newSVpv(s.c_str(), 0)));
} catch (Error & e) {
+26 -1
View File
@@ -58,6 +58,31 @@ readonly EXTRACTED_NIX_PATH="$(dirname "$0")"
readonly ROOT_HOME=~root
readonly PROXY_ENVIRONMENT_VARIABLES=(
http_proxy
https_proxy
ftp_proxy
no_proxy
HTTP_PROXY
HTTPS_PROXY
FTP_PROXY
NO_PROXY
)
SUDO_EXTRA_ENVIRONMENT_VARIABLES=()
setup_sudo_extra_environment_variables() {
local i=${#SUDO_EXTRA_ENVIRONMENT_VARIABLES[@]}
for variable in "${PROXY_ENVIRONMENT_VARIABLES[@]}"; do
if [ "x${!variable:-}" != "x" ]; then
SUDO_EXTRA_ENVIRONMENT_VARIABLES[i]="$variable=${!variable}"
i=$((i + 1))
fi
done
}
setup_sudo_extra_environment_variables
if [ -t 0 ] && [ -z "${NIX_INSTALLER_YES:-}" ]; then
readonly IS_HEADLESS='no'
else
@@ -361,7 +386,7 @@ _sudo() {
if is_root; then
env "$@"
else
sudo "$@"
sudo "${SUDO_EXTRA_ENVIRONMENT_VARIABLES[@]}" "$@"
fi
}
+4 -2
View File
@@ -9,6 +9,7 @@
#include "store-api.hh"
#include "command.hh"
#include "tarball.hh"
#include "fetch-to-store.hh"
namespace nix {
@@ -167,8 +168,9 @@ Bindings * MixEvalArgs::getAutoArgs(EvalState & state)
SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * baseDir)
{
if (EvalSettings::isPseudoUrl(s)) {
auto storePath = fetchers::downloadTarball(
state.store, EvalSettings::resolvePseudoUrl(s), "source", false).storePath;
auto accessor = fetchers::downloadTarball(
EvalSettings::resolvePseudoUrl(s)).accessor;
auto storePath = fetchToStore(*state.store, SourcePath(accessor), FetchMode::Copy);
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
+1 -1
View File
@@ -45,7 +45,7 @@ ref<InstallableValue> InstallableValue::require(ref<Installable> installable)
std::optional<DerivedPathWithInfo> InstallableValue::trySinglePathToDerivedPaths(Value & v, const PosIdx pos, std::string_view errorCtx)
{
if (v.type() == nPath) {
auto storePath = fetchToStore(*state->store, v.path());
auto storePath = fetchToStore(*state->store, v.path(), FetchMode::Copy);
return {{
.path = DerivedPath::Opaque {
.path = std::move(storePath),
+121
View File
@@ -0,0 +1,121 @@
#include "misc-store-flags.hh"
namespace nix::flag
{
static void hashFormatCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & format : hashFormats) {
if (hasPrefix(format, prefix)) {
completions.add(format);
}
}
}
Args::Flag hashFormatWithDefault(std::string && longName, HashFormat * hf)
{
assert(*hf == nix::HashFormat::SRI);
return Args::Flag {
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`). Default: `sri`.",
.labels = {"hash-format"},
.handler = {[hf](std::string s) {
*hf = parseHashFormat(s);
}},
.completer = hashFormatCompleter,
};
}
Args::Flag hashFormatOpt(std::string && longName, std::optional<HashFormat> * ohf)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`).",
.labels = {"hash-format"},
.handler = {[ohf](std::string s) {
*ohf = std::optional<HashFormat>{parseHashFormat(s)};
}},
.completer = hashFormatCompleter,
};
}
static void hashAlgoCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & algo : hashAlgorithms)
if (hasPrefix(algo, prefix))
completions.add(algo);
}
Args::Flag hashAlgo(std::string && longName, HashAlgorithm * ha)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`).",
.labels = {"hash-algo"},
.handler = {[ha](std::string s) {
*ha = parseHashAlgo(s);
}},
.completer = hashAlgoCompleter,
};
}
Args::Flag hashAlgoOpt(std::string && longName, std::optional<HashAlgorithm> * oha)
{
return Args::Flag {
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`). Can be omitted for SRI hashes.",
.labels = {"hash-algo"},
.handler = {[oha](std::string s) {
*oha = std::optional<HashAlgorithm>{parseHashAlgo(s)};
}},
.completer = hashAlgoCompleter,
};
}
Args::Flag fileIngestionMethod(FileIngestionMethod * method)
{
return Args::Flag {
.longName = "mode",
// FIXME indentation carefully made for context, this is messed up.
.description = R"(
How to compute the hash of the input.
One of:
- `nar` (the default): Serialises the input as an archive (following the [_Nix Archive Format_](https://edolstra.github.io/pubs/phd-thesis.pdf#page=101)) and passes that to the hash function.
- `flat`: Assumes that the input is a single file and directly passes it to the hash function;
)",
.labels = {"file-ingestion-method"},
.handler = {[method](std::string s) {
*method = parseFileIngestionMethod(s);
}},
};
}
Args::Flag contentAddressMethod(ContentAddressMethod * method)
{
return Args::Flag {
.longName = "mode",
// FIXME indentation carefully made for context, this is messed up.
.description = R"(
How to compute the content-address of the store object.
One of:
- `nar` (the default): Serialises the input as an archive (following the [_Nix Archive Format_](https://edolstra.github.io/pubs/phd-thesis.pdf#page=101)) and passes that to the hash function.
- `flat`: Assumes that the input is a single file and directly passes it to the hash function;
- `text`: Like `flat`, but used for
[derivations](@docroot@/glossary.md#store-derivation) serialized in store object and
[`builtins.toFile`](@docroot@/language/builtins.html#builtins-toFile).
For advanced use-cases only;
for regular usage prefer `nar` and `flat.
)",
.labels = {"content-address-method"},
.handler = {[method](std::string s) {
*method = ContentAddressMethod::parse(s);
}},
};
}
}
+21
View File
@@ -0,0 +1,21 @@
#include "args.hh"
#include "content-address.hh"
namespace nix::flag {
Args::Flag hashAlgo(std::string && longName, HashAlgorithm * ha);
static inline Args::Flag hashAlgo(HashAlgorithm * ha)
{
return hashAlgo("hash-algo", ha);
}
Args::Flag hashAlgoOpt(std::string && longName, std::optional<HashAlgorithm> * oha);
Args::Flag hashFormatWithDefault(std::string && longName, HashFormat * hf);
Args::Flag hashFormatOpt(std::string && longName, std::optional<HashFormat> * ohf);
static inline Args::Flag hashAlgoOpt(std::optional<HashAlgorithm> * oha)
{
return hashAlgoOpt("hash-algo", oha);
}
Args::Flag fileIngestionMethod(FileIngestionMethod * method);
Args::Flag contentAddressMethod(ContentAddressMethod * method);
}
+47 -17
View File
@@ -52,6 +52,27 @@ extern "C" {
namespace nix {
/**
* Returned by `NixRepl::processLine`.
*/
enum class ProcessLineResult {
/**
* The user exited with `:quit`. The REPL should exit. The surrounding
* program or evaluation (e.g., if the REPL was acting as the debugger)
* should also exit.
*/
Quit,
/**
* The user exited with `:continue`. The REPL should exit, but the program
* should continue running.
*/
Continue,
/**
* The user did not exit. The REPL should request another line of input.
*/
PromptAgain,
};
struct NixRepl
: AbstractNixRepl
#if HAVE_BOEHMGC
@@ -75,13 +96,13 @@ struct NixRepl
std::function<AnnotatedValues()> getValues);
virtual ~NixRepl();
void mainLoop() override;
ReplExitStatus mainLoop() override;
void initEnv() override;
StringSet completePrefix(const std::string & prefix);
bool getLine(std::string & input, const std::string & prompt);
StorePath getDerivationPath(Value & v);
bool processLine(std::string line);
ProcessLineResult processLine(std::string line);
void loadFile(const Path & path);
void loadFlake(const std::string & flakeRef);
@@ -246,7 +267,7 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
static bool isFirstRepl = true;
void NixRepl::mainLoop()
ReplExitStatus NixRepl::mainLoop()
{
if (isFirstRepl) {
std::string_view debuggerNotice = "";
@@ -287,15 +308,25 @@ void NixRepl::mainLoop()
// When continuing input from previous lines, don't print a prompt, just align to the same
// number of chars as the prompt.
if (!getLine(input, input.empty() ? "nix-repl> " : " ")) {
// ctrl-D should exit the debugger.
// Ctrl-D should exit the debugger.
state->debugStop = false;
state->debugQuit = true;
logger->cout("");
break;
// TODO: Should Ctrl-D exit just the current debugger session or
// the entire program?
return ReplExitStatus::QuitAll;
}
logger->resume();
try {
if (!removeWhitespace(input).empty() && !processLine(input)) return;
switch (processLine(input)) {
case ProcessLineResult::Quit:
return ReplExitStatus::QuitAll;
case ProcessLineResult::Continue:
return ReplExitStatus::Continue;
case ProcessLineResult::PromptAgain:
break;
default:
abort();
}
} catch (ParseError & e) {
if (e.msg().find("unexpected end of file") != std::string::npos) {
// For parse errors on incomplete input, we continue waiting for the next line of
@@ -352,7 +383,6 @@ bool NixRepl::getLine(std::string & input, const std::string & prompt)
};
setupSignals();
Finally resetTerminal([&]() { rl_deprep_terminal(); });
char * s = readline(prompt.c_str());
Finally doFree([&]() { free(s); });
restoreSignals();
@@ -483,10 +513,11 @@ void NixRepl::loadDebugTraceEnv(DebugTrace & dt)
}
}
bool NixRepl::processLine(std::string line)
ProcessLineResult NixRepl::processLine(std::string line)
{
line = trim(line);
if (line == "") return true;
if (line.empty())
return ProcessLineResult::PromptAgain;
_isInterrupted = false;
@@ -581,13 +612,13 @@ bool NixRepl::processLine(std::string line)
else if (state->debugRepl && (command == ":s" || command == ":step")) {
// set flag to stop at next DebugTrace; exit repl.
state->debugStop = true;
return false;
return ProcessLineResult::Continue;
}
else if (state->debugRepl && (command == ":c" || command == ":continue")) {
// set flag to run to next breakpoint or end of program; exit repl.
state->debugStop = false;
return false;
return ProcessLineResult::Continue;
}
else if (command == ":a" || command == ":add") {
@@ -730,8 +761,7 @@ bool NixRepl::processLine(std::string line)
else if (command == ":q" || command == ":quit") {
state->debugStop = false;
state->debugQuit = true;
return false;
return ProcessLineResult::Quit;
}
else if (command == ":doc") {
@@ -792,7 +822,7 @@ bool NixRepl::processLine(std::string line)
}
}
return true;
return ProcessLineResult::PromptAgain;
}
void NixRepl::loadFile(const Path & path)
@@ -923,7 +953,7 @@ std::unique_ptr<AbstractNixRepl> AbstractNixRepl::create(
}
void AbstractNixRepl::runSimple(
ReplExitStatus AbstractNixRepl::runSimple(
ref<EvalState> evalState,
const ValMap & extraEnv)
{
@@ -945,7 +975,7 @@ void AbstractNixRepl::runSimple(
for (auto & [name, value] : extraEnv)
repl->addVarToScope(repl->state->symbols.create(name), *value);
repl->mainLoop();
return repl->mainLoop();
}
}
+2 -2
View File
@@ -28,13 +28,13 @@ struct AbstractNixRepl
const SearchPath & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
static void runSimple(
static ReplExitStatus runSimple(
ref<EvalState> evalState,
const ValMap & extraEnv);
virtual void initEnv() = 0;
virtual void mainLoop() = 0;
virtual ReplExitStatus mainLoop() = 0;
};
}
+3 -11
View File
@@ -28,15 +28,7 @@ template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withTrace(PosIdx pos, const std::string_view text)
{
error.err.traces.push_front(
Trace{.pos = error.state.positions[pos], .hint = HintFmt(std::string(text)), .frame = false});
return *this;
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrameTrace(PosIdx pos, const std::string_view text)
{
error.err.traces.push_front(
Trace{.pos = error.state.positions[pos], .hint = HintFmt(std::string(text)), .frame = true});
Trace{.pos = error.state.positions[pos], .hint = HintFmt(std::string(text))});
return *this;
}
@@ -63,9 +55,9 @@ EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr
}
template<class T>
EvalErrorBuilder<T> & EvalErrorBuilder<T>::addTrace(PosIdx pos, HintFmt hint, bool frame)
EvalErrorBuilder<T> & EvalErrorBuilder<T>::addTrace(PosIdx pos, HintFmt hint)
{
error.addTrace(error.state.positions[pos], hint, frame);
error.addTrace(error.state.positions[pos], hint);
return *this;
}
+1 -1
View File
@@ -89,7 +89,7 @@ public:
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & withFrame(const Env & e, const Expr & ex);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & addTrace(PosIdx pos, HintFmt hint, bool frame = false);
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> & addTrace(PosIdx pos, HintFmt hint);
template<typename... Args>
[[nodiscard, gnu::noinline]] EvalErrorBuilder<T> &
+10
View File
@@ -127,6 +127,16 @@ struct EvalSettings : Config
Setting<unsigned int> maxCallDepth{this, 10000, "max-call-depth",
"The maximum function call depth to allow before erroring."};
Setting<bool> builtinsTraceDebugger{this, false, "debugger-on-trace",
R"(
If set to true and the `--debugger` flag is given,
[`builtins.trace`](@docroot@/language/builtins.md#builtins-trace) will
enter the debugger like
[`builtins.break`](@docroot@/language/builtins.md#builtins-break).
This is useful for debugging warnings in third-party Nix code.
)"};
};
extern EvalSettings evalSettings;
+67 -24
View File
@@ -3,6 +3,7 @@
#include "hash.hh"
#include "primops.hh"
#include "print-options.hh"
#include "shared.hh"
#include "types.hh"
#include "util.hh"
#include "store-api.hh"
@@ -416,7 +417,6 @@ EvalState::EvalState(
, buildStore(buildStore ? buildStore : store)
, debugRepl(nullptr)
, debugStop(false)
, debugQuit(false)
, trylevel(0)
, regexCache(makeRegexCache())
#if HAVE_BOEHMGC
@@ -467,13 +467,13 @@ EvalState::~EvalState()
void EvalState::allowPath(const Path & path)
{
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListInputAccessor>())
rootFS2->allowPath(CanonPath(path));
rootFS2->allowPrefix(CanonPath(path));
}
void EvalState::allowPath(const StorePath & storePath)
{
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListInputAccessor>())
rootFS2->allowPath(CanonPath(store->toRealPath(storePath)));
rootFS2->allowPrefix(CanonPath(store->toRealPath(storePath)));
}
void EvalState::allowAndSetStorePathString(const StorePath & storePath, Value & v)
@@ -792,18 +792,30 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
auto se = getStaticEnv(expr);
if (se) {
auto vm = mapStaticEnvBindings(symbols, *se.get(), env);
(debugRepl)(ref<EvalState>(shared_from_this()), *vm);
auto exitStatus = (debugRepl)(ref<EvalState>(shared_from_this()), *vm);
switch (exitStatus) {
case ReplExitStatus::QuitAll:
if (error)
throw *error;
throw Exit(0);
case ReplExitStatus::Continue:
break;
default:
abort();
}
}
}
void EvalState::addErrorTrace(Error & e, const char * s, const std::string & s2) const
template<typename... Args>
void EvalState::addErrorTrace(Error & e, const Args & ... formatArgs) const
{
e.addTrace(nullptr, s, s2);
e.addTrace(nullptr, HintFmt(formatArgs...));
}
void EvalState::addErrorTrace(Error & e, const PosIdx pos, const char * s, const std::string & s2, bool frame) const
template<typename... Args>
void EvalState::addErrorTrace(Error & e, const PosIdx pos, const Args & ... formatArgs) const
{
e.addTrace(positions[pos], HintFmt(s, s2), frame);
e.addTrace(positions[pos], HintFmt(formatArgs...));
}
template<typename... Args>
@@ -1186,6 +1198,18 @@ void ExprPath::eval(EvalState & state, Env & env, Value & v)
}
Env * ExprAttrs::buildInheritFromEnv(EvalState & state, Env & up)
{
Env & inheritEnv = state.allocEnv(inheritFromExprs->size());
inheritEnv.up = &up;
Displacement displ = 0;
for (auto from : *inheritFromExprs)
inheritEnv.values[displ++] = from->maybeThunk(state, up);
return &inheritEnv;
}
void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
{
v.mkAttrs(state.buildBindings(attrs.size() + dynamicAttrs.size()).finish());
@@ -1197,6 +1221,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Env & env2(state.allocEnv(attrs.size()));
env2.up = &env;
dynamicEnv = &env2;
Env * inheritEnv = inheritFromExprs ? buildInheritFromEnv(state, env2) : nullptr;
AttrDefs::iterator overrides = attrs.find(state.sOverrides);
bool hasOverrides = overrides != attrs.end();
@@ -1207,11 +1232,11 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
Displacement displ = 0;
for (auto & i : attrs) {
Value * vAttr;
if (hasOverrides && !i.second.inherited) {
if (hasOverrides && i.second.kind != AttrDef::Kind::Inherited) {
vAttr = state.allocValue();
mkThunk(*vAttr, env2, i.second.e);
mkThunk(*vAttr, *i.second.chooseByKind(&env2, &env, inheritEnv), i.second.e);
} else
vAttr = i.second.e->maybeThunk(state, i.second.inherited ? env : env2);
vAttr = i.second.e->maybeThunk(state, *i.second.chooseByKind(&env2, &env, inheritEnv));
env2.values[displ++] = vAttr;
v.attrs->push_back(Attr(i.first, vAttr, i.second.pos));
}
@@ -1243,9 +1268,15 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
}
}
else
for (auto & i : attrs)
v.attrs->push_back(Attr(i.first, i.second.e->maybeThunk(state, env), i.second.pos));
else {
Env * inheritEnv = inheritFromExprs ? buildInheritFromEnv(state, env) : nullptr;
for (auto & i : attrs) {
v.attrs->push_back(Attr(
i.first,
i.second.e->maybeThunk(state, *i.second.chooseByKind(&env, &env, inheritEnv)),
i.second.pos));
}
}
/* Dynamic attrs apply *after* rec and __overrides. */
for (auto & i : dynamicAttrs) {
@@ -1277,12 +1308,17 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
Env & env2(state.allocEnv(attrs->attrs.size()));
env2.up = &env;
Env * inheritEnv = attrs->inheritFromExprs ? attrs->buildInheritFromEnv(state, env2) : nullptr;
/* The recursive attributes are evaluated in the new environment,
while the inherited attributes are evaluated in the original
environment. */
Displacement displ = 0;
for (auto & i : attrs->attrs)
env2.values[displ++] = i.second.e->maybeThunk(state, i.second.inherited ? env : env2);
for (auto & i : attrs->attrs) {
env2.values[displ++] = i.second.e->maybeThunk(
state,
*i.second.chooseByKind(&env2, &env, inheritEnv));
}
auto dts = state.debugRepl
? makeDebugTraceStacker(
@@ -1359,7 +1395,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
state.positions[pos2],
state.positions[getPos()],
"while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath))
: nullptr;
@@ -1577,9 +1613,8 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
"while calling %s",
lambda.name
? concatStrings("'", symbols[lambda.name], "'")
: "anonymous lambda",
true);
if (pos) addErrorTrace(e, pos, "from call site%s", "", true);
: "anonymous lambda");
if (pos) addErrorTrace(e, pos, "from call site");
}
throw;
}
@@ -2339,7 +2374,14 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
auto dstPath = i != srcToStore.end()
? i->second
: [&]() {
auto dstPath = fetchToStore(*store, path.resolveSymlinks(), path.baseName(), FileIngestionMethod::Recursive, nullptr, repair);
auto dstPath = fetchToStore(
*store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(),
FileIngestionMethod::Recursive,
nullptr,
repair);
allowPath(dstPath);
srcToStore.insert_or_assign(path, dstPath);
printMsg(lvlChatty, "copied source '%1%' -> '%2%'", path, store->printStorePath(dstPath));
@@ -2787,10 +2829,11 @@ std::optional<std::string> EvalState::resolveSearchPathPath(const SearchPath::Pa
if (EvalSettings::isPseudoUrl(value)) {
try {
auto storePath = fetchers::downloadTarball(
store, EvalSettings::resolvePseudoUrl(value), "source", false).storePath;
auto accessor = fetchers::downloadTarball(
EvalSettings::resolvePseudoUrl(value)).accessor;
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy);
res = { store->toRealPath(storePath) };
} catch (FileTransferError & e) {
} catch (Error & e) {
logWarning({
.msg = HintFmt("Nix search path entry '%1%' cannot be downloaded, ignoring", value)
});
+6 -5
View File
@@ -11,6 +11,7 @@
#include "experimental-features.hh"
#include "input-accessor.hh"
#include "search-path.hh"
#include "repl-exit-status.hh"
#include <map>
#include <optional>
@@ -219,9 +220,8 @@ public:
/**
* Debugger
*/
void (* debugRepl)(ref<EvalState> es, const ValMap & extraEnv);
ReplExitStatus (* debugRepl)(ref<EvalState> es, const ValMap & extraEnv);
bool debugStop;
bool debugQuit;
int trylevel;
std::list<DebugTrace> debugTraces;
std::map<const Expr*, const std::shared_ptr<const StaticEnv>> exprEnvs;
@@ -432,10 +432,12 @@ public:
std::string_view forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx);
std::string_view forceStringNoCtx(Value & v, const PosIdx pos, std::string_view errorCtx);
template<typename... Args>
[[gnu::noinline]]
void addErrorTrace(Error & e, const char * s, const std::string & s2) const;
void addErrorTrace(Error & e, const Args & ... formatArgs) const;
template<typename... Args>
[[gnu::noinline]]
void addErrorTrace(Error & e, const PosIdx pos, const char * s, const std::string & s2, bool frame = false) const;
void addErrorTrace(Error & e, const PosIdx pos, const Args & ... formatArgs) const;
public:
/**
@@ -758,7 +760,6 @@ struct DebugTraceStacker {
DebugTraceStacker(EvalState & evalState, DebugTrace t);
~DebugTraceStacker()
{
// assert(evalState.debugTraces.front() == trace);
evalState.debugTraces.pop_front();
}
EvalState & evalState;
+37 -24
View File
@@ -1,20 +1,52 @@
lockFileStr: rootSrc: rootSubdir:
# This is a helper to callFlake() to lazily fetch flake inputs.
# The contents of the lock file, in JSON format.
lockFileStr:
# A mapping of lock file node IDs to { sourceInfo, subdir } attrsets,
# with sourceInfo.outPath providing an InputAccessor to a previously
# fetched tree. This is necessary for possibly unlocked inputs, in
# particular the root input, but also --override-inputs pointing to
# unlocked trees.
overrides:
let
lockFile = builtins.fromJSON lockFileStr;
# Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root
# node.
resolveInput = inputSpec:
if builtins.isList inputSpec
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node.
getInputByPath = nodeName: path:
if path == []
then nodeName
else
getInputByPath
# Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path);
allNodes =
builtins.mapAttrs
(key: node:
let
sourceInfo =
if key == lockFile.root
then rootSrc
else fetchTree (node.info or {} // removeAttrs node.locked ["dir"]);
if overrides ? ${key}
then
overrides.${key}.sourceInfo
else
# FIXME: remove obsolete node.info.
fetchTree (node.info or {} // removeAttrs node.locked ["dir"]);
subdir = if key == lockFile.root then rootSubdir else node.locked.dir or "";
subdir = overrides.${key}.dir or node.locked.dir or "";
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
@@ -24,25 +56,6 @@ let
(inputName: inputSpec: allNodes.${resolveInput inputSpec})
(node.inputs or {});
# Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root
# node.
resolveInput = inputSpec:
if builtins.isList inputSpec
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node.
getInputByPath = nodeName: path:
if path == []
then nodeName
else
getInputByPath
# Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path);
outputs = flake.outputs (inputs // { self = result; });
result =
+53 -31
View File
@@ -365,6 +365,7 @@ LockedFlake lockFlake(
std::map<InputPath, FlakeInput> overrides;
std::set<InputPath> explicitCliOverrides;
std::set<InputPath> overridesUsed, updatesUsed;
std::map<ref<Node>, StorePath> nodePaths;
for (auto & i : lockFlags.inputOverrides) {
overrides.insert_or_assign(i.first, FlakeInput { .ref = i.second });
@@ -535,11 +536,13 @@ LockedFlake lockFlake(
}
}
computeLocks(
mustRefetch
? getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath).inputs
: fakeInputs,
childNode, inputPath, oldLock, lockRootPath, parentPath, !mustRefetch);
if (mustRefetch) {
auto inputFlake = getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath);
nodePaths.emplace(childNode, inputFlake.storePath);
computeLocks(inputFlake.inputs, childNode, inputPath, oldLock, lockRootPath, parentPath, false);
} else {
computeLocks(fakeInputs, childNode, inputPath, oldLock, lockRootPath, parentPath, true);
}
} else {
/* We need to create a new lock file entry. So fetch
@@ -584,6 +587,7 @@ LockedFlake lockFlake(
flake. Also, unless we already have this flake
in the top-level lock file, use this flake's
own lock file. */
nodePaths.emplace(childNode, inputFlake.storePath);
computeLocks(
inputFlake.inputs, childNode, inputPath,
oldLock
@@ -596,11 +600,13 @@ LockedFlake lockFlake(
}
else {
auto [sourceInfo, resolvedRef, lockedRef] = fetchOrSubstituteTree(
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, *input.ref, useRegistries, flakeCache);
auto childNode = make_ref<LockedNode>(lockedRef, ref, false);
nodePaths.emplace(childNode, storePath);
node->inputs.insert_or_assign(id, childNode);
}
}
@@ -615,6 +621,8 @@ LockedFlake lockFlake(
// Bring in the current ref for relative path resolution if we have it
auto parentPath = canonPath(state.store->toRealPath(flake.storePath) + "/" + flake.lockedRef.subdir, true);
nodePaths.emplace(newLockFile.root, flake.storePath);
computeLocks(
flake.inputs,
newLockFile.root,
@@ -707,14 +715,6 @@ LockedFlake lockFlake(
flake.lockedRef.input.getRev() &&
prevLockedRef.input.getRev() != flake.lockedRef.input.getRev())
warn("committed new revision '%s'", flake.lockedRef.input.getRev()->gitRev());
/* Make sure that we picked up the change,
i.e. the tree should usually be dirty
now. Corner case: we could have reverted from a
dirty to a clean tree! */
if (flake.lockedRef.input == prevLockedRef.input
&& !flake.lockedRef.input.isLocked())
throw Error("'%s' did not change after I updated its 'flake.lock' file; is 'flake.lock' under version control?", flake.originalRef);
}
} else
throw Error("cannot write modified lock file of flake '%s' (use '--no-write-lock-file' to ignore)", topRef);
@@ -724,7 +724,11 @@ LockedFlake lockFlake(
}
}
return LockedFlake { .flake = std::move(flake), .lockFile = std::move(newLockFile) };
return LockedFlake {
.flake = std::move(flake),
.lockFile = std::move(newLockFile),
.nodePaths = std::move(nodePaths)
};
} catch (Error & e) {
e.addTrace({}, "while updating the lock file of flake '%s'", flake.lockedRef.to_string());
@@ -736,30 +740,48 @@ void callFlake(EvalState & state,
const LockedFlake & lockedFlake,
Value & vRes)
{
auto vLocks = state.allocValue();
auto vRootSrc = state.allocValue();
auto vRootSubdir = state.allocValue();
auto vTmp1 = state.allocValue();
auto vTmp2 = state.allocValue();
experimentalFeatureSettings.require(Xp::Flakes);
vLocks->mkString(lockedFlake.lockFile.to_string());
auto [lockFileStr, keyMap] = lockedFlake.lockFile.to_string();
emitTreeAttrs(
state,
lockedFlake.flake.storePath,
lockedFlake.flake.lockedRef.input,
*vRootSrc,
false,
lockedFlake.flake.forceDirty);
auto overrides = state.buildBindings(lockedFlake.nodePaths.size());
vRootSubdir->mkString(lockedFlake.flake.lockedRef.subdir);
for (auto & [node, storePath] : lockedFlake.nodePaths) {
auto override = state.buildBindings(2);
auto & vSourceInfo = override.alloc(state.symbols.create("sourceInfo"));
auto lockedNode = node.dynamic_pointer_cast<const LockedNode>();
emitTreeAttrs(
state,
storePath,
lockedNode ? lockedNode->lockedRef.input : lockedFlake.flake.lockedRef.input,
vSourceInfo,
false,
!lockedNode && lockedFlake.flake.forceDirty);
auto key = keyMap.find(node);
assert(key != keyMap.end());
override
.alloc(state.symbols.create("dir"))
.mkString(lockedNode ? lockedNode->lockedRef.subdir : lockedFlake.flake.lockedRef.subdir);
overrides.alloc(state.symbols.create(key->second)).mkAttrs(override);
}
auto & vOverrides = state.allocValue()->mkAttrs(overrides);
auto vCallFlake = state.allocValue();
state.evalFile(state.callFlakeInternal, *vCallFlake);
auto vTmp1 = state.allocValue();
auto vLocks = state.allocValue();
vLocks->mkString(lockFileStr);
state.callFunction(*vCallFlake, *vLocks, *vTmp1, noPos);
state.callFunction(*vTmp1, *vRootSrc, *vTmp2, noPos);
state.callFunction(*vTmp2, *vRootSubdir, vRes, noPos);
state.callFunction(*vTmp1, vOverrides, vRes, noPos);
}
static void prim_getFlake(EvalState & state, const PosIdx pos, Value * * args, Value & v)
+7
View File
@@ -103,6 +103,13 @@ struct LockedFlake
Flake flake;
LockFile lockFile;
/**
* Store paths of nodes that have been fetched in
* lockFlake(); in particular, the root node and the overriden
* inputs.
*/
std::map<ref<Node>, StorePath> nodePaths;
Fingerprint getFingerprint() const;
};
+30 -19
View File
@@ -55,7 +55,7 @@ FlakeRef parseFlakeRef(
{
auto [flakeRef, fragment] = parseFlakeRefWithFragment(url, baseDir, allowMissing, isFlake);
if (fragment != "")
throw Error("unexpected fragment '%s' in flake reference '%s'", fragment, url);
throw FlakeRefError("unexpected fragment '%s' in flake reference '%s'", fragment, url);
return flakeRef;
}
@@ -78,19 +78,25 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
std::string path = url;
std::string fragment = "";
std::map<std::string, std::string> query;
auto pathEnd = url.find_first_of("#?");
auto fragmentStart = pathEnd;
if (pathEnd != std::string::npos && url[pathEnd] == '?') {
fragmentStart = url.find("#");
}
auto pathEnd = url.find_first_of("?#");
if (pathEnd != std::string::npos) {
// There's something (either a query string or a fragment) in addition
// to the path
path = url.substr(0, pathEnd);
}
if (fragmentStart != std::string::npos) {
fragment = percentDecode(url.substr(fragmentStart+1));
}
if (pathEnd != std::string::npos && fragmentStart != std::string::npos) {
query = decodeQuery(url.substr(pathEnd+1, fragmentStart-pathEnd-1));
std::string non_path_part = url.substr(pathEnd + 1);
if (url[pathEnd] == '#') {
// Not query, just a fragment
fragment = percentDecode(non_path_part);
} else {
// We have a query, and maybe a fragment too
auto fragmentStart = non_path_part.find("#");
if (fragmentStart != std::string::npos) {
query = decodeQuery(non_path_part.substr(0, fragmentStart));
fragment = percentDecode(non_path_part.substr(fragmentStart+1));
} else {
query = decodeQuery(non_path_part);
}
}
}
if (baseDir) {
@@ -113,10 +119,10 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
found = true;
break;
} else if (pathExists(path + "/.git"))
throw Error("path '%s' is not part of a flake (neither it nor its parent directories contain a 'flake.nix' file)", path);
throw FlakeRefError("path '%s' is not part of a flake (neither it nor its parent directories contain a 'flake.nix' file)", path);
else {
if (lstat(path).st_dev != device)
throw Error("unable to find a flake before encountering filesystem boundary at '%s'", path);
throw FlakeRefError("unable to find a flake before encountering filesystem boundary at '%s'", path);
}
path = dirOf(path);
}
@@ -148,7 +154,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
if (subdir != "") {
if (parsedURL.query.count("dir"))
throw Error("flake URL '%s' has an inconsistent 'dir' parameter", url);
throw FlakeRefError("flake URL '%s' has an inconsistent 'dir' parameter", url);
parsedURL.query.insert_or_assign("dir", subdir);
}
@@ -171,11 +177,16 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
path = canonPath(path + "/" + getOr(query, "dir", ""));
}
fetchers::Attrs attrs;
attrs.insert_or_assign("type", "path");
attrs.insert_or_assign("path", path);
auto parsedURL = ParsedURL{
.url = url,
.base = url,
.scheme = "path",
.authority = "",
.path = path,
.query = query,
};
return std::make_pair(FlakeRef(fetchers::Input::fromAttrs(std::move(attrs)), ""), fragment);
return std::make_pair(FlakeRef(fetchers::Input::fromURL(parsedURL), getOr(parsedURL.query, "dir", "")), fragment);
};
+2
View File
@@ -15,6 +15,8 @@ class Store;
typedef std::string FlakeId;
MakeError(FlakeRefError, Error);
/**
* A flake reference specifies how to fetch a flake or raw source
* (e.g. from a Git repository). It is created from a URL-like syntax
+10 -9
View File
@@ -38,7 +38,7 @@ LockedNode::LockedNode(const nlohmann::json & json)
, isFlake(json.find("flake") != json.end() ? (bool) json["flake"] : true)
{
if (!lockedRef.input.isLocked())
throw Error("lock file contains mutable lock '%s'",
throw Error("lock file contains unlocked input '%s'",
fetchers::attrsToJSON(lockedRef.input.toAttrs()));
}
@@ -107,7 +107,7 @@ LockFile::LockFile(const nlohmann::json & json, const Path & path)
std::string inputKey = i.value();
auto k = nodeMap.find(inputKey);
if (k == nodeMap.end()) {
auto nodes = json["nodes"];
auto & nodes = json["nodes"];
auto jsonNode2 = nodes.find(inputKey);
if (jsonNode2 == nodes.end())
throw Error("lock file references missing node '%s'", inputKey);
@@ -134,10 +134,10 @@ LockFile::LockFile(const nlohmann::json & json, const Path & path)
// a bit since we don't need to worry about cycles.
}
nlohmann::json LockFile::toJSON() const
std::pair<nlohmann::json, LockFile::KeyMap> LockFile::toJSON() const
{
nlohmann::json nodes;
std::unordered_map<std::shared_ptr<const Node>, std::string> nodeKeys;
KeyMap nodeKeys;
std::unordered_set<std::string> keys;
std::function<std::string(const std::string & key, ref<const Node> node)> dumpNode;
@@ -194,12 +194,13 @@ nlohmann::json LockFile::toJSON() const
json["root"] = dumpNode("root", root);
json["nodes"] = std::move(nodes);
return json;
return {json, std::move(nodeKeys)};
}
std::string LockFile::to_string() const
std::pair<std::string, LockFile::KeyMap> LockFile::to_string() const
{
return toJSON().dump(2);
auto [json, nodeKeys] = toJSON();
return {json.dump(2), std::move(nodeKeys)};
}
LockFile LockFile::read(const Path & path)
@@ -210,7 +211,7 @@ LockFile LockFile::read(const Path & path)
std::ostream & operator <<(std::ostream & stream, const LockFile & lockFile)
{
stream << lockFile.toJSON().dump(2);
stream << lockFile.toJSON().first.dump(2);
return stream;
}
@@ -243,7 +244,7 @@ std::optional<FlakeRef> LockFile::isUnlocked() const
bool LockFile::operator ==(const LockFile & other) const
{
// FIXME: slow
return toJSON() == other.toJSON();
return toJSON().first == other.toJSON().first;
}
bool LockFile::operator !=(const LockFile & other) const
+4 -3
View File
@@ -59,14 +59,15 @@ struct LockFile
typedef std::map<ref<const Node>, std::string> KeyMap;
nlohmann::json toJSON() const;
std::pair<nlohmann::json, KeyMap> toJSON() const;
std::string to_string() const;
std::pair<std::string, KeyMap> to_string() const;
static LockFile read(const Path & path);
/**
* Check whether this lock file has any unlocked inputs.
* Check whether this lock file has any unlocked inputs. If so,
* return one.
*/
std::optional<FlakeRef> isUnlocked() const;
+6 -8
View File
@@ -5,13 +5,12 @@
namespace nix {
static const std::string attributeNamePattern("[a-zA-Z0-9_-]+");
static const std::regex lastAttributeRegex("(?:" + attributeNamePattern + "\\.)*(?!default)(" + attributeNamePattern +")(\\^.*)?");
static const std::regex lastAttributeRegex("^((?:" + attributeNamePattern + "\\.)*)(" + attributeNamePattern +")(\\^.*)?$");
static const std::string pathSegmentPattern("[a-zA-Z0-9_-]+");
static const std::regex lastPathSegmentRegex(".*/(" + pathSegmentPattern +")");
static const std::regex secondPathSegmentRegex("(?:" + pathSegmentPattern + ")/(" + pathSegmentPattern +")(?:/.*)?");
static const std::regex gitProviderRegex("github|gitlab|sourcehut");
static const std::regex gitSchemeRegex("git($|\\+.*)");
static const std::regex defaultOutputRegex(".*\\.default($|\\^.*)");
std::optional<std::string> getNameFromURL(const ParsedURL & url)
{
@@ -22,8 +21,11 @@ std::optional<std::string> getNameFromURL(const ParsedURL & url)
return url.query.at("dir");
/* If the fragment isn't a "default" and contains two attribute elements, use the last one */
if (std::regex_match(url.fragment, match, lastAttributeRegex))
return match.str(1);
if (std::regex_match(url.fragment, match, lastAttributeRegex)
&& match.str(1) != "defaultPackage."
&& match.str(2) != "default") {
return match.str(2);
}
/* If this is a github/gitlab/sourcehut flake, use the repo name */
if (std::regex_match(url.scheme, gitProviderRegex) && std::regex_match(url.path, match, secondPathSegmentRegex))
@@ -33,10 +35,6 @@ std::optional<std::string> getNameFromURL(const ParsedURL & url)
if (std::regex_match(url.scheme, gitSchemeRegex) && std::regex_match(url.path, match, lastPathSegmentRegex))
return match.str(1);
/* If everything failed but there is a non-default fragment, use it in full */
if (!url.fragment.empty() && !std::regex_match(url.fragment, defaultOutputRegex))
return url.fragment;
/* If there is no fragment, take the last element of the path */
if (std::regex_match(url.path, match, lastPathSegmentRegex))
return match.str(1);
+3
View File
@@ -94,6 +94,9 @@ static StringToken unescapeStr(SymbolTable & symbols, char * s, size_t length)
}
// yacc generates code that uses unannotated fallthrough.
#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
#define YY_USER_INIT initLoc(yylloc)
#define YY_USER_ACTION adjustLoc(yylloc, yytext, yyleng);
+87 -26
View File
@@ -70,10 +70,8 @@ void ExprOpHasAttr::show(const SymbolTable & symbols, std::ostream & str) const
str << ") ? " << showAttrPath(symbols, attrPath) << ")";
}
void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
void ExprAttrs::showBindings(const SymbolTable & symbols, std::ostream & str) const
{
if (recursive) str << "rec ";
str << "{ ";
typedef const decltype(attrs)::value_type * Attr;
std::vector<Attr> sorted;
for (auto & i : attrs) sorted.push_back(&i);
@@ -81,10 +79,37 @@ void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
std::string_view sa = symbols[a->first], sb = symbols[b->first];
return sa < sb;
});
std::vector<Symbol> inherits;
std::map<ExprInheritFrom *, std::vector<Symbol>> inheritsFrom;
for (auto & i : sorted) {
if (i->second.inherited)
str << "inherit " << symbols[i->first] << " " << "; ";
else {
switch (i->second.kind) {
case AttrDef::Kind::Plain:
break;
case AttrDef::Kind::Inherited:
inherits.push_back(i->first);
break;
case AttrDef::Kind::InheritedFrom: {
auto & select = dynamic_cast<ExprSelect &>(*i->second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*select.e);
inheritsFrom[&from].push_back(i->first);
break;
}
}
}
if (!inherits.empty()) {
str << "inherit";
for (auto sym : inherits) str << " " << symbols[sym];
str << "; ";
}
for (const auto & [from, syms] : inheritsFrom) {
str << "inherit (";
(*inheritFromExprs)[from->displ]->show(symbols, str);
str << ")";
for (auto sym : syms) str << " " << symbols[sym];
str << "; ";
}
for (auto & i : sorted) {
if (i->second.kind == AttrDef::Kind::Plain) {
str << symbols[i->first] << " = ";
i->second.e->show(symbols, str);
str << "; ";
@@ -97,6 +122,13 @@ void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
i.valueExpr->show(symbols, str);
str << "; ";
}
}
void ExprAttrs::show(const SymbolTable & symbols, std::ostream & str) const
{
if (recursive) str << "rec ";
str << "{ ";
showBindings(symbols, str);
str << "}";
}
@@ -152,15 +184,7 @@ void ExprCall::show(const SymbolTable & symbols, std::ostream & str) const
void ExprLet::show(const SymbolTable & symbols, std::ostream & str) const
{
str << "(let ";
for (auto & i : attrs->attrs)
if (i.second.inherited) {
str << "inherit " << symbols[i.first] << "; ";
}
else {
str << symbols[i.first] << " = ";
i.second.e->show(symbols, str);
str << "; ";
}
attrs->showBindings(symbols, str);
str << "in ";
body->show(symbols, str);
str << ")";
@@ -305,6 +329,12 @@ void ExprVar::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> &
this->level = withLevel;
}
void ExprInheritFrom::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
}
void ExprSelect::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
@@ -328,22 +358,47 @@ void ExprOpHasAttr::bindVars(EvalState & es, const std::shared_ptr<const StaticE
i.expr->bindVars(es, env);
}
std::shared_ptr<const StaticEnv> ExprAttrs::bindInheritSources(
EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (!inheritFromExprs)
return nullptr;
// the inherit (from) source values are inserted into an env of its own, which
// does not introduce any variable names.
// analysis must see an empty env, or an env that contains only entries with
// otherwise unused names to not interfere with regular names. the parser
// has already filled all exprs that access this env with appropriate level
// and displacement, and nothing else is allowed to access it. ideally we'd
// not even *have* an expr that grabs anything from this env since it's fully
// invisible, but the evaluator does not allow for this yet.
auto inner = std::make_shared<StaticEnv>(nullptr, env.get(), 0);
for (auto from : *inheritFromExprs)
from->bindVars(es, env);
return inner;
}
void ExprAttrs::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, env));
if (recursive) {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), recursive ? attrs.size() : 0);
auto newEnv = [&] () -> std::shared_ptr<const StaticEnv> {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs.size());
Displacement displ = 0;
for (auto & i : attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
Displacement displ = 0;
for (auto & i : attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
return newEnv;
}();
// No need to sort newEnv since attrs is in sorted order.
auto inheritFromEnv = bindInheritSources(es, newEnv);
for (auto & i : attrs)
i.second.e->bindVars(es, i.second.inherited ? env : newEnv);
i.second.e->bindVars(es, i.second.chooseByKind(newEnv, env, inheritFromEnv));
for (auto & i : dynamicAttrs) {
i.nameExpr->bindVars(es, newEnv);
@@ -351,8 +406,10 @@ void ExprAttrs::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv>
}
}
else {
auto inheritFromEnv = bindInheritSources(es, env);
for (auto & i : attrs)
i.second.e->bindVars(es, env);
i.second.e->bindVars(es, i.second.chooseByKind(env, env, inheritFromEnv));
for (auto & i : dynamicAttrs) {
i.nameExpr->bindVars(es, env);
@@ -409,16 +466,20 @@ void ExprCall::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> &
void ExprLet::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env)
{
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs->attrs.size());
auto newEnv = [&] () -> std::shared_ptr<const StaticEnv> {
auto newEnv = std::make_shared<StaticEnv>(nullptr, env.get(), attrs->attrs.size());
Displacement displ = 0;
for (auto & i : attrs->attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
Displacement displ = 0;
for (auto & i : attrs->attrs)
newEnv->vars.emplace_back(i.first, i.second.displ = displ++);
return newEnv;
}();
// No need to sort newEnv since attrs->attrs is in sorted order.
auto inheritFromEnv = attrs->bindInheritSources(es, newEnv);
for (auto & i : attrs->attrs)
i.second.e->bindVars(es, i.second.inherited ? env : newEnv);
i.second.e->bindVars(es, i.second.chooseByKind(newEnv, env, inheritFromEnv));
if (es.debugRepl)
es.exprEnvs.insert(std::make_pair(this, newEnv));
+49 -3
View File
@@ -135,6 +135,23 @@ struct ExprVar : Expr
COMMON_METHODS
};
/**
* A pseudo-expression for the purpose of evaluating the `from` expression in `inherit (from)` syntax.
* Unlike normal variable references, the displacement is set during parsing, and always refers to
* `ExprAttrs::inheritFromExprs` (by itself or in `ExprLet`), whose values are put into their own `Env`.
*/
struct ExprInheritFrom : ExprVar
{
ExprInheritFrom(PosIdx pos, Displacement displ): ExprVar(pos, {})
{
this->level = 0;
this->displ = displ;
this->fromWith = nullptr;
}
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env);
};
struct ExprSelect : Expr
{
PosIdx pos;
@@ -160,16 +177,40 @@ struct ExprAttrs : Expr
bool recursive;
PosIdx pos;
struct AttrDef {
bool inherited;
enum class Kind {
/** `attr = expr;` */
Plain,
/** `inherit attr1 attrn;` */
Inherited,
/** `inherit (expr) attr1 attrn;` */
InheritedFrom,
};
Kind kind;
Expr * e;
PosIdx pos;
Displacement displ; // displacement
AttrDef(Expr * e, const PosIdx & pos, bool inherited=false)
: inherited(inherited), e(e), pos(pos) { };
AttrDef(Expr * e, const PosIdx & pos, Kind kind = Kind::Plain)
: kind(kind), e(e), pos(pos) { };
AttrDef() { };
template<typename T>
const T & chooseByKind(const T & plain, const T & inherited, const T & inheritedFrom) const
{
switch (kind) {
case Kind::Plain:
return plain;
case Kind::Inherited:
return inherited;
default:
case Kind::InheritedFrom:
return inheritedFrom;
}
}
};
typedef std::map<Symbol, AttrDef> AttrDefs;
AttrDefs attrs;
std::unique_ptr<std::vector<Expr *>> inheritFromExprs;
struct DynamicAttrDef {
Expr * nameExpr, * valueExpr;
PosIdx pos;
@@ -182,6 +223,11 @@ struct ExprAttrs : Expr
ExprAttrs() : recursive(false) { };
PosIdx getPos() const override { return pos; }
COMMON_METHODS
std::shared_ptr<const StaticEnv> bindInheritSources(
EvalState & es, const std::shared_ptr<const StaticEnv> & env);
Env * buildInheritFromEnv(EvalState & state, Env & up);
void showBindings(const SymbolTable & symbols, std::ostream & str) const;
};
struct ExprList : Expr
+12 -1
View File
@@ -89,7 +89,7 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, Expr *
if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) {
if (!j->second.inherited) {
if (j->second.kind != ExprAttrs::AttrDef::Kind::Inherited) {
ExprAttrs * attrs2 = dynamic_cast<ExprAttrs *>(j->second.e);
if (!attrs2) dupAttr(attrPath, pos, j->second.pos);
attrs = attrs2;
@@ -118,13 +118,24 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, Expr *
auto ae = dynamic_cast<ExprAttrs *>(e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
dupAttr(ad.first, j2->second.pos, ad.second.pos);
jAttrs->attrs.emplace(ad.first, ad.second);
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
}
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(), ae->dynamicAttrs.begin(), ae->dynamicAttrs.end());
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
ae->inheritFromExprs->begin(), ae->inheritFromExprs->end());
}
} else {
dupAttr(attrPath, pos, j->second.pos);
}
+13 -3
View File
@@ -313,17 +313,27 @@ binds
if ($$->attrs.find(i.symbol) != $$->attrs.end())
state->dupAttr(i.symbol, state->at(@3), $$->attrs[i.symbol].pos);
auto pos = state->at(@3);
$$->attrs.emplace(i.symbol, ExprAttrs::AttrDef(new ExprVar(CUR_POS, i.symbol), pos, true));
$$->attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(new ExprVar(CUR_POS, i.symbol), pos, ExprAttrs::AttrDef::Kind::Inherited));
}
delete $3;
}
| binds INHERIT '(' expr ')' attrs ';'
{ $$ = $1;
/* !!! Should ensure sharing of the expression in $4. */
if (!$$->inheritFromExprs)
$$->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
$$->inheritFromExprs->push_back($4);
auto from = new nix::ExprInheritFrom(state->at(@4), $$->inheritFromExprs->size() - 1);
for (auto & i : *$6) {
if ($$->attrs.find(i.symbol) != $$->attrs.end())
state->dupAttr(i.symbol, state->at(@6), $$->attrs[i.symbol].pos);
$$->attrs.emplace(i.symbol, ExprAttrs::AttrDef(new ExprSelect(CUR_POS, $4, i.symbol), state->at(@6)));
$$->attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(
new ExprSelect(CUR_POS, from, i.symbol),
state->at(@6),
ExprAttrs::AttrDef::Kind::InheritedFrom));
}
delete $6;
}
+66 -41
View File
@@ -705,38 +705,53 @@ static RegisterPrimOp primop_genericClosure(PrimOp {
.args = {"attrset"},
.arity = 1,
.doc = R"(
Take an *attrset* with values named `startSet` and `operator` in order to
return a *list of attrsets* by starting with the `startSet` and recursively
applying the `operator` function to each `item`. The *attrsets* in the
`startSet` and the *attrsets* produced by `operator` must contain a value
named `key` which is comparable. The result is produced by calling `operator`
for each `item` with a value for `key` that has not been called yet including
newly produced `item`s. The function terminates when no new `item`s are
produced. The resulting *list of attrsets* contains only *attrsets* with a
unique key. For example,
`builtins.genericClosure` iteratively computes the transitive closure over an arbitrary relation defined by a function.
```
builtins.genericClosure {
startSet = [ {key = 5;} ];
operator = item: [{
key = if (item.key / 2 ) * 2 == item.key
then item.key / 2
else 3 * item.key + 1;
}];
}
```
evaluates to
```
[ { key = 5; } { key = 16; } { key = 8; } { key = 4; } { key = 2; } { key = 1; } ]
```
It takes *attrset* with two attributes named `startSet` and `operator`, and returns a list of attrbute sets:
- `startSet`:
The initial list of attribute sets.
- `operator`:
A function that takes an attribute set and returns a list of attribute sets.
It defines how each item in the current set is processed and expanded into more items.
Each attribute set in the list `startSet` and the list returned by `operator` must have an attribute `key`, which must support equality comparison.
The value of `key` can be one of the following types:
`key` can be one of the following types:
- [Number](@docroot@/language/values.md#type-number)
- [Boolean](@docroot@/language/values.md#type-boolean)
- [String](@docroot@/language/values.md#type-string)
- [Path](@docroot@/language/values.md#type-path)
- [List](@docroot@/language/values.md#list)
The result is produced by calling the `operator` on each `item` that has not been called yet, including newly added items, until no new items are added.
Items are compared by their `key` attribute.
Common usages are:
- Generating unique collections of items, such as dependency graphs.
- Traversing through structures that may contain cycles or loops.
- Processing data structures with complex internal relationships.
> **Example**
>
> ```nix
> builtins.genericClosure {
> startSet = [ {key = 5;} ];
> operator = item: [{
> key = if (item.key / 2 ) * 2 == item.key
> then item.key / 2
> else 3 * item.key + 1;
> }];
> }
> ```
>
> evaluates to
>
> ```nix
> [ { key = 5; } { key = 16; } { key = 8; } { key = 4; } { key = 2; } { key = 1; } ]
> ```
)",
.fun = prim_genericClosure,
});
@@ -760,15 +775,6 @@ static RegisterPrimOp primop_break({
auto & dt = state.debugTraces.front();
state.runDebugRepl(&error, dt.env, dt.expr);
if (state.debugQuit) {
// If the user elects to quit the repl, throw an exception.
throw Error(ErrorInfo{
.level = lvlInfo,
.msg = HintFmt("quit the debugger"),
.pos = nullptr,
});
}
}
// Return the value we were passed.
@@ -820,7 +826,7 @@ static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value * *
auto message = state.coerceToString(pos, *args[0], context,
"while evaluating the error message passed to builtins.addErrorContext",
false, false).toOwned();
e.addTrace(nullptr, HintFmt(message), true);
e.addTrace(nullptr, HintFmt(message));
throw;
}
}
@@ -879,7 +885,7 @@ static void prim_tryEval(EvalState & state, const PosIdx pos, Value * * args, Va
/* increment state.trylevel, and decrement it when this function returns. */
MaintainCount trylevel(state.trylevel);
void (* savedDebugRepl)(ref<EvalState> es, const ValMap & extraEnv) = nullptr;
ReplExitStatus (* savedDebugRepl)(ref<EvalState> es, const ValMap & extraEnv) = nullptr;
if (state.debugRepl && evalSettings.ignoreExceptionsDuringTry)
{
/* to prevent starting the repl from exceptions withing a tryEval, null it. */
@@ -995,6 +1001,10 @@ static void prim_trace(EvalState & state, const PosIdx pos, Value * * args, Valu
printError("trace: %1%", args[0]->string_view());
else
printError("trace: %1%", ValuePrinter(state, *args[0]));
if (evalSettings.builtinsTraceDebugger && state.debugRepl && !state.debugTraces.empty()) {
const DebugTrace & last = state.debugTraces.front();
state.runDebugRepl(nullptr, last.env, last.expr);
}
state.forceValue(*args[1], pos);
v = *args[1];
}
@@ -1006,6 +1016,12 @@ static RegisterPrimOp primop_trace({
Evaluate *e1* and print its abstract syntax representation on
standard error. Then return *e2*. This function is useful for
debugging.
If the
[`debugger-on-trace`](@docroot@/command-ref/conf-file.md#conf-debugger-on-trace)
option is set to `true` and the `--debugger` flag is given, the
interactive debugger will be started when `trace` is called (like
[`break`](@docroot@/language/builtins.md#builtins-break)).
)",
.fun = prim_trace,
});
@@ -1074,7 +1090,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value * *
e.addTrace(nullptr, HintFmt(
"while evaluating derivation '%s'\n"
" whose name attribute is located at %s",
drvName, pos), true);
drvName, pos));
throw;
}
}
@@ -1122,7 +1138,10 @@ drvName, Bindings * attrs, Value & v)
auto handleHashMode = [&](const std::string_view s) {
if (s == "recursive") ingestionMethod = FileIngestionMethod::Recursive;
else if (s == "flat") ingestionMethod = FileIngestionMethod::Flat;
else if (s == "text") {
else if (s == "git") {
experimentalFeatureSettings.require(Xp::GitHashing);
ingestionMethod = FileIngestionMethod::Git;
} else if (s == "text") {
experimentalFeatureSettings.require(Xp::DynamicDerivations);
ingestionMethod = TextIngestionMethod {};
} else
@@ -1232,8 +1251,7 @@ drvName, Bindings * attrs, Value & v)
} catch (Error & e) {
e.addTrace(state.positions[i->pos],
HintFmt("while evaluating attribute '%1%' of derivation '%2%'", key, drvName),
true);
HintFmt("while evaluating attribute '%1%' of derivation '%2%'", key, drvName));
throw;
}
}
@@ -2074,7 +2092,7 @@ static void prim_toFile(EvalState & state, const PosIdx pos, Value * * args, Val
})
: ({
StringSource s { contents };
state.store->addToStoreFromDump(s, name, TextIngestionMethod {}, HashAlgorithm::SHA256, refs, state.repair);
state.store->addToStoreFromDump(s, name, FileSerialisationMethod::Flat, TextIngestionMethod {}, HashAlgorithm::SHA256, refs, state.repair);
});
/* Note: we don't need to add `context' to the context of the
@@ -2231,7 +2249,14 @@ static void addPath(
});
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
auto dstPath = fetchToStore(*state.store, path.resolveSymlinks(), name, method, filter.get(), state.repair);
auto dstPath = fetchToStore(
*state.store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
name,
method,
filter.get(),
state.repair);
if (expectedHash && expectedStorePath != dstPath)
state.error<EvalError>(
"store path mismatch in (possibly filtered) path added from '%s'",
+7 -7
View File
@@ -9,6 +9,7 @@
#include "tarball.hh"
#include "url.hh"
#include "value-to-json.hh"
#include "fetch-to-store.hh"
#include <ctime>
#include <iomanip>
@@ -24,8 +25,6 @@ void emitTreeAttrs(
bool emptyRevFallback,
bool forceDirty)
{
assert(input.isLocked());
auto attrs = state.buildBindings(100);
state.mkStorePathString(storePath, attrs.alloc(state.sOutPath));
@@ -159,9 +158,9 @@ static void fetchTree(
}
input = fetchers::Input::fromAttrs(std::move(attrs));
} else {
if (!experimentalFeatureSettings.isEnabled(Xp::FetchTreeUrls))
if (!experimentalFeatureSettings.isEnabled(Xp::Flakes))
state.error<EvalError>(
"passing a string argument to 'fetchTree' requires the 'fetch-tree-urls' experimental feature"
"passing a string argument to 'fetchTree' requires the 'flakes' experimental feature"
).atPos(pos).debugThrow();
input = fetchers::Input::fromURL(url);
}
@@ -176,8 +175,8 @@ static void fetchTree(
fetcher = "fetchGit";
state.error<EvalError>(
"in pure evaluation mode, %s requires a locked input",
fetcher
"in pure evaluation mode, '%s' will not fetch unlocked input '%s'",
fetcher, input.to_string()
).atPos(pos).debugThrow();
}
@@ -410,6 +409,7 @@ static RegisterPrimOp primop_fetchTree({
> ```
)",
.fun = prim_fetchTree,
.experimentalFeature = Xp::FetchTree,
});
static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v,
@@ -472,7 +472,7 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v
// https://github.com/NixOS/nix/issues/4313
auto storePath =
unpack
? fetchers::downloadTarball(state.store, *url, name, (bool) expectedHash).storePath
? fetchToStore(*state.store, fetchers::downloadTarball(*url).accessor, FetchMode::Copy, name)
: fetchers::downloadFile(state.store, *url, name, (bool) expectedHash).storePath;
if (expectedHash) {
+20
View File
@@ -0,0 +1,20 @@
#pragma once
namespace nix {
/**
* Exit status returned from the REPL.
*/
enum class ReplExitStatus {
/**
* The user exited with `:quit`. The program (e.g., if the REPL was acting
* as the debugger) should exit.
*/
QuitAll,
/**
* The user exited with `:continue`. The program should continue running.
*/
Continue,
};
}
+5 -3
View File
@@ -7,6 +7,7 @@ namespace nix {
StorePath fetchToStore(
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name,
ContentAddressMethod method,
PathFilter * filter,
@@ -33,18 +34,19 @@ StorePath fetchToStore(
} else
debug("source path '%s' is uncacheable", path);
Activity act(*logger, lvlChatty, actUnknown, fmt("copying '%s' to the store", path));
Activity act(*logger, lvlChatty, actUnknown,
fmt(mode == FetchMode::DryRun ? "hashing '%s'" : "copying '%s' to the store", path));
auto filter2 = filter ? *filter : defaultPathFilter;
auto storePath =
settings.readOnlyMode
mode == FetchMode::DryRun
? store.computeStorePath(
name, *path.accessor, path.path, method, HashAlgorithm::SHA256, {}, filter2).first
: store.addToStore(
name, *path.accessor, path.path, method, HashAlgorithm::SHA256, {}, filter2, repair);
if (cacheKey)
if (cacheKey && mode == FetchMode::Copy)
fetchers::getCache()->add(store, *cacheKey, {}, storePath, true);
return storePath;
+3
View File
@@ -8,12 +8,15 @@
namespace nix {
enum struct FetchMode { DryRun, Copy };
/**
* Copy the `path` to the Nix store.
*/
StorePath fetchToStore(
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name = "source",
ContentAddressMethod method = FileIngestionMethod::Recursive,
PathFilter * filter = nullptr,
+6 -7
View File
@@ -45,12 +45,8 @@ static void fixupInput(Input & input)
// Check common attributes.
input.getType();
input.getRef();
if (input.getRev())
input.locked = true;
input.getRevCount();
input.getLastModified();
if (input.getNarHash())
input.locked = true;
}
Input Input::fromURL(const ParsedURL & url, bool requireTree)
@@ -140,6 +136,11 @@ bool Input::isDirect() const
return !scheme || scheme->isDirect(*this);
}
bool Input::isLocked() const
{
return scheme && scheme->isLocked(*this);
}
Attrs Input::toAttrs() const
{
return attrs;
@@ -222,8 +223,6 @@ std::pair<StorePath, Input> Input::fetch(ref<Store> store) const
input.to_string(), *prevRevCount);
}
input.locked = true;
return {std::move(storePath), input};
}
@@ -376,7 +375,7 @@ void InputScheme::clone(const Input & input, const Path & destDir) const
std::pair<StorePath, Input> InputScheme::fetch(ref<Store> store, const Input & input)
{
auto [accessor, input2] = getAccessor(store, input);
auto storePath = fetchToStore(*store, SourcePath(accessor), input2.getName());
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy, input2.getName());
return {storePath, input2};
}
+9 -3
View File
@@ -29,7 +29,6 @@ struct Input
std::shared_ptr<InputScheme> scheme; // note: can be null
Attrs attrs;
bool locked = false;
/**
* path of the parent of this input, used for relative path resolution
@@ -71,7 +70,7 @@ public:
* Check whether this is a "locked" input, that is,
* one that contains a commit hash or content hash.
*/
bool isLocked() const { return locked; }
bool isLocked() const;
bool operator ==(const Input & other) const;
@@ -121,7 +120,6 @@ public:
std::optional<std::string> getFingerprint(ref<Store> store) const;
};
/**
* The `InputScheme` represents a type of fetcher. Each fetcher
* registers with nix at startup time. When processing an `Input`,
@@ -196,6 +194,14 @@ struct InputScheme
*/
virtual std::optional<std::string> getFingerprint(ref<Store> store, const Input & input) const
{ return std::nullopt; }
/**
* Return `true` if this input is considered "locked", i.e. it has
* attributes like a Git revision or NAR hash that uniquely
* identify its contents.
*/
virtual bool isLocked(const Input & input) const
{ return false; }
};
void registerInputScheme(std::shared_ptr<InputScheme> && fetcher);
+8 -8
View File
@@ -51,33 +51,33 @@ void FilteringInputAccessor::checkAccess(const CanonPath & path)
struct AllowListInputAccessorImpl : AllowListInputAccessor
{
std::set<CanonPath> allowedPaths;
std::set<CanonPath> allowedPrefixes;
AllowListInputAccessorImpl(
ref<InputAccessor> next,
std::set<CanonPath> && allowedPaths,
std::set<CanonPath> && allowedPrefixes,
MakeNotAllowedError && makeNotAllowedError)
: AllowListInputAccessor(SourcePath(next), std::move(makeNotAllowedError))
, allowedPaths(std::move(allowedPaths))
, allowedPrefixes(std::move(allowedPrefixes))
{ }
bool isAllowed(const CanonPath & path) override
{
return path.isAllowed(allowedPaths);
return path.isAllowed(allowedPrefixes);
}
void allowPath(CanonPath path) override
void allowPrefix(CanonPath prefix) override
{
allowedPaths.insert(std::move(path));
allowedPrefixes.insert(std::move(prefix));
}
};
ref<AllowListInputAccessor> AllowListInputAccessor::create(
ref<InputAccessor> next,
std::set<CanonPath> && allowedPaths,
std::set<CanonPath> && allowedPrefixes,
MakeNotAllowedError && makeNotAllowedError)
{
return make_ref<AllowListInputAccessorImpl>(next, std::move(allowedPaths), std::move(makeNotAllowedError));
return make_ref<AllowListInputAccessorImpl>(next, std::move(allowedPrefixes), std::move(makeNotAllowedError));
}
bool CachingFilteringInputAccessor::isAllowed(const CanonPath & path)
+5 -4
View File
@@ -54,18 +54,19 @@ struct FilteringInputAccessor : InputAccessor
};
/**
* A wrapping `InputAccessor` that checks paths against an allow-list.
* A wrapping `InputAccessor` that checks paths against a set of
* allowed prefixes.
*/
struct AllowListInputAccessor : public FilteringInputAccessor
{
/**
* Grant access to the specified path.
* Grant access to the specified prefix.
*/
virtual void allowPath(CanonPath path) = 0;
virtual void allowPrefix(CanonPath prefix) = 0;
static ref<AllowListInputAccessor> create(
ref<InputAccessor> next,
std::set<CanonPath> && allowedPaths,
std::set<CanonPath> && allowedPrefixes,
MakeNotAllowedError && makeNotAllowedError);
using FilteringInputAccessor::FilteringInputAccessor;
+29 -8
View File
@@ -2,6 +2,7 @@
#include "fs-input-accessor.hh"
#include "input-accessor.hh"
#include "filtering-input-accessor.hh"
#include "memory-input-accessor.hh"
#include "cache.hh"
#include "finally.hh"
#include "processes.hh"
@@ -466,6 +467,22 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
else
throw Error("Commit signature verification on commit %s failed: %s", rev.gitRev(), output);
}
Hash treeHashToNarHash(const Hash & treeHash) override
{
auto accessor = getAccessor(treeHash, false);
fetchers::Attrs cacheKey({{"_what", "treeHashToNarHash"}, {"treeHash", treeHash.gitRev()}});
if (auto res = fetchers::getCache()->lookup(cacheKey))
return Hash::parseAny(fetchers::getStrAttr(*res, "narHash"), HashAlgorithm::SHA256);
auto narHash = accessor->hashPath(CanonPath::root);
fetchers::getCache()->upsert(cacheKey, fetchers::Attrs({{"narHash", narHash.to_string(HashFormat::SRI, true)}}));
return narHash;
}
};
ref<GitRepo> GitRepo::openRepo(const std::filesystem::path & path, bool create, bool bare)
@@ -942,17 +959,21 @@ ref<InputAccessor> GitRepoImpl::getAccessor(const Hash & rev, bool exportIgnore)
ref<InputAccessor> GitRepoImpl::getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError)
{
auto self = ref<GitRepoImpl>(shared_from_this());
/* In case of an empty workdir, return an empty in-memory tree. We
cannot use AllowListInputAccessor because it would return an
error for the root (and we can't add the root to the allow-list
since that would allow access to all its children). */
ref<InputAccessor> fileAccessor =
AllowListInputAccessor::create(
makeFSInputAccessor(path),
std::set<CanonPath> { wd.files },
std::move(makeNotAllowedError));
if (exportIgnore) {
wd.files.empty()
? makeEmptyInputAccessor()
: AllowListInputAccessor::create(
makeFSInputAccessor(path),
std::set<CanonPath> { wd.files },
std::move(makeNotAllowedError)).cast<InputAccessor>();
if (exportIgnore)
return make_ref<GitExportIgnoreInputAccessor>(self, fileAccessor, std::nullopt);
}
else {
else
return fileAccessor;
}
}
ref<GitFileSystemObjectSink> GitRepoImpl::getFileSystemObjectSink()
+6
View File
@@ -93,6 +93,12 @@ struct GitRepo
virtual void verifyCommit(
const Hash & rev,
const std::vector<fetchers::PublicKey> & publicKeys) = 0;
/**
* Given a Git tree hash, compute the hash of its NAR
* serialisation. This is memoised on-disk.
*/
virtual Hash treeHashToNarHash(const Hash & treeHash) = 0;
};
ref<GitRepo> getTarballCache();
+11 -9
View File
@@ -158,6 +158,8 @@ std::vector<PublicKey> getPublicKeys(const Attrs & attrs)
} // end namespace
static const Hash nullRev{HashAlgorithm::SHA1};
struct GitInputScheme : InputScheme
{
std::optional<Input> inputFromURL(const ParsedURL & url, bool requireTree) const override
@@ -282,11 +284,6 @@ struct GitInputScheme : InputScheme
return res;
}
std::optional<ExperimentalFeature> experimentalFeature() const override
{
return Xp::FetchTreeGit;
}
void clone(const Input & input, const Path & destDir) const override
{
auto repoInfo = getRepoInfo(input);
@@ -713,10 +710,12 @@ struct GitInputScheme : InputScheme
if (auto ref = repo->getWorkdirRef())
input.attrs.insert_or_assign("ref", *ref);
auto rev = repoInfo.workdirInfo.headRev.value();
/* Return a rev of 000... if there are no commits yet. */
auto rev = repoInfo.workdirInfo.headRev.value_or(nullRev);
input.attrs.insert_or_assign("rev", rev.gitRev());
input.attrs.insert_or_assign("revCount", getRevCount(repoInfo, repoInfo.url, rev));
input.attrs.insert_or_assign("revCount",
rev == nullRev ? 0 : getRevCount(repoInfo, repoInfo.url, rev));
verifyCommit(input, repo);
} else {
@@ -738,8 +737,6 @@ struct GitInputScheme : InputScheme
? getLastModified(repoInfo, repoInfo.url, *repoInfo.workdirInfo.headRev)
: 0);
input.locked = true; // FIXME
return {accessor, std::move(input)};
}
@@ -776,6 +773,11 @@ struct GitInputScheme : InputScheme
else
return std::nullopt;
}
bool isLocked(const Input & input) const override
{
return (bool) input.getRev();
}
};
static auto rGitInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GitInputScheme>()); });
+9
View File
@@ -280,6 +280,15 @@ struct GitArchiveInputScheme : InputScheme
return {accessor, input};
}
bool isLocked(const Input & input) const override
{
/* Since we can't verify the integrity of the tarball from the
Git revision alone, we also require a NAR hash for
locking. FIXME: in the future, we may want to require a Git
tree hash instead of a NAR hash. */
return input.getRev().has_value() && input.getNarHash().has_value();
}
std::optional<ExperimentalFeature> experimentalFeature() const override
{
return Xp::Flakes;
+6
View File
@@ -20,4 +20,10 @@ ref<MemoryInputAccessor> makeMemoryInputAccessor()
return make_ref<MemoryInputAccessorImpl>();
}
ref<InputAccessor> makeEmptyInputAccessor()
{
static auto empty = makeMemoryInputAccessor().cast<InputAccessor>();
return empty;
}
}
+2
View File
@@ -13,4 +13,6 @@ struct MemoryInputAccessor : InputAccessor
ref<MemoryInputAccessor> makeMemoryInputAccessor();
ref<InputAccessor> makeEmptyInputAccessor();
}
+5
View File
@@ -347,6 +347,11 @@ struct MercurialInputScheme : InputScheme
return makeResult(infoAttrs, std::move(storePath));
}
bool isLocked(const Input & input) const override
{
return (bool) input.getRev();
}
std::optional<std::string> getFingerprint(ref<Store> store, const Input & input) const override
{
if (auto rev = input.getRev())
+5
View File
@@ -87,6 +87,11 @@ struct PathInputScheme : InputScheme
writeFile((CanonPath(getAbsPath(input)) / path).abs(), contents);
}
bool isLocked(const Input & input) const override
{
return (bool) input.getNarHash();
}
CanonPath getAbsPath(const Input & input) const
{
auto path = getStrAttr(input.attrs, "path");
+114 -72
View File
@@ -9,6 +9,9 @@
#include "types.hh"
#include "split.hh"
#include "posix-source-accessor.hh"
#include "fs-input-accessor.hh"
#include "store-api.hh"
#include "git-utils.hh"
namespace nix::fetchers {
@@ -57,10 +60,8 @@ DownloadFileResult downloadFile(
throw;
}
// FIXME: write to temporary file.
Attrs infoAttrs({
{"etag", res.etag},
{"url", res.effectiveUri},
});
if (res.immutableUrl)
@@ -91,96 +92,102 @@ DownloadFileResult downloadFile(
storePath = std::move(info.path);
}
getCache()->add(
*store,
inAttrs,
infoAttrs,
*storePath,
locked);
if (url != res.effectiveUri)
/* Cache metadata for all URLs in the redirect chain. */
for (auto & url : res.urls) {
inAttrs.insert_or_assign("url", url);
infoAttrs.insert_or_assign("url", *res.urls.rbegin());
getCache()->add(
*store,
{
{"type", "file"},
{"url", res.effectiveUri},
{"name", name},
},
inAttrs,
infoAttrs,
*storePath,
locked);
}
return {
.storePath = std::move(*storePath),
.etag = res.etag,
.effectiveUrl = res.effectiveUri,
.effectiveUrl = *res.urls.rbegin(),
.immutableUrl = res.immutableUrl,
};
}
DownloadTarballResult downloadTarball(
ref<Store> store,
const std::string & url,
const std::string & name,
bool locked,
const Headers & headers)
{
Attrs inAttrs({
{"type", "tarball"},
{"_what", "tarballCache"},
{"url", url},
{"name", name},
});
auto cached = getCache()->lookupExpired(*store, inAttrs);
auto cached = getCache()->lookupExpired(inAttrs);
auto attrsToResult = [&](const Attrs & infoAttrs)
{
auto treeHash = getRevAttr(infoAttrs, "treeHash");
return DownloadTarballResult {
.treeHash = treeHash,
.lastModified = (time_t) getIntAttr(infoAttrs, "lastModified"),
.immutableUrl = maybeGetStrAttr(infoAttrs, "immutableUrl"),
.accessor = getTarballCache()->getAccessor(treeHash, false),
};
};
if (cached && !getTarballCache()->hasObject(getRevAttr(cached->infoAttrs, "treeHash")))
cached.reset();
if (cached && !cached->expired)
return {
.storePath = std::move(cached->storePath),
.lastModified = (time_t) getIntAttr(cached->infoAttrs, "lastModified"),
.immutableUrl = maybeGetStrAttr(cached->infoAttrs, "immutableUrl"),
};
/* We previously downloaded this tarball and it's younger than
`tarballTtl`, so no need to check the server. */
return attrsToResult(cached->infoAttrs);
auto res = downloadFile(store, url, name, locked, headers);
auto _res = std::make_shared<Sync<FileTransferResult>>();
std::optional<StorePath> unpackedStorePath;
time_t lastModified;
if (cached && res.etag != "" && getStrAttr(cached->infoAttrs, "etag") == res.etag) {
unpackedStorePath = std::move(cached->storePath);
lastModified = getIntAttr(cached->infoAttrs, "lastModified");
} else {
Path tmpDir = createTempDir();
AutoDelete autoDelete(tmpDir, true);
unpackTarfile(store->toRealPath(res.storePath), tmpDir);
auto members = readDirectory(tmpDir);
if (members.size() != 1)
throw nix::Error("tarball '%s' contains an unexpected number of top-level files", url);
auto topDir = tmpDir + "/" + members.begin()->name;
lastModified = lstat(topDir).st_mtime;
PosixSourceAccessor accessor;
unpackedStorePath = store->addToStore(name, accessor, CanonPath { topDir }, FileIngestionMethod::Recursive, HashAlgorithm::SHA256, {}, defaultPathFilter, NoRepair);
}
Attrs infoAttrs({
{"lastModified", uint64_t(lastModified)},
{"etag", res.etag},
auto source = sinkToSource([&](Sink & sink) {
FileTransferRequest req(url);
req.expectedETag = cached ? getStrAttr(cached->infoAttrs, "etag") : "";
getFileTransfer()->download(std::move(req), sink,
[_res](FileTransferResult r)
{
*_res->lock() = r;
});
});
if (res.immutableUrl)
infoAttrs.emplace("immutableUrl", *res.immutableUrl);
// TODO: fall back to cached value if download fails.
getCache()->add(
*store,
inAttrs,
infoAttrs,
*unpackedStorePath,
locked);
/* Note: if the download is cached, `importTarball()` will receive
no data, which causes it to import an empty tarball. */
TarArchive archive { *source };
auto parseSink = getTarballCache()->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
return {
.storePath = std::move(*unpackedStorePath),
.lastModified = lastModified,
.immutableUrl = res.immutableUrl,
};
auto res(_res->lock());
Attrs infoAttrs;
if (res->cached) {
/* The server says that the previously downloaded version is
still current. */
infoAttrs = cached->infoAttrs;
} else {
infoAttrs.insert_or_assign("etag", res->etag);
infoAttrs.insert_or_assign("treeHash", parseSink->sync().gitRev());
infoAttrs.insert_or_assign("lastModified", uint64_t(lastModified));
if (res->immutableUrl)
infoAttrs.insert_or_assign("immutableUrl", *res->immutableUrl);
}
/* Insert a cache entry for every URL in the redirect chain. */
for (auto & url : res->urls) {
inAttrs.insert_or_assign("url", url);
getCache()->upsert(inAttrs, infoAttrs);
}
// FIXME: add a cache entry for immutableUrl? That could allow
// cache poisoning.
return attrsToResult(infoAttrs);
}
// An input scheme corresponding to a curl-downloadable resource.
@@ -198,6 +205,8 @@ struct CurlInputScheme : InputScheme
virtual bool isValidURL(const ParsedURL & url, bool requireTree) const = 0;
static const std::set<std::string> specialParams;
std::optional<Input> inputFromURL(const ParsedURL & _url, bool requireTree) const override
{
if (!isValidURL(_url, requireTree))
@@ -220,8 +229,17 @@ struct CurlInputScheme : InputScheme
if (auto n = string2Int<uint64_t>(*i))
input.attrs.insert_or_assign("revCount", *n);
url.query.erase("rev");
url.query.erase("revCount");
if (auto i = get(url.query, "lastModified"))
if (auto n = string2Int<uint64_t>(*i))
input.attrs.insert_or_assign("lastModified", *n);
/* The URL query parameters serve two roles: specifying fetch
settings for Nix itself, and arbitrary data as part of the
HTTP request. Now that we've processed the Nix-specific
attributes above, remove them so we don't also send them as
part of the HTTP request. */
for (auto & param : allowedAttrs())
url.query.erase(param);
input.attrs.insert_or_assign("type", std::string { schemeName() });
input.attrs.insert_or_assign("url", url.to_string());
@@ -260,6 +278,11 @@ struct CurlInputScheme : InputScheme
url.query.insert_or_assign("narHash", narHash->to_string(HashFormat::SRI, true));
return url;
}
bool isLocked(const Input & input) const override
{
return (bool) input.getNarHash();
}
};
struct FileInputScheme : CurlInputScheme
@@ -275,10 +298,24 @@ struct FileInputScheme : CurlInputScheme
: (!requireTree && !hasTarballExtension(url.path)));
}
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & input) override
std::pair<ref<InputAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
{
auto input(_input);
/* Unlike TarballInputScheme, this stores downloaded files in
the Nix store directly, since there is little deduplication
benefit in using the Git cache for single big files like
tarballs. */
auto file = downloadFile(store, getStrAttr(input.attrs, "url"), input.getName(), false);
return {std::move(file.storePath), input};
auto narHash = store->queryPathInfo(file.storePath)->narHash;
input.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
auto accessor = makeStorePathAccessor(store, file.storePath);
accessor->setPathDisplay("«" + input.to_string() + "»");
return {accessor, input};
}
};
@@ -296,11 +333,13 @@ struct TarballInputScheme : CurlInputScheme
: (requireTree || hasTarballExtension(url.path)));
}
std::pair<StorePath, Input> fetch(ref<Store> store, const Input & _input) override
std::pair<ref<InputAccessor>, Input> getAccessor(ref<Store> store, const Input & _input) const override
{
Input input(_input);
auto url = getStrAttr(input.attrs, "url");
auto result = downloadTarball(store, url, input.getName(), false);
auto input(_input);
auto result = downloadTarball(getStrAttr(input.attrs, "url"), {});
result.accessor->setPathDisplay("«" + input.to_string() + "»");
if (result.immutableUrl) {
auto immutableInput = Input::fromURL(*result.immutableUrl);
@@ -314,7 +353,10 @@ struct TarballInputScheme : CurlInputScheme
if (result.lastModified && !input.attrs.contains("lastModified"))
input.attrs.insert_or_assign("lastModified", uint64_t(result.lastModified));
return {result.storePath, std::move(input)};
input.attrs.insert_or_assign("narHash",
getTarballCache()->treeHashToNarHash(result.treeHash).to_string(HashFormat::SRI, true));
return {result.accessor, input};
}
};
+8 -4
View File
@@ -2,11 +2,13 @@
#include "types.hh"
#include "path.hh"
#include "hash.hh"
#include <optional>
namespace nix {
class Store;
struct InputAccessor;
}
namespace nix::fetchers {
@@ -28,16 +30,18 @@ DownloadFileResult downloadFile(
struct DownloadTarballResult
{
StorePath storePath;
Hash treeHash;
time_t lastModified;
std::optional<std::string> immutableUrl;
ref<InputAccessor> accessor;
};
/**
* Download and import a tarball into the Git cache. The result is the
* Git tree hash of the root directory.
*/
DownloadTarballResult downloadTarball(
ref<Store> store,
const std::string & url,
const std::string & name,
bool locked,
const Headers & headers = {});
}
-2
View File
@@ -408,6 +408,4 @@ PrintFreed::~PrintFreed()
showBytes(results.bytesFreed));
}
Exit::~Exit() { }
}
+1 -9
View File
@@ -7,6 +7,7 @@
#include "common-args.hh"
#include "path.hh"
#include "derived-path.hh"
#include "exit.hh"
#include <signal.h>
@@ -15,15 +16,6 @@
namespace nix {
class Exit : public std::exception
{
public:
int status;
Exit() : status(0) { }
Exit(int status) : status(status) { }
virtual ~Exit();
};
int handleExceptions(const std::string & programName, std::function<void()> fun);
/**
+20 -8
View File
@@ -305,7 +305,8 @@ void BinaryCacheStore::addToStore(const ValidPathInfo & info, Source & narSource
StorePath BinaryCacheStore::addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair)
@@ -313,17 +314,27 @@ StorePath BinaryCacheStore::addToStoreFromDump(
std::optional<Hash> caHash;
std::string nar;
// Calculating Git hash from NAR stream not yet implemented. May not
// be possible to implement in single-pass if the NAR is in an
// inconvenient order. Could fetch after uploading, however.
if (hashMethod.getFileIngestionMethod() == FileIngestionMethod::Git)
unsupported("addToStoreFromDump");
if (auto * dump2p = dynamic_cast<StringSource *>(&dump)) {
auto & dump2 = *dump2p;
// Hack, this gives us a "replayable" source so we can compute
// multiple hashes more easily.
caHash = hashString(HashAlgorithm::SHA256, dump2.s);
switch (method.getFileIngestionMethod()) {
case FileIngestionMethod::Recursive:
//
// Only calculate if the dump is in the right format, however.
if (static_cast<FileIngestionMethod>(dumpMethod) == hashMethod.getFileIngestionMethod())
caHash = hashString(HashAlgorithm::SHA256, dump2.s);
switch (dumpMethod) {
case FileSerialisationMethod::Recursive:
// The dump is already NAR in this case, just use it.
nar = dump2.s;
break;
case FileIngestionMethod::Flat:
case FileSerialisationMethod::Flat:
{
// The dump is Flat, so we need to convert it to NAR with a
// single file.
StringSink s;
@@ -331,10 +342,11 @@ StorePath BinaryCacheStore::addToStoreFromDump(
nar = std::move(s.s);
break;
}
}
} else {
// Otherwise, we have to do th same hashing as NAR so our single
// hash will suffice for both purposes.
if (method != FileIngestionMethod::Recursive || hashAlgo != HashAlgorithm::SHA256)
if (dumpMethod != FileSerialisationMethod::Recursive || hashAlgo != HashAlgorithm::SHA256)
unsupported("addToStoreFromDump");
}
StringSource narDump { nar };
@@ -349,7 +361,7 @@ StorePath BinaryCacheStore::addToStoreFromDump(
*this,
name,
ContentAddressWithReferences::fromParts(
method,
hashMethod,
caHash ? *caHash : nar.first,
{
.others = references,
@@ -450,7 +462,7 @@ StorePath BinaryCacheStore::addToStore(
non-recursive+sha256 so we can just use the default
implementation of this method in terms of addToStoreFromDump. */
auto h = hashPath(accessor, path, method.getFileIngestionMethod(), hashAlgo, filter).first;
auto h = hashPath(accessor, path, method.getFileIngestionMethod(), hashAlgo, filter);
auto source = sinkToSource([&](Sink & sink) {
accessor.dumpPath(path, sink, filter);
+3 -2
View File
@@ -125,7 +125,8 @@ public:
StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair) override;
@@ -147,7 +148,7 @@ public:
void narFromPath(const StorePath & path, Sink & sink) override;
ref<SourceAccessor> getFSAccessor(bool requireValidPath) override;
ref<SourceAccessor> getFSAccessor(bool requireValidPath = true) override;
void addSignatures(const StorePath & storePath, const StringSet & sigs) override;
+5
View File
@@ -123,6 +123,11 @@ struct KeyedBuildResult : BuildResult
* The derivation we built or the store path we substituted.
*/
DerivedPath path;
// Hack to work around a gcc "may be used uninitialized" warning.
KeyedBuildResult(BuildResult res, DerivedPath path)
: BuildResult(std::move(res)), path(std::move(path))
{ }
};
}
+28 -12
View File
@@ -8,6 +8,7 @@
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "git.hh"
#include "compression.hh"
#include "daemon.hh"
#include "topo-sort.hh"
@@ -1311,12 +1312,13 @@ struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual In
StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair) override
{
auto path = next->addToStoreFromDump(dump, name, method, hashAlgo, references, repair);
auto path = next->addToStoreFromDump(dump, name, dumpMethod, hashMethod, hashAlgo, references, repair);
goal.addDependency(path);
return path;
}
@@ -2457,15 +2459,29 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
rewriteOutput(outputRewrites);
/* FIXME optimize and deduplicate with addToStore */
std::string oldHashPart { scratchPath->hashPart() };
auto got = ({
HashModuloSink caSink { outputHash.hashAlgo, oldHashPart };
auto got = [&]{
PosixSourceAccessor accessor;
dumpPath(
accessor, CanonPath { actualPath },
caSink,
outputHash.method.getFileIngestionMethod());
caSink.finish().first;
});
auto fim = outputHash.method.getFileIngestionMethod();
switch (fim) {
case FileIngestionMethod::Flat:
case FileIngestionMethod::Recursive:
{
HashModuloSink caSink { outputHash.hashAlgo, oldHashPart };
auto fim = outputHash.method.getFileIngestionMethod();
dumpPath(
accessor, CanonPath { actualPath },
caSink,
(FileSerialisationMethod) fim);
return caSink.finish().first;
}
case FileIngestionMethod::Git: {
return git::dumpHash(
outputHash.hashAlgo, accessor,
CanonPath { tmpDir + "/tmp" }).hash;
}
}
assert(false);
}();
ValidPathInfo newInfo0 {
worker.store,
@@ -2491,7 +2507,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
PosixSourceAccessor accessor;
HashResult narHashAndSize = hashPath(
accessor, CanonPath { actualPath },
FileIngestionMethod::Recursive, HashAlgorithm::SHA256);
FileSerialisationMethod::Recursive, HashAlgorithm::SHA256);
newInfo0.narHash = narHashAndSize.first;
newInfo0.narSize = narHashAndSize.second;
}
@@ -2515,7 +2531,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
PosixSourceAccessor accessor;
HashResult narHashAndSize = hashPath(
accessor, CanonPath { actualPath },
FileIngestionMethod::Recursive, HashAlgorithm::SHA256);
FileSerialisationMethod::Recursive, HashAlgorithm::SHA256);
ValidPathInfo newInfo0 { requiredFinalPath, narHashAndSize.first };
newInfo0.narSize = narHashAndSize.second;
auto refs = rewriteRefs();
+2 -2
View File
@@ -529,11 +529,11 @@ bool Worker::pathContentsGood(const StorePath & path)
if (!pathExists(store.printStorePath(path)))
res = false;
else {
HashResult current = hashPath(
Hash current = hashPath(
*store.getFSAccessor(), CanonPath { store.printStorePath(path) },
FileIngestionMethod::Recursive, info->narHash.algo);
Hash nullHash(HashAlgorithm::SHA256);
res = info->narHash == nullHash || info->narHash == current.first;
res = info->narHash == nullHash || info->narHash == current;
}
pathContentsGoodCache.insert_or_assign(path, res);
if (!res)
+14 -3
View File
@@ -11,6 +11,9 @@ std::string_view makeFileIngestionPrefix(FileIngestionMethod m)
return "";
case FileIngestionMethod::Recursive:
return "r:";
case FileIngestionMethod::Git:
experimentalFeatureSettings.require(Xp::GitHashing);
return "git:";
default:
throw Error("impossible, caught both cases");
}
@@ -51,6 +54,10 @@ ContentAddressMethod ContentAddressMethod::parsePrefix(std::string_view & m)
if (splitPrefix(m, "r:")) {
return FileIngestionMethod::Recursive;
}
else if (splitPrefix(m, "git:")) {
experimentalFeatureSettings.require(Xp::GitHashing);
return FileIngestionMethod::Git;
}
else if (splitPrefix(m, "text:")) {
return TextIngestionMethod {};
}
@@ -111,10 +118,10 @@ static std::pair<ContentAddressMethod, HashAlgorithm> parseContentAddressMethodP
}
auto parseHashAlgorithm_ = [&](){
auto hashTypeRaw = splitPrefixTo(rest, ':');
if (!hashTypeRaw)
auto hashAlgoRaw = splitPrefixTo(rest, ':');
if (!hashAlgoRaw)
throw UsageError("content address hash must be in form '<algo>:<hash>', but found: %s", wholeInput);
HashAlgorithm hashAlgo = parseHashAlgo(*hashTypeRaw);
HashAlgorithm hashAlgo = parseHashAlgo(*hashAlgoRaw);
return hashAlgo;
};
@@ -131,6 +138,10 @@ static std::pair<ContentAddressMethod, HashAlgorithm> parseContentAddressMethodP
auto method = FileIngestionMethod::Flat;
if (splitPrefix(rest, "r:"))
method = FileIngestionMethod::Recursive;
else if (splitPrefix(rest, "git:")) {
experimentalFeatureSettings.require(Xp::GitHashing);
method = FileIngestionMethod::Git;
}
HashAlgorithm hashAlgo = parseHashAlgorithm_();
return {
std::move(method),
+4 -4
View File
@@ -91,17 +91,17 @@ struct ContentAddressMethod
std::string_view renderPrefix() const;
/**
* Parse a content addressing method and hash type.
* Parse a content addressing method and hash algorithm.
*/
static std::pair<ContentAddressMethod, HashAlgorithm> parseWithAlgo(std::string_view rawCaMethod);
/**
* Render a content addressing method and hash type in a
* Render a content addressing method and hash algorithm in a
* nicer way, prefixing both cases.
*
* The rough inverse of `parse()`.
*/
std::string renderWithAlgo(HashAlgorithm ht) const;
std::string renderWithAlgo(HashAlgorithm ha) const;
/**
* Get the underlying way to content-address file system objects.
@@ -127,7 +127,7 @@ struct ContentAddressMethod
* ‘text:sha256:<sha256 hash of file contents>’
*
* - `FixedIngestionMethod`:
* ‘fixed:<r?>:<hash type>:<hash of file contents>’
* ‘fixed:<r?>:<hash algorithm>:<hash of file contents>’
*/
struct ContentAddress
{
+33 -25
View File
@@ -13,6 +13,7 @@
#include "archive.hh"
#include "derivations.hh"
#include "args.hh"
#include "git.hh"
namespace nix::daemon {
@@ -400,11 +401,25 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
logger->startWork();
auto pathInfo = [&]() {
// NB: FramedSource must be out of scope before logger->stopWork();
auto [contentAddressMethod, hashAlgo_] = ContentAddressMethod::parseWithAlgo(camStr);
auto hashAlgo = hashAlgo_; // work around clang bug
auto [contentAddressMethod, hashAlgo] = ContentAddressMethod::parseWithAlgo(camStr);
FramedSource source(from);
FileSerialisationMethod dumpMethod;
switch (contentAddressMethod.getFileIngestionMethod()) {
case FileIngestionMethod::Flat:
dumpMethod = FileSerialisationMethod::Flat;
break;
case FileIngestionMethod::Recursive:
dumpMethod = FileSerialisationMethod::Recursive;
break;
case FileIngestionMethod::Git:
// Use NAR; Git is not a serialization method
dumpMethod = FileSerialisationMethod::Recursive;
break;
default:
assert(false);
}
// TODO these two steps are essentially RemoteStore::addCAToStore. Move it up to Store.
auto path = store->addToStoreFromDump(source, name, contentAddressMethod, hashAlgo, refs, repair);
auto path = store->addToStoreFromDump(source, name, dumpMethod, contentAddressMethod, hashAlgo, refs, repair);
return store->queryPathInfo(path);
}();
logger->stopWork();
@@ -430,30 +445,23 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
hashAlgo = parseHashAlgo(hashAlgoRaw);
}
// Old protocol always sends NAR, regardless of hashing method
auto dumpSource = sinkToSource([&](Sink & saved) {
if (method == FileIngestionMethod::Recursive) {
/* We parse the NAR dump through into `saved` unmodified,
so why all this extra work? We still parse the NAR so
that we aren't sending arbitrary data to `saved`
unwittingly`, and we know when the NAR ends so we don't
consume the rest of `from` and can't parse another
command. (We don't trust `addToStoreFromDump` to not
eagerly consume the entire stream it's given, past the
length of the Nar. */
TeeSource savedNARSource(from, saved);
NullFileSystemObjectSink sink; /* just parse the NAR */
parseDump(sink, savedNARSource);
} else {
/* Incrementally parse the NAR file, stripping the
metadata, and streaming the sole file we expect into
`saved`. */
RegularFileSink savedRegular { saved };
parseDump(savedRegular, from);
if (!savedRegular.regular) throw Error("regular file expected");
}
/* We parse the NAR dump through into `saved` unmodified,
so why all this extra work? We still parse the NAR so
that we aren't sending arbitrary data to `saved`
unwittingly`, and we know when the NAR ends so we don't
consume the rest of `from` and can't parse another
command. (We don't trust `addToStoreFromDump` to not
eagerly consume the entire stream it's given, past the
length of the Nar. */
TeeSource savedNARSource(from, saved);
NullFileSystemObjectSink sink; /* just parse the NAR */
parseDump(sink, savedNARSource);
});
logger->startWork();
auto path = store->addToStoreFromDump(*dumpSource, baseName, method, hashAlgo);
auto path = store->addToStoreFromDump(
*dumpSource, baseName, FileSerialisationMethod::Recursive, method, hashAlgo);
logger->stopWork();
to << store->printStorePath(path);
@@ -485,7 +493,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
logger->startWork();
auto path = ({
StringSource source { s };
store->addToStoreFromDump(source, suffix, TextIngestionMethod {}, HashAlgorithm::SHA256, refs, NoRepair);
store->addToStoreFromDump(source, suffix, FileSerialisationMethod::Flat, TextIngestionMethod {}, HashAlgorithm::SHA256, refs, NoRepair);
});
logger->stopWork();
to << store->printStorePath(path);
+2 -2
View File
@@ -150,7 +150,7 @@ StorePath writeDerivation(Store & store,
})
: ({
StringSource s { contents };
store.addToStoreFromDump(s, suffix, TextIngestionMethod {}, HashAlgorithm::SHA256, references, repair);
store.addToStoreFromDump(s, suffix, FileSerialisationMethod::Flat, TextIngestionMethod {}, HashAlgorithm::SHA256, references, repair);
});
}
@@ -701,7 +701,7 @@ DerivationType BasicDerivation::type() const
floatingHashAlgo = dof.hashAlgo;
} else {
if (*floatingHashAlgo != dof.hashAlgo)
throw Error("all floating outputs must use the same hash type");
throw Error("all floating outputs must use the same hash algorithm");
}
},
[&](const DerivationOutput::Deferred &) {
+2 -1
View File
@@ -61,7 +61,8 @@ struct DummyStore : public virtual DummyStoreConfig, public virtual Store
virtual StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method = FileIngestionMethod::Recursive,
FileSerialisationMethod dumpMethod = FileSerialisationMethod::Recursive,
ContentAddressMethod hashMethod = FileIngestionMethod::Recursive,
HashAlgorithm hashAlgo = HashAlgorithm::SHA256,
const StorePathSet & references = StorePathSet(),
RepairFlag repair = NoRepair) override
+21 -8
View File
@@ -106,6 +106,8 @@ struct curlFileTransfer : public FileTransfer
this->result.data.append(data);
})
{
result.urls.push_back(request.uri);
requestHeaders = curl_slist_append(requestHeaders, "Accept-Encoding: zstd, br, gzip, deflate, bzip2, xz");
if (!request.expectedETag.empty())
requestHeaders = curl_slist_append(requestHeaders, ("If-None-Match: " + request.expectedETag).c_str());
@@ -182,6 +184,14 @@ struct curlFileTransfer : public FileTransfer
return ((TransferItem *) userp)->writeCallback(contents, size, nmemb);
}
void appendCurrentUrl()
{
char * effectiveUriCStr = nullptr;
curl_easy_getinfo(req, CURLINFO_EFFECTIVE_URL, &effectiveUriCStr);
if (effectiveUriCStr && *result.urls.rbegin() != effectiveUriCStr)
result.urls.push_back(effectiveUriCStr);
}
size_t headerCallback(void * contents, size_t size, size_t nmemb)
{
size_t realSize = size * nmemb;
@@ -196,6 +206,7 @@ struct curlFileTransfer : public FileTransfer
statusMsg = trim(match.str(1));
acceptRanges = false;
encoding = "";
appendCurrentUrl();
} else {
auto i = line.find(':');
@@ -360,14 +371,11 @@ struct curlFileTransfer : public FileTransfer
{
auto httpStatus = getHTTPStatus();
char * effectiveUriCStr = nullptr;
curl_easy_getinfo(req, CURLINFO_EFFECTIVE_URL, &effectiveUriCStr);
if (effectiveUriCStr)
result.effectiveUri = effectiveUriCStr;
debug("finished %s of '%s'; curl status = %d, HTTP status = %d, body = %d bytes",
request.verb(), request.uri, code, httpStatus, result.bodySize);
appendCurrentUrl();
if (decompressionSink) {
try {
decompressionSink->finish();
@@ -779,7 +787,10 @@ FileTransferResult FileTransfer::upload(const FileTransferRequest & request)
return enqueueFileTransfer(request).get();
}
void FileTransfer::download(FileTransferRequest && request, Sink & sink)
void FileTransfer::download(
FileTransferRequest && request,
Sink & sink,
std::function<void(FileTransferResult)> resultCallback)
{
/* Note: we can't call 'sink' via request.dataCallback, because
that would cause the sink to execute on the fileTransfer
@@ -829,11 +840,13 @@ void FileTransfer::download(FileTransferRequest && request, Sink & sink)
};
enqueueFileTransfer(request,
{[_state](std::future<FileTransferResult> fut) {
{[_state, resultCallback{std::move(resultCallback)}](std::future<FileTransferResult> fut) {
auto state(_state->lock());
state->quit = true;
try {
fut.get();
auto res = fut.get();
if (resultCallback)
resultCallback(std::move(res));
} catch (...) {
state->exc = std::current_exception();
}
+28 -5
View File
@@ -75,14 +75,34 @@ struct FileTransferRequest
struct FileTransferResult
{
/**
* Whether this is a cache hit (i.e. the ETag supplied in the
* request is still valid). If so, `data` is empty.
*/
bool cached = false;
/**
* The ETag of the object.
*/
std::string etag;
std::string effectiveUri;
/**
* All URLs visited in the redirect chain.
*/
std::vector<std::string> urls;
/**
* The response body.
*/
std::string data;
uint64_t bodySize = 0;
/* An "immutable" URL for this resource (i.e. one whose contents
will never change), as returned by the `Link: <url>;
rel="immutable"` header. */
/**
* An "immutable" URL for this resource (i.e. one whose contents
* will never change), as returned by the `Link: <url>;
* rel="immutable"` header.
*/
std::optional<std::string> immutableUrl;
};
@@ -116,7 +136,10 @@ struct FileTransfer
* Download a file, writing its data to a sink. The sink will be
* invoked on the thread of the caller.
*/
void download(FileTransferRequest && request, Sink & sink);
void download(
FileTransferRequest && request,
Sink & sink,
std::function<void(FileTransferResult)> resultCallback = {});
enum Error { NotFound, Forbidden, Misc, Transient, Interrupted };
};
+2 -2
View File
@@ -1094,8 +1094,8 @@ public:
this, {}, "hashed-mirrors",
R"(
A list of web servers used by `builtins.fetchurl` to obtain files by
hash. Given a hash type *ht* and a base-16 hash *h*, Nix will try to
download the file from *hashed-mirror*/*ht*/*h*. This allows files to
hash. Given a hash algorithm *ha* and a base-16 hash *h*, Nix will try to
download the file from *hashed-mirror*/*ha*/*h*. This allows files to
be downloaded even if they have disappeared from their original URI.
For example, given an example mirror `http://tarballs.nixos.org/`,
when building the derivation
+2 -1
View File
@@ -72,7 +72,8 @@ struct LegacySSHStore : public virtual LegacySSHStoreConfig, public virtual Stor
virtual StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method = FileIngestionMethod::Recursive,
FileSerialisationMethod dumpMethod = FileSerialisationMethod::Recursive,
ContentAddressMethod hashMethod = FileIngestionMethod::Recursive,
HashAlgorithm hashAlgo = HashAlgorithm::SHA256,
const StorePathSet & references = StorePathSet(),
RepairFlag repair = NoRepair) override
+1 -1
View File
@@ -43,7 +43,7 @@ public:
LocalFSStore(const Params & params);
void narFromPath(const StorePath & path, Sink & sink) override;
ref<SourceAccessor> getFSAccessor(bool requireValidPath) override;
ref<SourceAccessor> getFSAccessor(bool requireValidPath = true) override;
/**
* Creates symlink from the `gcRoot` to the `storePath` and
+56 -22
View File
@@ -1,5 +1,6 @@
#include "local-store.hh"
#include "globals.hh"
#include "git.hh"
#include "archive.hh"
#include "pathlocks.hh"
#include "worker-protocol.hh"
@@ -1097,19 +1098,29 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
if (info.ca) {
auto & specified = *info.ca;
auto actualHash = ({
HashModuloSink caSink {
specified.hash.algo,
std::string { info.path.hashPart() },
};
PosixSourceAccessor accessor;
dumpPath(
*getFSAccessor(false),
CanonPath { printStorePath(info.path) },
caSink,
specified.method.getFileIngestionMethod());
auto accessor = getFSAccessor(false);
CanonPath path { printStorePath(info.path) };
Hash h { HashAlgorithm::SHA256 }; // throwaway def to appease C++
auto fim = specified.method.getFileIngestionMethod();
switch (fim) {
case FileIngestionMethod::Flat:
case FileIngestionMethod::Recursive:
{
HashModuloSink caSink {
specified.hash.algo,
std::string { info.path.hashPart() },
};
dumpPath(*accessor, path, caSink, (FileSerialisationMethod) fim);
h = caSink.finish().first;
break;
}
case FileIngestionMethod::Git:
h = git::dumpHash(specified.hash.algo, *accessor, path).hash;
break;
}
ContentAddress {
.method = specified.method,
.hash = caSink.finish().first,
.hash = std::move(h),
};
});
if (specified.hash != actualHash.hash) {
@@ -1137,7 +1148,8 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
StorePath LocalStore::addToStoreFromDump(
Source & source0,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair)
@@ -1190,7 +1202,13 @@ StorePath LocalStore::addToStoreFromDump(
Path tempDir;
AutoCloseFD tempDirFd;
if (!inMemory) {
bool methodsMatch = ContentAddressMethod(FileIngestionMethod(dumpMethod)) == hashMethod;
/* If the methods don't match, our streaming hash of the dump is the
wrong sort, and we need to rehash. */
bool inMemoryAndDontNeedRestore = inMemory && methodsMatch;
if (!inMemoryAndDontNeedRestore) {
/* Drain what we pulled so far, and then keep on pulling */
StringSource dumpSource { dump };
ChainSource bothSource { dumpSource, source };
@@ -1199,17 +1217,23 @@ StorePath LocalStore::addToStoreFromDump(
delTempDir = std::make_unique<AutoDelete>(tempDir);
tempPath = tempDir + "/x";
restorePath(tempPath, bothSource, method.getFileIngestionMethod());
restorePath(tempPath, bothSource, dumpMethod);
dumpBuffer.reset();
dump = {};
}
auto [hash, size] = hashSink->finish();
auto [dumpHash, size] = hashSink->finish();
PosixSourceAccessor accessor;
auto desc = ContentAddressWithReferences::fromParts(
method,
hash,
hashMethod,
methodsMatch
? dumpHash
: hashPath(
accessor, CanonPath { tempPath },
hashMethod.getFileIngestionMethod(), hashAlgo),
{
.others = references,
// caller is not capable of creating a self-reference, because this is content-addressed without modulus
@@ -1235,10 +1259,20 @@ StorePath LocalStore::addToStoreFromDump(
autoGC();
if (inMemory) {
if (inMemoryAndDontNeedRestore) {
StringSource dumpSource { dump };
/* Restore from the buffer in memory. */
restorePath(realPath, dumpSource, method.getFileIngestionMethod());
auto fim = hashMethod.getFileIngestionMethod();
switch (fim) {
case FileIngestionMethod::Flat:
case FileIngestionMethod::Recursive:
restorePath(realPath, dumpSource, (FileSerialisationMethod) fim);
break;
case FileIngestionMethod::Git:
// doesn't correspond to serialization method, so
// this should be unreachable
assert(false);
}
} else {
/* Move the temporary path we restored above. */
moveFile(tempPath, realPath);
@@ -1246,8 +1280,8 @@ StorePath LocalStore::addToStoreFromDump(
/* For computing the nar hash. In recursive SHA-256 mode, this
is the same as the store hash, so no need to do it again. */
auto narHash = std::pair { hash, size };
if (method != FileIngestionMethod::Recursive || hashAlgo != HashAlgorithm::SHA256) {
auto narHash = std::pair { dumpHash, size };
if (dumpMethod != FileSerialisationMethod::Recursive || hashAlgo != HashAlgorithm::SHA256) {
HashSink narSink { HashAlgorithm::SHA256 };
dumpPath(realPath, narSink);
narHash = narSink.finish();
@@ -1367,7 +1401,7 @@ bool LocalStore::verifyStore(bool checkContents, RepairFlag repair)
PosixSourceAccessor accessor;
std::string hash = hashPath(
accessor, CanonPath { linkPath },
FileIngestionMethod::Recursive, HashAlgorithm::SHA256).first.to_string(HashFormat::Nix32, false);
FileIngestionMethod::Recursive, HashAlgorithm::SHA256).to_string(HashFormat::Nix32, false);
if (hash != link.name) {
printError("link '%s' was modified! expected hash '%s', got '%s'",
linkPath, link.name, hash);
+2 -1
View File
@@ -180,7 +180,8 @@ public:
StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair) override;
+2 -2
View File
@@ -151,7 +151,7 @@ void LocalStore::optimisePath_(Activity * act, OptimiseStats & stats,
PosixSourceAccessor accessor;
hashPath(
accessor, CanonPath { path },
FileIngestionMethod::Recursive, HashAlgorithm::SHA256).first;
FileSerialisationMethod::Recursive, HashAlgorithm::SHA256).first;
});
debug("'%1%' has hash '%2%'", path, hash.to_string(HashFormat::Nix32, true));
@@ -166,7 +166,7 @@ void LocalStore::optimisePath_(Activity * act, OptimiseStats & stats,
PosixSourceAccessor accessor;
hashPath(
accessor, CanonPath { linkPath },
FileIngestionMethod::Recursive, HashAlgorithm::SHA256).first;
FileSerialisationMethod::Recursive, HashAlgorithm::SHA256).first;
})))
{
// XXX: Consider overwriting linkPath with our valid version.
+21 -2
View File
@@ -13,6 +13,7 @@
#include "derivations.hh"
#include "pool.hh"
#include "finally.hh"
#include "git.hh"
#include "logging.hh"
#include "callback.hh"
#include "filetransfer.hh"
@@ -508,12 +509,30 @@ ref<const ValidPathInfo> RemoteStore::addCAToStore(
StorePath RemoteStore::addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method,
FileSerialisationMethod dumpMethod,
ContentAddressMethod hashMethod,
HashAlgorithm hashAlgo,
const StorePathSet & references,
RepairFlag repair)
{
return addCAToStore(dump, name, method, hashAlgo, references, repair)->path;
FileSerialisationMethod fsm;
switch (hashMethod.getFileIngestionMethod()) {
case FileIngestionMethod::Flat:
fsm = FileSerialisationMethod::Flat;
break;
case FileIngestionMethod::Recursive:
fsm = FileSerialisationMethod::Recursive;
break;
case FileIngestionMethod::Git:
// Use NAR; Git is not a serialization method
fsm = FileSerialisationMethod::Recursive;
break;
default:
assert(false);
}
if (fsm != dumpMethod)
unsupported("RemoteStore::addToStoreFromDump doesn't support this `dumpMethod` `hashMethod` combination");
return addCAToStore(dump, name, hashMethod, hashAlgo, references, repair)->path;
}
+3 -2
View File
@@ -87,7 +87,8 @@ public:
StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method = FileIngestionMethod::Recursive,
FileSerialisationMethod dumpMethod = FileSerialisationMethod::Recursive,
ContentAddressMethod hashMethod = FileIngestionMethod::Recursive,
HashAlgorithm hashAlgo = HashAlgorithm::SHA256,
const StorePathSet & references = StorePathSet(),
RepairFlag repair = NoRepair) override;
@@ -184,7 +185,7 @@ protected:
friend struct ConnectionHandle;
virtual ref<SourceAccessor> getFSAccessor(bool requireValidPath) override;
virtual ref<SourceAccessor> getFSAccessor(bool requireValidPath = true) override;
virtual void narFromPath(const StorePath & path, Sink & sink) override;
+24 -6
View File
@@ -12,7 +12,9 @@
#include "references.hh"
#include "archive.hh"
#include "callback.hh"
#include "git.hh"
#include "remote-store.hh"
#include "posix-source-accessor.hh"
// FIXME this should not be here, see TODO below on
// `addMultipleToStore`.
#include "worker-protocol.hh"
@@ -119,6 +121,9 @@ static std::string makeType(
StorePath StoreDirConfig::makeFixedOutputPath(std::string_view name, const FixedOutputInfo & info) const
{
if (info.method == FileIngestionMethod::Git && info.hash.algo != HashAlgorithm::SHA1)
throw Error("Git file ingestion must use SHA-1 hash");
if (info.hash.algo == HashAlgorithm::SHA256 && info.method == FileIngestionMethod::Recursive) {
return makeStorePath(makeType(*this, "source", info.references), info.hash, name);
} else {
@@ -166,7 +171,7 @@ std::pair<StorePath, Hash> StoreDirConfig::computeStorePath(
const StorePathSet & references,
PathFilter & filter) const
{
auto h = hashPath(accessor, path, method.getFileIngestionMethod(), hashAlgo, filter).first;
auto h = hashPath(accessor, path, method.getFileIngestionMethod(), hashAlgo, filter);
return {
makeFixedOutputPathFromCA(
name,
@@ -192,10 +197,23 @@ StorePath Store::addToStore(
PathFilter & filter,
RepairFlag repair)
{
FileSerialisationMethod fsm;
switch (method.getFileIngestionMethod()) {
case FileIngestionMethod::Flat:
fsm = FileSerialisationMethod::Flat;
break;
case FileIngestionMethod::Recursive:
fsm = FileSerialisationMethod::Recursive;
break;
case FileIngestionMethod::Git:
// Use NAR; Git is not a serialization method
fsm = FileSerialisationMethod::Recursive;
break;
}
auto source = sinkToSource([&](Sink & sink) {
dumpPath(accessor, path, sink, method.getFileIngestionMethod(), filter);
dumpPath(accessor, path, sink, fsm, filter);
});
return addToStoreFromDump(*source, name, method, hashAlgo, references, repair);
return addToStoreFromDump(*source, name, fsm, method, hashAlgo, references, repair);
}
void Store::addMultipleToStore(
@@ -355,9 +373,7 @@ ValidPathInfo Store::addToStoreSlow(
NullFileSystemObjectSink blank;
auto & parseSink = method.getFileIngestionMethod() == FileIngestionMethod::Flat
? (FileSystemObjectSink &) fileSink
: method.getFileIngestionMethod() == FileIngestionMethod::Recursive
? (FileSystemObjectSink &) blank
: (abort(), (FileSystemObjectSink &)*(FileSystemObjectSink *)nullptr); // handled both cases
: (FileSystemObjectSink &) blank; // for recursive or git we do recursive
/* The information that flows from tapped (besides being replicated in
narSink), is now put in parseSink. */
@@ -369,6 +385,8 @@ ValidPathInfo Store::addToStoreSlow(
auto hash = method == FileIngestionMethod::Recursive && hashAlgo == HashAlgorithm::SHA256
? narHash
: method == FileIngestionMethod::Git
? git::dumpHash(hashAlgo, accessor, srcPath).hash
: caHashSink.finish().first;
if (expectedCAHash && expectedCAHash != hash)
+13 -4
View File
@@ -466,14 +466,23 @@ public:
* in `dump`, which is either a NAR serialisation (if recursive ==
* true) or simply the contents of a regular file (if recursive ==
* false).
* `dump` may be drained
*
* \todo remove?
* `dump` may be drained.
*
* @param dumpMethod What serialisation format is `dump`, i.e. how
* to deserialize it. Must either match hashMethod or be
* `FileSerialisationMethod::Recursive`.
*
* @param hashMethod How content addressing? Need not match be the
* same as `dumpMethod`.
*
* @todo remove?
*/
virtual StorePath addToStoreFromDump(
Source & dump,
std::string_view name,
ContentAddressMethod method = FileIngestionMethod::Recursive,
FileSerialisationMethod dumpMethod = FileSerialisationMethod::Recursive,
ContentAddressMethod hashMethod = FileIngestionMethod::Recursive,
HashAlgorithm hashAlgo = HashAlgorithm::SHA256,
const StorePathSet & references = StorePathSet(),
RepairFlag repair = NoRepair) = 0;
@@ -772,7 +781,7 @@ protected:
* Helper for methods that are not unsupported: this is used for
* default definitions for virtual methods that are meant to be overriden.
*
* \todo Using this should be a last resort. It is better to make
* @todo Using this should be a last resort. It is better to make
* the method "virtual pure" and/or move it to a subclass.
*/
[[noreturn]] void unsupported(const std::string & op)
+1 -1
View File
@@ -35,7 +35,7 @@ public:
static std::set<std::string> uriSchemes()
{ return {"unix"}; }
ref<SourceAccessor> getFSAccessor(bool requireValidPath) override
ref<SourceAccessor> getFSAccessor(bool requireValidPath = true) override
{ return LocalFSStore::getFSAccessor(requireValidPath); }
void narFromPath(const StorePath & path, Sink & sink) override
-67
View File
@@ -544,73 +544,6 @@ nlohmann::json Args::toJSON()
return res;
}
static void hashFormatCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & format : hashFormats) {
if (hasPrefix(format, prefix)) {
completions.add(format);
}
}
}
Args::Flag Args::Flag::mkHashFormatFlagWithDefault(std::string &&longName, HashFormat * hf) {
assert(*hf == nix::HashFormat::SRI);
return Flag{
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`). Default: `sri`.",
.labels = {"hash-format"},
.handler = {[hf](std::string s) {
*hf = parseHashFormat(s);
}},
.completer = hashFormatCompleter,
};
}
Args::Flag Args::Flag::mkHashFormatOptFlag(std::string && longName, std::optional<HashFormat> * ohf) {
return Flag{
.longName = std::move(longName),
.description = "Hash format (`base16`, `nix32`, `base64`, `sri`).",
.labels = {"hash-format"},
.handler = {[ohf](std::string s) {
*ohf = std::optional<HashFormat>{parseHashFormat(s)};
}},
.completer = hashFormatCompleter,
};
}
static void hashAlgoCompleter(AddCompletions & completions, size_t index, std::string_view prefix)
{
for (auto & algo : hashAlgorithms)
if (hasPrefix(algo, prefix))
completions.add(algo);
}
Args::Flag Args::Flag::mkHashAlgoFlag(std::string && longName, HashAlgorithm * ha)
{
return Flag{
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`).",
.labels = {"hash-algo"},
.handler = {[ha](std::string s) {
*ha = parseHashAlgo(s);
}},
.completer = hashAlgoCompleter,
};
}
Args::Flag Args::Flag::mkHashAlgoOptFlag(std::string && longName, std::optional<HashAlgorithm> * oha)
{
return Flag{
.longName = std::move(longName),
.description = "Hash algorithm (`md5`, `sha1`, `sha256`, or `sha512`). Can be omitted for SRI hashes.",
.labels = {"hash-algo"},
.handler = {[oha](std::string s) {
*oha = std::optional<HashAlgorithm>{parseHashAlgo(s)};
}},
.completer = hashAlgoCompleter,
};
}
static void _completePath(AddCompletions & completions, std::string_view prefix, bool onlyDirs)
{
completions.setType(Completions::Type::Filenames);
+8 -11
View File
@@ -155,6 +155,8 @@ protected:
*/
using CompleterClosure = std::function<CompleterFun>;
public:
/**
* Description of flags / options
*
@@ -175,19 +177,10 @@ protected:
CompleterClosure completer;
std::optional<ExperimentalFeature> experimentalFeature;
static Flag mkHashAlgoFlag(std::string && longName, HashAlgorithm * ha);
static Flag mkHashAlgoFlag(HashAlgorithm * ha) {
return mkHashAlgoFlag("hash-algo", ha);
}
static Flag mkHashAlgoOptFlag(std::string && longName, std::optional<HashAlgorithm> * oha);
static Flag mkHashAlgoOptFlag(std::optional<HashAlgorithm> * oha) {
return mkHashAlgoOptFlag("hash-algo", oha);
}
static Flag mkHashFormatFlagWithDefault(std::string && longName, HashFormat * hf);
static Flag mkHashFormatOptFlag(std::string && longName, std::optional<HashFormat> * ohf);
};
protected:
/**
* Index of all registered "long" flag descriptions (flags like
* `--long`).
@@ -206,6 +199,8 @@ protected:
*/
virtual bool processFlag(Strings::iterator & pos, Strings::iterator end);
public:
/**
* Description of positional arguments
*
@@ -220,6 +215,8 @@ protected:
CompleterClosure completer;
};
protected:
/**
* Queue of expected positional argument forms.
*
+1 -1
View File
@@ -8,7 +8,7 @@ CanonPath CanonPath::root = CanonPath("/");
static std::string absPathPure(std::string_view path)
{
return canonPathInner(path, [](auto &, auto &){});
return canonPathInner<UnixPathTrait>(path, [](auto &, auto &){});
}
CanonPath::CanonPath(std::string_view raw)
+1 -1
View File
@@ -4,7 +4,7 @@
*
* Template implementations (as opposed to mere declarations).
*
* This file is an exmample of the "impl.hh" pattern. See the
* This file is an example of the "impl.hh" pattern. See the
* contributing guide.
*
* One only needs to include this when one is declaring a
+4 -8
View File
@@ -84,7 +84,9 @@ void AbstractConfig::reapplyUnknownSettings()
void Config::getSettings(std::map<std::string, SettingInfo> & res, bool overriddenOnly)
{
for (const auto & opt : _settings)
if (!opt.second.isAlias && (!overriddenOnly || opt.second.setting->overridden))
if (!opt.second.isAlias
&& (!overriddenOnly || opt.second.setting->overridden)
&& experimentalFeatureSettings.isEnabled(opt.second.setting->experimentalFeature))
res.emplace(opt.first, SettingInfo{opt.second.setting->to_string(), opt.second.setting->description});
}
@@ -334,14 +336,8 @@ template<> std::set<ExperimentalFeature> BaseSetting<std::set<ExperimentalFeatur
for (auto & s : tokenizeString<StringSet>(str)) {
if (auto thisXpFeature = parseExperimentalFeature(s); thisXpFeature) {
res.insert(thisXpFeature.value());
if (thisXpFeature.value() == Xp::Flakes) {
if (thisXpFeature.value() == Xp::Flakes)
res.insert(Xp::FetchTree);
res.insert(Xp::FetchTreeGit);
res.insert(Xp::FetchTreeUrls);
} else if (thisXpFeature.value() == Xp::FetchTree) {
res.insert(Xp::FetchTreeGit);
res.insert(Xp::FetchTreeUrls);
}
} else
warn("unknown experimental feature '%s'", s);
}
+3 -7
View File
@@ -11,9 +11,9 @@
namespace nix {
void BaseError::addTrace(std::shared_ptr<Pos> && e, HintFmt hint, bool frame)
void BaseError::addTrace(std::shared_ptr<Pos> && e, HintFmt hint)
{
err.traces.push_front(Trace { .pos = std::move(e), .hint = hint, .frame = frame });
err.traces.push_front(Trace { .pos = std::move(e), .hint = hint });
}
void throwExceptionSelfCheck(){
@@ -61,8 +61,7 @@ inline bool operator<(const Trace& lhs, const Trace& rhs)
// This formats a freshly formatted hint string and then throws it away, which
// shouldn't be much of a problem because it only runs when pos is equal, and this function is
// used for trace printing, which is infrequent.
return std::forward_as_tuple(lhs.hint.str(), lhs.frame)
< std::forward_as_tuple(rhs.hint.str(), rhs.frame);
return lhs.hint.str() < rhs.hint.str();
}
inline bool operator> (const Trace& lhs, const Trace& rhs) { return rhs < lhs; }
inline bool operator<=(const Trace& lhs, const Trace& rhs) { return !(lhs > rhs); }
@@ -373,7 +372,6 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
// prepended to each element of the trace
auto ellipsisIndent = " ";
bool frameOnly = false;
if (!einfo.traces.empty()) {
// Stack traces seen since we last printed a chunk of `duplicate frames
// omitted`.
@@ -384,7 +382,6 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
for (const auto & trace : einfo.traces) {
if (trace.hint.str().empty()) continue;
if (frameOnly && !trace.frame) continue;
if (!showTrace && count > 3) {
oss << "\n" << ANSI_WARNING "(stack trace truncated; use '--show-trace' to show the full trace)" ANSI_NORMAL << "\n";
@@ -400,7 +397,6 @@ std::ostream & showErrorInfo(std::ostream & out, const ErrorInfo & einfo, bool s
printSkippedTracesMaybe(oss, ellipsisIndent, count, skippedTraces, tracesSeen);
count++;
frameOnly = trace.frame;
printTrace(oss, ellipsisIndent, count, trace);
}
+1 -2
View File
@@ -64,7 +64,6 @@ void printCodeLines(std::ostream & out,
struct Trace {
std::shared_ptr<Pos> pos;
HintFmt hint;
bool frame;
};
inline bool operator<(const Trace& lhs, const Trace& rhs);
@@ -162,7 +161,7 @@ public:
addTrace(std::move(e), HintFmt(std::string(fs), args...));
}
void addTrace(std::shared_ptr<Pos> && e, HintFmt hint, bool frame = false);
void addTrace(std::shared_ptr<Pos> && e, HintFmt hint);
bool hasTrace() const { return !err.traces.empty(); }
+7
View File
@@ -0,0 +1,7 @@
#include "exit.hh"
namespace nix {
Exit::~Exit() {}
}
+19
View File
@@ -0,0 +1,19 @@
#pragma once
#include <exception>
namespace nix {
/**
* Exit the program with a given exit code.
*/
class Exit : public std::exception
{
public:
int status;
Exit() : status(0) { }
explicit Exit(int status) : status(status) { }
virtual ~Exit();
};
}
+7 -17
View File
@@ -78,23 +78,13 @@ constexpr std::array<ExperimentalFeatureDetails, numXpFeatures> xpFeatureDetails
.tag = Xp::FetchTree,
.name = "fetch-tree",
.description = R"(
Backwards-compatibility alias for
[fetch-tree-git](#xp-feature-fetch-tree-git) and
[fetch-tree-urls](#xp-feature-fetch-tree-urls).
)",
},
{
.tag = Xp::FetchTreeGit,
.name = "fetch-tree-git",
.description = R"(
Enable the use of the `git` [`fetchTree`](@docroot@/language/builtins.md#builtins-fetchTree) fetcher.
)",
},
{
.tag = Xp::FetchTreeUrls,
.name = "fetch-tree-urls",
.description = R"(
Enable the use of the [URL-like syntax](@docroot@/command-ref/new-cli/nix3-flake.html#url-like-syntax) in [`fetchTree`](@docroot@/language/builtins.md#builtins-fetchTree).
Enable the use of the [`fetchTree`](@docroot@/language/builtins.md#builtins-fetchTree) built-in function in the Nix language.
`fetchTree` exposes a generic interface for fetching remote file system trees from different types of remote sources.
The [`flakes`](#xp-feature-flakes) feature flag always enables `fetch-tree`.
This built-in was previously guarded by the `flakes` experimental feature because of that overlap.
Enabling just this feature serves as a "release candidate", allowing users to try it out in isolation.
)",
},
{
-2
View File
@@ -21,8 +21,6 @@ enum struct ExperimentalFeature
ImpureDerivations,
Flakes,
FetchTree,
FetchTreeGit,
FetchTreeUrls,
NixCommand,
GitHashing,
RecursiveNix,
Loaded 100 of 149 files, more files were not shown because too many files have changed in this diff. Show more