Compare commits

..
Author SHA1 Message Date
Sergei Zimmerman 6dfeff04c1 local-binary-cache-store: Check that paths don't escape the binary cache directory
Previously arguments to getFile, upsertFile weren't checked
to be inside the root directory. It's not a very big issue since
substituters/stores are already a trusted component and can't be
specified without being a trusted user. Still, it's nice to validate
the necessary preconditions. Also changes the binaryCacheDir to be a
std::filesystem::path. Note the gotcha with absolute paths and operator/.
2026-02-24 19:53:09 +03:00
Sergei Zimmerman 3d9174cec5 libutil: Accept std::filesystem::path in readFile 2026-02-24 19:53:08 +03:00
John Ericson 5ce241cbfd Merge pull request #15307 from nix-windows/windows-local-shorthand
windows: add a separate local_shorthand_path test
2026-02-23 19:42:48 +00:00
John Ericson 3bfd64c3cd Merge pull request #15323 from obsidiansystems/fix-eio-readline
libutil: treat EIO as EOF in `readLine`
2026-02-23 19:30:19 +00:00
Brian McKenna 3f419cfa4e ParsedURL::path <-> std::filesystem::path, use in StoreReference
This missing URL functionality allow us to properly fix the path
shorthand for local store URLs test case.
2026-02-23 13:32:07 -05:00
Amaan Qureshi 994137dcf7 libutil: treat EIO as EOF in readLine
Reading from a pty master returns `EIO` once the slave side closes, however, `readLine` lets it propagate as an uncaught `SysError`, which causes spurious build failures in gvisor and similar sandboxed environments where pty teardown races differently. This commit catches `EIO` inside the read lambda and maps it to a zero-length read, reusing the existing EOF path.
2026-02-23 13:30:05 -05:00
Sergei Zimmerman 9242d74bc1 Merge pull request #15321 from obsidiansystems/writefull-interrupt-test
tests: add `writeFull` interrupt-handling regression test
2026-02-23 18:29:27 +00:00
Amaan Qureshi 6cddf03b5a tests: add writeFull interrupt-handling regression test
This commit verifies that `writeFull` with `allowInterrupts=false` completes
successfully when the interrupt flag is set. This prevents regressions
like the one fixed by #15255 where `write()` called `checkInterrupt()`
unconditionally.
2026-02-23 12:20:54 -05:00
Bernardo Meurer afccf1d2d3 Merge pull request #15256 from NixOS/dependabot/github_actions/cachix/install-nix-action-31.9.1
build(deps): bump cachix/install-nix-action from 31.9.0 to 31.9.1
2026-02-23 14:09:16 +00:00
Bernardo Meurer de16ef8be6 Merge pull request #15257 from NixOS/dependabot/github_actions/korthout/backport-action-4.1.0
build(deps): bump korthout/backport-action from 4.0.1 to 4.1.0
2026-02-23 14:08:51 +00:00
Sergei Zimmerman 7cd7930344 Merge pull request #15319 from xokdvium/fix-interrupts-write-full
libutil: Fix writeFull to respect allowInterrupts
2026-02-23 14:03:30 +00:00
Sergei Zimmerman 658c775f01 libutil: Fix writeFull to respect allowInterrupts
c0e849b696 broke interrupt handling since
writeFull started throwing Interrupted even when allowInterrupts was false.
This would lead to exceptions leaking out when they must not (for example during
progress bar shutdown).
2026-02-23 15:34:18 +03:00
Sergei Zimmerman b1ad42e6d5 Merge pull request #15242 from obsidiansystems/fix-prefetch-segfault
libstore: guard against empty archive in unpack paths
2026-02-22 19:14:11 +00:00
John Ericson 761139f31c Merge pull request #15314 from obsidiansystems/fix-registry-symlink
registry: fix symlinked flake registry files broken by convert-end-exes
2026-02-21 20:06:31 +00:00
Amaan Qureshi c6d93828bd registry: fix symlinked flake registry files broken by convert-end-exes
This commit reverts to using `getFSSourceAccessor()` so absolute symlink targets resolve correctly, since `makeFSSourceAccessor(path)` roots the accessor at `path` and can't follow symlinks that escape it.
2026-02-21 12:14:42 -05:00
Sergei Zimmerman 614072adcb Merge pull request #15286 from NixOS/failed-values-v2
Introduce a "failed" value type (v2)
2026-02-21 09:05:15 +00:00
John Ericson 42f6e9933d Merge pull request #15311 from obsidiansystems/convert-end-exes
libutil: return `std::filesystem::path` from XDG directory helpers
2026-02-21 04:22:26 +00:00
siddhantCodesandAmaan Qureshi 6808bfab92 libutil: return std::filesystem::path from XDG directory helpers
`getCacheDir`, `getConfigDir`, `getDataDir`, `getStateDir`, and related functions now return `std::filesystem::path` and use `getEnvOs` for native OS string handling, letting callsites replace string concatenation with `operator/` and drop the ad-hoc `namespace nix::fs` alias from six CLI files. `expandTilde` is fixed to strip both `~` and `/` before joining with `operator/` (an absolute right-hand operand silently replaces the left-hand side), and `ExecutablePath` gains `parseAppend` for incremental `PATH` construction.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-20 22:35:47 -05:00
John Ericson 92942071c7 Merge pull request #15310 from obsidiansystems/pathlocks-wine
`PathLocks` allow to fail on Wine
2026-02-21 03:32:18 +00:00
John Ericson 8899af09c1 Merge pull request #15308 from obsidiansystems/fix-ioport-init
Fix initialization of `ioport` in `Worker` on Windows
2026-02-21 02:48:26 +00:00
John Ericson 6363c1bf00 PathLocks allow to fail on Wine
It needs functionality Wine doesn't implement yet. So let's just have a
wine-only warn-and-keep-going.
2026-02-20 21:45:04 -05:00
John Ericson 5adb6a36b6 Fix initialization of ioport in Worker on Windows 2026-02-20 18:56:00 -05:00
John Ericson b10cb6596e Merge pull request #15305 from obsidiansystems/file-system-prep
Centralize I/O error handling and make read/write functions portable
2026-02-20 17:31:21 +00:00
John Ericson c0e849b696 Centralize I/O error handling and make read/write functions portable
- Improve existing `read` and `readOffset` wrappers:
  - Unix: Add `EINTR` retry handling and `checkInterrupt`
  - Windows: Handle `ERROR_BROKEN_PIPE` as EOF, add `checkInterrupt`

- Add `write` wrapper with same treatment (`EINTR` on Unix, `checkInterrupt`)

- Improve many `windows/file-descriptor.cc` error messages with
  `descriptorToPath`

- Move `readFull`, `readLine`, `writeFull` to common file, using the
  platform wrappers instead of duplicating platform-specific logic.
  These ones don't need any `EINTR` or interrupt checking either. They
  only have `EGAIN` checking as Unix-specific code, but this is a temp
  hack to be removed per #12688, so its fine that it goes in the
  platform-agnostic file for now.

- Add `retryOnBlock` helper to abstract `EAGAIN`/`EWOULDBLOCK` poll-and-retry
  logic (Unix only, needed for buildhook workaround #12688)

- Simplify `FdSource::readUnbuffered` to just call `nix::read`

- Remove now-dead `EINTR` handling from `drainFD` and `copyFdRange`

- `writeErr` better impl on Windows
2026-02-20 11:39:05 -05:00
John Ericson 2470b7981a Merge pull request #15277 from puffnfresh/windows/remove-profiles-symlink
LocalStore: stop creating outdated profiles symlink
2026-02-20 05:52:49 +00:00
John Ericson 5c1939e315 Merge pull request #15301 from obsidiansystems/file-system-prep
Misc file system fixes, especially for windows
2026-02-20 01:28:58 +00:00
Brian McKenna 9799023545 LocalStore: stop creating outdated profiles symlink
The gcroots/profiles link became outdated in
aeb810b01e when the GC code started
reading directly from /profiles, and gcroots/profiles was even
partially deleted in that commit.
2026-02-20 11:48:13 +11:00
John Ericson ae4e229c9f Change writeFile(AutoCloseFD &, ...) to take a Descriptor
The new signature is:
  writeFile(Descriptor fd, std::string_view s, FsSync sync = FsSync::No, const Path * origPath = nullptr)

This uses `descriptorToPath` if `origPath` is not provided.
2026-02-19 19:30:55 -05:00
Amaan Qureshi b63f5d1914 libutil: refactor AutoCloseFD::fsync into standalone syncDescriptor
`AutoCloseFD::fsync()` contained platform-specific logic behind CPP
guards. This extracts it into a free function `syncDescriptor(Descriptor)`
with separate Unix and Windows implementations, and makes
`AutoCloseFD::fsync()` an inline wrapper that delegates to it. The
Windows implementation uses `FlushFileBuffers` (returns `BOOL`, true on
success) while Unix uses `::fsync` (or `F_FULLFSYNC` on macOS), so
splitting them avoids conflating the two calling conventions.
2026-02-19 19:23:49 -05:00
John Ericson 31d87afc5a openFileEnsureBeneathNoSymlinks now returns AutoCloseFD
This indicates that they are returning an owned handle (i.e. the caller
 should --- and will, thanks to RAII --- close it).

 `ntOpenAt` and friends (internal) also use `AutoCloseFD` for the same
 reason.
2026-02-19 19:08:31 -05:00
John Ericson 204618c9d8 Misc unit test improvements 2026-02-19 17:26:38 -05:00
John Ericson f0d90d3bdb Create two more FSSourceAccessorTests
Make sure we're testing non-directory roots.
2026-02-19 17:25:54 -05:00
John Ericson 0730dcb4a8 Skip chmodIfNeeded test on Windows
It doesn't do anything on Windows, it appears.
2026-02-19 17:21:46 -05:00
John Ericson 9b363e1e5c libutil-tests: Use FS_ROOT macros consistently in file-system tests
Add FS_ROOT_NO_TRAILING_SLASH macro and update isInDir and isDirOrInDir tests to use FS_ROOT and FS_SEP macros for cross-platform compatibility.
2026-02-19 17:21:12 -05:00
John Ericson 3df67a8347 Merge pull request #15298 from dramforever/parse-derivation-empty-special-msg
Add special error message for empty derivation file
2026-02-19 20:12:38 +00:00
Robert Hensing dd4b73a44d Add rl-next/c-api-recoverable-errors 2026-02-19 14:23:05 +03:00
Sergei Zimmerman fd4eee9d62 libexpr-tests: Add ValuePrintingTests.vFailed 2026-02-19 14:23:04 +03:00
Sergei Zimmerman 100e9a4436 Add tests/f/lang/eval-fail-memoised-error-trace-not-mutated.nix 2026-02-19 14:23:03 +03:00
Robert Hensing 17f344cdda libexpr: Add recoverable errors
This provides an explicit API for call-fail-retry-succeed evaluation
flows, such as currently used in NixOps4.

An alternative design would simply reset the `Value` to the original
thunk instead of `tFailed` under the condition of catching a
`RecoverableEvalError`.
That is somewhat simpler, but I believe the presence of `tFailed` is
beneficial for possible use in the repl; being able to show the error
sooner, without re-evaluation.

The hasPos method is required in order to avoid an include loop.
2026-02-19 14:23:02 +03:00
Eelco DolstraandRobert Hensing c33d9e31cc Introduce a "failed" value type
In the multithreaded evaluator, it's possible for multiple threads to
wait on the same thunk. If evaluation of the thunk results in an
exception, the waiting threads shouldn't try to re-force the thunk.
Instead, they should rethrow the same exception, without duplicating
any work.

Therefore, there is now a new value type `tFailed` that stores an
std::exception_ptr. If evaluation of a thunk/app results in an
exception, `forceValue()` overwrites the value with a `tFailed`. If
`forceValue()` encounters a `tFailed`, it rethrows the exception. So
you normally never need to check for failed values (since forcing them
causes a rethrow).

Co-authored-by: Robert Hensing <robert@roberthensing.nl>
2026-02-19 14:23:01 +03:00
Sergei Zimmerman 89158eedb5 treewide: Make exceptions cloneable
This is needed to make it possible to store exceptions in failed values
with each new rethrow getting a fresh copy of the exception object.
2026-02-19 14:22:58 +03:00
dramforever c33c82f345 Add special error message for empty derivation file
This rather commonly occurs when the computer was uncleanly shut down
and fsync was not enabled. Show the user a more actionable message than
"expected string 'D'".
2026-02-19 17:34:57 +08:00
John Ericson 40abcebbe1 Merge pull request #15295 from NixOS/expr-op-update-dont-mutate-result
libexpr: Use temporary value in ExprOpUpdate::eval
2026-02-19 09:14:44 +00:00
John Ericson 3bf690a407 Merge pull request #15294 from NixOS/less-filesystem-to-string
libutil: Drop superflous .string() in pathExists and pathAccessible
2026-02-19 02:33:55 +00:00
Jörg Thalheim 360ff05e73 Merge pull request #15296 from Mic92/fix-nix-build
Reapply "Use the hash modulo in the derivation outputs"
2026-02-19 01:38:29 +00:00
Jörg Thalheim 100e7cc337 Reapply "Use the hash modulo in the derivation outputs"
This reverts commit 4f91e9599f.

This broke:

when I applied this pr, `--print-out-paths` wouldn't print anything:

```
shell-for-nix-env % nix build --print-out-paths .#legacyPackages.aarch64-darwin.homeConfigurations.macos.activationPackage
```

After dropping the patches from my fork, it does again.
```
shell-for-nix-env % nix build --print-out-paths .#legacyPackages.aarch64-darwin.homeConfigurations.macos.activationPackage
/nix/store/s8mlcalszdml0v8172w4hwqnx0m6477r-home-manager-generation
```
2026-02-19 01:50:17 +01:00
Sergei Zimmerman 247cc7013a libexpr: Use temporary value in ExprOpUpdate::eval 2026-02-19 02:49:10 +03:00
Sergei Zimmerman 9e865ae4ff libutil: Drop superflous .string() in pathExists and pathAccessible 2026-02-19 02:13:12 +03:00
Eelco Dolstra fef83c9f9c Merge pull request #15287 from NixOS/narinfo-cache-meta-ttl
Add setting narinfo-cache-meta-ttl
2026-02-18 22:13:23 +00:00
Eelco Dolstra 8491e26cd4 Merge pull request #15291 from roberth/fix-evalstate-shared-from-this-usage
fix(EvalState): Use make_shared for enable_shared_from_this compatibi…
2026-02-18 22:06:03 +00:00
Eelco Dolstra e5fa203d7f Add release note 2026-02-18 22:24:42 +01:00
Robert Hensing 987ecca24a fix(EvalState): Use make_shared for enable_shared_from_this compatibility
EvalState inherits from enable_shared_from_this (added in b4c24a29 for
debugRepl), but was being stack-allocated or created with make_unique
in several places. This causes bad_weak_ptr errors when shared_from_this
is called.

Convert all EvalState allocations to use make_shared.
2026-02-18 22:14:20 +01:00
Jörg Thalheim a98b43b994 Merge pull request #12464 from obsidiansystems/build-trace-rework
Realisations use regular drv paths again
2026-02-18 20:54:56 +00:00
Eelco Dolstra 6733f2e5ce Add setting narinfo-cache-meta-ttl
This makes the current hard-coded 7-day `nix-cache-info` TTL
configurable, making `--offline` and `--refresh` do the right thing.
2026-02-18 21:30:37 +01:00
John Ericson 08ce8dbfba Merge pull request #15283 from obsidiansystems/filesytem-error-improvements
Filesystem error improvements
2026-02-18 18:29:44 +00:00
John EricsonandSergei Zimmerman bbcf2041e1 File system error improvements
- Make `descriptorToPath` cross-platform (renamed from
  `windows::handleToPath`). Uses `/proc/self/fd` on Linux and
  `F_GETPATH` on macOS. Add `HAVE_F_GETPATH` meson check.

  This is based on 7226a116a0, which was
  removed in 479c356510, but is now
  introduced more judiciously.

- Unix error messages in `readFull`, `writeFull`, `readLine` now include
  file paths via `descriptorToPath`.

- Convert `std::filesystem::filesystem_error` to `SystemError`

  Wrappers like `readLink`, `createDirs`, `DirectoryIterator`, etc. now
  catch `std::filesystem::filesystem_error` and rethrow as `SystemError`
  with the error code preserved. This ensures consistent exception types
  throughout the codebase.

  Call sites that previously caught `filesystem_error` and rethrew with
  `throw;` now throw `SystemError(e.code(), ...)` instead.

  Some call sites can stop catching `filesystem_error` at all,
  because they only call the wrapped functions.

- Rework `SystemError` constructors to auto-append error message

  The public `SystemError(std::error_code, ...)` constructor now
  automatically appends `errorCode.message()` to the error message.
  A protected constructor takes an explicit error message string for
  subclasses.

  `SysError` delegates to the protected constructor with `strerror(errNo)`.
  `WinError` delegates with `renderError(lastError)` (now static).

  This removes the need to manually append `e.code().message()` at call
  sites when converting `filesystem_error` to `SystemError`.

- Use perfect forwarding (`Args &&...` with `std::forward`) consistently
  in `BaseError`, `SystemError`, `SysError`, and `WinError` constructors.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-02-18 12:29:11 -05:00
John Ericson 96bcf5928f Merge pull request #15273 from NixOS/more-robust-ubsan-macro
libutil: More robust check for NIX_UBSAN_ENABLED
2026-02-18 16:15:26 +00:00
Sergei Zimmerman db853cf4fb libutil: More robust check for NIX_UBSAN_ENABLED
In 3df91bea62 I forgot that the header
might get included out-of-tree with -Wundef. Let's make this a public
config option for libutil as it can affect function bodies in headers.
2026-02-18 17:33:51 +03:00
John Ericson 663db5b48b Merge pull request #15278 from puffnfresh/windows/bar-log-format
Windows: don't use bar log format
2026-02-18 05:14:27 +00:00
Brian McKenna c486e78235 Windows: don't use bar log format
Relies on terminal features that don't always work on Windows.
2026-02-18 14:35:35 +11:00
John Ericson 4fff871383 Merge pull request #15274 from obsidiansystems/tryToBuild-raii
libstore: refactor `tryToBuild` with coroutine lambdas and RAII
2026-02-17 22:10:42 +00:00
Amaan Qureshi b9acea908e libstore: refactor tryToBuild with coroutine lambdas and RAII
`tryToBuild` threaded a single `PathLocks outputLocks` by reference
across all build phases and managed a `std::unique_ptr<Activity> actLock`
with explicit `if (!actLock)` guards and `.reset()` calls around the hook
retry loop. This commit introduces coroutine lambdas for the three phases:
`tryHookLoop` owns a `PathLocks` in a scoped block for the first attempt
and per-iteration in the retry loop, `tryBuildLocally` acquires its own
`PathLocks`, and the hook-wait `Activity` is a stack variable scoped to
the postpone block.
2026-02-17 16:23:44 -05:00
John Ericson c3f0670b4e Merge pull request #15266 from obsidiansystems/fix-maxjobs-error
libstore: structured diagnostics for local build rejection
2026-02-17 18:39:58 +00:00
Amaan Qureshi 7cd4359a8b libstore: structured diagnostics for local build rejection
When `max-jobs = 0` and no remote builders are available, Nix reported
"required system or feature not available" even though the system and
features matched fine. The `canBuildLocally` lambda returned a plain
`bool`, conflating a configuration knob (`max-jobs = 0`) with actual
incompatibility (wrong platform, missing features). It also short-circuited
on the first failing check, so a user with both a platform mismatch and
missing features would only see one of the two.

This commit replaces the bool with a `LocalBuildRejection` struct whose
`WrongLocalStore` variant collects all applicable failures into
`badPlatform`, `missingFeatures`, and an orthogonal `maxJobsZero` flag.
Platform mismatch and missing features now produce separate error
paragraphs, and all applicable reasons appear in a single message.

The local-build capability check also now returns
`std::variant<LocalBuildCapability, LocalBuildRejection>`, bundling
the `LocalStore &` and optional `ExternalBuilder *` together.
2026-02-17 12:54:24 -05:00
John Ericson 6e725093e6 Merge pull request #15143 from obsidiansystems/rootless-daemon-minimal
Support garbage collection in external daemon
2026-02-17 16:53:06 +00:00
Artemis Tosini 96fef69755 libstore: support searching for roots from an external daemon
This comes in two parts: a `nix store roots-daemon` command that
can run as root and list runtime roots,
and client logic to find runtime roots for a `LocalStore` by connecting
to that daemon.

This may be useful with an unprivileged nix daemon, as it would
otherwise be unable to find runtime roots from process open files
and maps.
2026-02-17 10:42:04 -05:00
John Ericson 16b0bb7548 Merge pull request #15270 from NixOS/inline-lookup-var
libexpr: Make sure `EvalState::lookupVar` is inlined
2026-02-17 15:12:00 +00:00
John Ericson ebcd31e434 Merge pull request #15271 from NixOS/faster-type-internal-type
libexpr: Optimise `Value::type()`, `ValueStorage::getInternalType()`
2026-02-17 15:11:23 +00:00
John Ericson f940ab5146 Merge pull request #15265 from xokdvium/libgit2-error
libfetchers/git-utils: Add GitError class for deduplicating error…
2026-02-17 15:06:31 +00:00
Sergei Zimmerman 3df91bea62 libexpr: Optimise Value::type(), ValueStorage::getInternalType()
Using nix::unreachable() in getInternalType() and type() turns
out to be quite expensive and prevents inlining. Also Value::type
got compiled to a jump table which has a high overhead from indirect
jumps. Using an explicit lookup table turns out to be more efficient.

This does mean that we lose out on nice diagnostics from nix::unreachable
calls, but this code is probably one of the hottests functions in the whole
evaluator, so I think the tradeoff is worth it. The nixUnreachableWhenHardened
boils down to nix::unreachable when UBSan is enabled so we still have good
coverage there.
2026-02-17 16:50:07 +03:00
Sergei Zimmerman aaabe82483 libexpr: Make sure EvalState::lookupVar is inlined
This makes sure that ExprVar::eval inlines lookupVar call. In practice
this seems to reduce instruction count by ~2%, though it doesn't have
a statistically significant impact on the wall time.
2026-02-17 15:32:26 +03:00
Sergei Zimmerman a81f83604b libexpr: Add marker values to InternalType enum
This reduces the churn when changing up the order of
values in a follow-up commit. This should have been done
from the start ideally to improve readability.
2026-02-17 13:32:45 +03:00
Sergei Zimmerman c1bfa30303 libfetchers/git-utils: Add GitError class for deduplicating error message printing
Consolidates all the error message formatting in one place. It was very weird
and tiring to remember to call git_error_last() in all the places.
2026-02-17 12:18:37 +03:00
John Ericson 509694d5f0 Merge pull request #15267 from obsidiansystems/fix-external-builders-path
tests: quote `PATH` in external-builders test heredoc
2026-02-17 05:53:17 +00:00
Amaan Qureshi 0b7629da08 tests: quote PATH in external-builders test heredoc
The external-builders test expands `$PATH` into a heredoc without quotes,
so any `PATH` entry containing spaces causes bash to parse the line as a
command instead of an assignment, failing the test.
2026-02-16 23:20:10 -05:00
dependabot[bot] 6429f2fd6c build(deps): bump korthout/backport-action from 4.0.1 to 4.1.0
Bumps [korthout/backport-action](https://github.com/korthout/backport-action) from 4.0.1 to 4.1.0.
- [Release notes](https://github.com/korthout/backport-action/releases)
- [Commits](https://github.com/korthout/backport-action/compare/c656f5d5851037b2b38fb5db2691a03fa229e3b2...01619ebc9a6e3f6820274221b9956b3e7365000a)

---
updated-dependencies:
- dependency-name: korthout/backport-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-16 22:11:56 +00:00
dependabot[bot] 1a2d73dc2b build(deps): bump cachix/install-nix-action from 31.9.0 to 31.9.1
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 31.9.0 to 31.9.1.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md)
- [Commits](https://github.com/cachix/install-nix-action/compare/4e002c8ec80594ecd40e759629461e26c8abed15...2126ae7fc54c9df00dd18f7f18754393182c73cd)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-version: 31.9.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-16 22:11:52 +00:00
Sergei Zimmerman e7e5eaaa37 Merge pull request #15255 from obsidiansystems/fix-repl-tab-crash
repl: catch all errors during tab completion
2026-02-16 21:58:22 +00:00
Jörg Thalheim 974545290e Merge pull request #15252 from obsidiansystems/fix-docker-compression
upload-release: disable containerd image store to preserve gzip layer compression
2026-02-16 21:26:31 +00:00
Amaan Qureshi be6e72f11b repl: prevent exceptions from escaping editline callbacks
The tab completion handler in `completePrefix` only caught `ParseError`,
`EvalError`, `BadURL`, and `FileNotFound`. Other error types like
`JSONParseError` (which derives from `Error`, not `EvalError`) escaped
the catch block and propagated through editline's C code as undefined
behavior, crashing the REPL. This happened when tab-completing
expressions like `(builtins.fromJSON "invalid").` where evaluation
throws a non-`EvalError` exception.

This commit marks `completionCallback` and `listPossibleCallback` as
`noexcept` with function-try-blocks that catch all exceptions at the
C/C++ boundary, preventing any exception from reaching editline.

Fixes #15133.
2026-02-16 16:02:37 -05:00
Sergei Zimmerman 27782fcc42 Merge pull request #15253 from obsidiansystems/fix-url-assertion
libflake: fix assertion crash when malformed URL falls through to path scheme
2026-02-16 20:49:49 +00:00
John Ericson 06d4d5779f Merge pull request #15251 from obsidiansystems/file-system-at
Split `file-system-at.{cc,hh}` from `file-descriptor.{cc,hh}`
2026-02-16 20:10:28 +00:00
Amaan Qureshi a32cd16f64 libflake: fix assertion crash when malformed URL falls through to path scheme
When a URL like `github:nixos/nixpkgs/nixpkgs.git?ref=<hash>` (using
`ref` instead of `rev`) failed the github input scheme, it fell
through to `parsePathFlakeRefWithFragment` which constructed a `path:`
`ParsedURL` with an empty authority but a relative path. This violated
RFC 3986 section 3.3 (authority present requires path starting with
`/`), causing an assertion failure in `renderAuthorityAndPath` when
`PathInputScheme` tried to format the URL for an error message.

This commit only sets the authority on absolute paths. Relative paths
get `std::nullopt` for authority, which is the correct representation
per the URL spec.

Fixes #15196. Fixes #14830.
2026-02-16 15:10:19 -05:00
Sergei Zimmerman 46a4a554ca Merge pull request #15237 from xokdvium/add-missing-temp-roots
Add missing temproots for cached sources and existing derivations
2026-02-16 19:35:15 +00:00
John Ericson cc0b489967 Merge pull request #15250 from obsidiansystems/assume-lchown
Remove suppport for not having `lchown`
2026-02-16 19:29:08 +00:00
John Ericson af7e585009 Split file-system-at.{cc,hh} from file-descriptor.{cc,hh}
`file-descriptor.{cc,hh}` was getting too big, split out
`file-system-at.{cc,hh}` for the FD-based file system stuff,
`file-descriptor.{cc,hh}` will only be for the fundamental primitives
that are file-system agnostic and work on almost all file types.

Review with `git show --color-moved` to see that this is indeed all
moving.
2026-02-16 14:21:52 -05:00
Amaan Qureshi 2ccb8a9a56 upload-release: disable containerd image store to preserve gzip layer compression
Docker 28+ defaults to the containerd image store, which pushes layers
uncompressed instead of gzip. The GHA runner image updated Docker to
29.x (actions/runner-images#13633), causing the `nixos/nix:2.33.3`
image to balloon from 138 MB to 505 MB, with all 70 layers pushed as
`application/vnd.docker.image.rootfs.diff.tar` instead of `.tar.gzip`.
OCI clients that only support gzip (e.g. `go-containerregistry`, used
by Concourse CI) fail with "gzip: invalid header".

This commit disables the containerd snapshotter in the release workflow
before any Docker operations, restoring the classic storage driver that
preserves gzip compression through the `docker load` / `docker push`
pipeline.

Fixes #15246
2026-02-16 14:08:08 -05:00
John Ericson fefa66880a Remove suppport for not having lchown
Linux, macOS, and all 3 BSDs have it (according to man page google
search), so let's just drop this. Support for not having it was added in
d03f0d4117 in 2006, things have changed in
the last 20 years!
2026-02-16 13:40:29 -05:00
John Ericson a53391fd0e Merge pull request #15247 from roberth/clarify-ref-upcasting
Better `ref` casting DX
2026-02-16 17:09:16 +00:00
Robert Hensing 771421a34e fix(ref): improve cast exception type and add demangled type names
When ref::cast() fails, the error message was cryptic ("null pointer
cast to ref"). Now it throws a proper bad_ref_cast (a std::bad_cast
subclass) with a clear message showing the actual types involved:

    ref<nix::Base> cannot be cast to ref<nix::Derived>

This also adds a demangle.hh utility.
2026-02-16 17:07:40 +01:00
Robert Hensing 5aaa0cc4a6 refactor(ref): clarify implicit conversion semantics with requires clause
ref<Derived> was already implicitly convertible to ref<Base>, but the
mechanism was unclear and error messages for rejected downcasts were
more cryptic than necessary. This change:

- Adds RefImplicitlyUpcastableTo concept to constrain the conversion
  operator, making the intent explicit and improving error messages
- Documents .cast() and .dynamic_pointer_cast() as alternatives for
  explicit downcasting
- Adds unit tests for covariance behavior
2026-02-16 16:43:08 +01:00
John Ericson 0749ec4e55 Merge pull request #15230 from obsidiansystems/new-wine
flake: Use Wine 11 for running mingw tests
2026-02-15 16:41:52 +00:00
Artemis Tosini 4cc97150df flake: Use Wine 11 for running mingw tests
Set wine_11 as the emulator for Windows.
2026-02-15 10:56:02 -05:00
John Ericson 2bbd1094a2 flake.lock: Update Nixpkgs
Flake lock file updates:

• Updated input 'nixpkgs':
    'https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz?narHash=sha256-Xu%2B7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4%3D' (2026-01-22)
  → 'https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz?narHash=sha256-WoiezqWJQ3OHILah%2Bp6rzNXdJceEAmAhyDFZFZ6pZzY%3D' (2026-02-14)

This will be needed to get Wine 11.
2026-02-15 10:53:15 -05:00
John Ericson 95251a51dd Merge pull request #15241 from obsidiansystems/fix-isindir
libutil: fix `isInDir` rejecting paths starting with dot
2026-02-15 15:52:37 +00:00
Amaan Qureshi 0e39aa2068 libstore: guard against empty archive in unpack paths
`DirectoryIterator` is dereferenced without an end check, which segfaults
when unpacking an empty or zero-sized archive. This commit adds an
emptiness check before the dereference in both `prefetchFile` and
`builtinUnpackChannel`, throwing a descriptive error instead.
Fixes #15116.
2026-02-15 10:16:54 -05:00
John Ericson 02d9f4ecb4 Merge pull request #15239 from xokdvium/fix-warnings-no-intereference-size
meson: Only enable -Wno-interference-size with GCC
2026-02-15 15:06:54 +00:00
John Ericson 3269c71e9d Merge pull request #15240 from xokdvium/fix-mtls-redirect-test
libstore-tests: Fix mTLS test for redirect, correctly propagate tries
2026-02-15 15:04:50 +00:00
Amaan Qureshi ad0055e67c libutil: fix isInDir rejecting paths starting with dot
The old check rejected any relative path whose first character was a
dot, producing false negatives for valid descendants like `.ssh` or
`.config`. This commit changes the logic such that now it inspects the
first path component via `path::begin()`, only rejects `.` and `..`
rather than anything dot-prefixed. Fixes #15207.
2026-02-15 10:04:08 -05:00
John Ericson 7c915b371d Merge pull request #15235 from obsidiansystems/os-environ
libutil-tests: Fix crash on Windows
2026-02-15 14:58:22 +00:00
Artemis TosiniandJohn Ericson 36d0e9580f Implement Pid::kill for Windows
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-14 20:39:32 -05:00
Artemis TosiniandJohn Ericson c9abefbc30 libutil-tests: Fix crash on Windows
libutil tests were crashing on Windows due to issues finding `environ`.
Replace process creation of `getEnv` with a new `getEnvOs` function that
uses native windows APIs.

Also convert a bunch of `RunOptions` fields to use `OsString` to better
reflect the underlying interfaces.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-14 20:39:32 -05:00
Sergei Zimmerman 6cbf80a0b0 Merge pull request #15219 from obsidiansystems/writeDerivation-lighter-read-only
Get rid of the settings-dependent `writeDerivation` wrapper
2026-02-14 21:27:52 +00:00
Sergei Zimmerman d3d63a4b5b libstore-tests: Fix mTLS test for redirect, correctly propagate tries
The fake cacert didn't have subjectAltName for 127.0.0.1, so the test
was failing for a different reason. Also `tries` setting wasn't being respected.
There's no callsite specifying it in the request, so just use the one specified
in the FileTransferSettings and remove the fields from the FileTransferRequest.
2026-02-15 00:08:21 +03:00
Sergei Zimmerman 6a5ee08737 meson: Only enable -Wno-interference-size with GCC
Clang doesn't recognise this option.
2026-02-14 23:42:28 +03:00
Sergei Zimmerman ac2dd58b6f Add missing temproots for cached sources and existing derivations 2026-02-14 12:09:24 +03:00
John Ericson 8fadcceb6d Merge pull request #15233 from obsidiansystems/remove-nixstore-global
libstore: remove `Settings::nixStore` in favor of `StoreConfigBase::getDefaultNixStoreDir`
2026-02-13 20:29:11 +00:00
John EricsonandAmaan Qureshi 4f91e9599f Revert "Use the hash modulo in the derivation outputs"
Fix #11897

As described in the issue, this makes for a simpler and much more
intuitive notion of a realisation key. This is better for pedagogy, and
interoperability between more tools.

The way the issue was written was that we would switch to only having
shallow realisations first, and then do this. But going to only shallow
realisations is more complex change, and it turns out we weren't even
testing for the benefits that derivation hashes (modulo FODs) provided
in the deep realisation case, so I now just want to do this first.

Doing this gets the binary cache data structures in order, which will
unblock the Hydra fixed-output-derivation tracking work. I don't want to
delay that work while I figure out the changes needed for
shallow-realisations only.

This reverts commit bab1cda0e6.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-13 15:20:00 -05:00
John Ericson 2913722781 Merge pull request #15229 from lisanna-dettwyler/fix-gc-dry-run
Emit basic dry run message for garbage collection
2026-02-13 20:19:49 +00:00
Amaan Qureshi 12f97382af libstore: remove Settings::nixStore in favor of StoreConfigBase::getDefaultNixStoreDir
This commit removes the `nixStore` member from `Settings` and instead
computes the default Nix store directory directly in
`StoreConfigBase::getDefaultNixStoreDir()` from env vars
(`NIX_STORE_DIR`, `NIX_STORE`) or the compile-time default. The method
is made public so callers that previously reached through the global
`settings.nixStore` can use it instead.

Progress on #5638
2026-02-13 14:45:49 -05:00
Lisanna Dettwyler fdfc772114 Emit basic dry run message for garbage collection
nix store gc: prints number of paths that would be freed, but not bytes
nix-collect-garbage: ditto
nix-store --gc: retains current behavior

It would be very non-trivial to also compute the bytes that would be
freed, due to hardlinking in the store.

Also adds checking for incompatible mixing of dry-run and max-freed
options.

Resolves #5704

Signed-off-by: Lisanna Dettwyler <lisanna.dettwyler@gmail.com>
2026-02-13 14:40:36 -05:00
John Ericson a4b1814d67 Merge pull request #15232 from obsidiansystems/inline-buildlocally
libstore: inline `willBuildLocally` and `canBuildLocally` into call sites
2026-02-13 19:29:15 +00:00
John Ericson 702ebdb11b Merge pull request #15231 from obsidiansystems/inline-getmachines
libstore: inline `getMachines` into call sites
2026-02-13 19:07:18 +00:00
Amaan QureshiandJohn Ericson 7106de16e6 libstore: inline willBuildLocally and canBuildLocally into call sites
This commit inlines `DerivationOptions::willBuildLocally` and
`DerivationOptions::canBuildLocally` into their sole call site in
`DerivationBuildingGoal::tryToBuild`. The `canBuildLocally` logic is now
a lambda capturing the surrounding context, and `willBuildLocally` is
replaced by `drvOptions.preferLocalBuild && canBuildLocally`.

Progress on #5638

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-13 13:45:33 -05:00
John Ericson b818594ba2 Merge pull request #15228 from obsidiansystems/profile-dirs-options
libstore: extract `ProfileDirsOptions` from `Settings`
2026-02-13 18:28:56 +00:00
Amaan QureshiandJohn Ericson 9ae12ede4c libstore: inline getMachines into call sites
This commit removes the `getMachines` free function and inlines `Machine::parseConfig({settings.thisSystem}, settings.getWorkerSettings().builders)` at its two call sites in `worker.cc` and `build-remote.cc`. The wrapper just forwarded to `Machine::parseConfig` with global settings, so inlining it removes an unnecessary layer of indirection and makes the global dependency explicit at each call site.

Progress on #5638

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-02-13 13:22:49 -05:00
John Ericson 20f7f33123 Merge pull request #15227 from obsidiansystems/narinfo-disk-cache-settings
libstore: extract `NarInfoDiskCacheSettings` from `Settings`
2026-02-13 17:58:33 +00:00
John EricsonandAmaan Qureshi 002cbefa9f libstore: extract ProfileDirsOptions from Settings
This commit moves `nixStateDir` and `useXDGBaseDirectories` into a dedicated `ProfileDirsOptions` struct and threads it through the profile directory functions (`profilesDir`, `rootProfilesDir`, `defaultChannelsDir`, `rootChannelsDir`, `getDefaultProfile`) so they no longer read from the global `Settings` object directly. This follows the same pattern as `LocalSettings`, `WorkerSettings`, and `NarInfoDiskCacheSettings`.

Progress on #5638

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-13 12:43:53 -05:00
John Ericson dc636dde10 libstore: extract NarInfoDiskCacheSettings from Settings
This commit moves `ttlNegativeNarInfoCache` and `ttlPositiveNarInfoCache` into a dedicated `NarInfoDiskCacheSettings` struct that `Settings` privately inherits from, following the same pattern as `LocalSettings`, `LogFileSettings`, and `WorkerSettings`.

`NarInfoDiskCache` now takes explicit `NarInfoDiskCacheSettings` and `SQLiteSettings` in its constructor instead of reading from the global. The singleton `getNarInfoDiskCache()` is replaced with a `NarInfoDiskCache::get()` static method that accepts these settings, though they are only used on the first call (subsequent calls return the cached instance regardless of arguments).

Progress on #5638
2026-02-13 12:12:34 -05:00
John Ericson a06ab4871c Merge pull request #15217 from amaanq/acquire-user-lock-state-dir
libstore: pass `stateDir` to `acquireUserLock` instead of using global
2026-02-13 01:31:41 +00:00
John Ericson ed22ef2b89 Merge pull request #15218 from obsidiansystems/read-only-per-store
libstore: make substitution use the per-store `getReadOnly` method
2026-02-13 01:31:32 +00:00
John Ericson 7926a629e2 Get rid of the settings-dependent writeDerivation wrapper
It was a crude hack that this one low-level function was dependent on
the high-level read-only mode setting --- all the more so because rather
than making derivation writing fail, that setting made it silently
"succeed" why not actually writing the derivation. (Also, for context,
we didn't have an such behavior for any other store-mutating operations,
just for this one function.)

I have gotten rid of the wrapper, and updated the call sites
accordingly.

- For the ones that should remain dependent on this setting, I made this
  explicit, and added a comment.

- For others, surrounding operations assumed writability (e.g. we had
  written something before, or were about to try to read back the
  written derivation after), and so I just made those do the underlying
  `Store::writeDerivation` operation.
2026-02-12 20:26:24 -05:00
Bernardo Meurer a8f305add3 Merge pull request #15216 from NixOS/fix-s3-conn-reuse
fix: #15208
2026-02-13 00:53:58 +00:00
Amaan Qureshi cecbe9f73a libstore: pass stateDir to acquireUserLock instead of using global
This makes `acquireUserLock` take an explicit stateDir parameter,
since it was previously reaching into the global settings object
just to read `nixStateDir` for constructing the userpool paths.

Progress on #5638
2026-02-12 19:43:40 -05:00
Amaan Qureshi 9ac91e36a9 libstore: make substitution use the per-store getReadOnly method
This commit introduces a `getReadOnly` method on the store config that returns if the current store is read only or not. This is then used in subtitution, so we fail gracefully with a nice error message if only the individual store is read-only.

As a bonus, it gets us one step closer to getting rid of the global because we can use the per-store method instead.

Progress on #5638
2026-02-12 19:43:20 -05:00
Bernardo Meurer Costa 759f6c856b feat(libstore/s3): use virtual-hosted-style URLs and add addressing-style option
S3 binary caches now use virtual-hosted-style URLs by default for
standard AWS endpoints. Path-style endpoints (s3.region.amazonaws.com)
only serve HTTP/1.1, preventing HTTP/2 multiplexing and causing TCP
TIME_WAIT socket exhaustion under high concurrency. Virtual-hosted-style
endpoints (bucket.s3.region.amazonaws.com) support HTTP/2, enabling
multiplexing with the existing CURLPIPE_MULTIPLEX configuration.

Add a new `addressing-style` store option (auto/path/virtual) to control
this behavior. `auto` (default) uses virtual-hosted-style for standard
AWS endpoints and path-style for custom endpoints. `path` forces
path-style for backwards compatibility. `virtual` forces virtual-hosted-
style for all endpoints including custom ones.

Fixes: https://github.com/NixOS/nix/issues/15208
2026-02-13 00:03:50 +00:00
Bernardo Meurer Costa 736abd50ff fix(libstore/filetransfer): enable TCP keep-alive on curl handles
Idle connections in libcurl's connection pool can be silently dropped by
the OS or intermediate firewalls/NATs before they can be reused, forcing
new TCP connections to be created. This is especially problematic for
HTTP/1.1 endpoints where multiplexing is unavailable.

Enable TCP keep-alive with a 60-second idle/interval on all curl easy
handles to prevent idle connection drops and improve connection reuse.
2026-02-12 22:52:48 +00:00
John Ericson a3d51172e9 Merge pull request #15211 from obsidiansystems/worker-settings
libstore: extract `WorkerSettings` from `Settings`
2026-02-12 21:15:43 +00:00
Sergei Zimmerman eae7e0151c Merge pull request #15213 from xokdvium/unhardcode-alignas-cache-line-size
Unhardcode alignas cache line size
2026-02-12 20:53:09 +00:00
Amaan Qureshi d3388d3d81 libstore: extract WorkerSettings from Settings
This commit  moves `pollInterval`, `maxSubstitutionJobs`, `postBuildHook`, and `logLines` into a dedicated `WorkerSettings` struct that `Settings` privately inherits from, as they are only used by the build worker subsystem. This follows the same pattern as `LocalSettings` and `LogFileSettings`.
2026-02-12 15:31:08 -05:00
Sergei Zimmerman 7352205ce9 libexpr: Replace hardcoded cache line size with std::hardware_destructive_interference_size
This expands to __GCC_DESTRUCTIVE_SIZE, which is also 64 (at least in the x86_64 stdenv).
Let the compiler decide what's the appropriate cache line size is. Also, on aarch64-darwin
the cache line size 128 bytes, so the previous fix didn't actually get rid of false sharing
reliably. Clang does this [1] [2], so it overestimates the sizes somewhat, but that's still enough
for avoiding false sharing on darwin.

[1]: https://github.com/llvm/llvm-project/blob/a289341dedf4d73955faefbb0b3c13881e13dd06/clang/lib/Frontend/InitPreprocessor.cpp#L1331-L1339
[2]: https://github.com/llvm/llvm-project/blob/6f51f8e0f93e6ff2baf0ed2f60021308920a557f/clang/lib/Basic/Targets/AArch64.h#L262-L264
2026-02-12 23:04:40 +03:00
Sergei Zimmerman f3f9eac8fc Merge pull request #15209 from obsidiansystems/http-store-port-ctor
libstore: add `HttpBinaryCacheStoreConfig` constructor that takes a ` ParsedURL`
2026-02-12 18:48:42 +00:00
Sergei Zimmerman df21c81191 libexpr: Fix some typos in value.hh 2026-02-12 20:51:38 +03:00
Amaan Qureshi 52b1906995 libstore: add HttpBinaryCacheStoreConfig constructor that takes a ParsedURL
In the https-store tests, a `TestHttpBinaryCacheStoreConfig` is constructed with a call to format to create the cache uri. This commit adds a constructor to `HttpBinaryCacheStoreConfig` to remove the need for this call, and updates the test type to leverage this so we're no longer manually calling fmt on a string to format the port.
2026-02-12 11:22:29 -05:00
John Ericson c756d02948 Merge pull request #15206 from obsidiansystems/injectable-filetransfer
libstore: make `FileTransfer` injectable into `HttpBinaryCacheStore`
2026-02-12 14:58:34 +00:00
Amaan Qureshi 403e30f136 libstore: make FileTransfer injectable into HttpBinaryCacheStore
This commit makes `FileTransfer` self-contained by giving it a reference
to `FileTransferSettings` instead of reading from the global. It also
adds an optional `FileTransfer` parameter to `HttpBinaryCacheStore` so
callers can inject their own instance.

The main motivation is test isolation. The HTTPS store tests now create
custom `FileTransferSettings` with the test CA certificate and pass it
through `makeFileTransfer()`, avoiding global state mutation entirely.
2026-02-11 19:00:53 -05:00
Sergei Zimmerman 3a60a04bf8 Merge pull request #15183 from obsidiansystems/newuidmap
Support build users on unprivileged users with subuid/subgid
2026-02-11 22:35:44 +00:00
Artemis Tosini c9526e289a Add new libexec/nix-nswrapper program
nix-nswrapper allows running nix in its own user namespace,
believing it is root and with access to build users for sandboxing
with auto-allocate-uids, while it is actually unprivileged.

It is used to wrap nix, and an example of its use has been
added to the unprivileged daemon functional tests.

Running it does not require any elevated privileges,
only uids and gids allocated in /etc/sub{uid,gid}
2026-02-11 16:53:08 -05:00
Eelco Dolstra d4a0024184 Merge pull request #15205 from NixOS/bump-file-limit-upstream
Increase the open file soft limit to the hard limit
2026-02-11 21:40:39 +00:00
Sergei Zimmerman d9651b1f82 Merge pull request #15193 from NixOS/restore-death-signal
DerivationBuilder: Preserve death signal across setuid,setgid
2026-02-11 21:26:32 +00:00
John Ericson 912c6c283d Merge pull request #15202 from obsidiansystems/migrate-ca-netrc-downloadspeed-filetransfer
libstore: migrate `caFile`, `netrcFile`, and `downloadSpeed` to `FileTransferSettings`
2026-02-11 21:09:06 +00:00
Eelco Dolstra 04fd722b1b Increase the open file soft limit to the hard limit
On some platforms (macOS), the default soft limit is very low, but the
hard limit is high. So let's just raise it the maximum permitted.
2026-02-11 21:55:57 +01:00
John Ericson 1a57df3473 Merge pull request #15203 from obsidiansystems/substituter-confs
Deduplicate `nix repl` and `nix log`
2026-02-11 20:41:56 +00:00
eveeifyeveandAmaan Qureshi 04d13a96e3 libstore: migrate caFile, netrcFile, and downloadSpeed to FileTransferSettings
The `caFile`, `netrcFile`, and `downloadSpeed` settings are only used by
the file transfer subsystem but lived in the global `Settings` class.
This moves them to `FileTransferSettings` where they belong.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-11 14:58:27 -05:00
Amaan Qureshi 46eabe34c2 libstore: move hashedMirrors to LocalSettings
`hashedMirrors` is only relevant to local builds (it is consumed by
`builtin:fetchurl` during derivation building) but lived in the global
`Settings` class. This moves it to `LocalSettings` where it belongs
and threads it through `BuiltinBuilderContext` so `fetchurl.cc` reads
it from the context instead of reaching into `settings` directly.
2026-02-11 14:58:27 -05:00
John Ericson ecdcdd82e0 Deduplicate nix repl and nix log
The underlying mechanism is now in a new `fetchBuildLog` function I put
in `libcmd`.

I am putting it in here and not in libstore because I have some doubts
about `getDefaultSubstituters`, so I would like to keep it in a more
"peripheral" part of the codebase for now.
2026-02-11 14:55:31 -05:00
Jörg Thalheim a4c421da22 Merge pull request #15201 from mkenigs/install-release-notes
beta nix-installer: add release-note
2026-02-11 19:07:49 +00:00
John Ericson ae4e4d9afd Merge pull request #15192 from obsidiansystems/store-reference-types
globals: change store settings to use `StoreReference` types directly
2026-02-11 17:49:07 +00:00
Matthew KenigsbergandCole Helbling fbd837c911 beta nix-installer: add release-note
Add a release note asking for help testing
https://github.com/NixOS/nix-installer

We're hoping to start recommending the Rust-based installer after one
release cycle.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2026-02-11 10:07:59 -07:00
Amaan Qureshi 857fd2a3a4 globals: change store settings to use StoreReference types directly
The `storeUri`, `substituters`, and `trustedSubstituters` settings now
store typed `StoreReference` values directly instead of raw strings,
so callers work with the real types without manual parsing.

This is a reworked version of #10761.
2026-02-11 12:05:50 -05:00
John Ericson 3c1ad7d978 Merge pull request #15197 from obsidiansystems/ssh-store-config-direct
nix-copy-closure: create `LegacySSHStoreConfig` directly
2026-02-11 16:15:29 +00:00
Amaan Qureshi 8020a847ab nix-copy-closure: create LegacySSHStoreConfig directly
Instead of constructing a `StoreReference` and letting `openStore()` resolve it,
this commit creates the store config directly and calls `openStore()` on it. This
avoids the indirection through the store registry.
2026-02-11 10:31:46 -05:00
John Ericson db8499e62f Merge pull request #15200 from obsidiansystems/remove-const-settings
store-config: remove unnecessary `const` from `Setting<>` fields
2026-02-11 15:29:59 +00:00
Amaan Qureshi 1add77677f store-config: remove unnecessary const from Setting<> fields
Stores hold their config as `ref<const Config>` or `const Config &`,
so `Setting<>` fields are already immutable after store construction.
The field-level `const` is redundant and prevents pre-construction
mutation which is sometimes useful. This commit updates these settings by dropping the `const` qualifier, as it's not needed.
2026-02-11 09:20:41 -05:00
John Ericson d5eda907ef Merge pull request #15195 from obsidiansystems/store-reference-args
globals: use `StoreReference` types in CLI argument handlers
2026-02-11 00:22:08 +00:00
Amaan Qureshi f9300514cd globals: use StoreReference types in CLI argument handlers
The CLI flags `--from`, `--to`, `--eval-store`, and substituter URIs now
parse to `StoreReference` at the argument boundary. `fetchClosure` uses
`StoreReference::parse` instead of `parseURL`. This also adds
`operator<=>` to `StoreReference`.
2026-02-10 18:37:34 -05:00
John Ericson 036a47be83 Merge pull request #15194 from obsidiansystems/nix-daemon-store-config
Make `nix daemon` a `StoreConfigCommand`
2026-02-10 23:13:15 +00:00
John Ericson c4e408459a Make nix daemon a StoreConfigCommand
This commit makes `nix daemon` inherit from `StoreConfigCommand`
instead of `Command`, so that it receives a `StoreConfig` to open
and serve stores with. This cleans up a few things (removes
`openUncachedStore` helper, passes `storeConfig` through
`daemonLoop`/`runDaemon` instead of opening stores ad-hoc) and will
allow further cleanups.
2026-02-10 17:34:13 -05:00
Sergei Zimmerman f0498b94d8 Merge pull request #14768 from pkpbynum/capi/copy-path
C API: Add copy_path to Store API
2026-02-10 22:04:22 +00:00
Sergei Zimmerman 34688ecf5f DerivationBuilder: Preserve death signal across setuid,setgid
It's apparently a common footgun in Linux that the death signal isn't
preserved across calls to setuid/setgid. If nix-build gets SIGKILL-ed
while a build is running that would lead to a runaway build process that
would get reparented to init/systemd.

This is pretty easy to reproduce with the following derivation:

derivation {
  name = "pdeathsig-repro";
  system = builtins.currentSystem;
  builder = "/bin/sh";
  args = [
    "-c"
    ''
      while :; do :; done
    ''
  ];
}

And the reproduction script:

sudo nix-build repro.nix &
sleep 3
BUILDER=$(pgrep -u nixbld1)
sudo kill -9 $(pgrep -f 'nix-build.*repro')
sleep 1
ps -p $BUILDER -o pid,ppid,user,comm

To address this we have to restore the death signal after all the calls
to setuid/setgid. This is done in a helper function preserveDeathSignal
that takes a callback to avoid code duplication.

See: https://github.com/golang/go/issues/9686
2026-02-11 00:56:34 +03:00
John Ericson 92d0fe000b Merge pull request #15188 from obsidiansystems/handleexceptions
Move `nix::handleExceptions` to libutil
2026-02-10 18:55:14 +00:00
John Ericson 75af0351ac Merge pull request #15089 from NixOS/sfp-global-settings
`std::filesystem::path` in some `Settings` fields
2026-02-10 18:43:55 +00:00
Artemis Tosini c79ff97c07 Move nix::handleExceptions to libutil
This is a fairly simple function, isolated from the rest of libmain
and could be useful if new programs are made that are not part of the
main nix-cli subproject.
2026-02-10 13:06:27 -05:00
John Ericson ef659136ca std::filesystem::path in some Settings fields 2026-02-10 12:50:17 -05:00
John Ericson 582e4fa0f6 Merge pull request #15187 from obsidiansystems/serve-unix-socket
Factor out `serveUnixSocket`
2026-02-10 17:03:36 +00:00
Amaan Qureshi 6674c23416 Factor out serveUnixSocket
This commit extracts the Unix domain socket server loop (`PeerInfo`,
`getPeerInfo`, and the systemd socket activation / poll / accept loop)
from `src/nix/unix/daemon.cc` into a reusable `unix::serveUnixSocket`
function in `libcmd`.
2026-02-10 11:25:57 -05:00
Eelco Dolstra b06d0f764f Merge pull request #15175 from KiaraGrouwstra/flake-ref-nixpkgs
check `isFlake` in `nixpkgsFlakeRef`
2026-02-10 14:03:37 +00:00
John Ericson 845d951682 Merge pull request #15180 from xokdvium/more-werror
meson: Add -Werror=return-type and -Werror=non-virtual-dtor flags
2026-02-09 21:48:15 +00:00
Sergei Zimmerman a900bf1548 meson: Add -Werror=return-type and -Werror=non-virtual-dtor flags
Some easy compile-time safety features to catch mistakes earlier.
Fixes some missing virtual destructors.
2026-02-10 00:02:00 +03:00
Sergei Zimmerman 36ad2962ca Merge pull request #14944 from iljah/patch-2
Use unreachable in nix::listNarImpl()
2026-02-09 20:12:33 +00:00
Sergei Zimmerman e4ce788f9d Merge pull request #15172 from NixOS/misc-fixes
libutil: Assorted collection of fixes, address UBSan failure in AutoDelete
2026-02-08 22:56:42 +00:00
Sergei Zimmerman 3cd840d7f1 libutil: Fix error message in readLinkAt
More correctly describes the error, since we are always reading a relative path.
2026-02-09 00:57:13 +03:00
Sergei Zimmerman 6b90755cad libutil: Fix uninitialised variable in AutoDelete
Diagnosed by UBSan in hydra [1]:

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /nix/store/m1k4nxs8r0fl0pjxqp5n37vxgms7gdlb-gcc-14.3.0/include/c++/14.3.0/bits/move.h:234:11 in
+(prefetch.sh:6) path=
++(prefetch.sh:6) onError
++(/build/source/tests/functional/common/functions.sh:241) set +x
prefetch.sh: test failed at:
  main in prefetch.sh:6

[1]: https://hydra.nixos.org/build/321098757/nixlog/1
2026-02-09 00:57:07 +03:00
cinereal 30e213c948 check isFlake in nixpkgsFlakeRef
To get `bashInteractive`, `nix develop` currently
[gets a flake reference to nixpkgs](https://github.com/NixOS/nix/blob/3b473c4be5e6abc58aa43271f35ff127af806c55/src/nix/develop.cc#L650-L658),
either from [`inputs.nixpkgs`](https://github.com/NixOS/nix/blob/3b473c4be5e6abc58aa43271f35ff127af806c55/src/libcmd/installable-flake.cc#L204)
or [`<nixpkgs>`](https://github.com/NixOS/nix/blob/3b473c4be5e6abc58aa43271f35ff127af806c55/src/libcmd/include/nix/cmd/installable-flake.hh#L87).

Currently, this is done by name for any (locked) reference.
For any `nixpkgs` reference lacking a `flake.nix`, `nix develop` thus errors:

```
error (ignored): path '.../flake.nix' does not exist
```

While the registry does not expose info to help check this,
flake inputs expose the `flake` boolean.
This means that given `inputs.<name>.flake = false;`,
we may know in advance that our reference should not be to a flake.

This change incorporates this, so that given `inputs.nixpkgs.flake = false;`,
`nix develop` will fall back `<nixpkgs>` to use its flake for `bashInteractive`.

While Nixpkgs itself of course does expose a flake,
this check is relevant in particular for dummy inputs
(e.g. inputs using `{ url = "file:/dev/null"; flake = false; }`),
which may be overridden by `--override-input` or `.follows`,
yet do not expose (flake) code themselves.

Signed-off-by: cinereal <cinereal@riseup.net>
2026-02-08 13:32:20 +01:00
Sergei Zimmerman 3b473c4be5 Merge pull request #14952 from koberbe/koberbe/master/SW-100752-nix-commands-help-is-missing-unformatted
Install nix-manual in default user profile
2026-02-07 15:37:54 +00:00
Ilja 23ddb0bfc7 Throw on unsupported type in nix::listNarImpl()
Prevent
```
../src/libutil/nar-accessor.cc: In function ”nix::ListNarResult<deep> nix::listNarImpl(SourceAccessor&, const CanonPath&) [with bool deep = true]”:
../src/libutil/nar-accessor.cc:335:1: varoitus: ei-void-tyyppisen funktion loppu saavutettu [-Wreturn-type]
  335 | }
      | ^
../src/libutil/nar-accessor.cc: In function ”nix::ListNarResult<deep> nix::listNarImpl(SourceAccessor&, const CanonPath&) [with bool deep = false]”:
../src/libutil/nar-accessor.cc:335:1: varoitus: ei-void-tyyppisen funktion loppu saavutettu [-Wreturn-type]
```
2026-02-07 17:31:45 +03:00
John Ericson e29bb23cf9 Merge pull request #15170 from obsidiansystems/should-resolve
Add `Derivation::shouldResolve()` method, use in `nix-shell`
2026-02-07 00:09:31 +00:00
John Ericson 08da3311b3 Add Derivation::shouldResolve() method, use in nix-shell
Extract the logic for determining whether a derivation should be resolved
before building into a dedicated method. Then use that to not resolve
unnecessarily in `nix-shell`.
2026-02-06 18:20:33 -05:00
John Ericson ba3dc07bf1 Merge pull request #15168 from roberth/fix-protocol-addToStore-version-comparisons
Add -Werror=c99-designator and fix brace elision warnings
2026-02-06 22:11:36 +00:00
Robert Hensing e5278ac66b Add -Werror=c99-designator and fix brace elision warnings
The recent conversion of WorkerProto::Version from unsigned int to a
struct exposed a latent issue: `.version = 16` was being interpreted
as aggregate initialization `{.major = 16, .minor = 0}` rather than
the intended wire format value.

This commit adds -Werror=c99-designator to catch this class of bugs at
compile time. (The bug itself was fixed in
7eb23c15f39ca413a5f3cc0d3ab630311b4709be.)

For background:

The hardcoded version was originally the integer 16, which in the old
wire format (major << 8 | minor) meant version 0.16. However, the
version checks only compared minor versions via GET_PROTOCOL_MINOR(),
so this worked by accident.

After the Version struct conversion, the aggregate initialization
{.major = 16, .minor = 0} happened to still work because 16 > 1 in
lexicographic comparison against {1, 16}.

The correct version is {1, 16} because:
- The worker protocol uses major version 1 (all checks are {1, x})
- Version 1.16 is when ultimate/sigs/ca fields were added
- This matches the serialization check `>= {1, 16}`
2026-02-06 16:23:16 -05:00
Sergei Zimmerman 2f49b730cf Merge pull request #15167 from NixOS/repl-last-loaded-fix
libcmd/repl: Fix issues with :ll before anything is loaded, get rid o…
2026-02-06 20:17:40 +00:00
John Ericson ca8e6cae91 Merge pull request #15161 from obsidiansystems/version-number-subtype
worker-protocol: embed features in `Version` and add `Number` inner type
2026-02-06 19:56:37 +00:00
Sergei Zimmerman bcc63908ba libcmd/repl: Fix issues with :ll before anything is loaded, get rid of store parameters to constructors
Fixes abort on :ll if nothing has been loaded yet. Also gets rid of
redundant openStore() calls that were dead code (store can be extracted
from EvalState already) and arguably openStore is a layer violation.

Also catches EPIPE in case the pager gets interrupted to avoid superfluous
error messages.
2026-02-06 22:29:20 +03:00
John EricsonandAmaan Qureshi 7eb23c15f3 worker-protocol: embed features in Version and add Number inner type
This commit embeds the negotiated `FeatureSet` directly into `WorkerProto::Version` and introduces a `Number` inner type with total ordering, so that `Version` itself (number + features) only has partial ordering. This is a follow-up to #15155, cleaning up the separate `features` fields on `ReadConn`/`WriteConn`.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-06 14:14:13 -05:00
John Ericson 103f912c40 Merge pull request #15163 from xokdvium/faster-nar-listing
NarIndexer: Implement skip
2026-02-06 18:48:47 +00:00
Eelco Dolstra 34cbfffa11 Merge pull request #15160 from NixOS/fix-flakeRefToString
builtins.flakeRefToString: Evaluate attributes
2026-02-06 18:21:20 +00:00
Sergei Zimmerman 499ffaf940 NarIndexer: Implement skip
This improves the performance of parseNarListing, which is used by commands
like `nix nar ls` when the underlying source allows cheap seeks (like StringSource
or FdSource that does lseek).

For `nix nar ls` of a NAR for linux source tarball this cuts down the runtime almost
in half (from 300ms -> 175ms).
2026-02-06 20:55:04 +03:00
Peter Bynum 72ab64b612 Add nix_store_copy_path C API 2026-02-06 11:59:37 -05:00
Eelco Dolstra 2989a23fca builtins.flakeRefToString: Evaluate attributes
Fixes "attribute 'x' is a thunk".
2026-02-06 16:30:19 +01:00
Eelco Dolstra bbb4b009ec Merge pull request #15157 from NixOS/respect-noexcept
BinaryCacheStore::queryPathInfoUncached(): Ensure noexcept
2026-02-06 08:49:59 +00:00
John Ericson 91c706852b Merge pull request #15155 from obsidiansystems/protocol-version-structs
worker-protocol: replace version bit-shifting with structs
2026-02-05 18:09:53 +00:00
John Ericson 80b944a3f6 Merge pull request #15156 from obsidiansystems/findroots-refactor
libstore: move logic of `findRuntimeRoots` to new file
2026-02-05 17:04:47 +00:00
John EricsonandAmaan Qureshi cccc9440d7 worker-protocol: replace version bit-shifting with structs
This commit replaces the `GET_PROTOCOL_MINOR(version)` macros with a proper `WorkerProto::Version` struct. As a bonus, this also fixes some version checks that were incorrectly ignoring the major version number.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-05 11:50:07 -05:00
Eelco Dolstra c21820db07 BinaryCacheStore::queryPathInfoUncached(): Ensure noexcept
Make sure we don't throw an exception, since that will terminate the
program.
2026-02-05 17:35:15 +01:00
John Ericson 4496a7eead Merge pull request #15101 from obsidiansystems/split-local-store-settings
libstore: split out local build and store related settings under `LocalSettings`
2026-02-05 16:23:59 +00:00
John Ericson d1ad4b183a Merge pull request #14769 from NixOS/nar-hash-cache
Replace fetchToStore cache by sourcePathToHash cache
2026-02-05 16:23:52 +00:00
Artemis Tosini 766316223c libstore: move logic of findRuntimeRoots to new file
This change is required for implementing the unprivileged garbage collection daemon,
but it may also be useful to reduce code duplication and separate out OS-specific
garbage collector roots implementations in the future.
2026-02-05 11:18:23 -05:00
Eelco Dolstra de6b5f60cd Replace fetchToStore cache by sourcePathToHash cache
Caching NAR hashes instead of store paths makes the cache more
general, because we can always compute the store path from the NAR
hash, but not the other way around. This is useful for lazy trees,
where we want to compute the NAR hash of an input with caching.
2026-02-05 16:45:50 +01:00
Amaan Qureshi afd40adc90 libstore: split out local build and store related settings under LocalSettings
The global `Settings` struct contained many settings that only apply to
local builds or the local store (sandbox configuration, GC settings,
build user groups, etc.). This commit extracts these into a dedicated
`LocalSettings` struct in its own header, along with `GCSettings` and
`AutoAllocateUidSettings`.

This improves code organization and prepares for eventually making these
per-store settings in the future. Settings are accessed via
`getLocalSettings()` from the global settings object or through
`LocalStoreConfig::getLocalSettings()` for store-specific access.
2026-02-05 10:38:08 -05:00
Jörg Thalheim dcc71da7e8 Merge pull request #15148 from Mic92/fix-tests
tests: fix URL literals in functional tests
2026-02-05 01:04:32 +00:00
Sergei Zimmerman ecda8c2329 Merge pull request #15149 from NixOS/align-up-robustness
libutil: Add overflow check to alignUp
2026-02-05 00:51:20 +00:00
Jörg Thalheim 0da728b1f5 tests: fix URL literals in functional tests 2026-02-05 00:52:02 +01:00
Sergei Zimmerman ea53914e47 Merge pull request #15147 from NixOS/fix-mingw-read-line
libutil: Fix mingw build
2026-02-04 23:14:52 +00:00
Sergei Zimmerman d77c131df3 libutil: Add overflow check to alignUp
Old code with size + (size % 8 ? 8 - (size % 8) : 0) also suffered from this.
2026-02-05 02:04:33 +03:00
Sergei Zimmerman 72c2954625 libutil: Fix mingw build
This was broken by b038500b47.
2026-02-05 00:53:23 +03:00
Eelco Dolstra 27d5cc39c8 Merge pull request #14957 from NixOS/invalidate-lstat-cache
SourceAccessor: Allow the lstat cache to be invalidated
2026-02-04 21:44:04 +00:00
John Ericson 25ab7f5850 Merge pull request #15038 from obsidiansystems/drainfd-improvements
More IO portability cleanups
2026-02-04 21:36:18 +00:00
Eelco Dolstra 139d05af6f SourceAccessor: Allow cached information to be invalidated
After lockFlake() creates flake.lock in a PosixSourceAccessor, it
needs to be able to invalidate the lstat cache.
2026-02-04 21:56:17 +01:00
John Ericson b489c8ea15 More IO portability cleanups
- options structs for `drainFD` (both versions)
- portable `read` wrapper
- portable `GetFileSize` wrapper
- dedup `readFile` and `drainFD` using the above
- Use `drainFD` in `PosixSourceAccessor`, avoiding manual IO
- Remove `fromDescriptorReadOnly` entirely!
2026-02-04 15:49:21 -05:00
John Ericson a4c0295822 Merge pull request #15060 from NixOS/read-link-at
Support `readLinkAt` and `openFileEnsureBeneathNoSymlinks` on Windows too
2026-02-04 20:40:08 +00:00
Eelco Dolstra 8336e71c19 Merge pull request #15135 from NixOS/dependabot/github_actions/docker/login-action-3.7.0
build(deps): bump docker/login-action from 3.6.0 to 3.7.0
2026-02-04 20:34:18 +00:00
John Ericson 037a19441a Merge pull request #15145 from obsidiansystems/storeconfigcommand
libcmd: add new `StoreConfigCommand` class
2026-02-04 20:09:48 +00:00
Artemis Tosini 124605dffc libcmd: add new StoreConfigCommand class
Useful for commands that need a `StoreConfig` but do not want to open
the store, as a `StoreCommand` would do.

At the same time, make copy commands more clear by making store choice
for `updateProfile` explicit and removing the unused `getDstStore` function
in `StoreCommand`. This was a layer violation, as `StoreCommand` does
not have a concept of a source/destination store distinction.
2026-02-04 14:31:51 -05:00
John Ericson 936f6c6c7d Merge pull request #15144 from obsidiansystems/readline-terminator
libutil: add terminator option to readLine
2026-02-04 17:48:28 +00:00
Artemis Tosini b038500b47 libutil: add terminator option to readLine
Some APIs use "lines" that end in `\0` instead of `\n`.
2026-02-04 11:58:34 -05:00
John Ericson a357d77492 Merge pull request #15141 from NixOS/open-new-file-for-write-helper
libutil: Add openNewFileForWrite helper function, wrap callsites
2026-02-04 03:23:40 +00:00
Sergei Zimmerman 47f261cc19 libutil: Add openNewFileForWrite helper function, wrap callsites
This is purely a fix to use CreateFileW in mingw builds. Also adds some
FIXMEs for suspicious symlink following on truncation that can probably
be tightened down without any problems (other than nix-channel), but for
now this is a no-op change other than consistently using O_CLOEXEC, which
is harmless.
2026-02-04 02:10:12 +03:00
Sergei Zimmerman 27435e0036 Merge pull request #15134 from pkpbynum/pb/fix-query-path-info-daemon
Fix: `QueryPathInfo` throws on invalid path error in daemon
2026-02-03 21:34:55 +00:00
John Ericson 0e2fc2a2f1 Merge pull request #15119 from obsidiansystems/canonicalize-pmd-options
libstore: introduce `CanonicalizePathMetadataOptions` for `canonicalisePathMetaData`
2026-02-03 20:00:27 +00:00
John Ericson 6d6cbf78cc Merge pull request #15137 from NixOS/pragma-once
input-cache.hh: Add missing `#pragma once`
2026-02-03 12:17:33 +00:00
Eelco Dolstra 39a9a004e2 input-cache.hh: Add missing #pragma once 2026-02-03 11:51:04 +01:00
dependabot[bot] 5f9483519a build(deps): bump docker/login-action from 3.6.0 to 3.7.0
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/5e57cd118135c172c3672efd75eb46360885c0ef...c94ce9fb468520275223c153574b00df6fe4bcc9)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-03 03:28:21 +00:00
Peter Bynum b9c77ecafc Fix QueryPathInfo in daemon 2026-02-02 20:52:43 -05:00
Eelco Dolstra d5ce1a79ec Merge pull request #15107 from bouk/fsync-key-generation
nix-store: fsync generated key files
2026-02-02 16:12:38 +00:00
John Ericson 663d27c9df Merge pull request #7892 from obsidiansystems/systemd-multi-socket
Support systemd socket activation with multiple sockets
2026-02-02 16:09:32 +00:00
John Ericson 4da0b36f83 Merge pull request #15127 from NixOS/s3-binary-cache-store-md5
libstore/s3-binary-cache-store: Add Content-MD5 header as message int…
2026-02-01 02:58:28 +00:00
John Ericson a2de09c9fa Support systemd socket activation with multiple sockets
We now support `LISTEN_FDS` values greater than 1, per the systemd
socket activation spec.

These changes are by @edolstra, taken from #5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.

Review with indentation ignored, because some code has moved inside a
new loop but otherwise is mostly unchanged.
2026-01-31 21:52:37 -05:00
Sergei Zimmerman 2ba65f1f26 libstore/s3-binary-cache-store: Add Content-MD5 header as message integrity check
aws-sdk-cpp used to include a checksum for uploads (CRC64 since ~September 2025).
Content-MD5 [1] should be universally supported by all s3 compatible services, since the SDK used
to include it unconditionally too.

[1]: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
2026-02-01 00:26:19 +03:00
John Ericson 77b6b01b72 Merge pull request #15118 from obsidiansystems/exit-status-flags
libstore: introduce `ExitStatusFlags` for exit status computation
2026-01-30 18:37:47 +00:00
Amaan Qureshi 5e7195e1a4 libstore: introduce ExitStatusFlags for exit status computation
This commit consolidates the four separate boolean flags
(`permanentFailure`, `timedOut`, `hashMismatch`, & `checkMismatch`) into
a single `ExitStatusFlags` struct with methods for computing exit status
codes and updating from failure status.
2026-01-30 12:52:51 -05:00
Amaan QureshiandJohn Ericson 78e8896d22 Move HashMismatch wire protocol back compat logic to better spot
The explicit serializer added in
bfdd124837 is the right place to adjust
values for sake of wire protocol compat. The protocol-agnostic `Worker`
code where it was before is the wrong spot.

(That spot was originally chosen because the back compat logic predates
having an explicit serializer for this data type to use instead.)

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-01-30 12:52:51 -05:00
John Ericson d84624d23d Merge pull request #15123 from obsidiansystems/gc-test-vm
Support gc-runtime functional tests in VMs
2026-01-30 17:42:19 +00:00
Artemis Tosini 9e8cf9055a tests/gc-functional: fix running on NixOS
This test insisted on placing profiles in NIX_STATE_DIR, but all
packages were removed from the profile immediately after so they did not
act as garbage collector roots. Switch to directly calling nix-build,
allowing the test to run in VMs without NIX_STATE_DIR.
2026-01-30 11:52:34 -05:00
John Ericson 22372d7889 Merge pull request #15122 from obsidiansystems/gc-test
libstore: fix runtime gc on non-standard store paths
2026-01-30 16:48:20 +00:00
Amaan Qureshi d09f03d742 libstore: introduce CanonicalizePathMetadataOptions for canonicalisePathMetaData
This commit refactors the `canonicalisePathMetaData` function to take an
options struct instead of individual parameters with platform-specific
`#ifdef`s.

The struct contains a `uidRange` field (Unix only) for build user
ownership validation, and an `ignoredAcls` field for ACLs to skip when
removing extended attributes
2026-01-30 11:06:01 -05:00
Artemis Tosini b026649c62 libstore: fix runtime gc roots on non-standard store paths
Due to a typo in quoteRegexChars, finding runtime garbage collection roots
was failing on paths that contained a dot, or any other regex chars that would
have to be replaced.

When fixing that error, also add tests to make sure gc continues to
work.
2026-01-30 11:05:10 -05:00
John Ericson d69ca7bf35 Remove obsolete CPP for Windows in nix-store
`LocalStore` and `canonicalisePathMetaData` are defined on Windows by
now, so we don't have to gate their usage like this.
2026-01-29 19:08:46 -05:00
John Ericson ee6cb7890a Merge pull request #15117 from obsidiansystems/move-impersonate-linux
libstore: add `PersonalityArgs` struct for `setPersonality`
2026-01-29 16:34:12 +00:00
Amaan Qureshi 351b8dd768 libstore: add PersonalityArgs struct for setPersonality
This introduces a `PersonalityArgs` struct to pass named arguments to `setPersonality`. The `impersonateLinux26` setting is now passed from the call site rather than read from settings inside the function.
2026-01-29 10:50:07 -05:00
John Ericson 1713f4c976 Merge pull request #15070 from amaanq/build-result-error-cleanup
build-result: Make `Failure` an alias for `BuildError`
2026-01-29 01:53:14 +00:00
Amaan Qureshi de88141cdf build-result: Make Failure an alias for BuildError and remove exception parameters from goal 2026-01-28 19:32:41 -05:00
John Ericson da9426b8fc Merge pull request #15095 from NixOS/small-fixes
Some small fixes
2026-01-28 21:27:06 +00:00
John Ericson bcbc8ae4e3 Merge pull request #15110 from obsidiansystems/pid-cleanup
libutil: add useful functions to Pid
2026-01-28 20:34:00 +00:00
Artemis Tosini 538e82aa0b libutil: add useful functions to Pid
The C++ rule of five suggests that when a custom destructor is needed
then several other functions are as well. The lack of those makes
certain operations challenging
2026-01-28 13:47:09 -05:00
Bouke van der Bijl 97f71909d7 nix-store: fsync generated key files
Fixes #15106
2026-01-28 11:59:46 +01:00
John Ericson c0ab135860 Some small fixes
I think this has to do with the 25.11 bump.
2026-01-27 00:06:59 -05:00
John Ericson 1100c9dc23 Support readLinkAt and openFileEnsureBeneathNoSymlinks on Windows too
This means that `RestoreSink` can work in the TOCTOU-resilliant way on
Windows too. And it also bodes will for the upcoming OS source accessor
improvements.

A few misc little refactors around error handling and whatnot are done
along the way too. (No more attempt to support pre Windows Vista! lol.)

This cannot be realiably automatically tested until we have a newer
version of Wine, but it does build, so I am inclined to say we just try
it for now.
2026-01-22 17:35:28 -05:00
Kai Oberbeckmann cd6eb7e473 Install nix-manual in default user profile
This makes man pages available in the default profile after using nix
installer.

Relates to: https://github.com/NixOS/nix/issues/12382
2026-01-09 16:16:44 +01:00
355 changed files with 9254 additions and 4966 deletions

No files matched your search

+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
uses: korthout/backport-action@c656f5d5851037b2b38fb5db2691a03fa229e3b2 # v4.0.1
uses: korthout/backport-action@01619ebc9a6e3f6820274221b9956b3e7365000a # v4.1.0
id: backport
with:
# Config README: https://github.com/korthout/backport-action#backport-action
+2 -2
View File
@@ -174,7 +174,7 @@ jobs:
echo "installer-url=file://$GITHUB_WORKSPACE/out" >> "$GITHUB_OUTPUT"
TARBALL_PATH="$(find "$GITHUB_WORKSPACE/out" -name 'nix*.tar.xz' -print | head -n 1)"
echo "tarball-path=file://$TARBALL_PATH" >> "$GITHUB_OUTPUT"
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31.9.0
- uses: cachix/install-nix-action@2126ae7fc54c9df00dd18f7f18754393182c73cd # v31.9.1
if: ${{ !matrix.experimental-installer }}
with:
install_url: ${{ format('{0}/install', steps.installer-tarball-url.outputs.installer-url) }}
@@ -222,7 +222,7 @@ jobs:
id: installer-tarball-url
run: |
echo "installer-url=file://$GITHUB_WORKSPACE/out" >> "$GITHUB_OUTPUT"
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31.9.0
- uses: cachix/install-nix-action@2126ae7fc54c9df00dd18f7f18754393182c73cd # v31.9.1
with:
install_url: ${{ format('{0}/install', steps.installer-tarball-url.outputs.installer-url) }}
install_options: ${{ format('--tarball-url-prefix {0}', steps.installer-tarball-url.outputs.installer-url) }}
+13 -2
View File
@@ -39,13 +39,24 @@ jobs:
role-to-assume: "arn:aws:iam::080433136561:role/nix-release"
role-session-name: nix-release-oidc-${{ github.run_id }}
aws-region: eu-west-1
- name: Disable containerd image store
run: |
# Docker 28+ defaults to the containerd image store, which
# pushes layers uncompressed instead of gzip. OCI clients
# that only support gzip (e.g. go-containerregistry) fail
# with "gzip: invalid header". Disabling the containerd
# snapshotter restores the classic storage driver, which
# preserves gzip-compressed layers through the
# `docker load` / `docker push` pipeline.
echo '{"features":{"containerd-snapshotter":false}}' | sudo tee /etc/docker/daemon.json > /dev/null
sudo systemctl restart docker
- name: Login to Docker Hub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ghcr.io
username: ${{ github.actor }}
+29
View File
@@ -0,0 +1,29 @@
---
synopsis: "Rust nix-installer in beta"
prs: []
---
The Rust-based rewrite of the Nix installer is now in beta.
We'd love help testing it out!
To test out the new installer, run:
```
curl -sSfL https://artifacts.nixos.org/nix-installer | sh -s -- install
```
This installer can be run even when you have an existing, script-based Nix installation without any adjustments.
This new installer also comes with the ability to uninstall your Nix installation; run:
```
/nix/nix-installer uninstall
```
This will get rid of your entire Nix installation (even if you installed over an existing, script-based installation).
This installer is a modified version of the [Determinate Nix Installer](https://github.com/DeterminateSystems/nix-installer) by Determinate Systems.
Thanks to Determinate Systems for all the investment they've put into the installer.
Source for the installer is in https://github.com/NixOS/nix-installer.
Report any issues in that repo.
For CI usage, a GitHub Action to install Nix using this installer is available at https://github.com/NixOS/nix-installer-action.
@@ -6,3 +6,4 @@ prs: [14766]
The C API now includes additional methods:
- `nix_store_query_path_from_hash_part()` - Get the full store path given its hash part
- `nix_store_copy_path()` - Copy a single store path between two stores, allows repairs and configuring signature checking
@@ -0,0 +1,23 @@
---
synopsis: "C API: Errors returned from your primops are not treated as recoverable by default"
prs: [15286, 13930]
---
Nix 2.34 by default remembers the error in the thunk that triggered it.
Previously the following sequence of events worked:
1. Have a thunk that invokes a primop that's defined through the C API
2. The primop returns an error
3. Force the thunk again
4. The primop returns a value
5. The thunk evaluated successfully
**Resolution**
C API consumers that rely on this must change their recoverable error calls:
```diff
-nix_set_err_msg(context, NIX_ERR_*, msg);
+nix_set_err_msg(context, NIX_ERR_RECOVERABLE, msg);
```
@@ -0,0 +1,6 @@
---
synopsis: New setting `narinfo-cache-meta-ttl`
prs: [15287]
---
The new setting `narinfo-cache-meta-ttl` controls how long binary cache metadata (i.e. `/nix-cache-info`) is cached locally, in seconds. This was previously hard-coded to 7 days, which is still the default. As a result, you can now use `nix store info --refresh` to check whether a binary cache is still valid.
+11
View File
@@ -0,0 +1,11 @@
---
synopsis: New command `nix store roots-daemon` for serving GC roots
prs: [15143]
---
New command [`nix store roots-daemon`](@docroot@/command-ref/new-cli/nix3-store-roots-daemon.md) runs a daemon that serves garbage collector roots over a Unix domain socket.
It enables the garbage collector to discover runtime roots when the main Nix daemon doesn't have `CAP_SYS_PTRACE` capability and therefore cannot scan `/proc`.
The garbage collector can be configured to use this daemon via the [`use-roots-daemon`](@docroot@/store/types/local-store.md#store-experimental-option-use-roots-daemon) store setting.
This feature requires the [`local-overlay-store` experimental feature](@docroot@/development/experimental-features.md#xp-feature-local-overlay-store).
@@ -0,0 +1,32 @@
---
synopsis: S3 binary caches now use virtual-hosted-style addressing by default
issues: [15208]
---
S3 binary caches now use virtual-hosted-style URLs
(`https://bucket.s3.region.amazonaws.com/key`) instead of path-style URLs
(`https://s3.region.amazonaws.com/bucket/key`) when connecting to standard AWS
S3 endpoints. This enables HTTP/2 multiplexing and fixes TCP connection
exhaustion (TIME_WAIT socket accumulation) under high-concurrency workloads.
A new `addressing-style` store option controls this behavior:
- `auto` (default): virtual-hosted-style for standard AWS endpoints, path-style
for custom endpoints.
- `path`: forces path-style addressing (deprecated by AWS).
- `virtual`: forces virtual-hosted-style addressing (bucket names must not
contain dots).
Bucket names containing dots (e.g., `my.bucket.name`) automatically fall back
to path-style addressing in `auto` mode, because dotted names create
multi-level subdomains that break TLS wildcard certificate validation.
Example using path-style for backwards compatibility:
```
s3://my-bucket/key?region=us-east-1&addressing-style=path
```
Additionally, TCP keep-alive is now enabled on all HTTP connections, preventing
idle connections from being silently dropped by intermediate network devices
(NATs, firewalls, load balancers).
-1
View File
@@ -358,7 +358,6 @@ dockerTools.buildLayeredImageWithNixDb {
extraCommands = ''
rm -rf nix-support
ln -s /nix/var/nix/profiles nix/var/nix/gcroots/profiles
'';
fakeRootCommands = ''
chmod 1777 tmp
Generated
+4 -4
View File
@@ -63,11 +63,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1769089682,
"narHash": "sha256-Xu+7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4=",
"rev": "078d69f03934859a181e81ba987c2bb033eebfc5",
"lastModified": 1771043024,
"narHash": "sha256-WoiezqWJQ3OHILah+p6rzNXdJceEAmAhyDFZFZ6pZzY=",
"rev": "3aadb7ca9eac2891d52a9dec199d9580a6e2bf44",
"type": "tarball",
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz"
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz"
},
"original": {
"type": "tarball",
+5
View File
@@ -115,6 +115,9 @@
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
}
// lib.optionalAttrs (crossSystem == "x86_64-w64-mingw32") {
emulator = pkgs: "${pkgs.buildPackages.wineWow64Packages.stable_11}/bin/wine";
};
overlays = [
(overlayFor (pkgs: pkgs.${stdenv}))
@@ -406,6 +409,8 @@
"nix-cmd" = { };
"nix-nswrapper" = { };
"nix-cli" = { };
"nix-everything" = { };
+4
View File
@@ -24,6 +24,10 @@ subproject('libcmd')
# Executables
subproject('nix')
if host_machine.system() == 'linux'
subproject('nswrapper')
endif
# Docs
if get_option('doc-gen')
subproject('internal-api-docs')
@@ -22,6 +22,8 @@ add_project_arguments(
'-Werror=undef',
'-Werror=unused-result',
'-Werror=sign-compare',
'-Werror=return-type',
'-Werror=non-virtual-dtor',
'-Wignored-qualifiers',
'-Wimplicit-fallthrough',
'-Wno-deprecated-declarations',
@@ -31,6 +33,13 @@ add_project_arguments(
# GCC doesn't benefit much from precompiled headers.
do_pch = cxx.get_id() == 'clang'
if cxx.get_id() == 'gcc'
add_project_arguments(
'-Wno-interference-size', # Used for C++ ABI only. We don't provide any guarantees about different march tunings.
language : 'cpp',
)
endif
# This is a clang-only option for improving build times.
# It forces the instantiation of templates in the PCH itself and
# not every translation unit it's included in.
@@ -40,6 +49,11 @@ do_pch = cxx.get_id() == 'clang'
# instantiations in libutil and libstore.
if cxx.get_id() == 'clang'
add_project_arguments('-fpch-instantiate-templates', language : 'cpp')
# Catch brace elision bugs: when WorkerProto::Version changed from `unsigned int`
# to `struct { unsigned int major; uint8_t minor; }`, `.version = 16` silently
# became `.version = {16, 0}` instead of failing, breaking protocol compatibility
# in a subtle way
add_project_arguments('-Werror=c99-designator', language : 'cpp')
endif
# Detect if we're using libstdc++ (GCC's standard library)
+3
View File
@@ -4,6 +4,7 @@
buildPackages,
cacert,
nix,
nixComponents2,
}:
let
@@ -11,6 +12,7 @@ let
installerClosureInfo = buildPackages.closureInfo {
rootPaths = [
nix
nixComponents2.nix-manual.man
cacert
];
};
@@ -42,6 +44,7 @@ runCommand "nix-binary-tarball-${version}" env ''
--subst-var-by cacert ${cacert}
substitute ${../scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by nix-manual ${nixComponents2.nix-manual.man} \
--subst-var-by cacert ${cacert}
if type -p shellcheck; then
+2
View File
@@ -418,6 +418,8 @@ in
nix-cmd = callPackage ../src/libcmd/package.nix { };
nix-nswrapper = callPackage ../src/nswrapper/package.nix { };
/**
The Nix command line interface. Note that this does not include its tests, whereas `nix-everything` does.
*/
+5
View File
@@ -31,6 +31,8 @@
nix-cmd,
nix-nswrapper,
nix-cli,
nix-functional-tests,
@@ -171,6 +173,9 @@ stdenv.mkDerivation (finalAttrs: {
# Forwarded outputs
ln -sT ${nix-manual} $doc
ln -sT ${nix-manual.man} $man
''
+ lib.optionalString stdenv.isLinux ''
lndir ${nix-nswrapper} $out
'';
passthru = {
+1
View File
@@ -57,6 +57,7 @@ let
"nix-flake"
"nix-flake-c"
"nix-flake-tests"
"nix-nswrapper"
"nix-main"
"nix-main-c"
"nix-cmd"
+3
View File
@@ -52,6 +52,7 @@ readonly PROFILE_FISH_PREFIXES=(
readonly PROFILE_NIX_FILE_FISH="$NIX_ROOT/var/nix/profiles/default/etc/profile.d/nix-daemon.fish"
readonly NIX_INSTALLED_NIX="@nix@"
readonly NIX_INSTALLED_NIX_MAN="@nix-manual@"
readonly NIX_INSTALLED_CACERT="@cacert@"
#readonly NIX_INSTALLED_NIX="/nix/store/byi37zv50wnfrpp4d81z3spswd5zva37-nix-2.3.6"
#readonly NIX_INSTALLED_CACERT="/nix/store/7pi45g541xa8ahwgpbpy7ggsl0xj1jj6-nss-cacert-3.49.2"
@@ -969,6 +970,8 @@ setup_default_profile() {
task "Setting up the default profile"
_sudo "to install a bootstrapping Nix in to the default profile" \
HOME="$ROOT_HOME" "$NIX_INSTALLED_NIX/bin/nix-env" -i "$NIX_INSTALLED_NIX"
_sudo "to install Nix man pages in to the default profile" \
HOME="$ROOT_HOME" "$NIX_INSTALLED_NIX/bin/nix-env" -i "$NIX_INSTALLED_NIX_MAN"
if [ -z "${NIX_SSL_CERT_FILE:-}" ] || ! [ -f "${NIX_SSL_CERT_FILE:-}" ] || cert_in_store; then
_sudo "to install a bootstrapping SSL certificate just for Nix in to the default profile" \
+43 -15
View File
@@ -4,6 +4,7 @@
#include "nix/cmd/command.hh"
#include "nix/cmd/legacy.hh"
#include "nix/cmd/markdown.hh"
#include "nix/store/globals.hh"
#include "nix/store/store-open.hh"
#include "nix/store/local-fs-store.hh"
#include "nix/store/derivations.hh"
@@ -63,6 +64,25 @@ void NixMultiCommand::run()
command->second->run();
}
StoreConfigCommand::StoreConfigCommand() {}
ref<StoreConfig> StoreConfigCommand::getStoreConfig()
{
if (!_storeConfig)
_storeConfig = createStoreConfig();
return ref<StoreConfig>(_storeConfig);
}
ref<StoreConfig> StoreConfigCommand::createStoreConfig()
{
return resolveStoreConfig(StoreReference{settings.storeUri.get()});
}
void StoreConfigCommand::run()
{
run(getStoreConfig());
}
StoreCommand::StoreCommand() {}
ref<Store> StoreCommand::getStore()
@@ -74,12 +94,20 @@ ref<Store> StoreCommand::getStore()
ref<Store> StoreCommand::createStore()
{
return openStore();
auto store = getStoreConfig()->openStore();
store->init();
return store;
}
void StoreCommand::run()
void StoreCommand::run(ref<StoreConfig> storeConfig)
{
run(getStore());
// We can either efficiently implement getStore/createStore with memoization,
// or use the StoreConfig passed in run.
// It's more efficient to memoize, especially since there are some direct users
// of getStore. The StoreConfig in both cases should be the same, though.
auto store = getStore();
assert(&*storeConfig == &store->config);
run(std::move(store));
}
CopyCommand::CopyCommand()
@@ -88,28 +116,28 @@ CopyCommand::CopyCommand()
.longName = "from",
.description = "URL of the source Nix store.",
.labels = {"store-uri"},
.handler = {&srcUri},
.handler = {[this](std::string s) { srcUri = StoreReference::parse(s); }},
});
addFlag({
.longName = "to",
.description = "URL of the destination Nix store.",
.labels = {"store-uri"},
.handler = {&dstUri},
.handler = {[this](std::string s) { dstUri = StoreReference::parse(s); }},
});
}
ref<Store> CopyCommand::createStore()
ref<StoreConfig> CopyCommand::createStoreConfig()
{
return srcUri.empty() ? StoreCommand::createStore() : openStore(srcUri);
return !srcUri ? StoreCommand::createStoreConfig() : resolveStoreConfig(StoreReference{*srcUri});
}
ref<Store> CopyCommand::getDstStore()
{
if (srcUri.empty() && dstUri.empty())
if (!srcUri && !dstUri)
throw UsageError("you must pass '--from' and/or '--to'");
return dstUri.empty() ? openStore() : openStore(dstUri);
return !dstUri ? openStore() : openStore(StoreReference{*dstUri});
}
EvalCommand::EvalCommand()
@@ -131,7 +159,7 @@ EvalCommand::~EvalCommand()
ref<Store> EvalCommand::getEvalStore()
{
if (!evalStore)
evalStore = evalStoreUrl ? openStore(*evalStoreUrl) : getStore();
evalStore = evalStoreUrl ? openStore(StoreReference{*evalStoreUrl}) : getStore();
return ref<Store>(evalStore);
}
@@ -257,18 +285,18 @@ MixProfile::MixProfile()
});
}
void MixProfile::updateProfile(const StorePath & storePath)
void MixProfile::updateProfile(Store & store_, const StorePath & storePath)
{
if (!profile)
return;
auto store = getDstStore().dynamic_pointer_cast<LocalFSStore>();
auto * store = dynamic_cast<LocalFSStore *>(&store_);
if (!store)
throw Error("'--profile' is not supported for this Nix store");
auto profile2 = absPath(*profile);
switchLink(profile2, createGeneration(*store, profile2, storePath));
}
void MixProfile::updateProfile(const BuiltPaths & buildables)
void MixProfile::updateProfile(Store & store, const BuiltPaths & buildables)
{
if (!profile)
return;
@@ -292,12 +320,12 @@ void MixProfile::updateProfile(const BuiltPaths & buildables)
throw UsageError(
"'--profile' requires that the arguments produce a single store path, but there are %d", result.size());
updateProfile(result[0]);
updateProfile(store, result[0]);
}
MixDefaultProfile::MixDefaultProfile()
{
profile = getDefaultProfile().string();
profile = getDefaultProfile(settings.getProfileDirsOptions()).string();
}
static constexpr auto environmentVariablesCategory = "Options that change environment variables";
+1 -1
View File
@@ -148,7 +148,7 @@ MixEvalArgs::MixEvalArgs()
)",
.category = category,
.labels = {"store-url"},
.handler = {&evalStoreUrl},
.handler = {[this](std::string s) { evalStoreUrl = StoreReference::parse(s); }},
});
}
+31
View File
@@ -0,0 +1,31 @@
#include "nix/cmd/get-build-log.hh"
#include "nix/store/log-store.hh"
#include "nix/store/store-open.hh"
namespace nix {
std::string fetchBuildLog(ref<Store> store, const StorePath & path, std::string_view what)
{
auto subs = getDefaultSubstituters();
subs.push_front(store);
for (auto & sub : subs) {
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo("Skipped '%s' which does not support retrieving build logs", sub->config.getHumanReadableURI());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(path);
if (!log)
continue;
printInfo("got build log for '%s' from '%s'", what, logSub.config.getHumanReadableURI());
return *log;
}
throw Error("build log of '%s' is not available", what);
}
} // namespace nix
+32 -16
View File
@@ -5,6 +5,7 @@
#include "nix/util/args.hh"
#include "nix/cmd/common-eval-args.hh"
#include "nix/store/path.hh"
#include "nix/store/store-reference.hh"
#include "nix/flake/lockfile.hh"
#include <optional>
@@ -40,28 +41,43 @@ struct NixMultiCommand : MultiCommand, virtual Command
// For the overloaded run methods
#pragma GCC diagnostic ignored "-Woverloaded-virtual"
/**
* A command that requires a \ref StoreConfig store configuration.
*/
struct StoreConfigCommand : virtual Command
{
StoreConfigCommand();
void run() override;
/**
* Return the default Nix store configuration.
*/
ref<StoreConfig> getStoreConfig();
virtual ref<StoreConfig> createStoreConfig();
/**
* Main entry point, with a `StoreConfig` provided
*/
virtual void run(ref<StoreConfig>) = 0;
private:
std::shared_ptr<StoreConfig> _storeConfig;
};
/**
* A command that requires a \ref Store "Nix store".
*/
struct StoreCommand : virtual Command
struct StoreCommand : virtual StoreConfigCommand
{
StoreCommand();
void run() override;
void run(ref<StoreConfig>) override;
/**
* Return the default Nix store.
*/
ref<Store> getStore();
/**
* Return the destination Nix store.
*/
virtual ref<Store> getDstStore()
{
return getStore();
}
virtual ref<Store> createStore();
ref<Store> createStore();
/**
* Main entry point, with a `Store` provided
*/
@@ -77,13 +93,13 @@ private:
*/
struct CopyCommand : virtual StoreCommand
{
std::string srcUri, dstUri;
std::optional<StoreReference> srcUri, dstUri;
CopyCommand();
ref<Store> createStore() override;
ref<StoreConfig> createStoreConfig() override;
ref<Store> getDstStore() override;
ref<Store> getDstStore();
};
/**
@@ -315,11 +331,11 @@ struct MixProfile : virtual StoreCommand
MixProfile();
/* If 'profile' is set, make it point at 'storePath'. */
void updateProfile(const StorePath & storePath);
void updateProfile(Store & store, const StorePath & storePath);
/* If 'profile' is set, make it point at the store path produced
by 'buildables'. */
void updateProfile(const BuiltPaths & buildables);
void updateProfile(Store & store, const BuiltPaths & buildables);
};
struct MixDefaultProfile : MixProfile
@@ -6,6 +6,7 @@
#include "nix/main/common-args.hh"
#include "nix/expr/search-path.hh"
#include "nix/expr/eval-settings.hh"
#include "nix/store/store-reference.hh"
#include <filesystem>
@@ -55,7 +56,7 @@ struct MixEvalArgs : virtual Args, virtual MixRepair
LookupPath lookupPath;
std::optional<std::string> evalStoreUrl;
std::optional<StoreReference> evalStoreUrl;
private:
struct AutoArgExpr
@@ -0,0 +1,23 @@
#pragma once
///@file
#include "nix/store/store-api.hh"
#include <string>
#include <string_view>
namespace nix {
/**
* Fetch the build log for a store path, searching the store and its
* substituters.
*
* @param store The store to search (and its substituters).
* @param path The store path to get the build log for.
* @param what A description of what we're fetching the log for (used in messages).
* @return The build log content.
* @throws Error if the build log is not available.
*/
std::string fetchBuildLog(ref<Store> store, const StorePath & path, std::string_view what);
} // namespace nix
+2
View File
@@ -9,6 +9,7 @@ headers = files(
'common-eval-args.hh',
'compatibility-settings.hh',
'editor-for.hh',
'get-build-log.hh',
'installable-attr-path.hh',
'installable-derived-path.hh',
'installable-flake.hh',
@@ -20,4 +21,5 @@ headers = files(
'network-proxy.hh',
'repl-interacter.hh',
'repl.hh',
'unix-socket-server.hh',
)
@@ -14,6 +14,7 @@ namespace detail {
struct ReplCompleterMixin
{
virtual StringSet completePrefix(const std::string & prefix) = 0;
virtual ~ReplCompleterMixin() = default;
};
}; // namespace detail
+1 -2
View File
@@ -25,7 +25,7 @@ struct AbstractNixRepl
* @todo this is a layer violation
*
* @param programName Name of the command, e.g. `nix` or `nix-env`.
* @param args aguments to the command.
* @param args arguments to the command.
*/
using RunNix =
void(const std::string & programName, const Strings & args, const std::optional<std::string> & input);
@@ -37,7 +37,6 @@ struct AbstractNixRepl
*/
static std::unique_ptr<AbstractNixRepl> create(
const LookupPath & lookupPath,
nix::ref<Store> store,
ref<EvalState> state,
std::function<AnnotatedValues()> getValues,
RunNix * runNix = nullptr);
@@ -0,0 +1,79 @@
#pragma once
///@file
#include "nix/util/file-descriptor.hh"
#include <filesystem>
#include <functional>
#include <optional>
#include <sys/types.h>
namespace nix::unix {
/**
* Information about the identity of the peer on a Unix domain socket connection.
*/
struct PeerInfo
{
std::optional<pid_t> pid;
std::optional<uid_t> uid;
std::optional<gid_t> gid;
};
/**
* Get the identity of the caller, if possible.
*/
PeerInfo getPeerInfo(Descriptor remote);
/**
* Callback type for handling new connections.
*
* The callback receives ownership of the connection and is responsible
* for handling it (e.g., forking a child process, spawning a thread, etc.).
*
* @param socket The accepted connection file descriptor.
* @param closeListeners A callback to close the listening sockets.
* Useful in forked child processes to release the bound sockets.
*/
using UnixSocketHandler = std::function<void(AutoCloseFD socket, std::function<void()> closeListeners)>;
/**
* Options for the serve loop.
*
* Only used if no systemd socket activation is detected.
*/
struct ServeUnixSocketOptions
{
/**
* The Unix domain socket path to create and listen on.
*/
std::filesystem::path socketPath;
/**
* Mode for the created socket file.
*/
mode_t socketMode = 0666;
};
/**
* Run a server loop that accepts connections and calls the handler for each.
*
* This function handles:
* - systemd socket activation (via LISTEN_FDS environment variable)
* - Creating and binding a Unix domain socket if no activation is detected
* - Polling for incoming connections
* - Accepting connections
*
* For each accepted connection, the handler is called with the connection
* file descriptor. The handler takes ownership of the file descriptor and
* is responsible for closing it when done.
*
* This function never returns normally. It runs until interrupted
* (e.g., via SIGINT), at which point it throws `Interrupted`.
*
* @param options Configuration for the server.
* @param handler Callback invoked for each accepted connection.
*/
[[noreturn]] void serveUnixSocket(const ServeUnixSocketOptions & options, UnixSocketHandler handler);
} // namespace nix::unix
+4 -2
View File
@@ -203,8 +203,10 @@ FlakeRef InstallableFlake::nixpkgsFlakeRef() const
if (auto nixpkgsInput = lockedFlake->lockFile.findInput({"nixpkgs"})) {
if (auto lockedNode = std::dynamic_pointer_cast<const flake::LockedNode>(nixpkgsInput)) {
debug("using nixpkgs flake '%s'", lockedNode->lockedRef);
return std::move(lockedNode->lockedRef);
if (lockedNode->isFlake) {
debug("using nixpkgs flake '%s'", lockedNode->lockedRef);
return std::move(lockedNode->lockedRef);
}
}
}
+3 -7
View File
@@ -583,16 +583,12 @@ static void throwBuildErrors(std::vector<KeyedBuildResult> & buildResults, const
auto failedResult = failed.begin();
if (failedResult != failed.end()) {
if (failed.size() == 1) {
failedResult->second->rethrow();
throw *failedResult->second;
} else {
StringSet failedPaths;
for (; failedResult != failed.end(); failedResult++) {
if (!failedResult->second->errorMsg.empty()) {
logError(
ErrorInfo{
.level = lvlError,
.msg = failedResult->second->errorMsg,
});
if (!failedResult->second->message().empty()) {
logError(failedResult->second->info());
}
failedPaths.insert(failedResult->first->path.to_string(store));
}
+7
View File
@@ -74,6 +74,7 @@ sources = files(
'command.cc',
'common-eval-args.cc',
'editor-for.cc',
'get-build-log.cc',
'installable-attr-path.cc',
'installable-derived-path.cc',
'installable-flake.cc',
@@ -86,6 +87,12 @@ sources = files(
'repl.cc',
)
if host_machine.system() != 'windows'
sources += files(
'unix/unix-socket-server.cc',
)
endif
subdir('include/nix/cmd')
subdir('nix-meson-build-support/export-all-symbols')
+9 -4
View File
@@ -40,8 +40,8 @@ void sigintHandler(int signo)
static detail::ReplCompleterMixin * curRepl; // ugly
#if !USE_READLINE
static char * completionCallback(char * s, int * match)
{
static char * completionCallback(char * s, int * match) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
@@ -73,10 +73,12 @@ static char * completionCallback(char * s, int * match)
*match = 0;
return nullptr;
} catch (...) {
return nullptr;
}
static int listPossibleCallback(char * s, char *** avp)
{
static int listPossibleCallback(char * s, char *** avp) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() > (std::numeric_limits<int>::max() / sizeof(char *)))
@@ -105,6 +107,9 @@ static int listPossibleCallback(char * s, char *** avp)
*avp = vp;
return ac;
} catch (...) {
*avp = nullptr;
return 0;
}
#endif
+19 -41
View File
@@ -12,9 +12,8 @@
#include "nix/expr/eval-settings.hh"
#include "nix/expr/attr-path.hh"
#include "nix/util/signals.hh"
#include "nix/store/store-open.hh"
#include "nix/store/log-store.hh"
#include "nix/cmd/common-eval-args.hh"
#include "nix/cmd/get-build-log.hh"
#include "nix/expr/get-drvs.hh"
#include "nix/store/derivations.hh"
#include "nix/store/globals.hh"
@@ -65,7 +64,7 @@ struct NixRepl : AbstractNixRepl, detail::ReplCompleterMixin, gc
const static int envSize = 32768;
std::shared_ptr<StaticEnv> staticEnv;
Value lastLoaded;
std::optional<Value> lastLoaded;
Env * env;
int displ;
StringSet varNames;
@@ -78,7 +77,6 @@ struct NixRepl : AbstractNixRepl, detail::ReplCompleterMixin, gc
NixRepl(
const LookupPath & lookupPath,
nix::ref<Store> store,
ref<EvalState> state,
std::function<AnnotatedValues()> getValues,
RunNix * runNix);
@@ -133,7 +131,6 @@ std::string removeWhitespace(std::string s)
NixRepl::NixRepl(
const LookupPath & lookupPath,
nix::ref<Store> store,
ref<EvalState> state,
std::function<NixRepl::AnnotatedValues()> getValues,
RunNix * runNix)
@@ -566,31 +563,9 @@ ProcessLineResult NixRepl::processLine(std::string line)
} else if (command == ":log") {
settings.readOnlyMode = true;
Finally roModeReset([&]() { settings.readOnlyMode = false; });
auto subs = getDefaultSubstituters();
subs.push_front(state->store);
bool foundLog = false;
RunPager pager;
for (auto & sub : subs) {
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo(
"Skipped '%s' which does not support retrieving build logs", sub->config.getHumanReadableURI());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.config.getHumanReadableURI());
logger->writeToStdout(*log);
foundLog = true;
break;
}
}
if (!foundLog)
throw Error("build log of '%s' is not available", drvPathRaw);
auto log = fetchBuildLog(state->store, drvPath, drvPathRaw);
logger->writeToStdout(log);
} else {
runNix("nix-shell", {drvPathRaw});
}
@@ -735,7 +710,7 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
try {
cwd = std::filesystem::current_path();
} catch (std::filesystem::filesystem_error & e) {
throw SysError("cannot determine current working directory");
throw SystemError(e.code(), "cannot determine current working directory");
}
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, cwd.string(), true);
@@ -773,11 +748,19 @@ void NixRepl::initEnv()
void NixRepl::showLastLoaded()
{
RunPager pager;
if (!lastLoaded)
throw Error("nothing has been loaded yet");
for (auto & i : *lastLoaded.attrs()) {
std::string_view name = state->symbols[i.name];
logger->cout(name);
RunPager pager;
try {
for (auto & i : *lastLoaded->attrs()) {
std::string_view name = state->symbols[i.name];
logger->cout(name);
}
} catch (SystemError & e) {
/* Ignore broken pipes when the pager gets interrupted. */
if (!e.is(std::errc::broken_pipe))
throw;
}
}
@@ -900,13 +883,9 @@ void NixRepl::runNix(const std::string & program, const Strings & args, const st
}
std::unique_ptr<AbstractNixRepl> AbstractNixRepl::create(
const LookupPath & lookupPath,
nix::ref<Store> store,
ref<EvalState> state,
std::function<AnnotatedValues()> getValues,
RunNix * runNix)
const LookupPath & lookupPath, ref<EvalState> state, std::function<AnnotatedValues()> getValues, RunNix * runNix)
{
return std::make_unique<NixRepl>(lookupPath, std::move(store), state, getValues, runNix);
return std::make_unique<NixRepl>(lookupPath, state, getValues, runNix);
}
ReplExitStatus AbstractNixRepl::runSimple(ref<EvalState> evalState, const ValMap & extraEnv)
@@ -919,7 +898,6 @@ ReplExitStatus AbstractNixRepl::runSimple(ref<EvalState> evalState, const ValMap
// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDelete)
auto repl = std::make_unique<NixRepl>(
lookupPath,
openStore(),
evalState,
getValues,
/*runNix=*/nullptr);
+126
View File
@@ -0,0 +1,126 @@
///@file
#include "nix/cmd/unix-socket-server.hh"
#include "nix/util/environment-variables.hh"
#include "nix/util/file-system.hh"
#include "nix/util/logging.hh"
#include "nix/util/signals.hh"
#include "nix/util/strings.hh"
#include "nix/util/unix-domain-socket.hh"
#include "nix/util/util.hh"
#include <sys/socket.h>
#include <sys/un.h>
#include <poll.h>
#if defined(__APPLE__) || defined(__FreeBSD__)
# include <sys/ucred.h>
#endif
namespace nix::unix {
PeerInfo getPeerInfo(Descriptor remote)
{
PeerInfo peer;
#if defined(SO_PEERCRED)
# if defined(__OpenBSD__)
struct sockpeercred cred;
# else
ucred cred;
# endif
socklen_t credLen = sizeof(cred);
if (getsockopt(remote, SOL_SOCKET, SO_PEERCRED, &cred, &credLen) == 0) {
peer.pid = cred.pid;
peer.uid = cred.uid;
peer.gid = cred.gid;
}
#elif defined(LOCAL_PEERCRED)
# if !defined(SOL_LOCAL)
# define SOL_LOCAL 0
# endif
xucred cred;
socklen_t credLen = sizeof(cred);
if (getsockopt(remote, SOL_LOCAL, LOCAL_PEERCRED, &cred, &credLen) == 0)
peer.uid = cred.cr_uid;
#endif
return peer;
}
[[noreturn]] void serveUnixSocket(const ServeUnixSocketOptions & options, UnixSocketHandler handler)
{
std::vector<AutoCloseFD> listeningSockets;
static constexpr int SD_LISTEN_FDS_START = 3;
// Handle socket-based activation by systemd.
auto listenFds = getEnv("LISTEN_FDS");
if (listenFds) {
if (getEnv("LISTEN_PID") != std::to_string(getpid()))
throw Error("unexpected systemd environment variables");
auto count = string2Int<unsigned int>(*listenFds);
assert(count);
for (unsigned int i = 0; i < count; ++i) {
AutoCloseFD fdSocket(SD_LISTEN_FDS_START + i);
closeOnExec(fdSocket.get());
listeningSockets.push_back(std::move(fdSocket));
}
}
// Otherwise, create and bind to a Unix domain socket.
else {
createDirs(options.socketPath.parent_path());
listeningSockets.push_back(createUnixDomainSocket(options.socketPath.string(), options.socketMode));
}
std::vector<struct pollfd> fds;
for (auto & i : listeningSockets)
fds.push_back({.fd = i.get(), .events = POLLIN});
// Loop accepting connections.
while (1) {
try {
checkInterrupt();
auto count = poll(fds.data(), fds.size(), -1);
if (count == -1) {
if (errno == EINTR)
continue;
throw SysError("polling for incoming connections");
}
for (auto & fd : fds) {
if (!fd.revents)
continue;
// Accept a connection.
struct sockaddr_un remoteAddr;
socklen_t remoteAddrLen = sizeof(remoteAddr);
AutoCloseFD remote = accept(fd.fd, (struct sockaddr *) &remoteAddr, &remoteAddrLen);
checkInterrupt();
if (!remote) {
if (errno == EINTR)
continue;
throw SysError("accepting connection");
}
handler(std::move(remote), [&]() { listeningSockets.clear(); });
}
} catch (Error & error) {
auto ei = error.info();
// FIXME: add to trace?
ei.msg = HintFmt("while processing connection: %1%", ei.msg.str());
logError(ei);
}
}
}
} // namespace nix::unix
+3 -1
View File
@@ -185,7 +185,9 @@ EvalState * nix_eval_state_build(nix_c_context * context, nix_eval_state_builder
return EvalState{
.fetchSettings = std::move(builder->fetchSettings),
.settings = std::move(builder->settings),
.state = nix::EvalState(builder->lookupPath, builder->store, self->fetchSettings, self->settings),
.statePtr = std::make_shared<nix::EvalState>(
builder->lookupPath, builder->store, self->fetchSettings, self->settings),
.state = *self->statePtr,
};
});
}
+2 -1
View File
@@ -24,7 +24,8 @@ struct EvalState
{
nix::fetchers::Settings fetchSettings;
nix::EvalSettings settings;
nix::EvalState state;
std::shared_ptr<nix::EvalState> statePtr;
nix::EvalState & state;
};
struct BindingsBuilder
+10 -2
View File
@@ -1,4 +1,5 @@
#include "nix/expr/attr-set.hh"
#include "nix/expr/eval-error.hh"
#include "nix/util/configuration.hh"
#include "nix/expr/eval.hh"
#include "nix/store/globals.hh"
@@ -107,8 +108,13 @@ static void nix_c_primop_wrapper(
f(userdata, &ctx, (EvalState *) &state, external_args.data(), vTmpPtr);
if (ctx.last_err_code != NIX_OK) {
/* TODO: Throw different errors depending on the error code */
state.error<nix::EvalError>("Error from custom function: %s", *ctx.last_err).atPos(pos).debugThrow();
if (ctx.last_err_code == NIX_ERR_RECOVERABLE) {
state.error<nix::RecoverableEvalError>("Recoverable error from custom function: %s", *ctx.last_err)
.atPos(pos)
.debugThrow();
} else {
state.error<nix::EvalError>("Error from custom function: %s", *ctx.last_err).atPos(pos).debugThrow();
}
}
if (!vTmp.isValid()) {
@@ -194,6 +200,8 @@ ValueType nix_get_type(nix_c_context * context, const nix_value * value)
switch (v.type()) {
case nThunk:
return NIX_TYPE_THUNK;
case nFailed:
return NIX_TYPE_FAILED;
case nInt:
return NIX_TYPE_INT;
case nFloat:
+2 -1
View File
@@ -100,7 +100,8 @@ typedef enum {
/** @brief External value from C++ plugins or C API
* @see Externals
*/
NIX_TYPE_EXTERNAL
NIX_TYPE_EXTERNAL,
NIX_TYPE_FAILED,
} ValueType;
// forward declarations
+2 -2
View File
@@ -43,7 +43,7 @@ RC_GTEST_FIXTURE_PROP(
auto * v = state.allocValue();
state.mkOutputString(*v, b, std::nullopt, mockXpSettings);
auto d = state.coerceToSingleDerivedPath(noPos, *v, "", mockXpSettings, true);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
RC_ASSERT(SingleDerivedPath{b} == d);
}
@@ -57,7 +57,7 @@ RC_GTEST_FIXTURE_PROP(
auto * v = state.allocValue();
state.mkOutputString(*v, b, outPath, mockXpSettings);
auto d = state.coerceToSingleDerivedPath(noPos, *v, "", mockXpSettings, true);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
RC_ASSERT(SingleDerivedPath{b} == d);
}
+2 -1
View File
@@ -37,7 +37,8 @@ static void BM_EvalDynamicAttrs(benchmark::State & state)
EvalSettings evalSettings{readOnlyMode};
evalSettings.nixPath = {};
EvalState st({}, store, fetchSettings, evalSettings, nullptr);
auto stPtr = std::make_shared<EvalState>(LookupPath{}, store, fetchSettings, evalSettings, nullptr);
auto & st = *stPtr;
Expr * expr = st.parseExprFromString(exprStr, st.rootPath(CanonPath::root));
Value v;
+4 -2
View File
@@ -16,7 +16,8 @@ struct GetDerivationsEnv
fetchers::Settings fetchSettings{};
bool readOnlyMode = true;
EvalSettings evalSettings{readOnlyMode};
EvalState state;
std::shared_ptr<EvalState> statePtr;
EvalState & state;
Bindings * autoArgs = nullptr;
Value attrsValue;
@@ -27,7 +28,8 @@ struct GetDerivationsEnv
settings.nixPath = {};
return settings;
}())
, state({}, store, fetchSettings, evalSettings, nullptr)
, statePtr(std::make_shared<EvalState>(LookupPath{}, store, fetchSettings, evalSettings, nullptr))
, state(*statePtr)
{
autoArgs = state.buildBindings(0).finish();
+102
View File
@@ -517,4 +517,106 @@ TEST_F(nix_api_expr_test, nix_expr_attrset_update)
assert_ctx_ok();
}
// The following is a test case for retryable thunks. This is a requirement
// for the current way in which NixOps4 evaluates its deployment expressions.
// An alternative strategy could be implemented, but unwinding the stack may
// be a more efficient way to deal with many suspensions/resumptions, compared
// to e.g. using a thread or coroutine stack for each suspended dependency.
// This test models the essential bits of a deployment tool that uses such
// a strategy.
// State for the retryable primop - simulates deployment resource availability
struct DeploymentResourceState
{
bool vm_created = false;
};
static void primop_load_resource_input(
void * user_data, nix_c_context * context, EvalState * state, nix_value ** args, nix_value * ret)
{
assert(context);
assert(state);
auto * resource_state = static_cast<DeploymentResourceState *>(user_data);
// Get the resource input name argument
std::string input_name;
if (nix_get_string(context, args[0], OBSERVE_STRING(input_name)) != NIX_OK)
return;
// Only handle "vm_id" input - throw for anything else
if (input_name != "vm_id") {
std::string error_msg = "unknown resource input: " + input_name;
nix_set_err_msg(context, NIX_ERR_NIX_ERROR, error_msg.c_str());
return;
}
if (resource_state->vm_created) {
// VM has been created, return the ID
nix_init_string(context, ret, "vm-12345");
} else {
// VM not created yet, fail with dependency error
nix_set_err_msg(context, NIX_ERR_RECOVERABLE, "VM not yet created");
}
}
TEST_F(nix_api_expr_test, nix_expr_thunk_re_evaluation_after_deployment)
{
// This test demonstrates NixOps4's requirement: a thunk calling a primop should be
// re-evaluable when deployment resources become available that were not available initially.
DeploymentResourceState resource_state;
PrimOp * primop = nix_alloc_primop(
ctx,
primop_load_resource_input,
1,
"loadResourceInput",
nullptr,
"load a deployment resource input",
&resource_state);
assert_ctx_ok();
nix_value * primopValue = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_init_primop(ctx, primopValue, primop);
assert_ctx_ok();
nix_value * inputName = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_init_string(ctx, inputName, "vm_id");
assert_ctx_ok();
// Create a single thunk by using nix_init_apply instead of nix_value_call
// This creates a lazy application that can be forced multiple times
nix_value * thunk = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_init_apply(ctx, thunk, primopValue, inputName);
assert_ctx_ok();
// First force: VM not created yet, should fail
nix_value_force(ctx, state, thunk);
ASSERT_EQ(NIX_ERR_NIX_ERROR, nix_err_code(ctx));
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("VM not yet created"));
// Clear the error context for the next attempt
nix_c_context_free(ctx);
ctx = nix_c_context_create();
// Simulate deployment process: VM gets created
resource_state.vm_created = true;
// Second force of the SAME thunk: this is where the "failed" value issue appears
// With failed value caching, this should fail because the thunk is marked as permanently failed
// Without failed value caching (or with retryable failures), this should succeed
nix_value_force(ctx, state, thunk);
// If we get here without error, the thunk was successfully re-evaluated
assert_ctx_ok();
std::string result;
nix_get_string(ctx, thunk, OBSERVE_STRING(result));
assert_ctx_ok();
ASSERT_STREQ("vm-12345", result.c_str());
}
} // namespace nixC
+1 -1
View File
@@ -756,7 +756,7 @@ TEST_F(PrimOpTest, langVersion)
TEST_F(PrimOpTest, storeDir)
{
auto v = eval("builtins.storeDir");
ASSERT_THAT(v, IsStringEq(settings.nixStore));
ASSERT_THAT(v, IsStringEq(state.store->storeDir));
}
TEST_F(PrimOpTest, nixVersion)
+2 -1
View File
@@ -27,7 +27,8 @@ static void BM_EvalManyBuiltinsMatchSameRegex(benchmark::State & state)
EvalSettings evalSettings{readOnlyMode};
evalSettings.nixPath = {};
EvalState st({}, store, fetchSettings, evalSettings, nullptr);
auto stPtr = std::make_shared<EvalState>(LookupPath{}, store, fetchSettings, evalSettings, nullptr);
auto & st = *stPtr;
Expr * expr = st.parseExprFromString(std::string(exprStr), st.rootPath(CanonPath::root));
Value v;
+16
View File
@@ -188,6 +188,22 @@ TEST_F(ValuePrintingTests, vBlackhole)
test(vBlackhole, "«potential infinite recursion»");
}
TEST_F(ValuePrintingTests, vFailed)
{
Value v;
try {
throw Error("nope");
} catch (...) {
v.mkFailed(std::current_exception(), nullptr);
}
// Historically, a tried and then ignored value (e.g. through tryEval) was
// reverted to the original thunk.
test(v, "«thunk»");
test(v, ANSI_MAGENTA "«thunk»" ANSI_NORMAL, PrintOptions{.ansiColors = true});
}
TEST_F(ValuePrintingTests, depthAttrs)
{
Value vOne;
+1 -2
View File
@@ -13,8 +13,7 @@ TEST_F(ValueTest, unsetValue)
{
Value unsetValue;
ASSERT_EQ(false, unsetValue.isValid());
ASSERT_EQ(nThunk, unsetValue.type(true));
ASSERT_DEATH(unsetValue.type(), "");
ASSERT_EQ(nThunk, unsetValue.type</*invalidIsThunk=*/true>());
}
TEST_F(ValueTest, vInt)
+2 -2
View File
@@ -11,7 +11,7 @@
namespace nix::eval_cache {
CachedEvalError::CachedEvalError(ref<AttrCursor> cursor, Symbol attr)
: EvalError(cursor->root->state, "cached failure of attribute '%s'", cursor->getAttrPathStr(attr))
: CloneableError(cursor->root->state, "cached failure of attribute '%s'", cursor->getAttrPathStr(attr))
, cursor(cursor)
, attr(attr)
{
@@ -70,7 +70,7 @@ struct AttrDb
{
auto state(_state->lock());
auto cacheDir = std::filesystem::path(getCacheDir()) / "eval-cache-v6";
auto cacheDir = getCacheDir() / "eval-cache-v6";
createDirs(cacheDir);
auto dbPath = cacheDir / (fingerprint.to_string(HashFormat::Base16, false) + ".sqlite");
+1
View File
@@ -114,5 +114,6 @@ template class EvalErrorBuilder<InfiniteRecursionError>;
template class EvalErrorBuilder<StackOverflowError>;
template class EvalErrorBuilder<InvalidPathError>;
template class EvalErrorBuilder<IFDError>;
template class EvalErrorBuilder<RecoverableEvalError>;
} // namespace nix
+8 -2
View File
@@ -133,11 +133,17 @@ class SampleStack : public EvalProfiler
FrameInfo getPrimOpFrameInfo(const PrimOp & primOp, std::span<Value *> args, PosIdx pos);
public:
SampleStack(EvalState & state, std::filesystem::path profileFile, std::chrono::nanoseconds period)
SampleStack(EvalState & state, const std::filesystem::path & profileFile, std::chrono::nanoseconds period)
: state(state)
, sampleInterval(period)
, profileFd([&]() {
AutoCloseFD fd = toDescriptor(open(profileFile.string().c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0660));
AutoCloseFD fd = openNewFileForWrite(
profileFile,
0660,
{
.truncateExisting = true,
.followSymlinksOnTruncate = true, /* FIXME: Probably shouldn't follow symlinks. */
});
if (!fd)
throw SysError("opening file %s", PathFmt(profileFile));
return fd;
+4 -3
View File
@@ -70,9 +70,10 @@ Strings EvalSettings::getDefaultNixPath()
}
};
add(std::filesystem::path{getNixDefExpr()} / "channels");
add(rootChannelsDir() / "nixpkgs", "nixpkgs");
add(rootChannelsDir());
add(getNixDefExpr() / "channels");
auto profilesDirOpts = settings.getProfileDirsOptions();
add(rootChannelsDir(profilesDirOpts) / "nixpkgs", "nixpkgs");
add(rootChannelsDir(profilesDirOpts));
return res;
}
+129 -151
View File
@@ -1,4 +1,5 @@
#include "nix/expr/eval.hh"
#include "nix/expr/eval-error.hh"
#include "nix/expr/eval-settings.hh"
#include "nix/expr/primops.hh"
#include "nix/expr/print-options.hh"
@@ -31,6 +32,7 @@
#include <algorithm>
#include <cstddef>
#include <cstdlib>
#include <exception>
#include <iostream>
#include <sstream>
#include <cstring>
@@ -155,6 +157,8 @@ std::string_view showType(ValueType type, bool withArticle)
return WA("a", "float");
case nThunk:
return WA("a", "thunk");
case nFailed:
return WA("an", "error");
}
unreachable();
}
@@ -318,7 +322,6 @@ EvalState::EvalState(
countCalls = getEnv("NIX_COUNT_CALLS").value_or("0") != "0";
static_assert(sizeof(Env) <= 16, "environment must be <= 16 bytes");
static_assert(sizeof(Counter) == 64, "counters must be 64 bytes");
/* Construct the Nix expression search path. */
assert(lookupPath.elements.empty());
@@ -467,7 +470,7 @@ void EvalState::addConstant(const std::string & name, Value * v, Constant info)
We might know the type of a thunk in advance, so be allowed
to just write it down in that case. */
if (auto gotType = v->type(true); gotType != nThunk)
if (auto gotType = v->type</*invalidIsThunk=*/true>(); gotType != nThunk)
assert(info.type == gotType);
/* Install value the base environment. */
@@ -886,7 +889,7 @@ void Value::mkPath(const SourcePath & path, EvalMemory & mem)
mkPath(&*path.accessor, StringData::make(mem, path.path.abs()));
}
inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
[[gnu::always_inline]] inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
{
for (auto l = var.level; l; --l, env = env->up)
;
@@ -904,11 +907,11 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
while (1) {
forceAttrs(*env->values[0], fromWith->pos, "while evaluating the first subexpression of a with expression");
if (auto j = env->values[0]->attrs()->get(var.name)) {
if (countCalls)
if (countCalls) [[unlikely]]
attrSelects[j->pos]++;
return j->value;
}
if (!fromWith->parentWith)
if (!fromWith->parentWith) [[unlikely]]
error<UndefinedVarError>("undefined variable '%1%'", symbols[var.name])
.atPos(var.pos)
.withFrame(*env, var)
@@ -1988,11 +1991,15 @@ void ExprOpUpdate::eval(EvalState & state, Env & env, Value & v)
UpdateQueue q;
evalForUpdate(state, env, q);
v.mkAttrs(&Bindings::emptyBindings);
Value vTmp;
vTmp.mkAttrs(&Bindings::emptyBindings);
for (auto & rhs : std::views::reverse(q)) {
/* Remember that queue is sorted rightmost attrset first. */
eval(state, /*v=*/v, /*v1=*/v, /*v2=*/rhs);
eval(state, /*v=*/vTmp, /*v1=*/vTmp, /*v2=*/rhs);
}
v = vTmp;
}
void Expr::evalForUpdate(EvalState & state, Env & env, UpdateQueue & q, std::string_view errorCtx)
@@ -2175,6 +2182,54 @@ void ExprBlackHole::eval(EvalState & state, [[maybe_unused]] Env & env, Value &
// always force this to be separate, otherwise forceValue may inline it and take
// a massive perf hit
[[gnu::noinline]]
void EvalState::handleEvalExceptionForThunk(Env * env, Expr * expr, Value & v, const PosIdx pos)
{
if (!env)
tryFixupBlackHolePos(v, pos);
auto e = std::current_exception();
Value * recovery = nullptr;
try {
std::rethrow_exception(e);
} catch (const RecoverableEvalError & e) {
recovery = allocValue();
} catch (...) {
}
if (recovery) {
recovery->mkThunk(env, expr);
}
v.mkFailed(e, recovery);
}
[[gnu::noinline]]
void EvalState::handleEvalExceptionForApp(Value & v, const Value & savedApp)
{
auto e = std::current_exception();
Value * recovery = nullptr;
try {
std::rethrow_exception(e);
} catch (const RecoverableEvalError & e) {
recovery = allocValue();
} catch (...) {
}
if (recovery) {
*recovery = savedApp;
}
v.mkFailed(e, recovery);
}
[[gnu::noinline]]
void EvalState::handleEvalFailed(Value & v, const PosIdx pos)
{
assert(v.isFailed());
if (auto recoveryValue = v.failed().recoveryValue) {
v = *recoveryValue;
forceValue(v, pos);
} else {
v.failed().rethrow();
}
}
void EvalState::tryFixupBlackHolePos(Value & v, PosIdx pos)
{
if (!v.isBlackhole())
@@ -2183,7 +2238,8 @@ void EvalState::tryFixupBlackHolePos(Value & v, PosIdx pos)
try {
std::rethrow_exception(e);
} catch (InfiniteRecursionError & e) {
e.atPos(positions[pos]);
if (!e.hasPos())
e.atPos(positions[pos]);
} catch (...) {
}
}
@@ -2406,8 +2462,7 @@ BackedStringView EvalState::coerceToString(
std::string_view errorCtx,
bool coerceMore,
bool copyToStore,
bool canonicalizePath,
bool strict)
bool canonicalizePath)
{
auto _level = addCallDepth(pos);
@@ -2434,28 +2489,14 @@ BackedStringView EvalState::coerceToString(
auto maybeString = tryAttrsToString(pos, v, context, coerceMore, copyToStore);
if (maybeString)
return std::move(*maybeString);
auto outPathAttr = v.attrs()->get(s.outPath);
if (!outPathAttr) {
auto i = v.attrs()->get(s.outPath);
if (!i) {
error<TypeError>(
"cannot coerce %1% to a string: %2%", showType(v), ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
try {
auto derivedPath = coerceToSingleDerivedPath(pos, v, errorCtx);
auto result = mkSingleDerivedPathStringRaw(derivedPath);
context.insert(
std::visit([](auto && o) -> NixStringContextElem { return std::move(o); }, std::move(derivedPath)));
return result;
} catch (Error & e) {
if (strict)
throw;
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
return coerceToString(
pos, *outPathAttr->value, context, errorCtx, coerceMore, copyToStore, canonicalizePath, strict);
}
return coerceToString(pos, *i->value, context, errorCtx, coerceMore, copyToStore, canonicalizePath);
}
if (v.type() == nExternal) {
@@ -2493,8 +2534,7 @@ BackedStringView EvalState::coerceToString(
"while evaluating one element of the list",
coerceMore,
copyToStore,
canonicalizePath,
strict);
canonicalizePath);
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
@@ -2581,131 +2621,63 @@ EvalState::coerceToStorePath(const PosIdx pos, Value & v, NixStringContext & con
error<EvalError>("path '%1%' is not in the Nix store", path).withTrace(pos, errorCtx).debugThrow();
}
SingleDerivedPath EvalState::coerceToSingleDerivedPath(
const PosIdx pos,
Value & v,
std::string_view errorCtx,
const ExperimentalFeatureSettings & xpSettings,
bool skipStringValidation)
std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedPathUnchecked(
const PosIdx pos, Value & v, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings)
{
forceValue(v, pos);
NixStringContext context;
auto s = forceString(v, context, pos, errorCtx, xpSettings);
auto csize = context.size();
if (csize != 1)
error<EvalError>("string '%s' has %d entries in its context. It should only have exactly one entry", s, csize)
.withTrace(pos, errorCtx)
.debugThrow();
auto derivedPath = std::visit(
overloaded{
[&](NixStringContextElem::Opaque && o) -> SingleDerivedPath { return std::move(o); },
[&](NixStringContextElem::DrvDeep &&) -> SingleDerivedPath {
error<EvalError>(
"string '%s' has a context which refers to a complete source and binary closure. This is not supported at this time",
s)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](NixStringContextElem::Built && b) -> SingleDerivedPath { return std::move(b); },
},
((NixStringContextElem &&) *context.begin()).raw);
return {
std::move(derivedPath),
std::move(s),
};
}
switch (v.type()) {
case nAttrs: {
if (!isDerivation(v)) {
error<TypeError>(
"cannot coerce a set that is not a package to a single derived path: %1%",
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
auto outPathAttr = v.attrs()->get(s.outPath);
if (!outPathAttr) {
error<TypeError>(
"cannot coerce %1% to a single derived path: no 'outPath' attribute: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
auto outPathDerivedPath =
coerceToSingleDerivedPath(pos, *outPathAttr->value, errorCtx, xpSettings, skipStringValidation);
auto drvPathAttr = v.attrs()->get(s.drvPath);
if (drvPathAttr) {
auto drvPathDerivedPath =
coerceToSingleDerivedPath(pos, *drvPathAttr->value, errorCtx, xpSettings, skipStringValidation);
std::visit(
overloaded{
[&](const SingleDerivedPath::Built & built) {
if (*built.drvPath != drvPathDerivedPath) {
error<EvalError>(
"relative outPath '%s' does not refer to drvPath '%s'",
outPathDerivedPath.to_string(*store),
drvPathDerivedPath.to_string(*store))
.withTrace(pos, errorCtx)
.debugThrow();
}
},
[&](const SingleDerivedPath::Opaque &) {
error<EvalError>(
"relative outPath '%s' does not refer to drvPath '%s'",
outPathDerivedPath.to_string(*store),
drvPathDerivedPath.to_string(*store))
.withTrace(pos, errorCtx)
.debugThrow();
},
},
outPathDerivedPath.raw());
}
return outPathDerivedPath;
}
case nString: {
auto s = v.string_view();
NixStringContext context;
copyContext(v, context, xpSettings);
auto csize = context.size();
if (csize != 1)
error<EvalError>(
"string '%s' has %d entries in its context. It should only have exactly one entry", s, csize)
.withTrace(pos, errorCtx)
.debugThrow();
auto derivedPath = std::visit(
SingleDerivedPath EvalState::coerceToSingleDerivedPath(const PosIdx pos, Value & v, std::string_view errorCtx)
{
auto [derivedPath, s_] = coerceToSingleDerivedPathUnchecked(pos, v, errorCtx);
auto s = s_;
auto sExpected = mkSingleDerivedPathStringRaw(derivedPath);
if (s != sExpected) {
/* `std::visit` is used here just to provide a more precise
error message. */
std::visit(
overloaded{
[&](NixStringContextElem::Opaque && o) -> SingleDerivedPath { return std::move(o); },
[&](NixStringContextElem::DrvDeep &&) -> SingleDerivedPath {
error<EvalError>(
"string '%s' has a context which refers to a complete source and binary closure. This is not "
"supported at this time",
s)
[&](const SingleDerivedPath::Opaque & o) {
error<EvalError>("path string '%s' has context with the different path '%s'", s, sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](NixStringContextElem::Built && b) -> SingleDerivedPath { return std::move(b); },
},
((NixStringContextElem &&) *context.begin()).raw);
if (!skipStringValidation) {
auto sExpected = mkSingleDerivedPathStringRaw(derivedPath);
if (s != sExpected) {
/* `std::visit` is used here just to provide a more precise
error message. */
std::visit(
overloaded{
[&](const SingleDerivedPath::Opaque & o) {
error<EvalError>("path string '%s' has context with the different path '%s'", s, sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](const SingleDerivedPath::Built & b) {
error<EvalError>(
"string '%s' has context with the output '%s' from derivation '%s', but the string is "
"not the right placeholder for this derivation output. It should be '%s'",
s,
b.output,
b.drvPath->to_string(*store),
sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
}},
derivedPath.raw());
}
}
return derivedPath;
}
default:
error<TypeError>(
"cannot coerce %1% to a single derived path: %2%", showType(v), ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
[&](const SingleDerivedPath::Built & b) {
error<EvalError>(
"string '%s' has context with the output '%s' from derivation '%s', but the string is not the right placeholder for this derivation output. It should be '%s'",
s,
b.output,
b.drvPath->to_string(*store),
sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
}},
derivedPath.raw());
}
return derivedPath;
}
// NOTE: This implementation must match eqValues!
@@ -2906,8 +2878,11 @@ void EvalState::assertEqValues(Value & v1, Value & v2, const PosIdx pos, std::st
}
return;
case nThunk: // Must not be left by forceValue
assert(false);
// Cannot be returned by forceValue().
case nThunk:
case nFailed:
unreachable();
default: // Note that we pass compiler flags that should make `default:` unreachable.
// Also note that this probably ran after `eqValues`, which implements
// the same logic more efficiently (without having to unwind stacks),
@@ -3001,8 +2976,11 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
// !!!
return v1.fpoint() == v2.fpoint();
case nThunk: // Must not be left by forceValue
assert(false);
// Cannot be returned by forceValue().
case nThunk:
case nFailed:
unreachable();
default: // Note that we pass compiler flags that should make `default:` unreachable.
error<EvalError>("eqValues: cannot compare %1% with %2%", showType(v1), showType(v2))
.withTrace(pos, errorCtx)
+25 -99
View File
@@ -1,6 +1,5 @@
#include "nix/expr/get-drvs.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/store/derived-path.hh"
#include "nix/store/derivations.hh"
#include "nix/store/store-api.hh"
#include "nix/store/path-with-outputs.hh"
@@ -22,13 +21,13 @@ PackageInfo::PackageInfo(EvalState & state, ref<Store> store, const std::string
, attrs(nullptr)
, attrPath("")
{
auto [drvStorePath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
auto [drvPath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
this->drvPath = {{Path{SingleDerivedPath::Opaque{drvStorePath}, drvStorePath}}};
this->drvPath = drvPath;
auto drv = store->derivationFromPath(drvStorePath);
auto drv = store->derivationFromPath(drvPath);
name = drvStorePath.name();
name = drvPath.name();
if (selectedOutputs.size() > 1)
throw Error("building more than one derivation output is not supported, in '%s'", drvPathWithOutputs);
@@ -37,18 +36,10 @@ PackageInfo::PackageInfo(EvalState & state, ref<Store> store, const std::string
auto i = drv.outputs.find(outputName);
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvStorePath), outputName);
auto & [outputName_, output] = *i;
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvPath), outputName);
auto & [outputName, output] = *i;
auto outStorePath = output.path(*store, drv.name, outputName_);
if (outStorePath) {
outPath = Path{SingleDerivedPath::Opaque{*outStorePath}, *outStorePath};
} else {
// CA derivation with unknown output path
outPath = Path{
SingleDerivedPath::Built{makeConstantStorePathRef(drvStorePath), outputName_},
std::nullopt};
}
outPath = {output.path(*store, drv.name, outputName)};
}
std::string PackageInfo::queryName() const
@@ -73,60 +64,26 @@ std::string PackageInfo::querySystem() const
return system;
}
std::optional<PackageInfo::Path> PackageInfo::queryDrvPathFlexible() const
std::optional<StorePath> PackageInfo::queryDrvPath() const
{
if (!drvPath && attrs) {
auto i = attrs->get(state->s.drvPath);
if (i) {
Value v;
v.mkAttrs(const_cast<Bindings *>(attrs));
std::optional<SingleDerivedPath> outPathDerivedPath;
if (auto i = attrs->get(state->s.drvPath)) {
NixStringContext context;
auto found = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the 'drvPath' attribute of a derivation");
try {
// Validate derivation structure
outPathDerivedPath = state->coerceToSingleDerivedPath(
noPos, v, "while evaluating the derivation");
found.requireDerivation();
} catch (Error & e) {
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
}
if (outPathDerivedPath) {
// Validation passed. Now get drvPath.
auto drvPathDerivedPath = state->coerceToSingleDerivedPath(
i->pos, *i->value, "while evaluating the 'drvPath' attribute of a derivation");
if (auto * opaque = std::get_if<SingleDerivedPath::Opaque>(&drvPathDerivedPath.raw())) {
opaque->path.requireDerivation();
drvPath = {{Path{std::move(drvPathDerivedPath), opaque->path}}};
} else {
// Dynamic derivation - drvPath is itself a derivation output
drvPath = {{Path{std::move(drvPathDerivedPath), std::nullopt}}};
}
} else {
// Fall back to old behavior
NixStringContext context;
auto found = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the 'drvPath' attribute of a derivation");
try {
found.requireDerivation();
} catch (Error & e) {
e.addTrace(state->positions[i->pos], "while evaluating the 'drvPath' attribute of a derivation");
throw;
}
drvPath = {{Path{SingleDerivedPath::Opaque{found}, std::move(found)}}};
e.addTrace(state->positions[i->pos], "while evaluating the 'drvPath' attribute of a derivation");
throw;
}
drvPath = {std::move(found)};
} else
drvPath = {std::nullopt};
}
return drvPath.value_or(std::nullopt);
}
std::optional<StorePath> PackageInfo::queryDrvPath() const
{
if (auto path = queryDrvPathFlexible())
return path->storePath;
return std::nullopt;
}
StorePath PackageInfo::requireDrvPath() const
{
if (auto drvPath = queryDrvPath())
@@ -134,49 +91,18 @@ StorePath PackageInfo::requireDrvPath() const
throw Error("derivation does not contain a 'drvPath' attribute");
}
PackageInfo::Path PackageInfo::queryOutPathFlexible() const
{
if (!outPath && attrs) {
Value v;
v.mkAttrs(const_cast<Bindings *>(attrs));
std::optional<SingleDerivedPath> derivedPath;
try {
derivedPath = state->coerceToSingleDerivedPath(
noPos, v, "while evaluating the derivation");
} catch (Error & e) {
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
}
if (derivedPath) {
if (auto * opaque = std::get_if<SingleDerivedPath::Opaque>(&derivedPath->raw())) {
outPath = Path{std::move(*derivedPath), opaque->path};
} else {
// Built path - no concrete output path available (placeholder)
outPath = Path{std::move(*derivedPath), std::nullopt};
}
} else {
// Fall back to old behavior
auto i = attrs->get(state->s.outPath);
NixStringContext context;
if (i) {
auto path = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation");
outPath = Path{SingleDerivedPath::Opaque{path}, std::move(path)};
}
}
}
if (!outPath)
throw Error("derivation does not have an 'outPath' attribute");
return *outPath;
}
StorePath PackageInfo::queryOutPath() const
{
auto path = queryOutPathFlexible();
if (!path.storePath)
if (!outPath && attrs) {
auto i = attrs->get(state->s.outPath);
NixStringContext context;
if (i)
outPath = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation");
}
if (!outPath)
throw UnimplementedError("CA derivations are not yet supported");
return *path.storePath;
return *outPath;
}
PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsToInstall)
+1 -1
View File
@@ -11,7 +11,7 @@ namespace nix {
* variable is set. This is to prevent contention on these counters
* when multi-threaded evaluation is enabled.
*/
struct alignas(64) Counter
struct alignas(std::hardware_destructive_interference_size) Counter
{
using value_type = uint64_t;
+1 -1
View File
@@ -14,7 +14,7 @@ namespace nix::eval_cache {
struct AttrDb;
class AttrCursor;
struct CachedEvalError : EvalError
struct CachedEvalError : CloneableError<CachedEvalError, EvalError>
{
const ref<AttrCursor> cursor;
const Symbol attr;
+15 -7
View File
@@ -18,7 +18,7 @@ class EvalErrorBuilder;
*
* Most subclasses should inherit from `EvalError` instead of this class.
*/
class EvalBaseError : public Error
class EvalBaseError : public CloneableError<EvalBaseError, Error>
{
template<class T>
friend class EvalErrorBuilder;
@@ -26,14 +26,14 @@ public:
EvalState & state;
EvalBaseError(EvalState & state, ErrorInfo && errorInfo)
: Error(errorInfo)
: CloneableError(errorInfo)
, state(state)
{
}
template<typename... Args>
explicit EvalBaseError(EvalState & state, const std::string & formatString, const Args &... formatArgs)
: Error(formatString, formatArgs...)
: CloneableError(formatString, formatArgs...)
, state(state)
{
}
@@ -60,23 +60,31 @@ MakeError(InfiniteRecursionError, EvalError);
* Inherits from EvalBaseError (not EvalError) because resource exhaustion
* should not be cached.
*/
struct StackOverflowError : public EvalBaseError
struct StackOverflowError : public CloneableError<StackOverflowError, EvalBaseError>
{
StackOverflowError(EvalState & state)
: EvalBaseError(state, "stack overflow; max-call-depth exceeded")
: CloneableError(state, "stack overflow; max-call-depth exceeded")
{
}
};
MakeError(IFDError, EvalBaseError);
struct InvalidPathError : public EvalError
/**
* An evaluation error which should be retried instead of rethrown.
*
* A RecoverableEvalError is not an EvalError, because we shouldn't cache it in
* the eval cache, as it should be retried anyway.
*/
MakeError(RecoverableEvalError, EvalBaseError);
struct InvalidPathError : public CloneableError<InvalidPathError, EvalError>
{
public:
Path path;
InvalidPathError(EvalState & state, const Path & path)
: EvalError(state, "path '%s' is not valid", path)
: CloneableError(state, "path '%s' is not valid", path)
{
}
};
+3
View File
@@ -33,6 +33,9 @@ using gc_allocator = std::allocator<T>;
struct gc
{};
struct gc_cleanup
{};
#endif
namespace nix {
+13 -5
View File
@@ -5,6 +5,7 @@
#include "nix/expr/eval.hh"
#include "nix/expr/eval-error.hh"
#include "nix/expr/eval-settings.hh"
#include <exception>
namespace nix {
@@ -91,18 +92,25 @@ void EvalState::forceValue(Value & v, const PosIdx pos)
Expr * expr = v.thunk().expr;
try {
v.mkBlackhole();
// checkInterrupt();
if (env) [[likely]]
expr->eval(*this, *env, v);
else
ExprBlackHole::throwInfiniteRecursionError(*this, v);
} catch (...) {
v.mkThunk(env, expr);
tryFixupBlackHolePos(v, pos);
handleEvalExceptionForThunk(env, expr, v, pos);
throw;
}
} else if (v.isApp())
callFunction(*v.app().left, *v.app().right, v, pos);
} else if (v.isApp()) {
Value savedApp = v;
try {
callFunction(*v.app().left, *v.app().right, v, pos);
} catch (...) {
handleEvalExceptionForApp(v, savedApp);
throw;
}
} else if (v.isFailed()) {
handleEvalFailed(v, pos);
}
}
[[gnu::always_inline]]
+41 -28
View File
@@ -652,8 +652,28 @@ public:
*/
inline void forceValue(Value & v, const PosIdx pos);
private:
/**
* Internal support function for forceValue
*
* This code is factored out so that it's not in the heavily inlined hot path.
*/
void handleEvalExceptionForThunk(Env * env, Expr * expr, Value & v, const PosIdx pos);
/**
* Internal support function for forceValue
*
* This code is factored out so that it's not in the heavily inlined hot path.
*/
void handleEvalExceptionForApp(Value & v, const Value & savedApp);
void handleEvalFailed(Value & v, PosIdx pos);
void tryFixupBlackHolePos(Value & v, PosIdx pos);
public:
/**
* Force a value, then recursively force list elements and
* attributes.
@@ -715,9 +735,6 @@ public:
* string. If `coerceMore` is set, also converts nulls, integers,
* booleans and lists to a string. If `copyToStore` is set,
* referenced paths are copied to the Nix store as a side effect.
* If `strict` is set, ensures that when coercing an attrset with
* both `outPath` and `drvPath`, the string context for `outPath`
* agrees with the `drvPath` attribute.
*/
BackedStringView coerceToString(
const PosIdx pos,
@@ -726,8 +743,7 @@ public:
std::string_view errorCtx,
bool coerceMore = false,
bool copyToStore = true,
bool canonicalizePath = true,
bool strict = false);
bool canonicalizePath = true);
StorePath copyPathToStore(NixStringContext & context, const SourcePath & path);
@@ -746,33 +762,30 @@ public:
StorePath coerceToStorePath(const PosIdx pos, Value & v, NixStringContext & context, std::string_view errorCtx);
/**
* Coerce to `SingleDerivedPath`.
*
* For strings: must be either a literal store path or a
* placeholder (see `DownstreamPlaceholder`). The string context
* must be exactly one element, which is either a
* `NixStringContextElem::Opaque` or `NixStringContextElem::Built`.
* (`NixStringContextElem::DrvDeep` is not permitted). The string
* is parsed based on the context --- the context is the source of
* truth, and ultimately tells us what we want, and then we ensure
* the string corresponds to it.
*
* For attrsets: must have an `outPath` attribute which is
* recursively coerced. If a `drvPath` attribute is present, it is
* also coerced and validated to agree with the `outPath` context.
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*
* @param skipStringValidation Skip validating that the string
* matches the expected placeholder for the context. This is
* exposed for unit testing only to avoid store IO.
* Part of `coerceToSingleDerivedPath()` without any store IO which is exposed for unit testing only.
*/
SingleDerivedPath coerceToSingleDerivedPath(
std::pair<SingleDerivedPath, std::string_view> coerceToSingleDerivedPathUnchecked(
const PosIdx pos,
Value & v,
std::string_view errorCtx,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings,
bool skipStringValidation = false);
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Coerce to `SingleDerivedPath`.
*
* Must be a string which is either a literal store path or a
* "placeholder (see `DownstreamPlaceholder`).
*
* Even more importantly, the string context must be exactly one
* element, which is either a `NixStringContextElem::Opaque` or
* `NixStringContextElem::Built`. (`NixStringContextEleme::DrvDeep`
* is not permitted).
*
* The string is parsed based on the context --- the context is the
* source of truth, and ultimately tells us what we want, and then
* we ensure the string corresponds to it.
*/
SingleDerivedPath coerceToSingleDerivedPath(const PosIdx pos, Value & v, std::string_view errorCtx);
#if NIX_USE_BOEHMGC
/** A GC root for the baseEnv reference. */
+4 -20
View File
@@ -2,7 +2,6 @@
///@file
#include "nix/expr/eval.hh"
#include "nix/store/derived-path.hh"
#include "nix/store/path.hh"
#include <string>
@@ -21,23 +20,10 @@ public:
private:
EvalState * state;
struct Path {
/**
* Underlying semantic path, valid in CA and IA cases.
*/
SingleDerivedPath derivedPath;
/**
* If we don't need a placeholder, we'll have a real store path, which we can remember here.
* For CA derivations using placeholders, this will be std::nullopt.
*/
std::optional<StorePath> storePath;
};
mutable std::string name;
mutable std::string system;
mutable std::optional<std::optional<Path>> drvPath;
mutable std::optional<Path> outPath;
mutable std::optional<std::optional<StorePath>> drvPath;
mutable std::optional<StorePath> outPath;
mutable std::string outputName;
Outputs outputs;
@@ -66,10 +52,8 @@ public:
std::string queryName() const;
std::string querySystem() const;
std::optional<StorePath> queryDrvPath() const;
std::optional<Path> queryDrvPathFlexible() const;
StorePath requireDrvPath() const;
StorePath queryOutPath() const;
Path queryOutPathFlexible() const;
std::string queryOutputName() const;
/**
* Return the unordered map of output names to (optional) output paths.
@@ -97,12 +81,12 @@ public:
void setDrvPath(StorePath path)
{
drvPath = {{Path{SingleDerivedPath::Opaque{path}, std::move(path)}}};
drvPath = {{std::move(path)}};
}
void setOutPath(StorePath path)
{
outPath = Path{SingleDerivedPath::Opaque{path}, std::move(path)};
outPath = {{std::move(path)}};
}
void setFailed()
+117 -52
View File
@@ -7,6 +7,7 @@
#include <cstring>
#include <memory>
#include <memory_resource>
#include <exception>
#include <span>
#include <string_view>
#include <type_traits>
@@ -35,27 +36,32 @@ class BindingsBuilder;
* about how this is mapped into the alignment bits to save significant memory.
* This also restricts the number of internal types represented with distinct memory layouts.
*/
typedef enum {
enum InternalType {
tUninitialized = 0,
/* layout: Single/zero field payload */
tInt = 1,
tBool,
tNull,
tFloat,
tFailed,
tExternal,
tPrimOp,
tAttrs,
/* layout: Pair of pointers payload */
tListSmall,
tFirstPairOfPointers,
tListSmall = tFirstPairOfPointers,
tPrimOpApp,
tApp,
tThunk,
tLambda,
tLastPairOfPointers = tLambda,
/* layout: Single untaggable field */
tListN,
tFirstSingleUntaggable,
tListN = tFirstSingleUntaggable,
tString,
tPath,
} InternalType;
tNumberOfInternalTypes, // Must be last
};
/**
* This type abstracts over all actual value types in the language,
@@ -64,6 +70,7 @@ typedef enum {
*/
typedef enum {
nThunk,
nFailed,
nInt,
nFloat,
nBool,
@@ -134,7 +141,7 @@ public:
virtual bool operator==(const ExternalValueBase & b) const noexcept;
/**
* Print the value as JSON. Defaults to unconvertable, i.e. throws an error
* Print the value as JSON. Defaults to unconvertible, i.e. throws an error
*/
virtual nlohmann::json
printValueAsJSON(EvalState & state, bool strict, NixStringContext & context, bool copyToStore = true) const;
@@ -420,6 +427,36 @@ struct ValueBase
size_t size;
Value * const * elems;
};
struct Failed : gc_cleanup
{
std::exception_ptr ex;
/**
* Optional value for recovering `RecoverableEvalError`
* Must be set iff `ex` is an instance of `RecoverableEvalError`.
*/
Value * recoveryValue;
Failed(std::exception_ptr ex, Value * recoveryValue)
: ex(ex)
, recoveryValue(recoveryValue)
{
}
[[noreturn]] void rethrow() const
{
try {
std::rethrow_exception(ex);
} catch (BaseError & e) {
/* Rethrow the copy of the exception - not the original one.
Stack tracing mechanisms rely on being able to modify the exceptions
they catch by reference. */
e.throwClone();
} catch (...) {
throw;
}
}
};
};
template<typename T>
@@ -446,6 +483,7 @@ struct PayloadTypeToInternalType
MACRO(PrimOp *, primOp, tPrimOp) \
MACRO(ValueBase::PrimOpApplicationThunk, primOpApp, tPrimOpApp) \
MACRO(ExternalValueBase *, external, tExternal) \
MACRO(ValueBase::Failed *, failed, tFailed) \
MACRO(NixFloat, fpoint, tFloat)
#define NIX_VALUE_PAYLOAD_TYPE(T, FIELD_NAME, DISCRIMINATOR) \
@@ -587,7 +625,7 @@ class alignas(16)
enum PrimaryDiscriminator : int {
pdUninitialized = 0,
pdSingleDWord, //< layout: Single/zero field payload
/* The order of these enumations must be the same as in InternalType. */
/* The order of these enumerations must be the same as in InternalType. */
pdListN, //< layout: Single untaggable field.
pdString,
pdPath,
@@ -633,7 +671,7 @@ class alignas(16)
template<InternalType type, typename T, typename U>
void setPairOfPointersPayload(T * firstPtrField, U * secondPtrField) noexcept
{
static_assert(type >= tListSmall && type <= tLambda);
static_assert(type >= tFirstPairOfPointers && type <= tLastPairOfPointers);
{
auto firstFieldPayload = std::bit_cast<PackedPointer>(firstPtrField);
assertAligned(firstFieldPayload);
@@ -642,7 +680,7 @@ class alignas(16)
{
auto secondFieldPayload = std::bit_cast<PackedPointer>(secondPtrField);
assertAligned(secondFieldPayload);
payload[1] = (type - tListSmall) | secondFieldPayload;
payload[1] = (type - tFirstPairOfPointers) | secondFieldPayload;
}
}
@@ -670,11 +708,11 @@ protected:
case pdListN:
case pdString:
case pdPath:
return static_cast<InternalType>(tListN + (pd - pdListN));
return static_cast<InternalType>(tFirstSingleUntaggable + (pd - pdListN));
case pdPairOfPointers:
return static_cast<InternalType>(tListSmall + (payload[1] & discriminatorMask));
return static_cast<InternalType>(tFirstPairOfPointers + (payload[1] & discriminatorMask));
[[unlikely]] default:
unreachable();
nixUnreachableWhenHardened();
}
}
@@ -750,6 +788,11 @@ protected:
path.path = std::bit_cast<const StringData *>(payload[1]);
}
void getStorage(Failed *& failed) const noexcept
{
failed = std::bit_cast<Failed *>(payload[1]);
}
void setStorage(NixInt integer) noexcept
{
setSingleDWordPayload<tInt>(integer.value);
@@ -799,6 +842,11 @@ protected:
{
setUntaggablePayload<pdPath>(path.accessor, path.path);
}
void setStorage(Failed * failed) noexcept
{
setSingleDWordPayload<tFailed>(std::bit_cast<PackedPointer>(failed));
}
};
/**
@@ -1027,7 +1075,7 @@ private:
T getStorage() const noexcept
{
if (getInternalType() != detail::payloadTypeToInternalType<T>) [[unlikely]]
unreachable();
nixUnreachableWhenHardened();
T out;
ValueStorage::getStorage(out);
return out;
@@ -1050,12 +1098,12 @@ public:
inline bool isThunk() const
{
return isa<tThunk>();
};
}
inline bool isApp() const
{
return isa<tApp>();
};
}
inline bool isBlackhole() const;
@@ -1063,61 +1111,66 @@ public:
inline bool isLambda() const
{
return isa<tLambda>();
};
}
inline bool isPrimOp() const
{
return isa<tPrimOp>();
};
}
inline bool isPrimOpApp() const
{
return isa<tPrimOpApp>();
};
}
inline bool isFailed() const
{
return isa<tFailed>();
}
/**
* Returns the normal type of a Value. This only returns nThunk if
* the Value hasn't been forceValue'd
*
* @param invalidIsThunk Instead of aborting an an invalid (probably
* @param invalidIsThunk Instead of UB an an invalid (probably
* 0, so uninitialized) internal type, return `nThunk`.
*/
inline ValueType type(bool invalidIsThunk = false) const
template<bool invalidIsThunk = false>
inline ValueType type() const
{
switch (getInternalType()) {
case tUninitialized:
break;
case tInt:
return nInt;
case tBool:
return nBool;
case tString:
return nString;
case tPath:
return nPath;
case tNull:
return nNull;
case tAttrs:
return nAttrs;
case tListSmall:
case tListN:
return nList;
case tLambda:
case tPrimOp:
case tPrimOpApp:
return nFunction;
case tExternal:
return nExternal;
case tFloat:
return nFloat;
case tThunk:
case tApp:
return nThunk;
/* Explicit lookup table. switch() might compile down (and it does at least with GCC 14)
to a jump table. Let's help the compiler a bit here. */
static constexpr auto table = [] {
std::array<ValueType, tNumberOfInternalTypes> t{};
t[tUninitialized] = nThunk;
t[tInt] = nInt;
t[tBool] = nBool;
t[tNull] = nNull;
t[tFloat] = nFloat;
t[tFailed] = nFailed;
t[tExternal] = nExternal;
t[tAttrs] = nAttrs;
t[tPrimOp] = nFunction;
t[tLambda] = nFunction;
t[tPrimOpApp] = nFunction;
t[tApp] = nThunk;
t[tThunk] = nThunk;
t[tListSmall] = nList;
t[tListN] = nList;
t[tString] = nString;
t[tPath] = nPath;
return t;
}();
auto it = getInternalType();
if (it == tUninitialized || it >= tNumberOfInternalTypes) [[unlikely]] {
if constexpr (invalidIsThunk)
return nThunk;
else
nixUnreachableWhenHardened();
}
if (invalidIsThunk)
return nThunk;
else
unreachable();
return table[it];
}
/**
@@ -1232,6 +1285,11 @@ public:
setStorage(n);
}
inline void mkFailed(std::exception_ptr e, Value * recovery) noexcept
{
setStorage(new Value::Failed(e, recovery));
}
bool isList() const noexcept
{
return isa<tListSmall, tListN>();
@@ -1346,6 +1404,13 @@ public:
{
return getStorage<Path>().accessor;
}
Failed & failed() const noexcept
{
auto p = getStorage<Failed *>();
assert(p);
return *p;
}
};
extern ExprBlackHole eBlackHole;
@@ -9,14 +9,14 @@
namespace nix {
class BadNixStringContextElem : public Error
class BadNixStringContextElem final : public CloneableError<BadNixStringContextElem, Error>
{
public:
std::string_view raw;
template<typename... Args>
BadNixStringContextElem(std::string_view raw_, const Args &... args)
: Error("")
: CloneableError("")
{
raw = raw_;
auto hf = HintFmt(args...);
+1 -1
View File
@@ -28,7 +28,7 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/big-objs')
# Check for each of these functions, and create a define like `#define HAVE_LCHOWN 1`.
# Check for each of these functions, and create a define like `#define HAVE_SYSCONF 1`.
check_funcs = [
'sysconf',
]
+1 -2
View File
@@ -23,13 +23,12 @@ SourcePath EvalState::storePath(const StorePath & path)
StorePath
EvalState::mountInput(fetchers::Input & input, const fetchers::Input & originalInput, ref<SourceAccessor> accessor)
{
auto storePath = fetchToStore(fetchSettings, *store, accessor, FetchMode::Copy, input.getName());
auto [storePath, narHash] = fetchToStore2(fetchSettings, *store, accessor, FetchMode::Copy, input.getName());
allowPath(storePath); // FIXME: should just whitelist the entire virtual store
storeFS->mount(CanonPath(store->printStorePath(storePath)), accessor);
auto narHash = store->queryPathInfo(storePath)->narHash;
input.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
if (originalInput.getNarHash() && narHash != *originalInput.getNarHash())
+14 -7
View File
@@ -562,6 +562,7 @@ static void prim_typeOf(EvalState & state, const PosIdx pos, Value ** args, Valu
v.mkStringNoCopy("float"_sds);
break;
case nThunk:
case nFailed:
unreachable();
}
}
@@ -1312,11 +1313,12 @@ static void prim_warn(EvalState & state, const PosIdx pos, Value ** args, Value
state.forceString(*args[0], pos, "while evaluating the first argument; the message passed to builtins.warn");
{
BaseError msg(std::string{msgStr});
msg.atPos(state.positions[pos]);
auto info = msg.info();
info.level = lvlWarn;
info.isFromExpr = true;
ErrorInfo info{
.level = lvlWarn,
.msg = HintFmt(std::string(msgStr)),
.pos = state.positions[pos],
.isFromExpr = true,
};
logWarning(info);
}
@@ -1814,8 +1816,13 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
drv.fillInOutputPaths(*state.store);
}
/* Write the resulting term into the Nix store directory. */
auto drvPath = writeDerivation(*state.store, drv, state.repair);
/* Write the resulting term into the Nix store directory.
Unless we are in read-only mode, that is, in which case we do not
write anything. Users commonly do this to speed up evaluation in
contexts where they don't actually want to build anything. */
auto drvPath =
settings.readOnlyMode ? computeStorePath(*state.store, drv) : state.store->writeDerivation(drv, state.repair);
auto drvPathS = state.store->printStorePath(drvPath);
printMsg(lvlChatty, "instantiated '%1%' -> '%2%'", drvName, drvPathS);
+13 -7
View File
@@ -191,19 +191,25 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value ** args
{.msg = HintFmt("attribute '%s' is missing in call to 'fetchClosure'", "fromStore"),
.pos = state.positions[pos]});
auto parsedURL = parseURL(*fromStoreUrl, /*lenient=*/true);
auto storeRef = StoreReference::parse(*fromStoreUrl);
if (parsedURL.scheme != "http" && parsedURL.scheme != "https"
&& !(getEnv("_NIX_IN_TEST").has_value() && parsedURL.scheme == "file"))
throw Error(
{.msg = HintFmt("'fetchClosure' only supports http:// and https:// stores"), .pos = state.positions[pos]});
if ([&] {
auto * specified = std::get_if<StoreReference::Specified>(&storeRef.variant);
return !specified
|| (specified->scheme != "http" && specified->scheme != "https"
&& !(getEnv("_NIX_IN_TEST").has_value() && specified->scheme == "file"));
}())
throw Error({
.msg = HintFmt("'fetchClosure' only supports http:// and https:// stores"),
.pos = state.positions[pos],
});
if (!parsedURL.query.empty())
if (!storeRef.params.empty())
throw Error(
{.msg = HintFmt("'fetchClosure' does not support URL query parameters (in '%s')", *fromStoreUrl),
.pos = state.positions[pos]});
auto fromStore = openStore(parsedURL.to_string());
auto fromStore = openStore(std::move(storeRef));
if (toPath)
runFetchClosureWithRewrite(state, pos, *fromStore, *fromPath, *toPath, v);
+5
View File
@@ -73,6 +73,11 @@ void printAmbiguous(EvalState & state, Value & v, std::ostream & str, std::set<c
str << "«potential infinite recursion»";
}
break;
case nFailed:
// Historically, a tried and then ignored value (e.g. through tryEval) was
// reverted to the original thunk.
str << "<CODE>";
break;
case nFunction:
if (v.isLambda()) {
str << "<LAMBDA>";
+15
View File
@@ -509,6 +509,17 @@ private:
}
}
void printFailed()
{
if (options.ansiColors)
output << ANSI_MAGENTA;
// Historically, a tried and then ignored value (e.g. through tryEval) was
// reverted to the original thunk.
output << "«thunk»";
if (options.ansiColors)
output << ANSI_NORMAL;
}
void printExternal(Value & v)
{
v.external()->print(output);
@@ -590,6 +601,10 @@ private:
printFunction(v);
break;
case nFailed:
printFailed();
break;
case nThunk:
printThunk(v);
break;
+1
View File
@@ -98,6 +98,7 @@ json printValueAsJSON(
break;
case nThunk:
case nFailed:
case nFunction:
state.error<TypeError>("cannot convert %1% to JSON", showType(v)).atPos(v.determinePos(pos)).debugThrow();
}
+4
View File
@@ -171,7 +171,11 @@ static void printValueAsXML(
break;
case nThunk:
// Historically, a tried and then ignored value (e.g. through tryEval) was
// reverted to the original thunk.
case nFailed:
doc.writeEmptyElement("unevaluated");
break;
}
}
+11 -5
View File
@@ -34,14 +34,20 @@ struct CacheImpl : Cache
Sync<State> _state;
CacheImpl()
/**
* This is a back-reference to the `Settings` that owns us.
*/
const Settings & settings;
CacheImpl(const Settings & _settings)
: settings(_settings)
{
auto state(_state.lock());
auto dbPath = (getCacheDir() / "fetcher-cache-v4.sqlite").string();
createDirs(dirOf(dbPath));
auto dbPath = getCacheDir() / "fetcher-cache-v4.sqlite";
createDirs(dbPath.parent_path());
state->db = SQLite(dbPath, {.useWAL = settings.useSQLiteWAL});
state->db = SQLite(dbPath, {.useWAL = nix::settings.useSQLiteWAL});
state->db.isCache();
state->db.exec(schema);
@@ -154,7 +160,7 @@ ref<Cache> Settings::getCache() const
{
auto cache(_cache.lock());
if (!*cache)
*cache = std::make_shared<CacheImpl>();
*cache = std::make_shared<CacheImpl>(*this);
return ref<Cache>(*cache);
}
+69 -18
View File
@@ -5,12 +5,12 @@
namespace nix {
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string & name, const std::string & fingerprint, ContentAddressMethod method, const std::string & path)
fetchers::Cache::Key
makeSourcePathToHashCacheKey(std::string_view fingerprint, ContentAddressMethod method, const CanonPath & path)
{
return fetchers::Cache::Key{
"fetchToStore",
{{"name", name}, {"fingerprint", fingerprint}, {"method", std::string{method.render()}}, {"path", path}}};
"sourcePathToHash",
{{"fingerprint", std::string(fingerprint)}, {"method", std::string{method.render()}}, {"path", path.abs()}}};
}
StorePath fetchToStore(
@@ -23,19 +23,45 @@ StorePath fetchToStore(
PathFilter * filter,
RepairFlag repair)
{
// FIXME: add an optimisation for the case where the accessor is
// a `PosixSourceAccessor` pointing to a store path.
return fetchToStore2(settings, store, path, mode, name, method, filter, repair).first;
}
std::pair<StorePath, Hash> fetchToStore2(
const fetchers::Settings & settings,
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name,
ContentAddressMethod method,
PathFilter * filter,
RepairFlag repair)
{
std::optional<fetchers::Cache::Key> cacheKey;
auto [subpath, fingerprint] = filter ? std::pair<CanonPath, std::optional<std::string>>{path.path, std::nullopt}
: path.accessor->getFingerprint(path.path);
if (fingerprint) {
cacheKey = makeFetchToStoreCacheKey(std::string{name}, *fingerprint, method, subpath.abs());
if (auto res = settings.getCache()->lookupStorePath(*cacheKey, store)) {
debug("store path cache hit for '%s'", path);
return res->storePath;
cacheKey = makeSourcePathToHashCacheKey(*fingerprint, method, subpath);
if (auto res = settings.getCache()->lookup(*cacheKey)) {
auto hash = Hash::parseSRI(fetchers::getStrAttr(*res, "hash"));
auto storePath =
store.makeFixedOutputPathFromCA(name, ContentAddressWithReferences::fromParts(method, hash, {}));
/* Add a temproot before the call to isValidPath to prevent accidental GC in case the
input is cached. Note that this must be done before to avoid races. */
if (mode != FetchMode::DryRun)
store.addTempRoot(storePath);
if (mode == FetchMode::DryRun || store.isValidPath(storePath)) {
debug(
"source path '%s' cache hit in '%s' (hash '%s')",
path,
store.printStorePath(storePath),
hash.to_string(HashFormat::SRI, true));
return {storePath, hash};
}
debug("source path '%s' not in store", path);
}
} else {
static auto barf = getEnv("_NIX_TEST_BARF_ON_UNCACHEABLE").value_or("") == "1";
@@ -53,16 +79,41 @@ StorePath fetchToStore(
auto filter2 = filter ? *filter : defaultPathFilter;
auto storePath = mode == FetchMode::DryRun
? store.computeStorePath(name, path, method, HashAlgorithm::SHA256, {}, filter2).first
: store.addToStore(name, path, method, HashAlgorithm::SHA256, {}, filter2, repair);
auto [storePath, hash] =
mode == FetchMode::DryRun
? [&]() {
auto [storePath, hash] =
store.computeStorePath(name, path, method, HashAlgorithm::SHA256, {}, filter2);
debug(
"hashed '%s' to '%s' (hash '%s')",
path,
store.printStorePath(storePath),
hash.to_string(HashFormat::SRI, true));
return std::make_pair(storePath, hash);
}()
: [&]() {
// FIXME: ideally addToStore() would return the hash
// right away (like computeStorePath()).
auto storePath = store.addToStore(name, path, method, HashAlgorithm::SHA256, {}, filter2, repair);
auto info = store.queryPathInfo(storePath);
assert(info->references.empty());
auto hash = method == ContentAddressMethod::Raw::NixArchive ? info->narHash : ({
if (!info->ca || info->ca->method != method)
throw Error("path '%s' lacks a CA field", store.printStorePath(storePath));
info->ca->hash;
});
debug(
"copied '%s' to '%s' (hash '%s')",
path,
store.printStorePath(storePath),
hash.to_string(HashFormat::SRI, true));
return std::make_pair(storePath, hash);
}();
debug(mode == FetchMode::DryRun ? "hashed '%s'" : "copied '%s' to '%s'", path, store.printStorePath(storePath));
if (cacheKey)
settings.getCache()->upsert(*cacheKey, {{"hash", hash.to_string(HashFormat::SRI, true)}});
if (cacheKey && mode == FetchMode::Copy)
settings.getCache()->upsert(*cacheKey, store, {}, storePath);
return storePath;
return {storePath, hash};
}
} // namespace nix
+4 -3
View File
@@ -339,9 +339,10 @@ std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(const Settings
// can reuse the existing nar instead of copying the unpacked
// input back into the store on every evaluation.
if (accessor->fingerprint) {
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive;
auto cacheKey = makeFetchToStoreCacheKey(getName(), *accessor->fingerprint, method, "/");
settings.getCache()->upsert(cacheKey, store, {}, storePath);
settings.getCache()->upsert(
makeSourcePathToHashCacheKey(
*accessor->fingerprint, ContentAddressMethod::Raw::NixArchive, CanonPath::root),
{{"hash", store.queryPathInfo(storePath)->narHash.to_string(HashFormat::SRI, true)}});
}
accessor->setPathDisplay("«" + to_string() + "»");
@@ -61,6 +61,11 @@ std::pair<CanonPath, std::optional<std::string>> FilteringSourceAccessor::getFin
return next->getFingerprint(prefix / path);
}
void FilteringSourceAccessor::invalidateCache(const CanonPath & path)
{
next->invalidateCache(prefix / path);
}
void FilteringSourceAccessor::checkAccess(const CanonPath & path)
{
if (!isAllowed(path))
+2 -2
View File
@@ -268,10 +268,10 @@ void Fetch::fetch(
return;
}
std::filesystem::path cacheDir = getCacheDir() / "git-lfs";
auto cacheDir = getCacheDir() / "git-lfs";
std::string key = hashString(HashAlgorithm::SHA256, pointerFilePath.rel()).to_string(HashFormat::Base16, false)
+ "/" + pointer->oid;
std::filesystem::path cachePath = cacheDir / key;
auto cachePath = cacheDir / key;
if (pathExists(cachePath)) {
debug("using cache entry %s -> %s", key, PathFmt(cachePath));
sink(readFile(cachePath));
+65 -46
View File
@@ -74,6 +74,29 @@ namespace nix {
struct GitSourceAccessor;
struct GitError final : public CloneableError<GitError, Error>
{
template<typename... Ts>
GitError(const git_error & error, Ts &&... args)
: CloneableError("")
{
auto hf = HintFmt(std::forward<Ts>(args)...);
err.msg = HintFmt("%1%: %2% (libgit2 error code = %3%)", Uncolored(hf.str()), error.message, error.klass);
}
template<typename... Ts>
GitError(Ts &&... args)
: GitError(
[]() -> const git_error & {
const git_error * p = git_error_last();
assert(p && "git_error_last() is unexpectedly null");
return *p;
}(),
std::forward<Ts>(args)...)
{
}
};
typedef std::unique_ptr<git_repository, Deleter<git_repository_free>> Repository;
typedef std::unique_ptr<git_tree_entry, Deleter<git_tree_entry_free>> TreeEntry;
typedef std::unique_ptr<git_tree, Deleter<git_tree_free>> Tree;
@@ -106,7 +129,7 @@ static void initLibGit2()
static std::once_flag initialized;
std::call_once(initialized, []() {
if (git_libgit2_init() < 0)
throw Error("initialising libgit2: %s", git_error_last()->message);
throw GitError("initialising libgit2");
});
}
@@ -114,7 +137,7 @@ static git_oid hashToOID(const Hash & hash)
{
git_oid oid;
if (git_oid_fromstr(&oid, hash.gitRev().c_str()))
throw Error("cannot convert '%s' to a Git OID", hash.gitRev());
throw GitError("cannot convert '%s' to a Git OID", hash.gitRev());
return oid;
}
@@ -122,8 +145,7 @@ static Object lookupObject(git_repository * repo, const git_oid & oid, git_objec
{
Object obj;
if (git_object_lookup(Setter(obj), repo, &oid, type)) {
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return obj;
}
@@ -133,8 +155,7 @@ static T peelObject(git_object * obj, git_object_t type)
{
T obj2;
if (git_object_peel((git_object **) (typename T::pointer *) Setter(obj2), obj, type)) {
auto err = git_error_last();
throw Error("peeling Git object '%s': %s", *git_object_id(obj), err->message);
throw Error("peeling Git object '%s'", *git_object_id(obj));
}
return obj2;
}
@@ -144,7 +165,7 @@ static T dupObject(typename T::pointer obj)
{
T obj2;
if (git_object_dup((git_object **) (typename T::pointer *) Setter(obj2), (git_object *) obj))
throw Error("duplicating object '%s': %s", *git_object_id((git_object *) obj), git_error_last()->message);
throw GitError("duplicating object '%s'", *git_object_id((git_object *) obj));
return obj2;
}
@@ -216,7 +237,7 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
Repository tmpRepo;
if (git_repository_init(Setter(tmpRepo), tmpDir.string().c_str(), options.bare))
throw Error("creating Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("creating Git repository %s", PathFmt(path));
try {
std::filesystem::rename(tmpDir, path);
} catch (std::filesystem::filesystem_error & e) {
@@ -226,7 +247,8 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
|| e.code() == std::errc::directory_not_empty) {
return;
} else
throw SysError("moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
throw SystemError(
e.code(), "moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
}
// we successfully moved the repository, so the temporary directory no longer exists.
delTmpDir.cancel();
@@ -266,7 +288,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
initRepoAtomically(path, options);
if (git_repository_open(Setter(repo), path.string().c_str()))
throw Error("opening Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("opening Git repository %s", PathFmt(path));
ObjectDb odb;
if (options.packfilesOnly) {
@@ -279,28 +301,28 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
*/
if (git_odb_new(Setter(odb)))
throw Error("creating Git object database: %s", git_error_last()->message);
throw GitError("creating Git object database");
if (git_odb_backend_pack(&packBackend, (path / "objects").string().c_str()))
throw Error("creating pack backend: %s", git_error_last()->message);
throw GitError("creating pack backend");
if (git_odb_add_backend(odb.get(), packBackend, 1))
throw Error("adding pack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding pack backend to Git object database");
} else {
if (git_repository_odb(Setter(odb), repo.get()))
throw Error("getting Git object database: %s", git_error_last()->message);
throw GitError("getting Git object database");
}
// mempack_backend will be owned by the repository, so we are not expected to free it ourselves.
if (git_mempack_new(&mempackBackend))
throw Error("creating mempack backend: %s", git_error_last()->message);
throw GitError("creating mempack backend");
if (git_odb_add_backend(odb.get(), mempackBackend, 999))
throw Error("adding mempack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding mempack backend to Git object database");
if (options.packfilesOnly) {
if (git_repository_set_odb(repo.get(), odb.get()))
throw Error("setting Git object database: %s", git_error_last()->message);
throw GitError("setting Git object database");
}
}
@@ -340,7 +362,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Indexer indexer;
git_indexer_progress stats;
if (git_indexer_new(Setter(indexer), pack_dir_path.c_str(), 0, nullptr, nullptr))
throw Error("creating git packfile indexer: %s", git_error_last()->message);
throw GitError("creating git packfile indexer");
// TODO: provide index callback for checkInterrupt() termination
// though this is about an order of magnitude faster than the packbuilder
@@ -348,15 +370,15 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
constexpr size_t chunkSize = 128 * 1024;
for (size_t offset = 0; offset < buf.size; offset += chunkSize) {
if (git_indexer_append(indexer.get(), buf.ptr + offset, std::min(chunkSize, buf.size - offset), &stats))
throw Error("appending to git packfile index: %s", git_error_last()->message);
throw GitError("appending to git packfile index");
checkInterrupt();
}
if (git_indexer_commit(indexer.get(), &stats))
throw Error("committing git packfile index: %s", git_error_last()->message);
throw GitError("committing git packfile index");
if (git_mempack_reset(mempackBackend))
throw Error("resetting git mempack backend: %s", git_error_last()->message);
throw GitError("resetting git mempack backend");
checkInterrupt();
}
@@ -449,7 +471,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
void setRemote(const std::string & name, const std::string & url) override
{
if (git_remote_set_url(*this, name.c_str(), url.c_str()))
throw Error("setting remote '%s' URL to '%s': %s", name, url, git_error_last()->message);
throw GitError("setting remote '%s' URL to '%s'", name, url);
}
Hash resolveRef(std::string ref) override
@@ -462,7 +484,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
// an object_id.
std::string peeledRef = ref + "^{commit}";
if (git_revparse_single(Setter(object), *this, peeledRef.c_str()))
throw Error("resolving Git reference '%s': %s", ref, git_error_last()->message);
throw GitError("resolving Git reference '%s'", ref);
auto oid = git_object_id(object.get());
return toHash(*oid);
}
@@ -471,11 +493,11 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
{
GitConfig config;
if (git_config_open_ondisk(Setter(config), configFile.string().c_str()))
throw Error("parsing .gitmodules file: %s", git_error_last()->message);
throw GitError("parsing .gitmodules file");
ConfigIterator it;
if (git_config_iterator_glob_new(Setter(it), config.get(), "^submodule\\..*\\.(path|url|branch)$"))
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
StringMap entries;
@@ -484,7 +506,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto err = git_config_next(&entry, it.get())) {
if (err == GIT_ITEROVER)
break;
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
}
entries.emplace(entry->name + 10, entry->value);
}
@@ -521,7 +543,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
git_oid headRev;
if (auto err = git_reference_name_to_id(&headRev, *this, "HEAD")) {
if (err != GIT_ENOTFOUND)
throw Error("resolving HEAD: %s", git_error_last()->message);
throw GitError("resolving HEAD");
} else
info.headRev = toHash(headRev);
@@ -544,7 +566,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
options.flags |= GIT_STATUS_OPT_INCLUDE_UNMODIFIED;
options.flags |= GIT_STATUS_OPT_EXCLUDE_SUBMODULES;
if (git_status_foreach_ext(*this, &options, &statusCallbackTrampoline, &statusCallback))
throw Error("getting working directory status: %s", git_error_last()->message);
throw GitError("getting working directory status");
/* Get submodule info. */
auto modulesFile = path / ".gitmodules";
@@ -587,8 +609,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto errCode = git_object_lookup(Setter(obj), *this, &oid, GIT_OBJECT_ANY)) {
if (errCode == GIT_ENOTFOUND)
return false;
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return true;
@@ -918,7 +939,7 @@ struct GitSourceAccessor : SourceAccessor
TreeEntry copy;
if (git_tree_entry_dup(Setter(copy), entry))
throw Error("dupping tree entry: %s", git_error_last()->message);
throw GitError("dupping tree entry");
auto entryName = std::string_view(git_tree_entry_name(entry));
@@ -948,7 +969,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -983,7 +1004,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -1016,7 +1037,7 @@ struct GitSourceAccessor : SourceAccessor
Blob blob;
if (git_tree_entry_to_object((git_object **) (git_blob **) Setter(blob), *state.repo, entry))
throw Error("looking up file '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up file '%s'", showPath(path));
return blob;
}
@@ -1068,7 +1089,7 @@ struct GitExportIgnoreSourceAccessor : CachingFilteringSourceAccessor
if (git_error_last()->klass == GIT_ENOTFOUND)
return false;
else
throw Error("looking up '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up '%s'", showPath(path));
} else {
// Official git will silently reject export-ignore lines that have
// values. We do the same.
@@ -1224,17 +1245,17 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
repo.emplace(parent.repoPool.get());
if (git_blob_create_from_stream(Setter(stream), **repo, nullptr))
throw Error("creating a blob stream object: %s", git_error_last()->message);
throw GitError("creating a blob stream object");
if (stream->write(stream.get(), contents.data(), contents.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
parent.totalBufSize -= contents.size();
contents.clear();
}
} else {
if (stream->write(stream.get(), data.data(), data.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
}
}
@@ -1256,7 +1277,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
acquires ownership and frees the stream. */
git_oid oid;
if (git_blob_create_from_stream_commit(&oid, crf->stream.release()))
throw Error("creating a blob object for '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for '%s'", path);
addNode(
*_state.lock(),
crf->path,
@@ -1270,8 +1291,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, crf->contents.data(), crf->contents.size()))
throw Error(
"creating a blob object for '%s' from in-memory buffer: %s", crf->path, git_error_last()->message);
throw GitError("creating a blob object for '%s' from in-memory buffer", crf->path);
addNode(
*_state.lock(),
@@ -1295,8 +1315,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, target.c_str(), target.size()))
throw Error(
"creating a blob object for tarball symlink member '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for tarball symlink member '%s'", path);
auto state(_state.lock());
addNode(*state, path, Child{GIT_FILEMODE_LINK, oid});
@@ -1357,19 +1376,19 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
// Write this directory.
git_treebuilder * b;
if (git_treebuilder_new(&b, *repo, nullptr))
throw Error("creating a tree builder: %s", git_error_last()->message);
throw GitError("creating a tree builder");
TreeBuilder builder(b);
for (auto & [name, child] : node.children) {
auto oid_p = std::get_if<git_oid>(&child.file);
auto oid = oid_p ? *oid_p : std::get<Directory>(child.file).oid.value();
if (git_treebuilder_insert(nullptr, builder.get(), name.c_str(), &oid, child.mode))
throw Error("adding a file to a tree builder: %s", git_error_last()->message);
throw GitError("adding a file to a tree builder");
}
git_oid oid;
if (git_treebuilder_write(&oid, builder.get()))
throw Error("creating a tree object: %s", git_error_last()->message);
throw GitError("creating a tree object");
node.oid = oid;
}(_state.lock()->root);
+10 -9
View File
@@ -37,15 +37,16 @@ namespace {
// old version of git, which will ignore unrecognized `-c` options.
const std::string gitInitialBranch = "__nix_dummy_branch";
bool isCacheFileWithinTtl(time_t now, const PosixStat & st)
static bool isCacheFileWithinTtl(const Settings & settings, time_t now, const PosixStat & st)
{
return st.st_mtime + static_cast<time_t>(settings.tarballTtl) > now;
}
std::filesystem::path getCachePath(std::string_view key, bool shallow)
{
return getCacheDir() / "gitv3"
/ (hashString(HashAlgorithm::SHA256, key).to_string(HashFormat::Nix32, false) + (shallow ? "-shallow" : ""));
auto name =
hashString(HashAlgorithm::SHA256, key).to_string(HashFormat::Nix32, false) + (shallow ? "-shallow" : "");
return getCacheDir() / "gitv3" / std::move(name);
}
// Returns the name of the HEAD branch.
@@ -105,7 +106,7 @@ bool storeCachedHead(const std::string & actualUrl, bool shallow, const std::str
return true;
}
std::optional<std::string> readHeadCached(const std::string & actualUrl, bool shallow)
static std::optional<std::string> readHeadCached(const Settings & settings, const std::string & actualUrl, bool shallow)
{
// Create a cache path to store the branch of the HEAD ref. Append something
// in front of the URL to prevent collision with the repository itself.
@@ -117,7 +118,7 @@ std::optional<std::string> readHeadCached(const std::string & actualUrl, bool sh
std::optional<std::string> cachedRef;
if (st) {
cachedRef = readHead(cacheDir);
if (cachedRef != std::nullopt && *cachedRef != gitInitialBranch && isCacheFileWithinTtl(now, *st)) {
if (cachedRef != std::nullopt && *cachedRef != gitInitialBranch && isCacheFileWithinTtl(settings, now, *st)) {
debug("using cached HEAD ref '%s' for repo '%s'", *cachedRef, actualUrl);
return cachedRef;
}
@@ -728,12 +729,12 @@ struct GitInputScheme : InputScheme
return revCount;
}
std::string getDefaultRef(const RepoInfo & repoInfo, bool shallow) const
std::string getDefaultRef(const Settings & settings, const RepoInfo & repoInfo, bool shallow) const
{
auto head = std::visit(
overloaded{
[&](const std::filesystem::path & path) { return GitRepo::openRepo(path, {})->getWorkdirRef(); },
[&](const ParsedURL & url) { return readHeadCached(url.to_string(), shallow); }},
[&](const ParsedURL & url) { return readHeadCached(settings, url.to_string(), shallow); }},
repoInfo.location);
if (!head) {
warn("could not read HEAD ref from repo at '%s', using 'master'", repoInfo.locationToArg());
@@ -783,7 +784,7 @@ struct GitInputScheme : InputScheme
auto originalRef = input.getRef();
bool shallow = getShallowAttr(input);
auto ref = originalRef ? *originalRef : getDefaultRef(repoInfo, shallow);
auto ref = originalRef ? *originalRef : getDefaultRef(settings, repoInfo, shallow);
input.attrs.insert_or_assign("ref", ref);
std::filesystem::path repoDir;
@@ -822,7 +823,7 @@ struct GitInputScheme : InputScheme
/* If the local ref is older than 'tarball-ttl' seconds, do a
git fetch to update the local ref to the remote ref. */
auto st = maybeStat(localRefFile);
doFetch = !st || !isCacheFileWithinTtl(now, *st);
doFetch = !st || !isCacheFileWithinTtl(settings, now, *st);
}
}
@@ -129,6 +129,25 @@ struct Settings : public Config
true,
Xp::Flakes};
Setting<unsigned int> tarballTtl{
this,
60 * 60,
"tarball-ttl",
R"(
The number of seconds a downloaded tarball is considered fresh. If
the cached tarball is stale, Nix checks whether it is still up
to date using the ETag header. Nix downloads a new version if
the ETag header is unsupported, or the cached ETag doesn't match.
Setting the TTL to `0` forces Nix to always check if the tarball is
up to date.
Nix caches tarballs in `$XDG_CACHE_HOME/nix/tarballs`.
Files fetched via `NIX_PATH`, `fetchGit`, `fetchMercurial`,
`fetchTarball`, and `fetchurl` respect this TTL.
)"};
ref<Cache> getCache() const;
ref<GitRepo> getTarballCache() const;
@@ -24,7 +24,17 @@ StorePath fetchToStore(
PathFilter * filter = nullptr,
RepairFlag repair = NoRepair);
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string & name, const std::string & fingerprint, ContentAddressMethod method, const std::string & path);
std::pair<StorePath, Hash> fetchToStore2(
const fetchers::Settings & settings,
Store & store,
const SourcePath & path,
FetchMode mode,
std::string_view name = "source",
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive,
PathFilter * filter = nullptr,
RepairFlag repair = NoRepair);
fetchers::Cache::Key
makeSourcePathToHashCacheKey(std::string_view fingerprint, ContentAddressMethod method, const CanonPath & path);
} // namespace nix
@@ -52,6 +52,8 @@ struct FilteringSourceAccessor : SourceAccessor
std::pair<CanonPath, std::optional<std::string>> getFingerprint(const CanonPath & path) override;
void invalidateCache(const CanonPath & path) override;
/**
* Call `makeNotAllowedError` to throw a `RestrictedPathError`
* exception if `isAllowed()` returns `false` for `path`.
@@ -1,3 +1,5 @@
#pragma once
#include "nix/fetchers/fetchers.hh"
namespace nix::fetchers {
+3 -2
View File
@@ -1,5 +1,6 @@
#include "nix/fetchers/fetchers.hh"
#include "nix/util/processes.hh"
#include "nix/util/environment-variables.hh"
#include "nix/util/users.hh"
#include "nix/fetchers/cache.hh"
#include "nix/store/globals.hh"
@@ -16,9 +17,9 @@ namespace nix::fetchers {
static RunOptions hgOptions(const Strings & args)
{
auto env = getEnv();
auto env = getEnvOs();
// Set HGPLAIN: this means we get consistent output from hg and avoids leakage from a user or system .hgrc.
env["HGPLAIN"] = "";
env[OS_STR("HGPLAIN")] = OS_STR("");
return {.program = "hg", .lookupPath = true, .args = args, .environment = env};
}
+5 -7
View File
@@ -165,14 +165,12 @@ struct PathInputScheme : InputScheme
// To prevent `fetchToStore()` copying the path again to Nix
// store, pre-create an entry in the fetcher cache.
accessor->fingerprint =
fmt("path:%s", store.queryPathInfo(*storePath)->narHash.to_string(HashFormat::SRI, true));
auto narHash = store.queryPathInfo(*storePath)->narHash.to_string(HashFormat::SRI, true);
accessor->fingerprint = fmt("path:%s", narHash);
settings.getCache()->upsert(
makeFetchToStoreCacheKey(
input.getName(), *accessor->fingerprint, ContentAddressMethod::Raw::NixArchive, "/"),
store,
{},
*storePath);
makeSourcePathToHashCacheKey(
*accessor->fingerprint, ContentAddressMethod::Raw::NixArchive, CanonPath::root),
{{"hash", narHash}});
/* Trust the lastModified value supplied by the user, if
any. It's not a "secure" attribute so we don't care. */
+1 -1
View File
@@ -92,7 +92,7 @@ void Registry::remove(const Input & input)
static std::filesystem::path getSystemRegistryPath()
{
return settings.nixConfDir / "registry.json";
return nixConfDir() / "registry.json";
}
static std::shared_ptr<Registry> getSystemRegistry(const Settings & settings)
+14
View File
@@ -283,4 +283,18 @@ TEST(to_string, doesntReencodeUrl)
ASSERT_EQ(unparsed, expected);
}
TEST(parseFlakeRef, malformedGithubUrlDoesNotCrash)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
fetchers::Settings fetchSettings;
// Using ref= instead of rev= with a github: URL should produce an
// error, not an assertion failure in renderAuthorityAndPath
// (https://github.com/NixOS/nix/issues/15196).
EXPECT_THROW(
parseFlakeRef(fetchSettings, "github:nixos/nixpkgs/nixpkgs.git?ref=aead170c1a49253ebfa5027010dfd89a77b73ca4"),
Error);
}
} // namespace nix
+1 -1
View File
@@ -31,7 +31,7 @@ namespace nix::flake {
// setting name -> setting value -> allow or ignore.
typedef std::map<std::string, std::map<std::string, bool>> TrustedList;
std::filesystem::path trustedListPath()
static std::filesystem::path trustedListPath()
{
return getDataDir() / "trusted-settings.json";
}
+1
View File
@@ -128,6 +128,7 @@ static void prim_flakeRefToString(EvalState & state, const PosIdx pos, Value **
state.forceAttrs(*args[0], noPos, "while evaluating the argument passed to builtins.flakeRefToString");
fetchers::Attrs attrs;
for (const auto & attr : *args[0]->attrs()) {
state.forceValue(*attr.value, attr.pos);
auto t = attr.value->type();
if (t == nInt) {
auto intValue = attr.value->integer().value;
+2
View File
@@ -876,6 +876,8 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
CanonPath((topRef.subdir == "" ? "" : topRef.subdir + "/") + "flake.lock"),
newLockFileS,
commitMessage);
flake.lockFilePath().invalidateCache();
}
/* Rewriting the lockfile changed the top-level
+1 -1
View File
@@ -205,7 +205,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
fetchSettings,
{
.scheme = "path",
.authority = ParsedURL::Authority{},
.authority = isAbsolute(path) ? std::optional{ParsedURL::Authority{}} : std::nullopt,
.path = splitString<std::vector<std::string>>(path, "/"),
.query = query,
.fragment = fragment,
+1 -15
View File
@@ -13,8 +13,6 @@
namespace nix {
int handleExceptions(const std::string & programName, std::function<void()> fun);
/**
* Don't forget to call initPlugins() after settings are initialized!
* @param loadConfig Whether to load configuration from `nix.conf`, `NIX_CONFIG`, etc. May be disabled for unit tests.
@@ -91,19 +89,7 @@ extern volatile ::sig_atomic_t blockInt;
struct GCResults;
struct PrintFreed
{
bool show;
const GCResults & results;
PrintFreed(bool show, const GCResults & results)
: show(show)
, results(results)
{
}
~PrintFreed();
};
void printFreed(bool dryRun, const GCResults & results);
#ifndef _WIN32
/**
+41 -37
View File
@@ -1,7 +1,9 @@
#include "nix/store/globals.hh"
#include "nix/util/current-process.hh"
#include "nix/util/executable-path.hh"
#include "nix/main/shared.hh"
#include "nix/store/store-api.hh"
#include "nix/store/store-open.hh"
#include "nix/store/gc-store.hh"
#include "nix/main/loggers.hh"
#include "nix/main/progress-bar.hh"
@@ -17,6 +19,10 @@
#include <sys/stat.h>
#include <unistd.h>
#include <signal.h>
#ifndef _WIN32
# include <sys/resource.h>
#endif
#ifdef __linux__
# include <features.h>
#endif
@@ -116,6 +122,26 @@ std::string getArg(const std::string & opt, Strings::iterator & i, const Strings
static void sigHandler(int signo) {}
#endif
/**
* Increase the open file soft limit to the hard limit. On some
* platforms (macOS), the default soft limit is very low, but the hard
* limit is high. So let's just raise it the maximum permitted.
*/
void bumpFileLimit()
{
#ifndef _WIN32
struct rlimit limit;
if (getrlimit(RLIMIT_NOFILE, &limit) != 0)
return;
if (limit.rlim_cur < limit.rlim_max) {
limit.rlim_cur = limit.rlim_max;
// Ignore errors, this is best effort.
setrlimit(RLIMIT_NOFILE, &limit);
}
#endif
}
void initNix(bool loadConfig)
{
/* Turn on buffering for cerr. */
@@ -183,6 +209,8 @@ void initNix(bool loadConfig)
now. In particular, store objects should be readable by
everybody. */
umask(0022);
bumpFileLimit();
}
LegacyArgs::LegacyArgs(
@@ -209,13 +237,13 @@ LegacyArgs::LegacyArgs(
.longName = "keep-going",
.shortName = 'k',
.description = "Keep going after a build fails.",
.handler = {&(bool &) settings.keepGoing, true},
.handler = {&(bool &) settings.getWorkerSettings().keepGoing, true},
});
addFlag({
.longName = "fallback",
.description = "Build from source if substitution fails.",
.handler = {&(bool &) settings.tryFallback, true},
.handler = {&(bool &) settings.getWorkerSettings().tryFallback, true},
});
auto intSettingAlias =
@@ -252,7 +280,7 @@ LegacyArgs::LegacyArgs(
.longName = "store",
.description = "The URL of the Nix store to use.",
.labels = {"store-uri"},
.handler = {&(std::string &) settings.storeUri},
.handler = {[](std::string s) { settings.storeUri = StoreReference::parse(s); }},
});
}
@@ -304,43 +332,16 @@ void printVersion(const std::string & programName)
std::cout << "System type: " << settings.thisSystem << "\n";
std::cout << "Additional system types: " << concatStringsSep(", ", settings.extraPlatforms.get()) << "\n";
std::cout << "Features: " << concatStringsSep(", ", cfg) << "\n";
std::cout << "System configuration file: " << (settings.nixConfDir / "nix.conf").string() << "\n";
std::cout << "User configuration files: " << concatStringsSep(":", settings.nixUserConfFiles) << "\n";
std::cout << "Store directory: " << settings.nixStore << "\n";
std::cout << "System configuration file: " << nixConfFile() << "\n";
std::cout << "User configuration files: "
<< os_string_to_string(ExecutablePath{.directories = nixUserConfFiles()}.render()) << "\n";
std::cout << "Store directory: " << resolveStoreConfig(StoreReference{settings.storeUri.get()})->storeDir
<< "\n";
std::cout << "State directory: " << settings.nixStateDir << "\n";
}
throw Exit();
}
int handleExceptions(const std::string & programName, std::function<void()> fun)
{
ReceiveInterrupts receiveInterrupts; // FIXME: need better place for this
ErrorInfo::programName = baseNameOf(programName);
std::string error = ANSI_RED "error:" ANSI_NORMAL " ";
try {
fun();
} catch (Exit & e) {
return e.status;
} catch (UsageError & e) {
logError(e.info());
printError("Try '%1% --help' for more information.", programName);
return 1;
} catch (BaseError & e) {
logError(e.info());
return e.info().status;
} catch (std::bad_alloc & e) {
printError(error + "out of memory");
return 1;
} catch (std::exception & e) {
printError(error + e.what());
return 1;
}
return 0;
}
RunPager::RunPager()
{
if (!isatty(STDOUT_FILENO))
@@ -395,9 +396,12 @@ RunPager::~RunPager()
}
}
PrintFreed::~PrintFreed()
void printFreed(bool dryRun, const GCResults & results)
{
if (show)
/* bytesFreed cannot be reliably computed without actually deleting store paths because of hardlinking. */
if (dryRun)
std::cout << fmt("%d store paths would be deleted\n", results.paths.size());
else
std::cout << fmt("%d store paths deleted, %s freed\n", results.paths.size(), renderSize(results.bytesFreed));
}
+1 -1
View File
@@ -73,7 +73,7 @@ std::function<void(siginfo_t * info, void * ctx)> stackOverflowHandler(defaultSt
void defaultStackOverflowHandler(siginfo_t * info, void * ctx)
{
char msg[] = "error: stack overflow (possible infinite recursion)\n";
[[gnu::unused]] auto res = write(2, msg, strlen(msg));
[[gnu::unused]] auto res = ::write(2, msg, strlen(msg));
_exit(1); // maybe abort instead?
}
+37 -11
View File
@@ -9,6 +9,7 @@
#include "nix/store/path.hh"
#include "nix/store/store-api.hh"
#include "nix/store/store-open.hh"
#include "nix/store/store-reference.hh"
#include "nix/store/build-result.hh"
#include "nix/store/local-fs-store.hh"
#include "nix/util/base-nix-32.hh"
@@ -47,14 +48,14 @@ Store * nix_store_open(nix_c_context * context, const char * uri, const char ***
if (uri_str.empty())
return new Store{nix::openStore()};
if (!params)
return new Store{nix::openStore(uri_str)};
auto storeRef = nix::StoreReference::parse(uri_str);
nix::Store::Config::Params params_map;
for (size_t i = 0; params[i] != nullptr; i++) {
params_map[params[i][0]] = params[i][1];
if (params) {
for (size_t i = 0; params[i] != nullptr; i++) {
storeRef.params[params[i][0]] = params[i][1];
}
}
return new Store{nix::openStore(uri_str, params_map)};
return new Store{nix::openStore(std::move(storeRef))};
}
NIXC_CATCH_ERRS_NULL
}
@@ -182,10 +183,8 @@ nix_err nix_store_realise(
assert(results.size() == 1);
// Check if any builds failed
for (auto & result : results) {
if (auto * failureP = result.tryGetFailure())
failureP->rethrow();
}
for (auto & result : results)
result.tryThrowBuildError();
if (callback) {
for (const auto & result : results) {
@@ -309,7 +308,12 @@ StorePath * nix_add_derivation(nix_c_context * context, Store * store, nix_deriv
if (context)
context->last_err_code = NIX_OK;
try {
auto ret = nix::writeDerivation(*store->ptr, derivation->drv, nix::NoRepair);
/* Quite dubious that users would want this to silently suceed
without actually writing the derivation if this setting is
set, but it was that way already, so we are doing this for
back-compat for now. */
auto ret = nix::settings.readOnlyMode ? nix::computeStorePath(*store->ptr, derivation->drv)
: store->ptr->writeDerivation(derivation->drv, nix::NoRepair);
return new StorePath{ret};
}
@@ -354,4 +358,26 @@ StorePath * nix_store_query_path_from_hash_part(nix_c_context * context, Store *
NIXC_CATCH_ERRS_NULL
}
nix_err nix_store_copy_path(
nix_c_context * context, Store * srcStore, Store * dstStore, const StorePath * path, bool repair, bool checkSigs)
{
if (context)
context->last_err_code = NIX_OK;
try {
if (srcStore == nullptr)
return nix_set_err_msg(context, NIX_ERR_UNKNOWN, "Source store is null");
if (dstStore == nullptr)
return nix_set_err_msg(context, NIX_ERR_UNKNOWN, "Destination store is null");
if (path == nullptr)
return nix_set_err_msg(context, NIX_ERR_UNKNOWN, "Store path is null");
auto repairFlag = repair ? nix::RepairFlag::Repair : nix::RepairFlag::NoRepair;
auto checkSigsFlag = checkSigs ? nix::CheckSigsFlag::CheckSigs : nix::CheckSigsFlag::NoCheckSigs;
nix::copyStorePath(*srcStore->ptr, *dstStore->ptr, path->path, repairFlag, checkSigsFlag);
}
NIXC_CATCH_ERRS
}
} // extern "C"
+13
View File
@@ -270,6 +270,19 @@ nix_derivation * nix_store_drv_from_store_path(nix_c_context * context, Store *
*/
StorePath * nix_store_query_path_from_hash_part(nix_c_context * context, Store * store, const char * hash);
/**
* @brief Copy a path from one store to another.
*
* @param[out] context Optional, stores error information
* @param[in] srcStore nix source store reference
* @param[in] dstStore nix destination store reference
* @param[in] path The path to copy
* @param[in] repair Whether to repair the path
* @param[in] checkSigs Whether to check path signatures are trusted before copying
*/
nix_err nix_store_copy_path(
nix_c_context * context, Store * srcStore, Store * dstStore, const StorePath * path, bool repair, bool checkSigs);
// cffi end
#ifdef __cplusplus
}
+29 -10
View File
@@ -9,11 +9,12 @@ void TestHttpBinaryCacheStore::init()
BinaryCacheStore::init();
}
ref<TestHttpBinaryCacheStore> TestHttpBinaryCacheStoreConfig::openTestStore() const
ref<TestHttpBinaryCacheStore> TestHttpBinaryCacheStoreConfig::openTestStore(ref<FileTransfer> fileTransfer) const
{
auto store = make_ref<TestHttpBinaryCacheStore>(
ref{// FIXME we shouldn't actually need a mutable config
std::const_pointer_cast<HttpBinaryCacheStore::Config>(shared_from_this())});
std::const_pointer_cast<HttpBinaryCacheStore::Config>(shared_from_this())},
fileTransfer);
store->init();
return store;
}
@@ -49,9 +50,11 @@ void HttpsBinaryCacheStoreTest::SetUp()
// clang-format off
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", caKey.string()});
openssl({"req", "-new", "-x509", "-days", "1", "-key", caKey.string(), "-out", caCert.string(), "-subj", "/CN=TestCA"});
auto serverExtFile = tmpDir / "server.ext";
writeFile(serverExtFile, "subjectAltName=DNS:localhost,IP:127.0.0.1");
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", serverKey.string()});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1"});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost,IP:127.0.0.1"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1", "-extfile", serverExtFile.string()});
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", clientKey.string()});
openssl({"req", "-new", "-key", clientKey.string(), "-out", (tmpDir / "client.csr").string(), "-subj", "/CN=TestClient"});
openssl({"x509", "-req", "-in", (tmpDir / "client.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", clientCert.string(), "-days", "1"});
@@ -79,16 +82,18 @@ void HttpsBinaryCacheStoreTest::SetUp()
for the port explicitly - this is enough. */
std::this_thread::sleep_for(std::chrono::milliseconds(50));
/* FIXME: Don't use global settings. Tests are not run concurrently, so this is fine for now. */
oldCaCert = settings.caFile;
settings.caFile = caCert.string();
/* Create custom FileTransferSettings with our test CA certificate.
This avoids mutating global settings. */
testFileTransferSettings = std::make_unique<FileTransferSettings>();
testFileTransferSettings->caFile = caCert;
testFileTransfer = makeFileTransfer(*testFileTransferSettings);
}
void HttpsBinaryCacheStoreTest::TearDown()
{
settings.caFile = oldCaCert;
serverPid.kill();
delTmpDir.reset();
testFileTransferSettings.reset();
}
std::vector<std::string> HttpsBinaryCacheStoreTest::serverArgs()
@@ -115,11 +120,25 @@ std::vector<std::string> HttpsBinaryCacheStoreMtlsTest::serverArgs()
return args;
}
ref<TestHttpBinaryCacheStoreConfig> HttpsBinaryCacheStoreTest::makeConfig(BinaryCacheStoreConfig::Params params)
ref<TestHttpBinaryCacheStoreConfig> HttpsBinaryCacheStoreTest::makeConfig()
{
auto res = make_ref<TestHttpBinaryCacheStoreConfig>("https", fmt("localhost:%d", port), std::move(params));
auto res = make_ref<TestHttpBinaryCacheStoreConfig>(
ParsedURL{
.scheme = "https",
.authority =
ParsedURL::Authority{
.host = "localhost",
.port = port,
},
},
TestHttpBinaryCacheStoreConfig::Params{});
res->pathInfoCacheSize = 0; /* We don't want any caching in tests. */
return res;
}
ref<TestHttpBinaryCacheStore> HttpsBinaryCacheStoreTest::openStore(ref<TestHttpBinaryCacheStoreConfig> config)
{
return config->openTestStore(ref<FileTransfer>{testFileTransfer});
}
} // namespace nix::testing
@@ -28,10 +28,10 @@ public:
TestHttpBinaryCacheStore & operator=(const TestHttpBinaryCacheStore &) = delete;
TestHttpBinaryCacheStore & operator=(TestHttpBinaryCacheStore &&) = delete;
TestHttpBinaryCacheStore(ref<HttpBinaryCacheStoreConfig> config)
TestHttpBinaryCacheStore(ref<HttpBinaryCacheStoreConfig> config, ref<FileTransfer> fileTransfer)
: Store{*config}
, BinaryCacheStore{*config}
, HttpBinaryCacheStore(config)
, HttpBinaryCacheStore(config, fileTransfer)
{
diskCache = nullptr; /* Disable caching, we'll be creating a new binary cache for each test. */
}
@@ -42,19 +42,13 @@ public:
class TestHttpBinaryCacheStoreConfig : public HttpBinaryCacheStoreConfig
{
public:
TestHttpBinaryCacheStoreConfig(
std::string_view scheme, std::string_view cacheUri, const Store::Config::Params & params)
TestHttpBinaryCacheStoreConfig(ParsedURL url, const Store::Config::Params & params)
: StoreConfig(params)
, HttpBinaryCacheStoreConfig(scheme, cacheUri, params)
, HttpBinaryCacheStoreConfig(url, params)
{
}
ref<TestHttpBinaryCacheStore> openTestStore() const;
ref<Store> openStore() const override
{
return openTestStore();
}
ref<TestHttpBinaryCacheStore> openTestStore(ref<FileTransfer> fileTransfer) const;
};
class HttpsBinaryCacheStoreTest : public virtual LibStoreNetworkTest
@@ -71,17 +65,29 @@ protected:
std::filesystem::path tmpDir, cacheDir;
std::filesystem::path caCert, caKey, serverCert, serverKey;
std::filesystem::path clientCert, clientKey;
std::string oldCaCert;
Pid serverPid;
uint16_t port = 8443;
std::shared_ptr<Store> localCacheStore;
/**
* Custom FileTransferSettings with the test CA certificate.
* This is used instead of modifying global settings.
*/
std::unique_ptr<FileTransferSettings> testFileTransferSettings;
/**
* FileTransfer instance using our test settings.
* Initialized in SetUp().
*/
std::shared_ptr<FileTransfer> testFileTransfer;
static void openssl(Strings args);
void SetUp() override;
void TearDown() override;
virtual std::vector<std::string> serverArgs();
ref<TestHttpBinaryCacheStoreConfig> makeConfig(BinaryCacheStoreConfig::Params params);
ref<TestHttpBinaryCacheStoreConfig> makeConfig();
ref<TestHttpBinaryCacheStore> openStore(ref<TestHttpBinaryCacheStoreConfig> config);
};
class HttpsBinaryCacheStoreMtlsTest : public HttpsBinaryCacheStoreTest
@@ -100,26 +100,26 @@ public:
writeProtoTest(STEM, VERSION, VALUE); \
}
#define VERSIONED_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE)
#define VERSIONED_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, (VERSION), VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, (VERSION), VALUE)
#define VERSIONED_READ_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE) \
TEST_F(FIXTURE, NAME##_json_read) \
{ \
readJsonTest(STEM, VALUE); \
#define VERSIONED_READ_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, (VERSION), VALUE) \
TEST_F(FIXTURE, NAME##_json_read) \
{ \
readJsonTest(STEM, VALUE); \
}
#define VERSIONED_WRITE_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, VERSION, VALUE) \
TEST_F(FIXTURE, NAME##_json_write) \
{ \
writeJsonTest(STEM, VALUE); \
#define VERSIONED_WRITE_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST_NO_JSON(FIXTURE, NAME, STEM, (VERSION), VALUE) \
TEST_F(FIXTURE, NAME##_json_write) \
{ \
writeJsonTest(STEM, VALUE); \
}
#define VERSIONED_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE)
#define VERSIONED_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, VERSION, VALUE) \
VERSIONED_READ_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, (VERSION), VALUE) \
VERSIONED_WRITE_CHARACTERIZATION_TEST(FIXTURE, NAME, STEM, (VERSION), VALUE)
} // namespace nix
+4 -4
View File
@@ -15,10 +15,10 @@ int testMainForBuidingPre(int argc, char ** argv)
}
// Disable build hook. We won't be testing remote builds in these unit tests. If we do, fix the above build hook.
settings.buildHook = {};
settings.getWorkerSettings().buildHook = {};
// No substituters, unless a test specifically requests.
settings.substituters = {};
settings.getWorkerSettings().substituters = {};
#ifdef __linux__ // should match the conditional around sandboxBuildDir declaration.
@@ -31,13 +31,13 @@ int testMainForBuidingPre(int argc, char ** argv)
// sandboxBuildDir = /build
// However, we have a rule that the store dir must not be inside the storeDir, so we need to pick a different
// sandboxBuildDir.
settings.sandboxBuildDir = "/test-build-dir-instead-of-usual-build-dir";
settings.getLocalSettings().sandboxBuildDir = "/test-build-dir-instead-of-usual-build-dir";
#endif
#ifdef __APPLE__
// Avoid this error, when already running in a sandbox:
// sandbox-exec: sandbox_apply: Operation not permitted
settings.sandboxMode = smDisabled;
settings.getLocalSettings().sandboxMode = smDisabled;
setEnv("_NIX_TEST_NO_SANDBOX", "1");
#endif
+7 -6
View File
@@ -1,6 +1,7 @@
#include <gtest/gtest.h>
#include "nix/store/build-result.hh"
#include "nix/util/tests/characterization.hh"
#include "nix/util/tests/json-characterization.hh"
namespace nix {
@@ -44,22 +45,22 @@ INSTANTIATE_TEST_SUITE_P(
std::pair{
"not-deterministic",
BuildResult{
.inner{BuildResult::Failure{
.inner{BuildResult::Failure{{
.status = BuildResult::Failure::NotDeterministic,
.errorMsg = "no idea why",
.msg = HintFmt("no idea why"),
.isNonDeterministic = false, // Note: This field is separate from the status
}},
}}},
.timesBuilt = 1,
},
},
std::pair{
"output-rejected",
BuildResult{
.inner{BuildResult::Failure{
.inner{BuildResult::Failure{{
.status = BuildResult::Failure::OutputRejected,
.errorMsg = "no idea why",
.msg = HintFmt("no idea why"),
.isNonDeterministic = false,
}},
}}},
.timesBuilt = 3,
.startTime = 30,
.stopTime = 50,
Loaded 100 of 355 files, more files were not shown because too many files have changed in this diff. Show more