Commit Graph

3351 Commits

Author SHA1 Message Date
John Ericson
c3f0670b4e Merge pull request #15266 from obsidiansystems/fix-maxjobs-error
libstore: structured diagnostics for local build rejection
2026-02-17 18:39:58 +00:00
Amaan Qureshi
7cd4359a8b libstore: structured diagnostics for local build rejection
When `max-jobs = 0` and no remote builders are available, Nix reported
"required system or feature not available" even though the system and
features matched fine. The `canBuildLocally` lambda returned a plain
`bool`, conflating a configuration knob (`max-jobs = 0`) with actual
incompatibility (wrong platform, missing features). It also short-circuited
on the first failing check, so a user with both a platform mismatch and
missing features would only see one of the two.

This commit replaces the bool with a `LocalBuildRejection` struct whose
`WrongLocalStore` variant collects all applicable failures into
`badPlatform`, `missingFeatures`, and an orthogonal `maxJobsZero` flag.
Platform mismatch and missing features now produce separate error
paragraphs, and all applicable reasons appear in a single message.

The local-build capability check also now returns
`std::variant<LocalBuildCapability, LocalBuildRejection>`, bundling
the `LocalStore &` and optional `ExternalBuilder *` together.
2026-02-17 12:54:24 -05:00
John Ericson
6e725093e6 Merge pull request #15143 from obsidiansystems/rootless-daemon-minimal
Support garbage collection in external daemon
2026-02-17 16:53:06 +00:00
Artemis Tosini
96fef69755 libstore: support searching for roots from an external daemon
This comes in two parts: a `nix store roots-daemon` command that
can run as root and list runtime roots,
and client logic to find runtime roots for a `LocalStore` by connecting
to that daemon.

This may be useful with an unprivileged nix daemon, as it would
otherwise be unable to find runtime roots from process open files
and maps.
2026-02-17 10:42:04 -05:00
Amaan Qureshi
0b7629da08 tests: quote PATH in external-builders test heredoc
The external-builders test expands `$PATH` into a heredoc without quotes,
so any `PATH` entry containing spaces causes bash to parse the line as a
command instead of an assignment, failing the test.
2026-02-16 23:20:10 -05:00
Amaan Qureshi
be6e72f11b repl: prevent exceptions from escaping editline callbacks
The tab completion handler in `completePrefix` only caught `ParseError`,
`EvalError`, `BadURL`, and `FileNotFound`. Other error types like
`JSONParseError` (which derives from `Error`, not `EvalError`) escaped
the catch block and propagated through editline's C code as undefined
behavior, crashing the REPL. This happened when tab-completing
expressions like `(builtins.fromJSON "invalid").` where evaluation
throws a non-`EvalError` exception.

This commit marks `completionCallback` and `listPossibleCallback` as
`noexcept` with function-try-blocks that catch all exceptions at the
C/C++ boundary, preventing any exception from reaching editline.

Fixes #15133.
2026-02-16 16:02:37 -05:00
Bernardo Meurer Costa
759f6c856b feat(libstore/s3): use virtual-hosted-style URLs and add addressing-style option
S3 binary caches now use virtual-hosted-style URLs by default for
standard AWS endpoints. Path-style endpoints (s3.region.amazonaws.com)
only serve HTTP/1.1, preventing HTTP/2 multiplexing and causing TCP
TIME_WAIT socket exhaustion under high concurrency. Virtual-hosted-style
endpoints (bucket.s3.region.amazonaws.com) support HTTP/2, enabling
multiplexing with the existing CURLPIPE_MULTIPLEX configuration.

Add a new `addressing-style` store option (auto/path/virtual) to control
this behavior. `auto` (default) uses virtual-hosted-style for standard
AWS endpoints and path-style for custom endpoints. `path` forces
path-style for backwards compatibility. `virtual` forces virtual-hosted-
style for all endpoints including custom ones.

Fixes: https://github.com/NixOS/nix/issues/15208
2026-02-13 00:03:50 +00:00
Artemis Tosini
c9526e289a Add new libexec/nix-nswrapper program
nix-nswrapper allows running nix in its own user namespace,
believing it is root and with access to build users for sandboxing
with auto-allocate-uids, while it is actually unprivileged.

It is used to wrap nix, and an example of its use has been
added to the unprivileged daemon functional tests.

Running it does not require any elevated privileges,
only uids and gids allocated in /etc/sub{uid,gid}
2026-02-11 16:53:08 -05:00
Sergei Zimmerman
bcc63908ba libcmd/repl: Fix issues with :ll before anything is loaded, get rid of store parameters to constructors
Fixes abort on :ll if nothing has been loaded yet. Also gets rid of
redundant openStore() calls that were dead code (store can be extracted
from EvalState already) and arguably openStore is a layer violation.

Also catches EPIPE in case the pager gets interrupted to avoid superfluous
error messages.
2026-02-06 22:29:20 +03:00
Jörg Thalheim
dcc71da7e8 Merge pull request #15148 from Mic92/fix-tests
tests: fix URL literals in functional tests
2026-02-05 01:04:32 +00:00
Jörg Thalheim
0da728b1f5 tests: fix URL literals in functional tests 2026-02-05 00:52:02 +01:00
Artemis Tosini
9e8cf9055a tests/gc-functional: fix running on NixOS
This test insisted on placing profiles in NIX_STATE_DIR, but all
packages were removed from the profile immediately after so they did not
act as garbage collector roots. Switch to directly calling nix-build,
allowing the test to run in VMs without NIX_STATE_DIR.
2026-01-30 11:52:34 -05:00
Artemis Tosini
b026649c62 libstore: fix runtime gc roots on non-standard store paths
Due to a typo in quoteRegexChars, finding runtime garbage collection roots
was failing on paths that contained a dot, or any other regex chars that would
have to be replaced.

When fixing that error, also add tests to make sure gc continues to
work.
2026-01-30 11:05:10 -05:00
John Ericson
9e9b6d44f8 Merge pull request #15094 from amaanq/git-signing-isolation
tests/functional: isolate git tests from host signing config
2026-01-27 18:11:25 +00:00
Amaan Qureshi
ac9682c52f tests/functional: isolate git tests from host signing config
Currently, tests fail when the host system has `commit.gpgsign` or
`tag.gpgsign` enabled at the system level (in my case, a custom path
located at `/etc/git/config`), since the signing key is unavailable in
the test sandbox.

The tests set `HOME=$TEST_HOME` to isolate themselves, which bypasses
the user-level git config (`~/.gitconfig`). However, if a user sets the
system-level config via `GIT_CONFIG_GLOBAL` or `GIT_CONFIG_SYSTEM`, it
still applies, causing commits to fail when signing is enabled there.

In this PR, I explicitly set `GIT_CONFIG_GLOBAL` and `GIT_CONFIG_SYSTEM`
to `/dev/null` so that the user's system config is never read from or
written to. I've also replaced `git config --global protocol.file.allow
always` with `GIT_CONFIG_*` environment variables to avoid writing to
`/dev/null`.
2026-01-27 12:32:41 -05:00
Amaan Qureshi
d1348a2477 tests/functional: fix nix-shell fixed-output derivation test
The test was checking for `$stdenv` but the `fixed` derivation doesn't
actually have stdenv, it just has `FOO`. I've updated it to check the
value of `FOO` instead.
2026-01-27 10:35:06 -05:00
John Ericson
929022c8f8 Merge pull request #15092 from NixOS/improve-error-messages
libexpr/parser: Use readable tokens in error messages instead of inte…
2026-01-27 00:01:21 +00:00
Sergei Zimmerman
68cf0a7f8a libexpr/parser: Use readable tokens in error messages instead of internal token names
Very low-hanging fruit for improving parser error messages.
2026-01-27 02:11:40 +03:00
John Ericson
395eef30f1 Merge pull request #14688 from NixOS/nixpkgs-25.11
flake: Bump nixpkgs to 25.11
2026-01-26 22:21:25 +00:00
Jörg Thalheim
ffe97db4f9 nix-store --print-env: fix shell quoting on _args output
The previous implementation double-quoted the _args variable by escaping
each argument individually and then wrapping them all in single quotes,
producing output like: _args=''-e' 'arg1' 'arg2''

This fix concatenates all arguments into a single string first, then
escapes that string once, producing correct output like:
_args='-e arg1 arg2'

This prevents potential command injection issues when the output is
sourced in shell scripts.

Fixes #14327
2026-01-26 15:08:08 -05:00
Sergei Zimmerman
00f67ee5d5 tests/functional: Require newer daemon version for empty error message bugfix 2026-01-25 23:56:44 +03:00
Sergei Zimmerman
d69001600b tests/nixos/functional/unpriviledged-daemon: Use nixStoreMountOpts instead of readOnlyNixStore
This option is not available in 25.11:

> Please use the `boot.nixStoreMountOpts' option to define mount options for the Nix store, including 'ro'
2026-01-25 22:24:34 +03:00
Taeer Bar-Yam
c1ab73f921 tests: Update version requirements on tests 2026-01-25 22:19:29 +03:00
Taeer Bar-Yam
3cb27988fb update error message of new daemon 2026-01-25 22:19:22 +03:00
Taeer Bar-Yam
fb6274b312 fix nix-serve with hacky workaround 2026-01-25 22:09:13 +03:00
Taeer Bar-Yam
7985873f73 inputDerivation is fixed upstream
fixed in nixpkgs PR #469652
2026-01-25 22:08:21 +03:00
Taeer Bar-Yam
d5544919e4 tests: minio: mc config host add -> mc alias set
`mc config host add` has been removed
SEE: https://github.com/minio/mc/issues/5206
2026-01-25 22:07:11 +03:00
Sergei Zimmerman
d45004f5ec treewide: Apply formatter diffs
Also disable some churny formatters on some specific files.
2026-01-25 22:03:16 +03:00
Sergei Zimmerman
e3b788b4ca tests/nixos/s3-binary-cache-store: Drop superfluous prints
As requested in review.
2026-01-25 19:40:30 +03:00
Bernardo Meurer Costa
3b8b764e29 feat(libstore/aws-creds): route AWS CRT logs through Nix logger
Previously AWS CRT logs went directly to stderr via ApiHandle::InitializeLogging,
causing log spam that didn't respect Nix's verbosity settings.

This implements a custom aws_logger using the aws-c-common C API that:
- Routes all AWS logs through nix::logger
- Maps AWS log levels conservatively (ERROR/WARN -> lvlInfo) since the SDK
  treats expected conditions like missing IMDS as errors
- Prefixes messages with (aws) for clarity
- Respects Nix's verbosity flags (-v, -vv, etc.)
2026-01-25 19:40:29 +03:00
Sergei Zimmerman
c4c0aee4f1 tests/nixos: Drop otherNixes.nix_2_3, replace with 2_18
Since [1] there's no way to run 2.3 anymore and overrides wouldn't be very
helpful. Let's instead use 2.18, which is the baseline for nixpkgs.
2026-01-25 00:50:55 +03:00
Sergei Zimmerman
0f22d60c7e tests/nixos: Specify -f argument to mount
Otherwise we barf on btrfs:

machine # [   17.027621] EXT4-fs error (device vdb): ext4_lookup:1819: inode #2476: comm nix: iget: checksum invalid
machine # error: getting status of '/mnt/nix/store/j8645yndikbrvn292zgvyv64xrrmwdcb-bash-5.3p3': Bad message
machine # checking '/nix/store/m3954qff15v7z1l6lpyqf8v2h47c7hv2-mailcap-2.1.54'...
machine # checking '/nix/store/xh1ff9c9c0yv1wxrwa5gnfp092yagh7v-tzdata-2025b'...
machine # [   17.172793] EXT4-fs error (device vdb): ext4_lookup:1819: inode #1777: comm nix: iget: checksum invalid
machine # error: getting status of '/mnt/nix/store/xh1ff9c9c0yv1wxrwa5gnfp092yagh7v-tzdata-2025b/share/zoneinfo/leap-seconds.list': Bad message
2026-01-25 00:50:54 +03:00
John Ericson
b7d07e42dc Merge pull request #15071 from roberth/fix-concurrent-failure-bug
tests: fix sandbox-paths in cancelled-builds test
2026-01-24 19:48:25 +00:00
Robert Hensing
7b4444f174 tests: fix sandbox-paths in cancelled-builds test
Don't add the whole store to sandbox-paths unconditionally. Exposing
the entire store defeats the purpose of sandboxing, and when the test
store is the same as the system store (NixOS VM), it causes an obscure
"Permission denied" error.

Only add sandbox-paths when not on NixOS, indicating a separate test
store that needs access to system store build tools.
2026-01-24 19:55:50 +01:00
Artemis Tosini
94907eb37a Add new VM test with unprivileged daemon user
All current NixOS functional VM tests have a daemon as root with the
tests running as different unprivileged users.
The new `functional_unprivileged-daemon` test runs the daemon and the
nix functional tests as separate unprivileged users.
Users may want to run an unprivileged daemon on non-NixOS systems
where the administrator does not fully trust nix, but multiple users
want to use nix for their own purposes. It could also be useful in
concert with an overlay-mount store, where the nix daemon cannot
modify the derivations used by the system, and thus a nix vulnerability
would not lead to root code execution.
2026-01-23 13:31:16 -05:00
John Ericson
83360cd7b7 Merge pull request #14972 from roberth/fix-concurrent-failure-bug
Fix concurrent builder failure empty message bugs
2026-01-23 00:50:47 +00:00
Sergei Zimmerman
6dd89b5a2a libutil: Add PathFmt wrapped type for formatting fs::path, fix all double-quoting issues
This will once and for all get rid of all double-quoting issues. On windows the quoting
is doubly bad because it escaped all \ to \\, which is very bad for error messages. In
order to prevent future regression std::filesystem::path formatting now must use a special
type PathFmt (like Magenta). In the future we could even change how we render filesystem paths.
2026-01-21 06:06:19 +03:00
Robert Hensing
3c3ceb18e9 DerivationTrampolineGoal: improve error message wording
Change "cannot build missing derivation" to "failed to obtain derivation of"
since the path (e.g. '...drv^out') is a derivation output, not a derivation.

The message could be improved further to resolve ambiguity when multiple
outputOf links are involved, but for now we err on the side of brevity
since this message is already merged into larger error messages with
other context from the Worker and CLI.
2026-01-14 20:42:20 +01:00
Robert Hensing
68f549def4 buildPathsWithResults: don't report cancelled goals as failures
When !keepGoing and a goal fails, other goals are cancelled and
remain with exitCode == ecBusy. These cancelled goals have a default
BuildResult::Failure{} with empty errorMsg.

Previously, buildPathsWithResults would return these cancelled goals,
and throwBuildErrors would report them as failures. When only one such
cancelled goal was present, it would throw an error with an empty
message like:

    error: build of '/nix/store/...drv^*' failed:

Now we skip goals with ecBusy since their state is indeterminate.
Cancelled goals could be reported, but this keeps the output relevant.
Other indeterminate goal states were already not being reported, for
instance: derivations that weren't started for being blocked on a
concurrency limit, or blocked on a currently building dependency.
2026-01-14 20:42:20 +01:00
Robert Hensing
3fd85c7d64 tests: don't expect cancelled goals to be reported as failures
When keepGoing=false and a build fails, other goals are cancelled.
Previously, these cancelled goals were reported in the "build of ...
failed" error message alongside actual failures. This was misleading
since cancelled goals didn't actually fail - they were never tried.

Update the test to expect only the actual failure (hash mismatch) to
be reported, not the cancelled goals.
2026-01-14 20:42:20 +01:00
Robert Hensing
25eb07a91b DerivationTrampolineGoal: use doneFailure to set buildResult
DerivationTrampolineGoal is the top-level goal whose buildResult is
returned by buildPathsWithResults. When it failed without setting
buildResult.inner, buildPathsWithResults would return failures with
empty errorMsg, producing error messages like:

  error: failed to build attribute 'checks.x86_64-linux.foo',
  build of '/nix/store/...drv^*' failed:

(note the empty message after "failed:")

Use the new doneFailure helper to ensure buildResult is populated
with meaningful error information.
2026-01-14 20:42:20 +01:00
Robert Hensing
b19bfc6373 fetchTree: improve relative path error wording
Avoid implying that relative paths could work if a base directory
were defined. The file: scheme fundamentally does not support them.
2026-01-12 20:16:26 +01:00
Robert Hensing
23a7178eb4 fetchTree: reject relative file: paths for tarballs
Relative paths like `file:./foo.tar.gz` have never worked for tarballs
because curl rejects relative file: URLs. Previously this resulted in
cryptic curl errors. Now we reject them early with a clear message
explaining that relative paths are not supported because there is no
defined base directory to resolve them against.

See https://github.com/NixOS/nix/issues/12281
2026-01-12 03:31:42 +01:00
Sergei Zimmerman
920c5ceb0c Merge pull request #14961 from NixOS/readdir-nonexistent-fix
libutil/union-source-accessor: Barf on non-existent directories
2026-01-11 18:15:22 +00:00
Sergei Zimmerman
4ab2cdacfc libutil/union-source-accessor: Barf on non-existent directories
Previously builtins.readDir would return an empty attribute set
instead of barfing on non-existent paths. This is a regression from
2.32 for impure eval.
2026-01-09 20:19:32 +03:00
Eelco Dolstra
2417ee4732 Move {init,create}GitRepo to tests/functional/common/functions.sh 2026-01-09 14:57:21 +01:00
Eelco Dolstra
7ce871ee86 Fix structured-attrs test failure in dev shell
Fixes "error: cannot create symlink '.../tests/functional/result';
already exists".
2026-01-07 18:00:27 +01:00
Graham Christensen
05df7d716a Auto-replace actually existing store path hashes 2026-01-06 17:26:20 +01:00
Sergei Zimmerman
9859068689 tests/functional: Reduce max-call-depth for stack overflow tests
This OOMs VM tests in CI and it's just wasteful.
2026-01-06 02:34:48 +03:00
John Ericson
6c884fff0c Merge pull request #14802 from NixOS/improve-timeouts-test
Improve the timeouts test
2026-01-05 19:29:16 +00:00