We are now seeing. I guess we are out with the cache. When the API responds with 418 (I'm a teapot)
it seems like the only reasonable solution is to oblige.
error: unable to download 'http://127.0.0.1:37515/7ms9f25xyxavf32pvdc3vb28nzzmkbn3.narinfo': HTTP error 418
response body:
GitHub API error: GitHub Actions Cache throttled Magic Nix Cache. Not trying to use it again on this run.
(cherry picked from commit dae41e06e8)
Fix intermittent SIGSEGV (exit code 139) on macOS when running
nix-shell and shebang tests inside the nix sandbox.
The foo, bar, and ruby test scripts were created without shebangs,
which causes intermittent crashes when executed via command
substitution on macOS. Adding proper shebangs resolves the flakiness.
Potentially closes: #13106
(cherry picked from commit 7b3d7eb634)
I messed up and accidentally configured the S3 client to use the same
host as the nix-releases bucket, but nix-channels is us-east-1 and
nix-releases is eu-west-1.
(cherry picked from commit 0900638f1d)
This workflow is supposed to automate release uploads by using OIDC
for AWS setup. DockerHub still uses long-lived credentials, but that's
not fixable. In a follow-up we could set up release uploads to GHCR too.
(cherry picked from commit 4599daa10e)
Previously it was only Eeclo doing releases that were signed with
B541D55301270E0BCF15CA5D8170B4726D7198DE. Other linux distributions
have the expectation (rightfully so) that our tags are signed. Let's
document this.
We could do cross-signing to make tracing the chain of trust easier
for all Nix team members [1].
[1]: https://nixos.org/community/teams/nix/
(cherry picked from commit 6cb8b58a47)
This allows for testing with a local minio deployment like:
./upload-release.pl --skip-docker --skip-git --s3-endpoint http://localhost:9000 --s3-host localhost:9000 1821360
(cherry picked from commit d19b8d5f99)
Best reviewed with -w --color-moved. This just moves the code
into a separate workflow. This will allow us to reuse it in
the release job for github releng of releases.
(cherry picked from commit 745983dfc0)
This reduces duplication and pins the underlying version of magic-nix-cache,
as we already do with other actions.
(cherry picked from commit ad5c6a53b9)
Until these repos are potentially merged, this is good for dogfooding
alongside the experimental installer. It also uses the more official
`artifacts.nixos.org` endpoint to install stable releases now
More immediately though, we need a patch for the experimental installer
to really work in CI at all, and that hasn't landed in a tag yet. So,
this lets us use it right from `main`!
(cherry picked from commit 92d7381826)
This fixes out-of-date information that is no longer true, and makes the
up-to-date information more accessible.
(cherry picked from commit c5fa5e503a)
Fixes#14628
- Remove mdbook-linkcheck dependency and configuration (was blocking
upgrades to mdbook 0.5.0+, configured with warning-policy = "ignore"
due to false positives, and redundant with lychee-based link checking)
- Update substitute.py and anchors.jq to handle 'items' (mdbook 0.5.x)
in addition to 'sections' (mdbook 0.4.x), as per mdbook 0.5.0
changelog: "Book::sections was renamed to Book::items"
https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#05-migration-guide
This gets us back to pre-2.23 behavior of this primop.
Done by inlining the code of `nix::dirOf` from 2.2-maintenance.
(cherry picked from commit a33fccf55a)
These will change in the next commit to fix the silent regression from 2.23
in the handling of multiple subsequent path separators.
(cherry picked from commit 86f090837b)
The `showBytes()` function was redundant with `renderSize()` as the
latter automatically selects the appropriate unit (KiB, MiB, GiB, etc.)
based on the value, whereas `showBytes()` always formatted as MiB
regardless of size.
Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
(cherry picked from commit f234633e27)
With the migration to /nix/var/nix/builds we now have failing builds
when the derivation name is too long.
This change removes the derivation name from the temporary build to have
a predictable prefix length:
Also see: https://github.com/NixOS/infra/pull/764
for context.
(cherry picked from commit 725a2f379f)
This allows the weird network or DNS server fallback mechanism inside
glibc to work, and prevents a "Resolving timed out after 5000
milliseconds" error. Read on for details.
The DNS request stuff (dns-hosts) in glibc uses this fallback procedure
to minimize network RTT in the ideal case while dealing with
ill-behaving networks and DNS servers gracefully (see resolv.conf(5)):
- Use sendmmsg() to send UDP DNS requests for IPv4 and IPv6 in parallel
- If that times out (meaning that none or only one of the responses have
been received), send the requests one by one, waiting for the response
before sending the next request ("single-request")
- If that still times out, try to use a different socket (hence
different address) for each request ("single-request-reopen")
The default timeout inside glibc is 5 seconds. Therefore, setting
connect-timeout, and therefore CURLOPT_CONNECTTIMEOUT to 5 seconds
prevents the single-request fallback, and setting it to even 10 seconds
prevents the single-request-reopen fallback as well.
The fallback decision is saved by glibc, but only thread-locally, and
libcurl starts a new thread for getaddrinfo() for each connection.
Therefore for every connection the fallback starts from sendmmsg() all
over again. And since these are considered to have timed out by libcurl,
even though getaddrinfo() might return a successful result, it is not
cached in libcurl.
While a user could tweak these with resolv.conf(5) options (e.g. using
networking.resolvconf.extraOptions in NixOS), and indeed that is
probably needed to avoid annoying delays, it still means that the
default connect-timeout of 5 is too low. Raise it to give fallback a
chance.
(cherry picked from commit 7295034362)
This reverts commit bdbc739d6e.
Such a change needs more thought put into it. By versioning
shared libraries we'd make a false impression that libraries
themselves are actually versioned and have some sort of stable
ABI, which is not the case.
This will be useful when C bindings become stable, but as long
as they are experimental it does not make sense to set SONAME.
Also this change should not have been backported, since it's
severely breaking.
(cherry picked from commit 0db2b8c8fe)
On macOS, poll() is fundamentally broken for HUP detection. It loses event
subscriptions when EVFILT_READ fires without matching the requested events
in the pollfd. This causes daemon processes to linger after client disconnect.
This commit replaces poll() with kqueue on macOS, which is what poll()
uses internally but without the bugs. The kqueue implementation uses
EVFILT_READ which works for both sockets and pipes, avoiding EVFILT_SOCK
which only works for sockets.
On Linux and other platforms, we continue using poll() with the standard
POSIX behavior where POLLHUP is always reported regardless of requested events.
Based on work from the Lix project (https://git.lix.systems/lix-project/lix)
commit 69ba3c92db3ecca468bcd5ff7849fa8e8e0fc6c0
Fixes: https://github.com/NixOS/nix/issues/13847
Related: https://git.lix.systems/lix-project/lix/issues/729
Apple bugs: rdar://37537852 (poll), FB17447257 (poll)
Co-authored-by: Jade Lovelace <jadel@mercury.com>
(cherry picked from commit 1286d5db78)
This would print erroneous and misleading diagnostics like:
> error (ignored): error: '--arg' and '--argstr' are incompatible with flakes
When run with --expr/--file. Since this installable is used to get the
bash package it doesn't make sense to check this.
(cherry picked from commit b6f98b52a4)
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
→ 'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26)
This update contains d1266642a8722f2a05e311fa151c1413d2b9653c, which
is necessary for the TOML timestamps to get tested via nixpkgsLibTests job.
(cherry picked from commit 625477a7df)
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].
1. Added more regression tests for timestamp formats.
Special attention needs to be paid to the precision
of the subsecond range for local-time. Prior versions select the closest
(upwards) multiple of 3 with a hard cap of 9 digits.
2. Normalize local datetime and offset datetime to always
use the uppercase separator `T`. This is actually the issue
surfaced in [2]. This canonicalization is basically a requirement
by (a certain reading) of rfc3339 section 5.6 [3].
3. If using toml11 >= 4.0 also keep the old behavior wrt
to the number of digits used for subsecond part of the local-time.
Newer versions cap it at 6 digits unconditionally.
[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339
(cherry picked from commit dc769d72cb)
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.
(cherry picked from commit 7ed0229d1a)
About time we upgraded our nixpkgs flake input. Ideally
we'd have automation to do this.
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/adaa24fbf46737f3f1b5497bf64bae750f82942e?narHash=sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY%3D' (2025-05-13)
→ 'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
(cherry picked from commit fc33681583)
Prior patches in 54dc5314e8
and 6db6190002 fixed the default
system double for i686 and ppc/ppc64. This also patch also covers
32 bit arm and mips. ARM cpu names are taken from host_machine.cpu()
for a lack of a better option, but host_machine.cpu_family() is
preferred, since that is supposed to be somewhat standard for cross
files. Endianness is handled correctly by looking at host_machine.endian().
This also updates the documentation to be up to date to how system cpu
is translated from the host_machine specification.
(cherry picked from commit 60d124b36e)
CI on release branches should be stable, otherwise backporting
might become flaky and unreliable. Dogfooding only really makes
sense for CI on master branch, where failures are not as tedious
to work around.
I've missed this while reviewing 6db6190002.
I only built big endian ppc64, so that didn't occur to me.
From meson manual:
> Those porting from autotools should note that Meson does not add
> endianness to the name of the cpu_family. For example, autotools will
> call little endian PPC64 "ppc64le", Meson will not, you must also check
> the .endian() value of the machine for this information.
This code should handle that correctly.
(cherry picked from commit ebd311b7b7)
* It is tough to contribute to a project that doesn't use a formatter,
* It is extra hard to contribute to a project which has configured the formatter, but ignores it for some files
* Code formatting makes it harder to hide obscure / weird bugs by accident or on purpose,
Let's rip the bandaid off?
Note that PRs currently in flight should be able to be merged relatively easily by applying `clang-format` to their tip prior to merge.
Co-authored-by: Graham Christensen <graham@grahamc.com>
Previously, `nix help shell` failed with "Nix has no subcommand 'shell'" despite `nix shell --help`
working correctly. This happened because the `shell` command is actually an alias for `env shell`,
and the help system wasn't resolving aliases when looking up documentation.
This patch modifies the `showHelp` function to check for and resolve aliases before generating the
manpage name, ensuring that shorthand commands like `shell` get proper help documentation.
Closes: #13431
(cherry picked from commit d678b071d6)
We’re already allowing `/tmp` anyway, so this should be harmless,
and it fixes a regression in the default configuration caused by
moving the build directories out of `temp-dir`. (For instance, that
broke the Lix `guessOrInventPath.sockets` test.)
Note that removing `/tmp` breaks quite a few builds, so although it may
be a good idea in general it would require work on the Nixpkgs side.
Fixes: 749afbbe99fd7b45f828b72628252feba9241362
Change-Id: I6a6a69645f429bc50d4cb24283feda3d3091f534
(This is a cherry-pick of commit d1db3e5fa3faa43b3d2f2e2e843e9cfc1e6e1b71)
Lix patch: https://gerrit.lix.systems/c/lix/+/3500
(cherry picked from commit 5cd94436f5)
c39cc00404 has added assertions for
all Value accesses and the following case has started failing with
an `unreachable`:
(/tmp/fun.nix):
```nix
{a}: a
```
```
$ nix eval --impure --expr 'import /tmp/fun.nix {a="a";b="b";}'
```
This would crash:
```
terminating due to unexpected unrecoverable internal error: Unexpected condition in getStorage at ../include/nix/expr/value.hh:844
```
This is not a regression, but rather surfaces an existing problem, which previously
was left undiagnosed. In the case of an import `fun` is the `import` primOp, so that read is invalid
and previously this resulted in an access into an inactive union member, which is UB.
The correct thing to use is `vCur`. Identical problem also affected the case of a missing argument.
Add previously failing test cases to the functional/lang test suite.
Fixes#13448.
(cherry picked from commit 6e78cc90d3)
Fixes:
[261/394] Linking target src/libexpr/libnixexpr.so
In function ‘copy’,
inlined from ‘__ct ’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:688:23,
inlined from ‘operator+’ at /nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/basic_string.h:3735:43,
inlined from ‘operator()’ at ../src/libexpr/primops/fetchClosure.cc:127:58,
inlined from ‘prim_fetchClosure’ at ../src/libexpr/primops/fetchClosure.cc:132:88:
/nix/store/24sdvjs6rfqs69d21gdn437mb3vc0svh-gcc-14.2.1.20250322/include/c++/14.2.1.20250322/bits/char_traits.h:427:56: warning: ‘__builtin_memcpy’ writing 74 bytes into a region of size 16 overflows the destination [-Wstringop-overflow=]
427 | return static_cast<char_type*>(__builtin_memcpy(__s1, __s2, __n));
| ^
../src/libexpr/primops/fetchClosure.cc: In function ‘prim_fetchClosure’:
../src/libexpr/primops/fetchClosure.cc:132:88: note: at offset 16 into destination object ‘<anonymous>’ of size 32
132 | fromPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
| ^
(cherry picked from commit aa18dc54dc)
-p preserves xattrs and acls which can be incompatible between
filesystems
Unfortunately keep -p on darwin because the bsd coreutils do not
support --preserve.
Fixes#13426
(cherry picked from commit 87299e466d)
The use of R"(...)" added a bunch of unnecessary whitespace, e.g.
error:
Unable to start any build;
either increase '--max-jobs' or enable remote builds.
For more information run 'man nix.conf' and search for '/machines'.
Now we get
error: Unable to start any build; either increase '--max-jobs' or enable remote builds.
For more information run 'man nix.conf' and search for '/machines'.
This shaves off a very significand amount of memory used
for evaluation as well as reduces the GC-managed heap.
Previously the union discriminator (InternalType) was
stored as a separate field in the Value, which takes up
whole 8 bytes due to padding needed for member alignment.
This effectively wasted 7 whole bytes of memory. Instead
of doing that InternalType is instead packed into pointer
alignment niches. As it turns out, there's more than enough
unused bits there for the bit packing to be effective.
See the doxygen comment in the ValueStorage specialization
for more details.
This does not add any performance overhead, even though
we now consistently assert the InternalType in all getters.
This can also be made atomic with a double width compare-and-swap
instruction on x86_64 (CMPXCHG16B instruction) for parallel evaluation.
This factors out most of the value representation into a mixin class.
`finishValue` is now gone for good and replaced with a simple template
function `setStorage` which derives the type information/disriminator from
the type of the argument. Likewise, reading of the value goes through function
template `getStorage`.
An empty type `Null` is introduced to make the bijection InternalType <-> C++ type
complete.
Previous use of symlink_status() always translated into a stat call, leading
to huge performance penalties for by-name-overlay in nixpkgs. The comment
below references the possible caching, but that seemed to be erroneous, since
the correct way to make use of the caching API is by calling a bunch of `is_*`
functions [1]. For example, here's how libstdc++ does that [2], [3].
This translates to great nixpkgs eval performance improvements:
```
Benchmark 1: GC_INITIAL_HEAP_SIZE=4G result/bin/nix-instantiate ../nixpkgs -A hello --readonly-mode
Time (mean ± σ): 186.7 ms ± 6.7 ms [User: 121.3 ms, System: 64.9 ms]
Range (min … max): 179.4 ms … 201.6 ms 16 runs
Benchmark 2: GC_INITIAL_HEAP_SIZE=4G nix-instantiate ../nixpkgs -A hello --readonly-mode
Time (mean ± σ): 230.6 ms ± 5.0 ms [User: 126.9 ms, System: 103.1 ms]
Range (min … max): 225.1 ms … 241.4 ms 13 runs
```
[1]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0317r1.html
[2]: 8ea555b7b4/libstdc%2B%2B-v3/include/bits/fs_dir.h (L341-L348)
[3]: 8ea555b7b4/libstdc%2B%2B-v3/include/bits/fs_dir.h (L161-L163)
These wrappers don't seem to be used anywhere in and out of tree.
Also the declaration in the header has an incorrect function name `maybeParseFlake`.
Closes#11948
Fixes
error:
… while processing sandbox path '/private/tmp/nix-shell.0MDgyx/nix-test/ca/repl/store/nix/var/nix/builds/nix-build-simple.drv-65916-3910734210' (/private/tmp/nix-shell.0MDgyx/nix-test/ca/repl/store)
error: 'nix' is too short to be a valid store path
which happened because we were now putting the build directory
underneath the store directory.
If a build directory is accessible to other users it is possible to
smuggle data in and out of build directories. Usually this is only
a build purity problem, but in combination with other issues it can
be used to break out of a build sandbox. to prevent this we default
to using a subdirectory of nixStateDir (which is more restrictive).
(cherry picked from pennae Lix commit 55b416f6897fb0d8a9315a530a9b7f0914458ded)
(store setting done by roberth)
In the case the deletion fails, we should ensure that the temporary
directory cannot be used for nefarious purposes.
Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
Signed-off-by: Raito Bezarius <raito@lix.systems>
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.
To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.
Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.
This avoids a TOCTOU which is part of the attack chain of CVE-????.
Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
`writeFile` lose its `sync` boolean flag to make things simpler.
A new `writeFileAndSync` function is created and all call sites are
converted to it.
Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
We use it immediately for the build temporary directory.
Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.
Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.
Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
Running parallel nix in nix can lead to multiple instances trying to
create the state directories and failing on the `createSymlink` step,
because the link already exists.
`replaceSymlink` is already idempotent, so let's use that.
Resolves#2706
libstdc++'s std::stable_sort and new builtins.sort implementation
special-case ranges with length less than or equal to 16 and delegate
to insertionsort.
Having a larger e2e test would allow catching sort stability issues
at functional level as well.
This prevents C++ level undefined behavior from affecting
the evaluator. Stdlib implementation details should not affect
eval, regardless of the build platform. Even erroneous usage
of `builtins.sort` should not make it possible to crash the
evaluator or produce results that depend on the host platform.
Unlike std::sort and std::stable_sort, this implementation
does not lead to out-of-bounds memory reads or other undefined
behavior when the predicate is not strict weak ordering.
This makes it possible to use this function in libexpr for
builtins.sort, where an incorrectly implemented comparator
in the user nix code currently can crash and burn the evaluator
by invoking C++ UB.
Makes the behavoral change of #13263 without the underlying refactor.
Hopefully this clearly safe from a perf and GC perspective, and will
make it easier to benchmark #13263.
This overload isn't actually necessary anywhere and
doesn't make much sense. The pointers to `Value`s are
themselves const, but the `Value`s are mutable.
A non-const member function implies that the object itself
can be modified but this doesn't make much sense considering
the return type: `Value * const * `, which is a pointer
to a constant array of pointers to mutable values.
It seems obvious that erasing any output paths from pathsInChroot needs
to happen after getPathsInSandbox(), not before.
Signed-off-by: Samuli Thomasson <samuli.thomasson@pm.me>
This adds a meson.format file that mostly mirrors the projects
meson style and a pre-commit hook to enforce this style.
Some low-diff files are formatted.
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:
nix-functional-tests> grep: repl_output: No such file or directory
nix-functional-tests> +(repl.sh:174) cat repl_output
This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.
Invoking `:ll` will start a pager with all variables which have just
been loaded by `:lf`, `:l`, or by a flake provided to `nix repl` as an
argument.
https://github.com/NixOS/nix/issues/11404
When we run `nix repl nixpkgs` we get "Added 6 variables". This is not
useful as it doesn't tell us which variables the flake has exported to
our global repl scope.
This patch prints the name of each variable that was just loaded. We
currently cap printing to 20 variables in order to avoid excessive
prints.
https://github.com/NixOS/nix/issues/11404
`getPrimOp` function was basically identical to existing
`Value::primOpAppPrimOp` modulo some trivial differences.
Makes sense to reuse existing code for that.
There's actually no mutation happening so there's no point in using
a mutable shared_ptr. Furthermore, this makes it much more evident to
the reader that no actual mutation (especially in multithreaded case)
is happening.
Also get rid of redundant constructor that isn't actually used anywhere
other than `Pos::operator std::shared_ptr<Pos>` which just passes in &*this,
(identical to just `this`), which can't be nullptr.
Judging by the comment for `makeEmptySourceAccessor` the prefix has
to be empty:
> Return a source accessor that contains only an empty root directory.
Fixes#13295.
Since the migration to meson precompiled-headers.h isn't actually used
anymore and is just confusing. Meson can't handle shared pch across
subprojects [1] and without that there's no performances benefit of PCH
at all. Also rolling our own support for that isn't trivial. See [2] for
an example of how that would look like.
[1]: https://github.com/mesonbuild/meson/issues/4350
[2]: 22bc8b6473/plugins/meson.build
Clang doesn't like the double indent that is needed for the `if...else`
that is CPP'd away. Adding braces is fine in the `if...else...` case,
and fine as a naked block in the CPP'd away case, and properly-indented
both ways.
These tests have been collected from nixpkgs f870c6ccc8951fc48aeb293cf3e98ade6ac42668
usage of builtins.match for x86_64-linux eval system. At most 2 matching and
non-matching cases are included for each encountered regex. This should
hopefully add more confidence when possibly trying to switch the regex implementation
in the future.
This is the utility changes from #9968, which were easier to rebase
first.
I (@Ericson2314) didn't write this code; I just rebased it.
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Audrey Dutcher <audrey@rhelmot.io>
Allows to copy the archive to a remote host and not get
error: cannot add path '/nix/store/01x2k4nlxcpyd85nnr0b9gm89rm8ff4x-source' because it lacks a signature by a trusted key
Before we got something like this but only inside the VM test:
vm-test-run-functional-tests-on-nixos_user> machine # fetchGit.sh: line 286: unexpected EOF while looking for matching `)'
We now try to do not too much in a single line, so that the bash parser does not get confused.
This also seems more readable and better quoted.
This makes the profiler much more useful by actually distiguishing
different derivations being evaluated. This does make the implementation
a bit more convoluted, but I think it's worth it.
Sometimes the profiler might want to do evaluation (e.g. for getting
derivation names). This is not ideal, but is really necessary
to make the profiler stack traces useful for end users.
...and also NIX_STATE_HOME in nix-profile.fish. This is directly
translated from the bash scripts and makes the fish scripts equivalent
in functionality to the bash scripts.
Note that nix-profile.fish checks for NIX_STATE_HOME and
nix-profile-daemon.fish does not, so the two scripts are no longer
identical.
This was discussed in https://github.com/NixOS/nix/issues/8034. I
personally like `PRJ_ROOT`, which hopefully avoids some ambiguity
around with subflakes.
I only implemented this for `nix fmt` because it doesn't let you point
at a flake not on your filesystem.
macOS compilation fixes
This method does *not* create a new type of goal. We instead just make
`DerivationGoal` more sophisticated, which is much easier to do now that
`DerivationBuildingGoal` has been split from it (and so many fields are
gone, or or local variables instead).
This avoids the need for a secondarily trampoline goal that interacted
poorly with `addWantedOutputs`. That, I hope, will mean the bugs from
before do not reappear.
There may in fact be a reason to introduce such a trampoline in the
future, but it would only happen in conjunction with getting rid of
`addWantedOutputs`.
Restores the functionality (and tests) that was reverted in
f4f28cdd0e.
This patch adds support for a native stack sampling
profiler to the evaluator, which saves a collapsed stack
profile information to a configurable location.
Introduced options (in `EvalSettings`):
- `eval-profile-file` - path to the collected profile file.
- `eval-profiler-frequency` - sampling frequency.
- `eval-profiler` - enumeration option for enabling the profiler.
Currently only `flamegraph` is supported, but having this an
enumeration rather than a boolean switch leaves the door open
for other profiler variants (e.g. tracy).
Profile includes the following information on best-effort basis (e.g. some lambdas might
have an undefined name). Callstack information contains:
- Call site location (where the function gets called).
- Primop/lambda name of the function being called.
- Functors/partial applications don't have a name attached to them unlike special-cased primops and lambads.
For cases where callsite location isn't available we have to resort to providing
the location where the lambda itself is defined. This removes some of the confusing
`«none»:0` locations in the profile from previous attempts.
Example usage with piping directly into zstd for compression:
```
nix eval --no-eval-cache nixpkgs#nixosTests.gnome \
--eval-profiler flamegraph \
--eval-profile-file >(zstd -of nix.profile.zstd)
```
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Commit b36637c8f7 set
`__ETC_PROFILE_NIX_SOURCED` globally, but this is not enough to prevent
the script from being run again by child shells, because the
variable was not exported and thus not inherited by any child process.
Exporting the variable also agrees with the bash scripts.
Notably, the old behavior broke `nix develop -c fish` in some cases,
because the profile bin directory got prepended to the path, causing
binaries from the profile to override binareis from the devshell.
This wires up the {pre,post}FunctionCallHook machinery
in EvalState::callFunction and migrates FunctionCallTrace
to use the new EvalProfiler mechanisms for tracing.
Note that branches when the hook gets called are marked with [[unlikely]]
as a hint to the compiler that this is not a hot path. For non-tracing
evaluation this should be a 100% predictable branch, so the performance
cost is nonexistent.
Some measurements to prove support this point:
```
nix build .#nix-cli
nix build github:nixos/nix/d692729759e4e370361cc5105fbeb0e33137ca9e#nix-cli --out-link before
```
(Before)
```
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g before/bin/nix eval nixpkgs#gnome --no-eval-cache" --warmup 4
Benchmark 1: GC_INITIAL_HEAP_SIZE=16g before/bin/nix eval nixpkgs#gnome --no-eval-cache
Time (mean ± σ): 2.517 s ± 0.032 s [User: 1.464 s, System: 0.476 s]
Range (min … max): 2.464 s … 2.557 s 10 runs
```
(After)
```
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g result/bin/nix eval nixpkgs#gnome --no-eval-cache" --warmup 4
Benchmark 1: GC_INITIAL_HEAP_SIZE=16g result/bin/nix eval nixpkgs#gnome --no-eval-cache
Time (mean ± σ): 2.499 s ± 0.022 s [User: 1.448 s, System: 0.478 s]
Range (min … max): 2.472 s … 2.537 s 10 runs
```
This patch adds an EvalProfiler and MultiEvalProfiler that can be used
to insert hooks into the evaluation for the purposes of function tracing
(what function-trace currently does) or for flamegraph/tracy profilers.
See the following commits for how this is supposed to be integrated into
the evaluator and performance considerations.
Ensure relative path inputs are relative to the parent node's _actual_
`outPath`, instead of the subtly different `sourceInfo.outPath`.
Additionally, non-flake inputs now also have a `sourceInfo` attribute.
This fixes the relationship between `self.outPath` and
`self.sourceInfo.outPath` in some edge cases.
Fixes#13164
Previous code had a sneaky bug due to which no caching
actually happened:
```cpp
auto linesForInput = (*lines)[origin->offset];
```
That should have been:
```cpp
auto & linesForInput = (*lines)[origin->offset];
```
See [1].
Now that it also makes sense to make the cache bound in side
in order not to memoize all the sources without freeing any memory.
The default cache size has been chosen somewhat arbitrarily to be ~64k
origins. For reference, 25.05 nixpkgs has ~50k .nix files.
Simple benchmark:
```nix
let
pkgs = import <nixpkgs> { };
in
builtins.foldl' (acc: el: acc + el.line) 0 (
builtins.genList (x: builtins.unsafeGetAttrPos "gcc" pkgs) 10000
)
```
(After)
```
$ hyperfine "result/bin/nix eval -f ./test.nix"
Benchmark 1: result/bin/nix eval -f ./test.nix
Time (mean ± σ): 292.7 ms ± 3.9 ms [User: 131.0 ms, System: 120.5 ms]
Range (min … max): 288.1 ms … 300.5 ms 10 runs
```
(Before)
```
hyperfine "nix eval -f ./test.nix"
Benchmark 1: nix eval -f ./test.nix
Time (mean ± σ): 666.7 ms ± 6.4 ms [User: 428.3 ms, System: 191.2 ms]
Range (min … max): 659.7 ms … 681.3 ms 10 runs
```
If the origin happens to be a `all-packages.nix` or similar in size then the
difference is much more dramatic.
[1]: 22e3f0e987
Try to make `DerivationGoal` care less whether we're working from an
in-memory derivation or not.
It's a clean-up in its own right, but it will also help with other
cleanups under the umbrella of #12628.
Now, each class provides the initial coroutine by value. This avoids
some sketchy virtual function stuff, and will also be further put to
good use in the next commit.
As summarized in
https://github.com/NixOS/nix/issues/77#issuecomment-2843228280 the
motivation is that the complicated retry logic this introduced was
making the cleanup task #12628 harder to accomplish. It was not easy to
ascertain just what policy / semantics the extra control-flow was
implementing, in order to figure out a different way to implementing it
either.
After talking to Eelco about it, he decided we could just....get rid of
the feature entirely! It's a bit scary removing a decade+ old feature,
but I think he is right. See the release notes for more explanation.
This reverts commit 299141ecbd.
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Leverage #10766 to show how we can now resolve a store configuration
without actually opening the store for that resolved configuration.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Splicing the list element to the back can be done in
a much simpler and concise way without the need for
erasing and re-inserting the element. Doing it this
way is equivalent to just moving node pointers around,
whereas inserting/erasing allocates/deallocates new nodes.
This renders the rl-next notes when `officialRelease = false`, which
corresponds to the case where we're not on a release branch.
Previously we had disabled this behavior because changelog-d is
somewhat of a heavy dependency, being the only Haskell package.
However, we now have new circumstances that topple the tradeoff.
- We render `master` docs to https://nix.dev/manual/nix/development/release-notes/rl-next.html
- `.#manual` is a separate build now, so `nix build nix/foo` is
not affected by the increased closure of build input outputs.
Because of these factors, I believe adding this functionality back
is more valuable, as we can use it to
- Previous release notes
- Showcase the upcoming release to the community
The existing header is a bit too big. Now the following use-cases are
separated, and get their own headers:
- Using or implementing an arbitrary store: remaining `store-api.hh`
This is closer to just being about the `Store` (and `StoreConfig`)
classes, as one would expect.
- Opening a store from a textual description: `store-open.hh`
Opening an aribtrary store implementation like this requires some sort
of store registration mechanism to exists, but the caller doesn't need
to know how it works. This just exposes the functions which use such a
mechanism, without exposing the mechanism itself
- Registering a store implementation: `store-registration.hh`
This requires understanding how the mechanism actually works, and the
mechanism in question involves templated machinery in headers we
rather not expose to things that don't need it, as it would slow down
compilation for no reason.
I can't find a good way to benchmark in isolation from the
git cache, but common sense dictates that creating (and destroying)
a 131KiB std::vector for each regular file from the archive imposes
quite a significant overhead regardless of the IO bound git cache.
AFAICT there is no reason to keep a copy of the data since
it always gets fed into the sink and there are no coroutines/threads
in sight.
warning: installing Nix as root is not supported by this script!
performing a single-user installation of Nix...
copying Nix to /nix/store.......................................................
......
warning: the group 'nixbld' specified in 'build-users-group' does not exist
warning: the group 'nixbld' specified in 'build-users-group' does not exist
installing 'nix-2.28.3'
error: the group 'nixbld' specified in 'build-users-group' does not exist
/tmp/nix-binary-tarball-unpack.2j3lCU0A89/unpack/nix-2.28.3-x86_64-linux/install: unable to install Nix into your default profile
The nixos/nix docker image is built using `buildLayeredImage`, which spreads the nix store over a configured number of layers. This number was set to create an image with 100 layers. Because there is a limit of (typically) 127 layers in AUFS, this only left 27 layers to build on top. At the same time, nearly half of the created layers were only <100kb in size, many even <10kb, negating the intended advantage in cachability.
This commit moves the tradeoff a bit by reducing the number of layers to 70.
Layer sizes for the 2.28.3 nixos/nix image: https://hub.docker.com/layers/nixos/nix/2.28.3/images/sha256-d078d7153763895fce17c5fbbdeb86fcfcac414ca0ba875d413c1df57be19931
As it turns out using `std::regex` is actually the bottleneck
for root discovery. Just substituting `std::` -> `boost::`
makes root discovery twice as fast (3x if counting only userspace time).
Some rather ad-hoc measurements to motivate the switch:
(On master)
```
nix build github:nixos/nix/1e822bd4149a8bce1da81ee2ad9404986b07914c#nix-cli --out-link result-1e822bd4149a8bce1da81ee2ad9404986b07914c
taskset -c 2,3 hyperfine "result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0"
Benchmark 1: result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run --max 0
Time (mean ± σ): 481.6 ms ± 3.9 ms [User: 336.2 ms, System: 142.0 ms]
Range (min … max): 474.6 ms … 487.7 ms 10 runs
```
(After this patch)
```
taskset -c 2,3 hyperfine "result/bin/nix store gc --dry-run --max 0"
Benchmark 1: result/bin/nix store gc --dry-run --max 0
Time (mean ± σ): 254.7 ms ± 9.7 ms [User: 111.1 ms, System: 141.3 ms]
Range (min … max): 246.5 ms … 281.3 ms 10 runs
```
`boost::regex` is a drop-in replacement for `std::regex`, but much faster.
Doing a simple before/after comparison doesn't surface any change in behavior:
```
result/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
result-1e822bd4149a8bce1da81ee2ad9404986b07914c/bin/nix store gc --dry-run -vvvvv --max 0 |& grep "got additional" | wc -l
```
Remove outdated and no longer relevant TODO. It's more confusing
now, since symbol table must now be addressed by uint32_t indices
in order to keep Attr size down to 16 bytes on 64 bit machines.
This patch finally applies the transition to std::less<>,
which is a transparent comparator. There's no functional
change and string lookups in sets are now more efficient
and don't produce temporaries (e.g. set.find(std::string_view{"key"})).
Unfortunately Feature is just an alias to `std::string`
and not a new-type, so a ton of code relies on it being
exactly a `std::string`.
Using transparent comparators just for StringSet necessitates
using it here as well.
The intention is to switch to transparent comparators from N3657 for
ordered set containers for strings and using the alias consistently
would simplify things.
This trades off some executable size for measurable lexer performance
improvements.
Note on the explicitly enabling 8bit scanner.
This is needed due to the default behavior of flex (excerpt from the manual [1]):
> Flex’s default behavior is to generate an 8-bit scanner unless you
> use the ‘-Cf’ or ‘-CF’, in which case flex defaults to generating
> 7-bit scanners unless your site was always configured to generate 8-bit
> scanners.
Some quantifyable metrics:
Nixpkgs revision: a6e3f45acf4e817532a861ab0eda4ab5485fecc1
Parsing the largest file in nixpkgs: pkgs/development/haskell-modules/hackage-packages.nix.
(Before this patch)
```
$ nix build github:nixos/nix/9fe3077d4#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2518 result/lib/libnixexpr.so
$ nix build github:nixos/nix/9fe3077d4#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
result/bin/nix-instantiate --parse \
../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
Time (mean ± σ): 375.5 ms ± 6.3 ms [User: 316.9 ms, System: 56.7 ms]
Range (min … max): 368.5 ms … 388.3 ms 10 runs
```
(After the patch)
```
$ nix build .#nix-expr
$ du --apparent-size result/lib/libnixexpr.so
2685 result/lib/libnixexpr.so
$ nix build .#nix-cli
$ taskset -c 2,3 hyperfine "GC_INITIAL_HEAP_SIZE=16g \
result/bin/nix-instantiate --parse \
../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix > /dev/null"
Time (mean ± σ): 326.8 ms ± 4.9 ms [User: 269.5 ms, System: 55.3 ms]
Range (min … max): 319.7 ms … 335.5 ms 10 runs
```
Overall, the change is roughly:
- 2518KiB -> 2685KiB ~ 150 KiB of machine code
- 375ms -> 325ms ~ 50ms
The perf uplift for eval-heavy test cases is obviously less noticeable,
but it doesn't make sense not to take this free perf win.
[1]: https://westes.github.io/flex/manual/Options-Affecting-Scanner-Behavior.html#Options-Affecting-Scanner-Behavior
When a PUT is redirected, some of the data can be sent by curl before headers are read. This means the subsequent PUT operation needs to seek back to origin.
The current example relies upon [nixfmt's deprecated tree traversal
behavior](https://github.com/NixOS/nixfmt/pull/240). The simplest
alternative is the new `nixfmt-tree` wrapper for `nixfmt`/`treefmt`.
Since we dropped fs::symlink_exists, we no longer have a need for the fs
namespace. Having less abstractions makes it easier to lookup the
functions in reference documentations.
As it turns out the orignal implementation of symlink_exists cannot be
used in Nix because it did now std::filesystem::filesystem_error.
The new implementation fixes that but is now actually the same as
pathExists except for the path type.
`nix formatter build` is sort of like `nix build`: it builds, links, and
prints a path to the formatter program:
$ nix formatter build
/nix/store/cb9w44vkhk2x4adfxwgdkkf5gjmm856j-treefmt/bin/treefmt
Note that unlike `nix build`, this prints the full path to the program,
not just the store path (in the example above that would be
`/nix/store/cb9w44vkhk2x4adfxwgdkkf5gjmm856j-treefmt`).
Motivation
----------
I maintain a vim plugin that automatically runs `nix fmt` on files on
save. Since `nix fmt` can be quite slow due to nix evaluation, I choose
to cache the `nix fmt `entrypoint. This was very awkward to do, see the
implementation for details:
7864607231/lua/null-ls/builtins/formatting/nix_flake_fmt.lua (L83-L110).
I recently discovered that my implementation was buggy (it didn't handle
flakes that expose a `formatter` package, such as nixpkgs), so I had to
rework the implementation:
https://github.com/nvimtools/none-ls.nvim/pull/272.
With the new `nix formatter build` command, I can delete all this akward
code, and it will be easier for other folks to build performant editor
integrations for `nix fmt`.
This refactor shouldn't change much except add a new `nix formatter run`
command. This creates space for the new `nix formatter build` command,
which I'll be introducing in the next commit.
Before the change `nix` was stripping warning flags
reported by `gcc-14` too eagerly:
$ nix build -f. texinfo4
error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
last 25 log lines:
> 1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
> | ~~~~~~~~^~~~
> window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type []
> 1887 | &replen);
> | ^~~~~~~
> | |
> | int *
After the change the compiler flag remains:
$ ~/patched.nix build -f. texinfo4
error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
last 25 log lines:
> 1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
> | ~~~~~~~~^~~~
> window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type [-Wincompatible-pointer-types]
> 1887 | &replen);
> | ^~~~~~~
> | |
> | int *
Note the difference in flag rendering around the warning.
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda has a
good sumamry of why it happens. Befomre the change `nix` was handling
just one form or URL separator:
$ printf '\e]8;;http://example.com\e\\This is a link\e]8;;\e\\\n'
Now it also handled another for (used by gcc-14`):
printf '\e]8;;http://example.com\aThis is a link\e]8;;\a\n'
While at it fixed accumulation of trailing escape `\e\\` symbol.
sometimes it's these little things that let beginners stumble at the
first step...
mentioning one potentially foreign concept while introducing an entirely
new concept is asking enough already.
The explicit include is needed for clangd to not get confused somehow,
which is also what threw me off initially and made me pick the wrong
constructor.
The (pointer, number, number) constructor first constructs a C string
and then takes a substring from that, but we didn't specify that the
buffer needs to be NUL-terminated, and then what would be the point of
the size argument anyway...
basic_string.h:
> basic_string(const _Tp& __t, size_type __pos, size_type __n,
> const _Alloc& __a = _Alloc())
> : basic_string(_S_to_string_view(__t).substr(__pos, __n), __a) { }
Valgrind on nixops4/rust/nix-flake tests:
==1344422== Conditional jump or move depends on uninitialised value(s)
==1344422== at 0x48513E8: strlen (vg_replace_strmem.c:505)
==1344422== by 0x488E941: UnknownInlinedFun (char_traits.h:391)
==1344422== by 0x488E941: UnknownInlinedFun (string_view:141)
==1344422== by 0x488E941: UnknownInlinedFun (basic_string.h:790)
==1344422== by 0x488E941: nix_flake_reference_and_fragment_from_string (nix_api_flake.cc:81)
==1344422== by 0x127332: nix_flake::FlakeReference::parse_with_fragment (lib.rs:123)
It seems that the intention was to format a number in base 8 (as
suggested by the %o format specifier), but `perms` is a `std::string`
and not a number. Looks like `rawMode` is the correct thing to use here.
This name is close to the Nixpkgs lib function `escapeShellArg`,
making it easier to find.
A friendlier function with the same behavior as lib could be added
later.
Fixes
../src/libstore/unix/build/derivation-builder.cc:1130:86: warning: the compiler can assume that the address of ‘nix::DerivationBuilderParams::drv’ will never be NULL [-Waddress]
1130 | if (useChroot && settings.preBuildHook != "" && dynamic_cast<const Derivation *>(&drv)) {
| ^~~~
Assuming this check was left over from the time `drv` could be a
`BasicDerivation`.
I split it out before to try to separate the building logic, but now we
have the much better `DerivationBuilder` abstraction for that. With that
change, I think `LocalDerivationGoal` has outlived its usefulness.
We just inline it back into `DerivationGoal`, and do so with minimal
`#ifdef` for Windows.
Note that the order of statements in `~DerivationGoal` is different than
it was after the `~LocalDerivationGoal` split, but it is *restored* to
the way it original was before --- evidently I did the split slightly
wrong, but nobody noticed, probably because the order doesn't actually
matter.
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
... by moving our stuff out of the way from upstream's
`nixComponents` and `nixDependencies` attrsets.
(I prefer not to use overlays, but let's make it work this way
first)
Now that `DerivationBuilder` is created after the underlying data has
already been initialized, we can just refer this data normally, with a
direct reference.
Only `parsedDrv` takes a (borrowing) pointer, because independent of
initialization the derivation may or may not have structured attrs.
While it seems unlikely that `$USER` will be unset while `$HOME` is set,
as `$USER` is not used in the script and as `nix-profile-daemon.fish` is
not checking `$USER`, it seems better to remove this check.
`nix-profile.fish` and `nix-profile-daemon.fish` now become identical.
It seems reasonable to add both `$HOME/.profile/bin` and
`@localstatedir@/nix/profiles/default/bin` to `$PATH` for both user
local and daemon based nix execution. Nix daemon execution mode does
not affect these path.
The building logic is now free of the scheduling logic!
(The interface between them is just what is in the new header. This
makes it much easier to audit, and shrink over time.)
We have a new `DerivationBuilder` struct, and `DerivationBuilderParams`
`DerivationBuilderCallbacks` supporting it.
`LocalDerivationGoal` doesn't subclass any of these, so we are ready to
now move them out to a new file!
Now, most of it is in two new functions:
`LocalDerivationGoal::{,un}repareBuild`.
This might seems like a step backwards from coroutines --- now we have
more functions, and are stuck with class vars --- but I don't think it
needs to be.
There's a few options here:
- (Re)introduce coroutines for the isolated building logic. We could use the
same coroutines types, or simpler ones specialized to this use-case.
The `tryLocalBuild` caller can still use `Goal::Co`, and just will
manually "pump" this inner coroutine.
- Return closures from each step. This is sort of like coroutines by
hand, but it still allows us to stop writing down the local variables
in each type.
Being able to fully-use RAII again would be very nice!
- Keep top-level first-order functions like now, but make more
functional. Instead of having one state object (`DerivationBuilder`)
for all steps (setup, run, teardown), we can have separate structs for
the live variables at each point we consume and return.
This at least avoids "are these variables active at this time?"
questions, but doesn't give us the full benefit of RAII as we must
manually ensure FIFO create/destroy orders still.
One thing to note is that by keeping the `outputLock` unlocking in
`tryLocalBuild`, we are arguably uncovering a rebuild scheduling vs
building distinction, as the output locks are pretty squarely a
scheduling concern. It's nice that the builder doesn't need to know
about them at all.
We had fields set to the same values before in our test data. This is
not a problem per-se, but does mean we wouldn't catch certain mixups.
Now, the fields are set to distinct values (where possible), which makes
the test more robust.
This requires that we refer to the `sourceInfo` instead of the
`result`. However, `sourceInfo` does not create a chain of basedir
resolution, so we add that back with `flakeDir`.
Only a much smaller `StructuredAttrs` remains, the rest is is now moved
to `DerivationOptions`.
This gets us quite close to `std::optional<StructuredAttrs>` and
`DerivationOptions` being included in `Derivation` as fields.
Now, both the unit and functional tests relating to derivation options
are tested both ways -- with input addressing and content-addressing
derivations.
This is a small optimization used when we're signing a narinfo for
multiple keys in one go. Using this sign variant, we only compute the
NAR fingerprint once, then sign it with all the keys.
It is just use for adding context to errors, but we have `addTrace` to
do that. Let the callers do that instead.
The callers doing so is a bit duplicated, yes, but this will get better
once `DerivationOptions` is included in `Derivation`.
This fixes evaluation for Windows. There are unfortunately deps that
still don't build, but this can be fixed next.
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/52faf482a3889b7619003c0daec593a1912fddc1?narHash=sha256-6hl6L/tRnwubHcA4pfUUtk542wn2Om%2BD4UnDhlDW9BE%3D' (2025-03-30)
→ 'github:NixOS/nixpkgs/f675531bc7e6657c10a18b565cfebd8aa9e24c14?narHash=sha256-gbl9hE39nQRpZaLjhWKmEu5ejtQsgI5TWYrIVVJn30U%3D' (2025-04-09)
The STSProfileCredentialsProviders allows to assume a specific IAM role
when accessing an S3 repository. Sometimes this is needed to obtain the
permissions to operate on the bucket.
The "derivation" is one of the key concepts and captures the most distinctive aspect of Nix:
that we work with a certain type data (linked files) in a certain manner (using pure functions).
Here we finally arrange all the important pieces to show how they belong
together, while referring to the respective reference documentation for details.
This change also unbreaks downstream links to `//glossary#gloss-derivation`,
which had been broken by removing the glossary entry
Nix shipping with Yet Another Programming Language is often questioned
among beginners. This change highlights distinctive aspects of the Nix
language to ease the learning curve and better orient readers around
what really matters for using Nix.
Since it's on topic, this change also polishes the wording on the motivation for string contexts.
This change follows the definition from aptitude, but using precise notions from Nix:
> package managers deal with packages: collections of files that are
> bundled together and can be installed and removed as a group.
> [...]
> If a package A depends upon another package B, then B is required
> for A to operate properly.
> [...]
> The job of a package manager is to present an interface which assists
> the user in managing the collection of packages installed on his or her system.
>
> -- <https://www.debian.org/doc/manuals/aptitude/pr01s02.en.html>
An interesting addition:
> Packages are abstractions defining the granularity at which users can act
> (add, remove, upgrade, etc.) on available software.
> A distribution is a collection of packages maintained (hopefully) coherently.
>
> -- Package Upgrades in FOSS Distributions: Details and Challenges
> (Roberto Di Cosmo, Stefano Zacchiroli; 2009) <https://arxiv.org/pdf/0902.1610>
Notably these quotes and this change don't say anything about installation,
or what it means for software to be available. In practice, this is
handled downstream, e.g. in NixOS or Home Manager. Nix historically
provides rudimentary facilities for package management such as
`nix-env`, but I claim they are widely agreed upon being discouraged,
with plenty of arguments provided in <https://stop-using-nix-env.privatevoid.net>.
Similarly, the specific structure of packages is determined downstream,
since Nix is policy-free:
> Nix is policy-free; it provides mechanisms to implement various deployment policies, but does not enforce a specific one.
>
> -- The Purely Functional Software Deployment Model (Eelco Dolstra; 2006) <https://edolstra.github.io/pubs/phd-thesis.pdf>
Specifically, Nix mechanisms do not define what a package is supposed to be:
> It's worth noting that the Nix language is intended as a DSL for package and configuration management, but it has no notions of "packages" or "configurations".
>
> -- <https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0dbc00d>
This is why we say, Nix *allows* denoting packages in a certain way, but
doesn't enforce any particular way.
Rather than "mounting" the store inside an empty virtual filesystem,
just return the store as a virtual filesystem. This is more modular.
(FWIW, it also supports two long term hopes of mind:
1. More capability-based Nix language mode. I dream of a "super pure
eval" where you can only use relative path literals (See #8738), and
any `fetchTree`-fetched stuff + the store are all disjoint (none is
mounted in another) file systems.
2. Windows, where the store dir may include drive letters, etc., and is
thus unsuitable to be the prefix of any `CanonPath`s.
)
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Add a `secretKeyFiles` URI parameter in the store URIs receiving a
coma-separated list of Nix signing keyfiles.
For instance:
nix copy --to "file:///tmp/store?secret-keys=/tmp/key1,/tmp/key2" \
"$(nix build --print-out-paths nixpkgs#hello)"
The keys passed through this new store URI parameter are merged with
the key specified in the `secretKeyFile` parameter, if any.
We'd like to rotate the signing key for cache.nixos.org. To simplify
the transition, we'd like to sign the new paths with two keys: the new
one and the current one. With this, the cache can support nix
configurations only trusting the new key and legacy configurations
only trusting the current key.
See https://github.com/NixOS/rfcs/pull/149 for more informations
behind the motivation.
nix> meson.build:216: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:222: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:235: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:236: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:242: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
Revert most of "Hack together a fix for the public headers"
- The `libmain` change is kept, and one more libmain change is made.
(Need to update Meson and Nix per the package alike).
- The S3 situation is fixed in a different way: the variable is public
now, used in the header, and fixed accordingly.
- Fix TODO for `HAVE_EMBEDDED_SANDBOX_SHELL`
This reverts commit 2b51250534.
Previously, when lock file contained invalid JSON nix reported a parser
error without specifying the file it came from.
This change adds flake.lock file path to the error message to avoid
confusion.
Some of the maintainer attribute names got changed in nixos-unstable
(e.g. "edolstra" is now "eelco") but we want this flake to work on
nixos-24.11. So just get rid of them.
This is required for other bindings like nix-flake-c to hook into
nix-expr-c appropriately.
The `_internal` part should be a sufficient deterrent normally,
and it may also be useful for bindings that migrate from the C++
interface.
For people self-hosting caches that can be occasionally down, the
default timeout is very long. This is annoying if you are trying to
update your binary cache at the same time you are trying to update
another machine. Same if cachix has one of its rare hiccups.
We tested this value of 5s in srvos now for years and we like to travel
around the world with shitty internet, so it should be still reasonable
high.
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
Previously, when users entered an incomplete expression in the REPL,
the continuation prompt was just 10 blank spaces, which looked invisible
and gave the impression that the REPL had stalled.
This change updates the prompt to " > ", aligning it visually
with 'nix-repl> ' and clearly indicating that the REPL is waiting for
more input.
Fixes: https://github.com/NixOS/nix/issues/12702
There are two big changes:
1. Public and private config is now separated. Configuration variables
that are only used internally do not go in a header which is
installed.
(Additionally, libutil has a unix-specific private config header,
which should only be used in unix-specific code. This keeps things a
bit more organized, in a purely private implementation-internal way.)
2. Secondly, there is no more `-include`. There are very few config
items that need to be publically exposed, so now it is feasible to
just make the headers that need them just including the (public)
configuration header.
And there are also a few more small cleanups on top of those:
- The configuration files have better names.
- The few CPP variables that remain exposed in the public headers are
now also renamed to always start with `NIX_`. This ensures they should
not conflict with variables defined elsewhere.
- We now always use `#if` and not `#ifdef`/`#ifndef` for our
configuration variables, which helps avoid bugs by requiring that
variables must be defined in all cases.
See comments on the script; this is supposed to avoid breaking muscle
memory without complicating the build system (which proved harder than I
thought too) or not doing the header hygiene change at all.
link-headers: use pathlib consistenly and fix type errors
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
- Since it's now private, give it a rename. Note that I want to switch the
word order on the public ones too.
- Since it is only needed by two files, just include there rather than
the nasty blanket-forced thing.
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:
/home/thomas/.nix-profile/bin
/nix/var/nix/profiles/default/bin
Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.
This makes it behave the same as nix-daemon. Opening the store in the
parent can cause a SIGBUS in libsqlite in the child:
#0 0x00007f141cf6f789 in __memset_avx2_unaligned_erms () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
#1 0x00007f141c322fe8 in walIndexAppend () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#2 0x00007f141c3711a2 in pagerWalFrames () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#3 0x00007f141c38317e in sqlite3PagerCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#4 0x00007f141c383555 in sqlite3BtreeCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#5 0x00007f141c384797 in sqlite3VdbeHalt () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#6 0x00007f141c3b8f60 in sqlite3VdbeExec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#7 0x00007f141c3bbfef in sqlite3_step () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#8 0x00007f141c3bd0e5 in sqlite3_exec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#9 0x00007f141da140e0 in nix::SQLiteTxn::commit() () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
#10 0x00007f141d9ce69c in nix::LocalStore::registerValidPaths(std::map<nix::StorePath, nix::ValidPathInfo, std::less<nix::StorePath>, std::allocator<std::pair<nix::StorePath const, nix::ValidPathInfo> > > const&)::{lambda()#1}::operator()() const () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
- Some headers were completely redundant and have been removed.
- Other headers have been turned private.
- Unnecessary meson.build code has been removed.
- libutil-tests now has a private config header, where previously
it had none. This removes the need to expose a package version
macro publicly.
We rely on `yyerror()` instead.
> The variable yynerrs contains the number of syntax errors reported so
> far.
> Normally this variable is global; but if you request a pure parser
> (see A Pure (Reentrant) Parser) then it is a local variable which only
> the actions can access.
https://www.gnu.org/software/bison/manual/html_node/Error-Reporting-Function.html
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.
Otherwise without the change the test fails on nix-2.26 as:
error: derivation contains an illegal reference specifier 'dev'
Note: the error message does not match intended change.
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.
Today's `nixpkgs` example:
Before the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation contains an illegal reference specifier 'man'
After the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.
It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.
Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843
Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.
We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.
Co-authored-by: Mic92 <joerg@thalheim.io>
Fixes https://github.com/NixOS/nix/issues/8946
See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.
Without this :u, :sh and :i repl commands fail with:
> Cannot run 'nix-shell'/`nix-env` because no method of calling the Nix
> CLI was provided. This is a configuration problem pertaining to how
> this program was built.
Remove the default ctor argument as it evidently makes catching
refactoring bugs much harder. `NixRepl` implementation lives completely
in `repl.cc`, so we can be as explicit as necessary.
We want the $doc, $man outputs to be symlinks pointing to nix-manual and
nix-manual.man. Creating the directories first makes the `ln` command
produce symlink $doc/${nix-manual} instead.
```
$file /nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
/nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man:
symbolic link to /nix/store/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
```
This is the reason `nix-env --help` is once again broken on 2.26/2.27/master
after 4108529.
Instead of calling `worker.waitForAWhile(shared_from_this())` etc.,
the subclasses of Goal instead call protected functions defined in Goal
that abstract over these.
The code for awaiting has also been heavily simplified.
Instead of calling `addWaitee`, then suspending,
`co_await await(waitees)` is called once, which also handles the suspend.
The end-goal is to remove all manual `co_await Suspend{}`s.
We now see exception beeing thrown when remote building in master
because of writing to a non-blocking file descriptor from our json logger.
> #0 0x00007f2ea97aea9c in __pthread_kill_implementation () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #1 0x00007f2ea975c576 in raise () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #2 0x00007f2ea9744935 in abort () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #3 0x00007f2ea99e8c2b in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #4 0x00007f2ea99f820a in __cxxabiv1::__terminate(void (*)()) () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #5 0x00007f2ea99f8275 in std::terminate() () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #6 0x00007f2ea99f84c7 in __cxa_throw () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #7 0x00007f2eaa5035c2 in nix::writeFull (fd=2, s=..., allowInterrupts=true) at ../unix/file-descriptor.cc:43
> #8 0x00007f2eaa5633c4 in nix::JSONLogger::write (this=this@entry=0x249a7d40, json=...) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
> #9 0x00007f2eaa5658d7 in nix::JSONLogger::logEI (this=<optimized out>, ei=...) at ../logging.cc:242
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> #11 nix::handleExceptions (programName="nix", fun=...) at ../shared.cc:343
> #12 0x0000000000465b1f in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/allocator.h:163
> (gdb) frame 10
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> 108 logEI(ei);
So far only drainFD sets the non-blocking flag on a "readable" file descriptor,
while this is a "writeable" file descriptor.
It's not clear to me yet, why we see logs after that point, but it's
also not that bad to handle EAGAIN in read/write functions after all.
Now that we have coroutines, we can go back to loops and regular RAII,
which is must less error-proone!
I look forward to removing the other instances!
Perhaps more significantly, it no longer knows about
`LocalDerivationGoal`, and without any effort it also compiles on
Windows just fine. (`local-derivation-goal.{cc,hh}` is currently skipped
on Windows.)
Default: istty(stdout)
This refactors `nix develop` internals a bit to use the `json` type
more. The assertion now operates in the in-memory json instead of
re-parsing it. While this is technically a weaker guarantee, we
should be able to rely on the library to get this right. It's its
most essential purpose.
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.
While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.
As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.
Some ad-hoc benchmarking:
After this commit:
```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
Time (mean ± σ): 784.2 ms ± 7.1 ms [User: 561.4 ms, System: 147.7 ms]
Range (min … max): 773.5 ms … 792.6 ms 10 runs
```
On master 3604c7c51:
```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
Time (mean ± σ): 22.914 s ± 0.178 s [User: 18.524 s, System: 4.151 s]
Range (min … max): 22.738 s … 23.290 s 10 runs
```
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.
The simplification here is due to a long-standing bug, but it is not
worth fixing the bug at this time. Instead we've finally written up an
issue for the bug, and referenced the issue number in the code.
In the local building case, there is many things which can through
`BuildError`, but in the hook case there is just this one. We can
therefore simplify the code by "cinching" down the logic just to the
spot the error is thrown.
There is other code outside `libstore/build` which also uses
`BuildError`, but I believe those cases are mistakes. The point of
`BuildError` is the narrow technical use-cases of "errors which should
not be fatal with `--keep-going`". Using it outside the
building/scheduling code doesn't really make sense in that regard. It
seems likely that those usages were instead merely because "oh, this
error has something to do with building, so I guess `BuildError` is
better than `Error`".
It is quite likely that I myself used `BuildError` incorrectly as
described above :).
The variables are only set by CGroup mechanisms in `killSandbox` in the
local build. In the build hook case, these variables will not be set, so
there is nothing to do.
We can move this method from `LocalStore` to `Store` --- even if we only
want the actual builder to sign things in many cases, there is no reason
to try to enforce this policy by spurious moving the method to a
subclass.
Now, we might technically sign class, but CA derivations is
experimental, and @Ericson2314 is going to revisit all this stuff with
issue #11896 anyways.
- `chrootParentDir` can be a local variable instead of a class variable.
- `getChildStatus` can be inlined. Again, we have the `assert(!hook);`
in the local building case, which makes for a simpler thing inlined.
Thanks to the previous commit, we can inline all these small callbacks.
In the build-hook case, they were empty, and now they disappear
entirely.
While `LocalDerivationGoal` can be used in the hook case (we use it
based on whether we have a local store, not based on whether we are
using the build hook, a decision which comes later), the previous
commit's inline moved the code into a spot where we know we are cleaning
up after local building, *not* after running the build hook. This allows
for much more simplification.
The basic idea is that while we have duplicated this function, we now
have one call-site in the local build case, and one call site in the
build hook case. This unlocks big opportunities to specialize each copy,
since they really shouldn't be doing the same things. By the time we are
are done, there should not be much duplication left.
See #12628 for further info.
This is supposed to firstly improve the docs as they are, and secondly
hint at how the core conceptual information ought to be moved to the
store derivation section of the manual.
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
This variable was once used to communicate already acquired store path
locks between Nix and the build hook, but this hasn't been the case
since 9bcb4d2dd9. So let's get rid of
it.
It was first introduced in 19e0ce2c03
In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.
Sample output:
Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]: 0# 0x000000000076876A in nix
1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
10# 0x000000000087A563 in nix
11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
13# 0x00000000006F4DF5 in nix
Co-authored-by: eldritch horrors <pennae@lix.systems>
1. Fix confusing wording that might imply unnecessary double-hashing.
2. Add references to specifics of base-32 encoding.
3. Fix incorrect description that sha256 hash of `fingerprint` is
truncated. "Truncated" is actual wording used in Nix theses, but it has
unusual meaning, that is better conveyed by word "compressed", which is
used by the reference C++ implementation.
4. Clarify details of base16 encoding.
Doing this makes catching non-obvious bugs easier. GHA CI workload is
already a concern and there isn't much benefit in running the tests with
and without sanitizers at the same time, so UBSAN is enabled for default
checks.
This change doesn't affect production builds in any way, but is rather a
step in the direction of improving automated testing during development.
Relates to #10969.
This fixes a few of the property tests, now that the property tests
are actually generating arbitrary data - some of that data now
requiring experimental features to function properly.
Here we're switching to combinators instead of dereference operator.
It turns out the dereference operator was being executed upon test
setup, meaning that we were only using a only single value for each of
the executions of the property tests! Really not good.
And on Windows, we instead get:
operator* is not allowed in this context
ff6af6fc68/src/gen/detail/GenerationHandler.cpp (L16C31-L16C71)
Now a few of the property tests fail, because we're generating cases
which haven't been exercised before.
I refactored the way that input resolution works in `DerivationGoal`. To
be honest, it is probably unclear to the reader whether this new way is
better or worse. I suppose *intrinsic* motivation, I can say that
- the more structured use of `inputGoal` (a local variable) is better
than the shotgrun approach with `inputDrvOutputs`
- A virtual `waiteeDone` was a hack, and now it's gone.
However, the *real* motivation of this is not the above things, but that
it is needed for my mammoth refactor fixing #11897 and #11928.
It is nice that this step could come first, rather than making that
refactor even bigger.
The bug reappeared after all, and the fix introduced a different bug. I
just reverted on 2.27 first, in #12576, but upon further introspection
and discussion with @roberth, with preparing for and travelling to
Planet Nix I will not be able to fix it on `master` soon enough for a
revert to not be warranted here in the meantime also.
This reverts commit c98525235f.
This should fix a few packaging regressions.
`dev` also includes a merged `includes/`, which may be helpful until
inter-component includes are fixed properly.
It's not very clear what the ownership model is here, but one thing
is certain: `.up` can't be destroyed before the StaticEnv that refers
to it is.
Changing a non-owning pointer to taking shared ownership of the parent
`StaticEnv` prevents the `.up` from being freed.
I'm not a huge fan of the inverted ownership, where child `StaticEnv`
takes a refcount of the parent, but this seems like the least intrusive
way to fix the use-after-free.
This shouldn't cause any shared_ptr cycles to appear (hopefully).
When #9863 converted the `Nix::Store` free functions into member functions, the
implicit `this` argument was not accounted for when iterating over the variable
number of arguments in some functions.
Seems like this got dropped at some point during meson migration, so
put it back in the build system.
Drop all tests for `parseGitUrl`, since that function doesn't exist
and migrating doesn't look sensible because git-lfs stuff seems to use
`ParsedURL`.
Note that in pure mode, we don't need to use the union FS even when
using a chroot store, since the user shouldn't have access to the
physical /nix/store.
This makes `nix.version` quicker to evaluate, which should speed up
package listing operations.
If you want an accurate count, use `lib.optionals` in your override
instead of `null` values.
By appending a colon to MANPATH NIX_MAN_DIR gets prepended to the
final MANPATH before default search paths.
This makes man still consider default search paths, but prefers
NIX_MAN_DIR (if it exists).
It still makes sense to point NIX_MAN_DIR to a correct location
by moving man pages build from nix-manual.man to nix-cli.man, but
this should fix most common use-cases where nix is installed globally.
... as intended.
Requirements:
- don't build fresh libraries for each git commit
- have git commit in the CLI
Bug:
- echo ${version} went into the wrong file => use the fact that it's
a symlink, not just for reading but also for writing.
Logging to another Logger was kind of nonsensical - it was really just
an easy way to get it to write its output to stderr, but that only
works if the underlying logger writes to stderr.
This change is needed to make it easy to log JSON output somewhere
else (like a file or socket).
It is unused in Nix currently, but will be used in Hydra. This reflects
what Hydra does in https://github.com/NixOS/hydra/pull/1387.
We may probably to use it more widely for better SSH store performance,
but this needs to be subject to more testing before we do that.
This is a first step towards PR #10760, and the issues it addresses.
See the Doxygen for details.
Thanks to these changes, we are able to drastically restrict how the
rest of the code-base uses `ParseDerivation`.
Co-Authored-By: HaeNoe <git@haenoe.party>
It's best we teach users that the "foo" derivation is less than pure in the sense that it cannot be built just on any system, in particular that builders cannot be selected arbitrarily but based on their system-features. The `"recursive-nix"` system-feature is automatically defined by `--extra-experimental-features recursive-nix`
I do not believe there is any problem with computing
`hashDerivationModulo` the normal way with impure derivations.
Conversely, the way this used to work is very suspicious because two
almost-equal derivations that only differ in depending on different
impure derivations could have the same drv hash modulo. That is very
suspicious because there is no reason to think those two different
impure derivations will end up producing the same content-addressed
data!
Co-authored-by: Alain Zscheile <zseri.devel@ytrizja.de>
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.
"content-address*ed*" derivation is misleading because all derivations
are *themselves* content-addressed. What may or may not be
content-addressed is not derivation itself, but the *output* of the
derivation.
The outputs are not *part* of the derivation (for then the derivation
wouldn't be complete before we built it) but rather separate entities
produced by the derivation.
"content-adddress*ed*" is not correctly because it can only describe
what the derivation *is*, and that is not what we are trying to do.
"content-address*ing*" is correct because it describes what the
derivation *does* --- it produces content-addressed data.
This is a big step documenting the store layer on its own, separately from the evaluator (and `builtins.derivation`).
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.
It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.
`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
This uses the single-threaded C-based routines from libblake3.
This is not optimal performance-wise but should be a good starting point
for nix compatibility with BLAKE3 hashing until a more performant
implementation based on the multi-threaded BLAKE3 routines
(written in Rust) can be developed.
This fixes dynamic derivations, reverting #9081.
I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.
Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.
The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.
This reverts commit 8440afbed7.
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
There seems to be no good reason for `nix-profile.fish` and
`nix-profile-daemon.fish` to differ in how they look for the location of
the `ca-bundle.crt` that might be installed by one of the packages.
As `$NIX_PROFILES` points to user local paths, not checking there is
strictly less useful, it seems?
This allows a flake to specify that it needs Git submodules to be
enabled (or disabled, if we ever change the default) on the top-level
flake. This requires the input to be refetched, but since the first
fetch is lazy, this shouldn't be expensive.
Currently the only attribute allowed by `inputs.self` is `submodules`,
but more can be added in the future (e.g. a `lazy` attribute to opt in
to lazy tree behaviour).
Fixes#5312, #9842.
- To remove the stale label, just leave a new comment.
- _How to find the right people to ping?_ → [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org).
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on [Matrix - #users:nixos.org](https://matrix.to/#/#users:nixos.org).
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents:none
name:Check Docker secrets present for installer tests
description:"Git ref to build the docker image from"
required:true
type:string
is_master:
description:"Whether run from master branch"
required:true
type:boolean
secrets:
DOCKERHUB_USERNAME:
required:true
DOCKERHUB_TOKEN:
required:true
permissions:{}
jobs:
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
@@ -31,7 +31,7 @@ Today, a world-wide developer community contributes to Nix and the ecosystem tha
- [Nixpkgs](https://github.com/NixOS/nixpkgs) is [the largest, most up-to-date free software repository in the world](https://repology.org/repositories/graphs)
- [NixOS](https://github.com/NixOS/nixpkgs/tree/master/nixos) is a Linux distribution that can be configured fully declaratively
- [Discourse](https://discourse.nixos.org/)
-[Matrix](https://matrix.to/#/#nix:nixos.org)
- Matrix: [#users:nixos.org](https://matrix.to/#/#users:nixos.org) for user support and [#nix-dev:nixos.org](https://matrix.to/#/#nix-dev:nixos.org) for development
synopsis: "Temporary build directories no longer include derivation names"
prs: [13839]
---
Temporary build directories created during derivation builds no longer include the derivation name in their path to avoid build failures when the derivation name is too long. This change ensures predictable prefix lengths for build directories under `/nix/var/nix/builds`.
@@ -53,6 +53,11 @@ This command has the following operations:
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
> **Note**
>
> Downloaded channel contents are cached.
> Use `--tarball-ttl` or the [`tarball-ttl` configuration option](@docroot@/command-ref/conf-file.md#conf-tarball-ttl) to change the validity period of cached downloads.
-`--list-generations`
Prints a list of all the current existing generations for the
@@ -22,11 +22,11 @@ It is based on the current generation of the active [profile](@docroot@/command-
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
- By default, *args* is a set of names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
@@ -66,11 +66,11 @@ The arguments *args* map to store paths in a number of possible ways:
This can be used to override the priority of the derivations being installed.
This is useful if *args* are [store paths], which don't have any priority information.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store paths] that point to [store derivations][storederivation], then those store derivations are [realised], and the resulting output paths are installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] that do not point to store derivations, then these are [realised] and installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [store derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Example:
@@ -122,6 +122,8 @@ The arguments *args* map to store paths in a number of possible ways:
> You can use `native-ccacheStdenvPackages` to drastically improve rebuild time.
> You can use `native-ccacheStdenv` to drastically improve rebuild time.
> By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
@@ -195,28 +195,38 @@ Nix uses a string with the following format to identify the *system type* or *pl
<cpu>-<os>[-<abi>]
```
It is set when Nix is compiled for the given system, and based on the output of [`config.guess`](https://github.com/nixos/nix/blob/master/config/config.guess) ([upstream](https://git.savannah.gnu.org/cgit/config.git/tree/config.guess)):
It is set when Nix is compiled for the given system, and based on the output of Meson's [`host_machine` information](https://mesonbuild.com/Reference-manual_builtin_host_machine.html)>
```
<cpu>-<vendor>-<os>[<version>][-<abi>]
```
When Nix is built such that `./configure` is passed any of the `--host`, `--build`, `--target` options, the value is based on the output of [`config.sub`](https://github.com/nixos/nix/blob/master/config/config.sub) ([upstream](https://git.savannah.gnu.org/cgit/config.git/tree/config.sub)):
When cross-compiling Nix with Meson for local development, you need to specify a [cross-file](https://mesonbuild.com/Cross-compilation.html) using the `--cross-file` option. Cross-files define the target architecture and toolchain. When cross-compiling Nix with Nix, Nixpkgs takes care of this for you.
In the nix flake we also have some cross-compilation targets available:
For historic reasons and backward-compatibility, some CPU and OS identifiers are translated from the GNU Autotools naming convention in [`configure.ac`](https://github.com/nixos/nix/blob/master/configure.ac) as follows:
For historic reasons and backward-compatibility, some CPU and OS identifiers are translated as follows:
@@ -30,7 +30,7 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> src
> ├── libexpr
> │ ├── meson.build
> │ ├── value/context.hh
> │ ├── include/nix/expr/value/context.hh
> │ ├── value/context.cc
> │ …
> │
@@ -46,8 +46,12 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> │ │
> │ ├── libexpr-test-support
> │ │ ├── meson.build
> │ │ ├── include/nix/expr
> │ │ │ ├── meson.build
> │ │ │ └── tests
> │ │ │ ├── value/context.hh
> │ │ │ …
> │ │ └── tests
> │ │ ├── value/context.hh
> │ │ ├── value/context.cc
> │ │ …
> │ │
@@ -59,7 +63,7 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> ```
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `src/libexpr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/libexpr-test-support/tests/value/context.{hh,cc}`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/include/nix/expr/value/context.hh` and `src/libexpr/value/context.cc`, we write tests for it in `src/libexpr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/libexpr-test-support/include/nix/expr/tests/value/context.hh` and `src/libexpr-test-support/tests/value/context.cc`.
Data for unit tests is stored in a `data` subdir of the directory for each unit test executable.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/libstore-tests/data`.
@@ -67,7 +71,7 @@ The path to the `src/${library_name_without-nix}-test/data` directory is passed
Note that each executable only gets the data for its tests.
The unit test libraries are in `src/${library_name_without-nix}-test-support`.
All headers are in a `tests` subdirectory so they are included with `#include "tests/"`.
All headers are in a `tests` subdirectory so they are included with `#include "nix/tests/"`.
The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing.
But organizing the tests this way has one big benefit:
@@ -164,7 +168,7 @@ $ checkPhase
Sometimes it is useful to group related tests so they can be easily run together without running the entire test suite.
Each test group is in a subdirectory of `tests`.
For example, `tests/functional/ca/meson.build` defines a `ca` test group for content-addressed derivation outputs.
For example, `tests/functional/ca/meson.build` defines a `ca` test group for content-addressing derivation outputs.
Software Heritage's writing on [*Intrinsic and Extrinsic identifiers*](https://www.softwareheritage.org/2020/07/09/intrinsic-vs-extrinsic-identifiers) is also a good introduction to the value of content-addressing over other referencing schemes.
Besides content addressing, the Nix store also uses [input addressing](#gloss-input-addressed-store-object).
The industry term for storage and retrieval systems using [content addressing](#gloss-content-address). A Nix store also has [input addressing](#gloss-input-addressed-store-object), and metadata.
- [derivation]{#gloss-derivation}
A description of a build task. The result of a derivation is a
store object. Derivations declared in Nix expressions are specified
using the [`derivation` primitive](./language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-build`, or explicitly by `nix-instantiate`).
A derivation can be thought of as a [pure function](https://en.wikipedia.org/wiki/Pure_function) that produces new [store objects][store object] from existing store objects.
Derivations are implemented as [operating system processes that run in a sandbox](@docroot@/store/building.md#builder-execution).
This sandbox by default only allows reading from store objects specified as inputs, and only allows writing to designated [outputs][output] to be [captured as store objects](@docroot@/store/building.md#processing-outputs).
A derivation is typically specified as a [derivation expression] in the [Nix language], and [instantiated][instantiate] to a [store derivation].
There are multiple ways of obtaining store objects from store derivatons, collectively called [realisation][realise].
[derivation]: #gloss-derivation
- [store derivation]{#gloss-store-derivation}
A [derivation] represented as a `.drv` file in the [store].
It has a [store path], like any [store object].
It is the [instantiated][instantiate] form of a derivation.
A [directed acyclic graph](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAG) is graph whose edges are given a direction ("a to b" is not the same edge as "b to a"), and for which no possible path (created by joining together edges) forms a cycle.
DAGs are very important to Nix.
In particular, the non-self-[references][reference] of [store object][store object] form a cycle.
- [derivation path]{#gloss-derivation-path}
A [store path] which uniquely identifies a [store derivation].
See [Referencing Store Derivations](@docroot@/store/derivation/index.md#derivation-path) for details.
A [derivation] where a cryptographic hash of the [output] is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network.
A [store derivation] where a cryptographic hash of the [output] is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network.
- [store]{#gloss-store}
@@ -84,6 +116,12 @@
[store]: #gloss-store
- [Nix instance]{#gloss-nix-instance}
<!-- ambiguous -->
1. An installation of Nix, which includes the presence of a [store], and the Nix package manager which operates on that store.
A local Nix installation and a [remote builder](@docroot@/advanced-topics/distributed-builds.md) are two examples of Nix instances.
2. A running Nix process, such as the `nix` command.
- [binary cache]{#gloss-binary-cache}
A *binary cache* is a Nix store which uses a different format: its
@@ -130,15 +168,17 @@
- [input-addressed store object]{#gloss-input-addressed-store-object}
See [input-addressing derivation outputs](store/derivation/outputs/input-address.md) for details.
- [content-addressed store object]{#gloss-content-addressed-store-object}
A [store object] which is [content-addressed](#gloss-content-address),
i.e. whose [store path] is determined by its contents.
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
This includes derivations, the outputs of [content-addressing derivations](#gloss-content-addressing-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
See [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md) for details.
@@ -188,35 +228,37 @@
>
> The contents of a `.nix` file form a Nix expression.
Nix expressions specify [derivations][derivation], which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
Nix expressions specify [derivation expressions][derivation expression], which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
> **Example**
>
> Building and deploying software using Nix entails writing Nix expressions as a high-level description of packages and compositions thereof.
> Building and deploying software using Nix entails writing Nix expressions to describe [packages][package] and compositions thereof.
- [reference]{#gloss-reference}
A [store object] `O` is said to have a *reference* to a store object `P` if a [store path] to `P` appears in the contents of `O`.
An edge from one [store object] to another.
Store objects can refer to both other storeobjects and themselves.
References from a store object to itself are called *self-references*.
References other than a self-reference must not form a cycle.
See [References](@docroot@/store/store-object.md#references) for details.
[reference]: #gloss-reference
See [References](@docroot@/store/store-object.md#references) for details.
- [reachable]{#gloss-reachable}
A store path `Q` is reachable from another store path `P` if `Q`
is in the *closure* of the *references* relation.
See [References](@docroot@/store/store-object.md#references) for details.
- [closure]{#gloss-closure}
The closure of a store path is the set of store paths that are
directly or indirectly “reachable” from that store path; that is,
it’s the closure of the path under the *references* relation. For
a package, the closure of its derivation is equivalent to the
build-time dependencies, while the closure of its output path is
build-time dependencies, while the closure of its [output path] is
equivalent to its runtime dependencies. For correct deployment it
is necessary to deploy whole closures, since otherwise at runtime
files could be missing. The command `nix-store --query --requisites ` prints out
@@ -226,18 +268,31 @@
to a store object at path `Q`, then `Q` is in the closure of `P`. Further, if `Q`
references `R` then `R` is also in the closure of `P`.
See [References](@docroot@/store/store-object.md#references) for details.
[closure]: #gloss-closure
- [requisite]{#gloss-requisite}
A store object [reachable] by a path (chain of references) from a given [store object].
The [closure] is the set of requisites.
See [References](@docroot@/store/store-object.md#references) for details.
- [referrer]{#gloss-reference}
A reversed edge from one [store object] to another.
- [output]{#gloss-output}
A [store object] produced by a [derivation].
A [store object] produced by a [store derivation].
See [the `outputs` argument to the `derivation` function](@docroot@/language/derivations.md#attr-outputs) for details.
[output]: #gloss-output
- [output path]{#gloss-output-path}
The [store path] to the [output] of a [derivation].
The [store path] to the [output] of a [store derivation].
[output path]: #gloss-output-path
@@ -246,14 +301,11 @@
- [deriving path]{#gloss-deriving-path}
Deriving paths are a way to refer to [store objects][store object] that ar not yet [realised][realise].
This is necessary because, in general and particularly for [content-addressed derivations][content-addressed derivation], the [output path] of an [output] is not known in advance.
There are two forms:
Deriving paths are a way to refer to [store objects][store object] that might not yet be [realised][realise].
- *constant*: just a [store path]
It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store.
See [Deriving Path](./store/derivation/index.md#deriving-path) for details.
- *output*: a pair of a [store path] to a [derivation] and an [output] name.
Not to be confused with [derivation path].
- [deriver]{#gloss-deriver}
@@ -301,7 +353,7 @@
See [Nix Archive](store/file-system-object/content-address.html#serial-nix-archive) for details.
- [`∅`]{#gloss-emtpy-set}
- [`∅`]{#gloss-empty-set}
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile.
@@ -311,18 +363,17 @@
- [package]{#package}
1. A software package; a collection of files and other data.
A software package; files that belong together for a particular purpose, and metadata.
2. A [package attribute set].
Nix represents files as [file system objects][file system object], and how they belong together is encoded as [references][reference] between [store objects][store object] that contain these file system objects.
- [package attribute set]{#package-attribute-set}
The [Nix language] allows denoting packages in terms of [attribute sets](@docroot@/language/types.md#attribute-set) containing:
- attributes that refer to the files of a package, typically in the form of [derivation outputs](#output),
- attributes with metadata, such as information about how the package is supposed to be used.
An [attribute set](@docroot@/language/types.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
- attributes that refer to the files of a [package], typically in the form of [derivation outputs](#output),
- attributes that declare something about how the package is supposed to be installed or used,
- other metadata or arbitrary attributes.
The exact shape of these attribute sets is up to convention.
If the special attribute `__structuredAttrs` is set to `true`, the other derivation
attributes are serialised into a file in JSON format.
This obviates the need for [`passAsFile`](#adv-attr-passAsFile) since JSON files have no size restrictions, unlike process environments.
It also makes it possible to tweak derivation settings in a structured way;
see [`outputChecks`](#adv-attr-outputChecks) for example.
See the [corresponding section in the derivation page](@docroot@/store/derivation/index.md#structured-attrs) for further details.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.
## Output checks
See the [corresponding section in the derivation output page](@docroot@/store/derivation/outputs/index.md).
These attributes declare that the derivation is a so-called *fixed-output derivation* (FOD), which means that a cryptographic hash of the output is already known in advance.
As opposed to regular derivations, the [`builder`] executable of a fixed-output derivation has access to the network.
Nix computes a cryptographic hash of its output and compares that to the hash declared with these attributes.
If there is a mismatch, the derivation fails.
The rationale for fixed-output derivations is derivations such as
those produced by the `fetchurl` function. This function downloads a
file from a given URL. To ensure that the downloaded file has not
been modified, the caller must also specify a cryptographic hash of
If a `fetchurl` derivation was treated like a normal derivation, the
output paths of the derivation and *all derivations depending on it*
would change. For instance, if we were to change the URL of the
Glibc source distribution in Nixpkgs (a package on which almost all
other packages depend) massive rebuilds would be needed. This is
unfortunate for a change which we know cannot have a real effect as
it propagates upwards through the dependency graph.
For fixed-output derivations, on the other hand, the name of the
output path only depends on the `outputHash*` and `name` attributes,
while all other attributes are ignored for the purpose of computing
the output path. (The `name` attribute is included because it is
part of the path.)
As an example, here is the (simplified) Nix expression for
`fetchurl`:
```nix
{ stdenv, curl }: # The curl program is used for downloading.
{ url, sha256 }:
stdenv.mkDerivation {
name = baseNameOf (toString url);
builder = ./builder.sh;
buildInputs = [ curl ];
# This is a fixed-output derivation; the output must be a regular
# file with SHA256 hash sha256.
outputHashMode = "flat";
outputHashAlgo = "sha256";
outputHash = sha256;
inherit url;
}
```
The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
The "nix32" encoding is an adaptation of base-32 encoding.
The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings, and the [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
It can currently be `"sha1"`, `"sha256"`, `"sha512"`, or `null`.
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format.
The `outputHashMode` attribute determines how the hash is computed.
> For example, in [nix.conf](../command-ref/conf-file.md) you could add:
>
> ```
> extra-experimental-features = ca-derivations
> ```
If this attribute is set to `true`, then the derivation
outputs will be stored in a content-addressed location rather than the
traditional input-addressed one.
Setting this attribute also requires setting
[`outputHashMode`](#adv-attr-outputHashMode)
and
[`outputHashAlgo`](#adv-attr-outputHashAlgo)
like for *fixed-output derivations* (see above).
It also implicitly requires that the machine to build the derivation must have the `ca-derivations` [system feature](@docroot@/command-ref/conf-file.md#conf-system-features).
- [`passAsFile`]{#adv-attr-passAsFile}\
A list of names of attributes that should be passed via files rather
than environment variables. For example, if you have
```nix
passAsFile = ["big"];
big = "a very long string";
```
then when the builder runs, the environment variable `bigPath`
will contain the absolute path to a temporary file containing `a
very long string`. That is, for any attribute *x* listed in
`passAsFile`, Nix will pass an environment variable `xPath`
holding the path of the file containing the value of attribute
*x*. This is useful when you need to pass large strings to a
builder, since most operating systems impose a limit on the size
of the environment (typically, a few hundred kilobyte).
If this attribute is set to `true` and [distributed building is enabled](@docroot@/command-ref/conf-file.md#conf-builders), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
This is useful for derivations that are cheapest to build locally.
If this attribute is set to `false`, then Nix will always build this derivation (locally or remotely); it will not try to substitute its outputs.
This is useful for derivations that are cheaper to build than to substitute.
This attribute can be ignored by setting [`always-allow-substitutes`](@docroot@/command-ref/conf-file.md#conf-always-allow-substitutes) to `true`.
> **Note**
>
> If set to `false`, the [`builder`] should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
If the special attribute `__structuredAttrs` is set to `true`, the other derivation
attributes are serialised into a file in JSON format. The environment variable
`NIX_ATTRS_JSON_FILE` points to the exact location of that file both in a build
and a [`nix-shell`](../command-ref/nix-shell.md). This obviates the need for
[`passAsFile`](#adv-attr-passAsFile) since JSON files have no size restrictions,
unlike process environments.
It also makes it possible to tweak derivation settings in a structured way; see
[`outputChecks`](#adv-attr-outputChecks) for example.
As a convenience to Bash builders,
Nix writes a script that initialises shell variables
corresponding to all attributes that are representable in Bash. The
environment variable `NIX_ATTRS_SH_FILE` points to the exact
location of the script, both in a build and a
[`nix-shell`](../command-ref/nix-shell.md). This includes non-nested
(associative) arrays. For example, the attribute `hardening.format = true`
ends up as the Bash associative array element `${hardening[format]}`.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.
- [`outputChecks`]{#adv-attr-outputChecks}\
When using [structured attributes](#adv-attr-structuredAttrs), the `outputChecks`
attribute allows defining checks per-output.
@@ -341,6 +157,8 @@ Derivations can declare some infrequently used optional attributes.
If this attribute is set to `true` and [distributed building is enabled](@docroot@/command-ref/conf-file.md#conf-builders), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
This is useful for derivations that are cheapest to build locally.
If this attribute is set to `false`, then Nix will always build this derivation (locally or remotely); it will not try to substitute its outputs.
This is useful for derivations that are cheaper to build than to substitute.
This attribute can be ignored by setting [`always-allow-substitutes`](@docroot@/command-ref/conf-file.md#conf-always-allow-substitutes) to `true`.
> **Note**
>
> If set to `false`, the [`builder`] should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
If a derivation has the `requiredSystemFeatures` attribute, then Nix will only build it on a machine that has the corresponding features set in its [`system-features` configuration](@docroot@/command-ref/conf-file.md#conf-system-features).
@@ -370,6 +206,171 @@ Derivations can declare some infrequently used optional attributes.
ensures that the derivation can only be built on a machine with the `kvm` feature.
As discussed in [Derivation Outputs and Types of Derivations](@docroot@/store/derivation/outputs/index.md), there are multiples kinds of derivations / kinds of derivation outputs.
The choice of the following attributes determines which kind of derivation we are making.
- [`__contentAddressed`]
- [`outputHash`]
- [`outputHashAlgo`]
- [`outputHashMode`]
The three types of derivations are chosen based on the following combinations of these attributes.
> This method is part of the [`git-hashing`][xp-feature-git-hashing] experimental feature.
See [content-addressing store objects](@docroot@/store/store-object/content-address.md) for more information about the process this flag controls.
- [`outputHashAlgo`]{#adv-attr-outputHashAlgo}
This specifies the hash algorithm used to digest the [file system object] data of a content-addressing derivation output.
This works in conjunction with [`outputHashMode`](#adv-attr-outputHashAlgo).
Specifying one without the other is an error (unless `outputHash` is also specified and includes its own hash algorithm as described below).
The `outputHashAlgo` attribute specifies the hash algorithm used to compute the hash.
It can currently be `"blake3"`, `"sha1"`, `"sha256"`, `"sha512"`, or `null`.
`outputHashAlgo` can only be `null` when `outputHash` follows the SRI format, because in that case the choice of hash algorithm is determined by `outputHash`.
This will specify the output hash of the single output of a [fixed-output derivation].
The `outputHash` attribute must be a string containing the hash in either hexadecimal or "nix32" encoding, or following the format for integrity metadata as defined by [SRI](https://www.w3.org/TR/SRI/).
The "nix32" encoding is an adaptation of base-32 encoding.
> **Note**
>
> The [`convertHash`](@docroot@/language/builtins.md#builtins-convertHash) function shows how to convert between different encodings.
> The [`nix-hash` command](../command-ref/nix-hash.md) has information about obtaining the hash for some contents, as well as converting to and from encodings.
Evaluation is the process of turning a Nix expression into a [Nix value](types.md).
This happens by a number of rules, such as:
- Constructing values from literals.
For example the number literal `1` is turned into the number value `1`.
- Applying operators
For example the addition operator `+` is applied to two number values to produce a new number value.
- Applying built-in functions
For example the expression `builtins.isInt 1` is evaluated to `true`.
- Applying user-defined functions
For example the expression `(x: x + 1) 10` can[*](#laziness) be thought of rewriting `x` in the function body to the argument, `10 + 1`, which is then evaluated to `11`.
These rules are applied as needed, driven by the specific use of the expression. For example, this can occur in the Nix command line interface or interactively with the [repl (read-eval-print loop)](@docroot@/command-ref/new-cli/nix3-repl.md), which is a useful tool when learning about evaluation.
# Details
## Values {#values}
Nix values can be thought of as a subset of Nix expressions.
For example, the expression `1 + 2` is not a value, because it can be reduced to `3`. The expression `3` is a value, because it cannot be reduced any further.
Evaluation normally happens by applying rules to the "head" of the expression, which is the outermost part of the expression. The head of an expression like `[ 1 2 ]` is the list literal (`[ a1 a2 ]`), for `1 + 2` it is the addition operator (`+`), and for `f 1` it is the function application "operator" (` `).
After applying all possible rules to the head until no rules can be applied, the expression is in "weak head normal form" (WHNF). This means that the outermost constructor of the expression is evaluated, but the inner values may or may not be. "Weak" only signifies that the expression may be a function. This is an historical or academic artifact, and Nix has no use for the non-weak "head normal form".
## Laziness and thunks {#laziness}
The Nix language implements _call by need_ (as opposed to _call by value_ or _call by reference_). <!-- No wikipedia link, which would be a huge distraction. --> Call by need is commonly known as laziness in functional programming, as it is a specific implementation of the concept where evaluation is deferred until the result is required, aiming to only evaluate the parts of an expression that are needed to produce the final result.
Furthermore, the result of evaluation is preserved, in values, in `let` bindings, in function _parameters_, which behave a lot like `let` bindings, but with the notable exception of function _calls_. Results of function calls rely on being put into `let` bindings, etc to be reused. <!-- which would be prohibitively expensive and too strict, or we wouldn't have a cache key for the argument -->
When discussing the process of evaluation in lower level terms, we may define values not as a subset of expressions, but separately, where each "value" is either a data constructor, a function or a _thunk_. A thunk is a delayed computation, represented by an expression reference and a "closure" – the values for the lexical scope around the delayed expression.
As a user of the language, you generally don't have to think about thunks, as they are not part of the language semantics, but you may encounter them in the repl, in the [C API] or in discussions.
## Strictness
Instead of thinking about thunks, it is often more productive to think in terms of _strictness_.
This term is used in functional programming to refer to the opposite of laziness, i.e. not just for something like error propagation. It refers to the need to evaluate certain expressions before evaluation can produce any result.
Statements about strictness usually implicitly refer to weak head normal form.
For example, we can say that the following function is strict in its argument:
```nix
x:isAttrsx||isFunctionx
```
The above function must be strict in its argument `x` because determining its type requires evaluating `x` to at least some degree.
The following function is not strict in its argument:
```nix
x:{isOk=isAttrsx||isFunctionx;}
```
It is not strict, because it can return the attribute set before evaluating `x`.
The attribute value for `isOk`_is_ strict in `x`.
A function with a _set pattern_ is always strict in its argument, as a consequence of checking the argument's type and/or attribute names:
```nix
letf={...}:"ok";
inf(throw"kablam")
=>error:kablam
```
However, a set pattern does not add any strictness beyond WHNF of the attribute set argument.
The Nix language is designed for conveniently creating and composing *derivations*– precise descriptions of how contents of existing files are used to derive new files.
The Nix language is designed for conveniently creating and composing [derivations](@docroot@/glossary.md#gloss-derivation)– precise descriptions of how contents of existing files are used to derive new files.
> **Tip**
>
@@ -11,7 +11,14 @@ The language is:
- *domain-specific*
It comes with [built-in functions](@docroot@/language/builtins.md) to integrate with the Nix store, which manages files and performs the derivations declared in the Nix language.
The Nix language is purpose-built for working with text files.
Its most characteristic features are:
- [File system path primitives](@docroot@/language/types.md#type-path), for accessing source files
- [Indented strings](@docroot@/language/string-literals.md) and [string interpolation](@docroot@/language/string-interpolation.md), for creating file contents
- [Strings with contexts](@docroot@/language/string-context.md), for transparently linking files
It comes with [built-in functions](@docroot@/language/builtins.md) to integrate with the [Nix store](@docroot@/store/index.md), which manages files and enables [realising](@docroot@/glossary.md#gloss-realise) derivations declared in the Nix language.
The idea is that a user can combine together values to create a build instructions for derivations without manually keeping track of where they come from.
Then the Nix language implicitly does that bookkeeping to efficiently obtain the closure of derivation inputs.
The idea is that a user can reference other files when creating text files through Nix expressions, without manually keeping track of the exact paths.
Nix will ensure that the all referenced files are accessible – that all [store paths](@docroot@/glossary.md#gloss-store-path) are [valid](@docroot@/glossary.md#gloss-validity).
> **Note**
>
@@ -115,7 +115,7 @@ It creates an [attribute set] representing the string context, which can be insp
## Clearing string contexts
[`buitins.unsafeDiscardStringContext`](./builtins.md#builtins-unsafeDiscardStringContext) will make a copy of a string, but with an empty string context.
[`builtins.unsafeDiscardStringContext`](./builtins.md#builtins-unsafeDiscardStringContext) will make a copy of a string, but with an empty string context.
The returned string can be used in more ways, e.g. by operators that require the string context to be empty.
The requirement to explicitly discard the string context in such use cases helps ensure that string context elements are not lost by mistake.
The "unsafe" marker is only there to remind that Nix normally guarantees that dependencies are tracked, whereas the returned string has lost them.
For historical reasons, [derivations](@docroot@/glossary.md#gloss-store-derivation) are stored on-disk in [ATerm](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html) format.
For historical reasons, [store derivations][storederivation] are stored on-disk in [ATerm](https://homepages.cwi.nl/~daybuild/daily-books/technology/aterm-guide/aterm-guide.html) format.
## The ATerm format used
Derivations are serialised in one of the following formats:
@@ -17,3 +19,20 @@ Derivations are serialised in one of the following formats:
The only `version-string`s that are in use today are for [experimental features](@docroot@/development/experimental-features.md):
- `"xp-dyn-drv"` for the [`dynamic-derivations`](@docroot@/development/experimental-features.md#xp-feature-dynamic-derivations) experimental feature.
## Use for encoding to store object
When derivation is encoded to a [store object] we make the following choices:
- The store path name is the derivation name with `.drv` suffixed at the end
Indeed, the ATerm format above does *not* contain the name of the derivation, on the assumption that a store path will also be provided out-of-band.
- The derivation is content-addressed using the ["Text" method] of content-addressing derivations
Currently we always encode derivations to store object using the ATerm format (and the previous two choices),
but we reserve the option to encode new sorts of derivations differently in the future.
@@ -24,7 +24,7 @@ is a JSON object with the following fields:
*`method`:
For an output which will be [content addresed], a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
For an output which will be [content addressed], a string representing the [method](@docroot@/store/store-object/content-address.md) of content addressing that is chosen.
@@ -7,7 +7,7 @@ The format of this specification is close to [Extended Backus–Naur form](https
Regular users do *not* need to know this information --- store paths can be treated as black boxes computed from the properties of the store objects they refer to.
But for those interested in exactly how Nix works, e.g. if they are reimplementing it, this information can be useful.
[store path](@docroot@/store/store-path.md)
[store path]: @docroot@/store/store-path.md
## Store path proper
@@ -20,14 +20,17 @@ where
-`store-dir` = the [store directory](@docroot@/store/store-path.md#store-directory)
-`digest` = base-32 representation of the first 160 bits of a [SHA-256] hash of `fingerprint`
-`digest` = base-32 representation of the compressed to 160 bits [SHA-256] hash of `fingerprint`
This the hash part of the store name
For the definition of the hash compression algorithm, please refer to the section 5.1 of
the [Nix thesis](https://edolstra.github.io/pubs/phd-thesis.pdf), which also defines the
specifics of base-32 encoding. Note that base-32 encoding processes the hash bytestring from
the end, while base-16 processes in from the beginning.
## Fingerprint
- ```ebnf
fingerprint = type ":" sha256 ":" inner-digest ":" store ":" name
fingerprint = type ":sha256:" inner-digest ":" store ":" name
```
Note that it includes the location of the store as well as the name to make sure that changes to either of those are reflected in the hash
@@ -53,7 +56,7 @@ where
method of content addressing store objects,
if the hash algorithm is [SHA-256].
Just like in the "Text" case, we can have the store objects referenced by their paths.
Additionally, we can have an optional `:self` label to denote selfreference.
Additionally, we can have an optional `:self` label to denote self-reference.
- ```ebnf
| "output:" id
@@ -70,7 +73,8 @@ where
`id` is the name of the output (usually, "out").
For content-addressed store objects, `id`, is always "out".
- `inner-digest` = base-16 representation of a SHA-256 hash of `inner-fingerprint`
- `inner-digest` = base-16 representation of a SHA-256 hash of `inner-fingerprint`.
The base-16 encoding uses lower-cased hex digits.
## Inner fingerprint
@@ -82,7 +86,7 @@ where
- if `type` = `"source:" ...`:
the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
- To operate on a flake outside the current directory, you must now pass `--flake path/to/flake`.
- The flake-specific flags `--recreate-lock-file` and `--update-input` have been removed from all commands operating on installables.
They are superceded by `nix flake update`.
They are superseded by `nix flake update`.
- Commit signature verification for the [`builtins.fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) is added as the new [`verified-fetches` experimental feature](@docroot@/development/experimental-features.md#xp-feature-verified-fetches).
- Modify `nix derivation {add,show}` JSON format [#9866](https://github.com/NixOS/nix/issues/9866) [#10722](https://github.com/NixOS/nix/pull/10722)
The JSON format for derivations has been slightly revised to better conform to our [JSON guidelines](@docroot@/development/cli-guideline.md#returning-future-proof-json).
In particular, the hash algorithm and content addressing method of content-addresed derivation outputs are now separated into two fields `hashAlgo` and `method`,
In particular, the hash algorithm and content addressing method of content-addressed derivation outputs are now separated into two fields `hashAlgo` and `method`,
rather than one field with an arcane `:`-separated format.
This JSON format is only used by the experimental `nix derivation` family of commands, at this time.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
# Contributors
## Contributors
This release was made possible by the following 58 contributors:
- Handle the case where a chroot store is used and some inputs are in the "host" `/nix/store` [#12512](https://github.com/NixOS/nix/pull/12512)
The evaluator now presents a "union" filesystem view of the `/nix/store` in the host and the chroot.
This change also removes some hacks that broke `builtins.{path,filterSource}` in chroot stores [#11503](https://github.com/NixOS/nix/issues/11503).
- `nix flake prefetch` now has a `--out-link` option [#12443](https://github.com/NixOS/nix/pull/12443)
- Set `FD_CLOEXEC` on sockets created by curl [#12439](https://github.com/NixOS/nix/pull/12439)
Curl created sockets without setting `FD_CLOEXEC`/`SOCK_CLOEXEC`. This could previously cause connections to remain open forever when using commands like `nix shell`. This change sets the `FD_CLOEXEC` flag using a `CURLOPT_SOCKOPTFUNCTION` callback.
This is an atypical release, and for almost all intents and purposes, it is just a continuation of 2.27; not a feature release.
We had originally set the goal of making 2.27 the Nixpkgs default for NixOS 25.05, but dependents that link to Nix need certain _interface breaking_ changes in the C++ headers. This is not something we should do in a patch release, so this is why we branched 2.28 right off 2.27 instead of `master`.
This completes the infrastructure overhaul for the [RFC 132](https://github.com/NixOS/rfcs/blob/master/rfcs/0132-meson-builds-nix.md) switchover to meson as our build system.
## Major changes
- Unstable C++ API reworked
[#12836](https://github.com/NixOS/nix/pull/12836)
[#12798](https://github.com/NixOS/nix/pull/12798)
[#12773](https://github.com/NixOS/nix/pull/12773)
Now the C++ interface confirms to common conventions much better than before:
- All headers are expected to be included with the initial `nix/`, e.g. as `#include "nix/....hh"` (what Nix's headers now do) or `#include <nix/....hh>` (what downstream projects may choose to do).
Likewise, the pkg-config files have `-I${includedir}` not `-I${includedir}/nix` or similar.
Including without the `nix/` like before sometimes worked because of how for `#include` C pre-process checks the directory containing the current file, not just the lookup path, but this was not reliable.
- All configuration headers are included explicitly by the (regular) headers that need them.
There is no more need to pass `-include` to force additional files to be included.
- The public, installed configuration headers no longer contain implementation-specific details that are not relevant to the API.
The vast majority of definitions that were previously in there are now moved to new headers that are not installed, but used during Nix's own compilation only.
The remaining macro definitions are renamed to have `NIX_` as a prefix.
- The name of the Nix component the header comes from
(e.g. `util`, `store`, `expr`, `flake`, etc.)
is now part of the path to the header, coming after `nix` and before the header name
(or rest of the header path, if it is already in a directory).
Here is a contrived diff showing a few of these changes at once:
```diff
@@ @@
-#include "derived-path.hh"
+#include "nix/store/derived-path.hh"
@@ @@
+// Would include for the variables used before. But when other headers
+// need these variables. those will include these config themselves.
+#include "nix/store/config.hh"
+#include "nix/expr/config.hh"
@@ @@
-#include "config.hh"
+// Additionally renamed to distinguish from components' config headers.
+#include "nix/util/configuration.hh"
@@ @@
-#if HAVE_ACL_SUPPORT
+#if NIX_SUPPORT_ACL
@@ @@
-#if HAVE_BOEHMGC
+#if NIX_USE_BOEHMGC
@@ @@
#endif
#endif
@@ @@
-const char *s = "hi from " SYSTEM;
+const char *s = "hi from " NIX_LOCAL_SYSTEM;
```
- C API `nix_flake_init_global` removed [#5638](https://github.com/NixOS/nix/issues/5638) [#12759](https://github.com/NixOS/nix/pull/12759)
In order to improve the modularity of the code base, we are removing a use of global state, and therefore the `nix_flake_init_global` function.
Instead, use `nix_flake_settings_add_to_eval_state_builder`.
Although this change is not as critical, we figured it would be good to do this API change at the same time, also.
Also note that we try to keep the C API compatible, but we decided to break this function because it was young and likely not in widespread use yet. This frees up time to make important progress on the rest of the C API.
## Contributors
This earlier-than-usual release was made possible by the following 16 contributors:
After the special backport-based release of Nix 2.28 (timed to coincide with Nixpkgs 25.05), the release process is back to normal with 2.29.
As such, we have slightly more weeks of work from `master` (since 2.28 was branched from 2.27) than usual.
This fact is counterbalanced by the fact that most of those changes are bug fixes rather than larger new features.
- Prettified JSON output on the terminal [#12555](https://github.com/NixOS/nix/issues/12555) [#12652](https://github.com/NixOS/nix/pull/12652)
This makes the output easier to read.
Scripts are mostly unaffected because for those, stdout will be a file or a pipe, not a terminal, and for those, the old single-line behavior applies.
`--json --pretty` can be passed to enable it even if the output is not a terminal.
If your script creates a pseudoterminal for Nix's stdout, you can pass `--no-pretty` to disable the new behavior.
- Repl: improve continuation prompt for incomplete expressions [#12846](https://github.com/NixOS/nix/pull/12846)
Improved REPL user experience by updating the continuation prompt from invisible blank spaces to a visible `" > "`, enhancing clarity when entering multi-line expressions.
- REPL `:load-flake` and `:reload` now work together [#8753](https://github.com/NixOS/nix/issues/8753) [#13180](https://github.com/NixOS/nix/pull/13180)
Previously, `:reload` only reloaded the files specified with `:load` (or on the command line).
Now, it also works with the flakes specified with `:load-flake` (or on the command line).
This makes it correctly reload everything that was previously loaded, regardless of what sort of thing (plain file or flake) each item is.
- Increase retry delays on HTTP 429 Too Many Requests [#13052](https://github.com/NixOS/nix/pull/13052)
When downloading Nix, the retry delay was previously set to 0.25 seconds. It has now been increased to 1 minute to better handle transient CI errors, particularly on GitHub.
- S3: opt-in the STSProfileCredentialsProvider [#12646](https://github.com/NixOS/nix/pull/12646)
Added support for STS-based authentication for S3-based binary caches, i.e. enabling seamless integration with `aws sso login`.
- Reduce connect timeout for http substituter [#12876](https://github.com/NixOS/nix/pull/12876)
Previously, the Nix setting `connect-timeout` had no limit. It is now set to `5s`, offering a more practical default for users self-hosting binary caches, which may occasionally become unavailable, such as during updates.
- C API: functions for locking and loading a flake [#10435](https://github.com/NixOS/nix/issues/10435) [#12877](https://github.com/NixOS/nix/pull/12877) [#13098](https://github.com/NixOS/nix/pull/13098)
This release adds functions to the C API for handling the loading of flakes. Previously, this had to be worked around by using `builtins.getFlake`.
C API consumers and language bindings now have access to basic locking functionality.
It does not expose the full locking API, so that the implementation can evolve more freely.
Locking is controlled with the functions, which cover the common use cases for consuming a flake:
-`nix_flake_lock_flags_set_mode_check`
-`nix_flake_lock_flags_set_mode_virtual`
-`nix_flake_lock_flags_set_mode_write_as_needed`
-`nix_flake_lock_flags_add_input_override`, which also enables `virtual`
This change also introduces the new `nix-fetchers-c` library, whose single purpose for now is to manage the (`nix.conf`) settings for the built-in fetchers.
More details can be found in the [C API documentation](@docroot@/c-api.md).
- No longer copy flakes that are in the nix store [#10435](https://github.com/NixOS/nix/issues/10435) [#12877](https://github.com/NixOS/nix/pull/12877) [#13098](https://github.com/NixOS/nix/pull/13098)
Previously, we would duplicate entries like `path:/nix/store/*` back into the Nix store.
This was prominently visible for pinned system flake registry entries in NixOS, e.g., when running `nix run nixpkgs#hello`.
- Consistently preserve error messages from cached evaluation [#12762](https://github.com/NixOS/nix/issues/12762) [#12809](https://github.com/NixOS/nix/pull/12809)
In one code path, we are not returning the errors cached from prior evaluation, but instead throwing generic errors stemming from the lack of value (due to the error).
These generic error messages were far less informative.
Now we consistently return the original error message.
The implementation for blake3 hashing is now multi-threaded and used memory-mapped IO.
Benchmark results can be found the [pull request](https://github.com/NixOS/nix/pull/12676).
- Fix progress bar for S3 binary caches and make file transfers interruptible [#12877](https://github.com/NixOS/nix/issues/12877) [#13098](https://github.com/NixOS/nix/issues/13098) [#12538](https://github.com/NixOS/nix/pull/12538)
The progress bar now correctly display upload/download progress for S3 up/downloads. S3 uploads are now interruptible.
- Add host attribute of github/gitlab flakerefs to URL serialization [#12580](https://github.com/NixOS/nix/pull/12580)
Resolved an issue where `github:` or `gitlab:` URLs lost their `host` attribute when written to a lockfile, resulting in invalid URLs.
- Multiple signatures support in store urls [#12976](https://github.com/NixOS/nix/pull/12976)
Added support for a `secretKeyFiles` URI parameter in Nix store URIs, allowing multiple signing key files to be specified as a comma-separated list.
This enables signing paths with multiple keys. This helps with [RFC #149](https://github.com/NixOS/rfcs/pull/149) to enable binary cache key rotation in the NixOS infra.
- nix flake show now skips over import-from-derivation [#4265](https://github.com/NixOS/nix/issues/4265) [#12583](https://github.com/NixOS/nix/pull/12583)
Previously, if a flake contained outputs relying on [import from derivation](@docroot@/language/import-from-derivation.md) during evaluation, `nix flake show` would fail to display the rest of the flake. The updated behavior skips such outputs, allowing the rest of the flake to be shown.
Note that unlike `nix build`, this prints the full path to the program, not just the store path (in the example above that would be `/nix/store/cb9w44vkhk2x4adfxwgdkkf5gjmm856j-treefmt`).
- Amend OSC 8 escape stripping for xterm-style separator [#13109](https://github.com/NixOS/nix/pull/13109)
Improve terminal escape code filtering to understand a second type of hyperlink escape codes.
This in particular prevents parts of GCC 14's diagnostics from being improperly filtered away.
## Contributors
This release was made possible by the following 40 contributors:
The directory in which temporary build directories are created no longer defaults
to `TMPDIR` or `/tmp`, to avoid builders making their directories
world-accessible. This behavior allowed escaping the build sandbox and can
cause build impurities even when not used maliciously. We now default to `builds`
in `NIX_STATE_DIR` (which is `/nix/var/nix/builds` in the default configuration).
- Deprecate manually making structured attrs using the `__json` attribute [#13220](https://github.com/NixOS/nix/pull/13220)
The proper way to create a derivation using [structured attrs] in the Nix language is by using `__structuredAttrs = true` with [`builtins.derivation`].
However, by exploiting how structured attrs are implementated, it has also been possible to create them by setting the `__json` environment variable to a serialized JSON string.
This sneaky alternative method is now deprecated, and may be disallowed in future versions of Nix.
- Rename `nix profile install` to [`nix profile add`] [#13224](https://github.com/NixOS/nix/pull/13224)
The command `nix profile install` has been renamed to [`nix profile add`] (though the former is still available as an alias). This is because the verb "add" is a better antonym for the verb "remove" (i.e. `nix profile remove`). Nix also does not have install hooks or general behavior often associated with "installing".
## Performance improvements
This release has a number performance improvements, in particular:
- Reduce the size of value from 24 to 16 bytes [#13407](https://github.com/NixOS/nix/pull/13407)
This shaves off a very significant amount of memory used for evaluation (~20% percent reduction in maximum heap size and ~17% in total bytes).
The Nix evaluator now supports [stack sampling evaluation profiling](@docroot@/advanced-topics/eval-profiler.md) via the [`--eval-profiler flamegraph`] setting.
It outputs collapsed call stack information to the file specified by
[`--eval-profile-file`] (`nix.profile` by default) in a format directly consumable
by `flamegraph.pl` and compatible tools like [speedscope](https://speedscope.app/).
Sampling frequency can be configured via [`--eval-profiler-frequency`] (99 Hz by default).
Unlike the existing [`--trace-function-calls`], this profiler includes the name of the function
being called when it's available.
- [`nix repl`] prints which variables were loaded [#11406](https://github.com/NixOS/nix/pull/11406)
Instead of `Added <n> variables` it now prints the first 10 variables that were added to the global scope.
This is useful when using [`nix flake archive`] with the destination set to a remote store.
- Emit warnings for IFDs with [`trace-import-from-derivation`] option [#13279](https://github.com/NixOS/nix/pull/13279)
While we have the setting [`allow-import-from-derivation`] to deny import-from-derivation (IFD), sometimes users would like to observe IFDs during CI processes to gradually phase out the idiom. The new setting `trace-import-from-derivation`, when set, logs a simple warning to the console.
New setting [`json-log-path`] that sends a copy of all Nix log messages (in JSON format) to a file or Unix domain socket.
- Non-flake inputs now contain a `sourceInfo` attribute [#13164](https://github.com/NixOS/nix/issues/13164) [#13170](https://github.com/NixOS/nix/pull/13170)
Flakes have always had a `sourceInfo` attribute which describes the source of the flake.
The `sourceInfo.outPath` is often identical to the flake's `outPath`. However, it can differ when the flake is located in a subdirectory of its source.
Non-flake inputs (i.e. inputs with [`flake = false`]) can also be located at some path _within_ a wider source.
This usually happens when defining a relative path input within the same source as the parent flake, e.g. `inputs.foo.url = ./some-file.nix`.
Such relative inputs will now inherit their parent's `sourceInfo`.
This also means it is now possible to use `?dir=subdir` on non-flake inputs.
This iterates on the work done in 2.26 to improve relative path support ([#10089](https://github.com/NixOS/nix/pull/10089)),
and resolves a regression introduced in 2.28 relating to nested relative path inputs ([#13164](https://github.com/NixOS/nix/issues/13164)).
Previously it used libstdc++'s `std::stable_sort()`. However, that implementation is not reliable if the user-supplied comparison function is not a strict weak ordering.
Since Nix 1.3 ([commit `299141e`] in 2013) Nix has attempted to mix together upstream fresh builds and downstream substitutions when remote substuters contain an "incomplete closure" (have some store objects, but not the store objects they reference).
This feature is now removed.
In the worst case, removing this feature could cause more building downstream, but it should not cause outright failures, since this is not happening for opaque store objects that we don't know how to build if we decide not to substitute.
In practice, however, we doubt even more building is very likely to happen.
Remote stores that are missing dependencies in arbitrary ways (e.g. corruption) don't seem to be very common.
On the contrary, when remote stores fail to implement the [closure property](@docroot@/store/store-object.md#closure-property), it is usually an *intentional* choice on the part of the remote store, because it wishes to serve as an "overlay" store over another store, such as `https://cache.nixos.org`.
If an "incomplete closure" is encountered in that situation, the right fix is not to do some sort of "franken-building" as this feature implemented, but instead to make sure both substituters are enabled in the settings.
(In the future, we should make it easier for remote stores to indicate this to clients, to catch settings that won't work in general before a missing dependency is actually encountered.)
## Contributors
This release was made possible by the following 32 contributors:
- Each input must be [realised] prior to building the derivation in question.
[realised]: @docroot@/glossary.md#gloss-realise
- Once this is done, the derivation is *normalized*, replacing each input deriving path with its store path, which we now know from realising the input.
## Builder Execution
The [`builder`](./derivation/index.md#builder) is executed as follows:
- A temporary directory is created where the build will take place. The
current directory is changed to this directory.
See the per-store [`build-dir`](@docroot@/store/types/local-store.md#store-local-store-build-dir) setting for more information.
- The environment is cleared and set to the derivation attributes, as
specified above.
- In addition, the following variables are set:
-`NIX_BUILD_TOP` contains the path of the temporary directory for
this build.
- Also, `TMPDIR`, `TEMPDIR`, `TMP`, `TEMP` are set to point to the
temporary directory. This is to prevent the builder from
accidentally writing temporary files anywhere else. Doing so
might cause interference by other processes.
-`PATH` is set to `/path-not-set` to prevent shells from
initialising it to their built-in default value.
-`HOME` is set to `/homeless-shelter` to prevent programs from
using `/etc/passwd` or the like to find the user's home
directory, which could cause impurity. Usually, when `HOME` is
set, it is used as the location of the home directory, even if
it points to a non-existent path.
-`NIX_STORE` is set to the path of the top-level Nix store
directory (typically, `/nix/store`).
-`NIX_ATTRS_JSON_FILE` & `NIX_ATTRS_SH_FILE` if `__structuredAttrs`
is set to `true` for the derivation. A detailed explanation of this
behavior can be found in the
[section about structured attrs](@docroot@/language/advanced-attributes.md#adv-attr-structuredAttrs).
- For each output declared in `outputs`, the corresponding
environment variable is set to point to the intended path in the
Nix store for that output. Each output path is a concatenation
of the cryptographic hash of all build inputs, the `name`
attribute and the output name. (The output name is omitted if
it’s `out`.)
- If an output path already exists, it is removed. Also, locks are
acquired to prevent multiple [Nix instances][Nix instance] from performing the same
build at the same time.
- A log of the combined standard output and error is written to
`/nix/var/log/nix`.
- The builder is executed with the arguments specified by the
attribute `args`. If it exits with exit code 0, it is considered to
have succeeded.
- The temporary directory is removed (unless the `-K` option was
specified).
## Processing outputs
If the builder exited successfully, the following steps happen in order to turn the output directories left behind by the builder into proper store objects:
- **Normalize the file permissions**
Nix sets the last-modified timestamp on all files
in the build result to 1 (00:00:01 1/1/1970 UTC), sets the group to
the default group, and sets the mode of the file to 0444 or 0555
(i.e., read-only, with execute permission enabled if the file was
originally executable). Any possible `setuid` and `setgid`
bits are cleared.
> **Note**
>
> Setuid and setgid programs are not currently supported by Nix.
> This is because the Nix archives used in deployment have no concept of ownership information,
> and because it makes the build result dependent on the user performing the build.
- **Calculate the references**
Nix scans each output path for
references to input paths by looking for the hash parts of the input
paths. Since these are potential runtime dependencies, Nix registers
them as dependencies of the output paths.
Nix also scans for references to other outputs' paths in the same way, because outputs are allowed to refer to each other.
If the outputs' references to each other form a cycle, this is an error, because the references of store objects much be acyclic.
Besides functioning as a [content-addressed store], the Nix store layer works as a [build system].
Other systems (like Git or IPFS) also store and transfer immutable data, but they don't concern themselves with *how* that data was created.
This is where Nix distinguishes itself.
*Derivations* represent individual build steps, and *deriving paths* are needed to refer to the *outputs* of those build steps before they are built.
<!-- The two concepts need to be introduced together because, as described below, each depends on the other. -->
## Store Derivation {#store-derivation}
A derivation is a specification for running an executable on precisely defined input to produce on more [store objects][store object].
These store objects are known as the derivation's *outputs*.
Derivations are *built*, in which case the process is spawned according to the spec, and when it exits, required to leave behind files which will (after post-processing) become the outputs of the derivation.
This process is described in detail in [Building](@docroot@/store/building.md).
<!--
Some of these things are described directly below, but we envision with more material the exposition will probably want to migrate to separate pages benough this.
See outputs spec for an example of this one that migrated to its own page.
-->
A derivation consists of:
- A name
- An [inputs specification][inputs], a set of [deriving paths][deriving path]
- An [outputs specification][outputs], specifying which outputs should be produced, and various metadata about them.
- The ["system" type][system] (e.g. `x86_64-linux`) where the executable is to run.
- The [process creation fields]: to spawn the arbitrary process which will perform the build step.
Derivations are always referred to by the [store path] of the store object they are encoded to.
See the [encoding section](#derivation-encoding) for more details on how this encoding works, and thus what exactly what store path we would end up with for a given derivation.
The store path of the store object which encodes a derivation is often called a *derivation path* for brevity.
## Deriving path {#deriving-path}
Deriving paths are a way to refer to [store objects][store object] that may or may not yet be [realised][realise].
There are two forms:
- [*constant*]{#deriving-path-constant}: just a [store path].
It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store.
- [*output*]{#deriving-path-output}: a pair of a [store path] to a [store derivation] and an [output] name.
In pseudo code:
```typescript
typeOutputName=String;
typeConstantPath={
path: StorePath;
};
typeOutputPath={
drvPath: StorePath;
output: OutputName;
};
typeDerivingPath=ConstantPath|OutputPath;
```
Deriving paths are necessary because, in general and particularly for [content-addressing derivations][content-addressing derivation], the [store path] of an [output] is not known in advance.
We can use an output deriving path to refer to such an output, instead of the store path which we do not yet know.
[deriving path]: #deriving-path
[validity]: @docroot@/glossary.md#gloss-validity
## Parts of a derivation
A derivation is constructed from the parts documented in the following subsections.
### Inputs {#inputs}
The inputs are a set of [deriving paths][deriving path], referring to all store objects needed in order to perform this build step.
The [process creation fields] will presumably include many [store paths][store path]:
- The path to the executable normally starts with a store path
- The arguments and environment variables likely contain many other store paths.
But rather than somehow scanning all the other fields for inputs, Nix requires that all inputs be explicitly collected in the inputs field. It is instead the responsibility of the creator of a derivation (e.g. the evaluator) to ensure that every store object referenced in another field (e.g. referenced by store path) is included in this inputs field.
### System {#system}
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
A necessary condition for Nix to schedule a given derivation on some [Nix instance] is for the "system" of that derivation to match that instance's [`system` configuration option] or [`extra-platforms` configuration option].
By putting the `system` in each derivation, Nix allows *heterogenous* build plans, where not all steps can be run on the same machine or same sort of machine.
Nix can schedule builds such that it automatically builds on other platforms by [forwarding build requests](@docroot@/advanced-topics/distributed-builds.md) to other Nix instances.
### Process creation fields {#process-creation-fields}
These are the three fields which describe how to spawn the process which (along with any of its own child processes) will perform the build.
You may note that this has everything needed for an `execve` system call.
#### Builder {#builder}
This is the path to an executable that will perform the build and produce the [outputs].
#### Arguments {#args}
Command-line arguments to be passed to the [`builder`](#builder) executable.
Note that these are the arguments after the first argument.
The first argument passed to the `builder` will be the value of `builder`, as per the usual convention on Unix.
See [Wikipedia](https://en.wikipedia.org/wiki/Argv) for details.
#### Environment Variables {#env}
Environment variables which will be passed to the [builder](#builder) executable.
#### Structured Attributes {#structured-attrs}
Nix also has special support for embedding JSON in the derivations.
The environment variable `NIX_ATTRS_JSON_FILE` points to the exact location of that file both in a build and a [`nix-shell`](@docroot@/command-ref/nix-shell.md).
As a convenience to Bash builders, Nix writes a script that initialises shell variables corresponding to all attributes that are representable in Bash.
The environment variable `NIX_ATTRS_SH_FILE` points to the exact location of the script, both in a build and a [`nix-shell`](@docroot@/command-ref/nix-shell.md).
This includes non-nested (associative) arrays.
For example, the attribute `hardening.format = true` ends up as the Bash associative array element `${hardening[format]}`.
### Placeholders
Placeholders are opaque values used within the [process creation fields] to [store objects] for which we don't yet know [store path]s.
They are strings in the form `/<hash>` that are embedded anywhere within the strings of those fields, and we are [considering](https://github.com/NixOS/nix/issues/12361) to add store-path-like placeholders.
> **Note**
>
> Output Deriving Path exist to solve the same problem as placeholders --- that is, referring to store objects for which we don't yet know a store path.
> They also have a string syntax with `^`, [described in the encoding section](#deriving-path-encoding).
> We could use that syntax instead of `/<hash>` for placeholders, but its human-legibility would cause problems.
There are two types of placeholder, corresponding to the two cases where this problem arises:
- [Output placeholder]{#output-placeholder}:
This is a placeholder for a derivation's own output.
- [Input placeholder]{#input-placeholder}:
This is a placeholder to a derivation's non-constant [input],
i.e. an input that is an [output derived path].
> **Explanation**
>
> In general, we need to [realise] a [store object] in order to be sure to have a store object for it.
> But for these two cases this is either impossible or impractical:
>
> - In the output case this is impossible:
>
> We cannot build the output until we have a correct derivation, and we cannot have a correct derivation (without using placeholders) until we have the output path.
>
> - In the input case this is impractical:
>
> If we always build a dependency first, and then refer to its output by store path, we would lose the ability for a derivation graph to describe an entire build plan consisting of multiple build steps.
## Encoding
### Derivation {#derivation-encoding}
There are two formats, documented separately:
- The legacy ["ATerm" format](@docroot@/protocols/derivation-aterm.md)
- The experimental, currently under development and changing [JSON format](@docroot@/protocols/json/derivation.md)
Every derivation has a canonical choice of encoding used to serialize it to a store object.
This ensures that there is a canonical [store path] used to refer to the derivation, as described in [Referencing derivations](#derivation-path).
> **Note**
>
> Currently, the canonical encoding for every derivation is the "ATerm" format,
> but this is subject to change for the types of derivations which are not yet stable.
Regardless of the format used, when serializing a derivation to a store object, that store object will be content-addressed.
In the common case, the inputs to store objects are either:
- [constant deriving paths](#deriving-path-constant) for content-addressed source objects, which are "initial inputs" rather than the outputs of some other derivation
- the outputs of other derivations
If those other derivations *also* abide by this common case (and likewise for transitive inputs), then the entire closure of the serialized derivation will be content-addressed.
### Deriving Path {#deriving-path-encoding}
- *constant*
Constant deriving paths are encoded simply as the underlying store path is.
Thus, we see that every encoded store path is also a valid encoded (constant) deriving path.
- *output*
Output deriving paths are encoded by
- encoding of a store path referring to a derivation
- a `^` separator (or `!` in some legacy contexts)
- the name of an output of the previously referred derivation
So far, we have used store paths to refer to derivations.
That works because we've implicitly assumed that all derivations are created *statically* --- created by some mechanism out of band, and then manually inserted into the store.
But what if derivations could also be created dynamically within Nix?
In other words, what if derivations could be the outputs of other derivations?
> **Note**
>
> In the parlance of "Build Systems à la carte", we are generalizing the Nix store layer to be a "Monadic" instead of "Applicative" build system.
How should we refer to such derivations?
A deriving path works, the same as how we refer to other derivation outputs.
But what about a dynamic derivations output?
(i.e. how do we refer to the output of a derivation, which is itself an output of a derivation?)
For that we need to generalize the definition of deriving path, replacing the store path used to refer to the derivation with a nested deriving path:
```diff
type OutputPath = {
- drvPath: StorePath;
+ drvPath: DerivingPath;
output: OutputName;
};
```
Now, the `drvPath` field of `OutputPath` is itself a `DerivingPath` instead of a `StorePath`.
With that change, here is updated definition:
```typescript
typeOutputName=String;
typeConstantPath={
path: StorePath;
};
typeOutputPath={
drvPath: DerivingPath;
output: OutputName;
};
typeDerivingPath=ConstantPath|OutputPath;
```
Under this extended model, `DerivingPath`s are thus inductively built up from a root `ConstantPath`, wrapped with zero or more outer `OutputPath`s.
The content-addressing of an output only depends on that store object itself, not any other information external (such has how it was made, when it was made, etc.).
As a consequence, a store object will be content-addressed the same way regardless of whether it was manually inserted into the store, outputted by some derivation, or outputted by a some other derivation.
The output spec for a content-addressed output must contains the following field:
- *method*: how the data of the store object is digested into a content address
The possible choices of *method* are described in the [section on content-addressing store objects](@docroot@/store/store-object/content-address.md).
Given the method, the output's name (computed from the derivation name and output spec mapping as described above), and the data of the store object, the output's store path will be computed as described in that section.
## Fixed-output content-addressing {#fixed}
In this case the content address of the *fixed* in advanced by the derivation itself.
In other words, when the derivation has finished [building](@docroot@/store/building.md), and the provisional output' content-address is computed as part of the process to turn it into a *bona fide* store object, the calculated content address must much that given in the derivation, or the build of that derivation will be deemed a failure.
The output spec for an output with a fixed content addresses additionally contains:
- *hash*, the hash expected from digesting the store object's file system objects.
This hash may be of a freely-chosen hash algorithm (that Nix supports)
> **Design note**
>
> In principle, the output spec could also specify the references the store object should have, since the references and file system objects are equally parts of a content-addressed store object proper that contribute to its content-addressed.
> However, at this time, the references are not done because all fixed content-addressed outputs are required to have no references (including no self-reference).
>
> Also in principle, rather than specifying the references and file system object data with separate hashes, a single hash that constraints both could be used.
> This could be done with the final store path's digest, or better yet, the hash that will become the store path's digest before it is truncated.
>
> These possible future extensions are included to elucidate the core property of fixed-output content addressing --- that all parts of the output must be cryptographically fixed with one or more hashes --- separate from the particulars of the currently-supported store object content-addressing schemes.
### Design rationale
What is the purpose of fixing an output's content address in advanced?
In abstract terms, the answer is carefully controlled impurity.
Unlike a regular derivation, the [builder] executable of a derivation that produced fixed outputs has access to the network.
The outputs' guaranteed content-addresses are supposed to mitigate the risk of the builder being given these capabilities;
regardless of what the builder does *during* the build, it cannot influence downstream builds in unanticipated ways because all information it passed downstream flows through the outputs whose content-addresses are fixed.
If a `fetchurl` derivation's outputs were [input-addressed][input addressing], the output paths of the derivation and of *all derivations depending on it* would change.
For instance, if we were to change the URL of the Glibc source distribution in Nixpkgs (a package on which almost all other packages depend on Linux) massive rebuilds would be needed.
This is unfortunate for a change which we know cannot have a real effect as it propagates upwards through the dependency graph.
For content-addressed outputs (fixed or floating), on the other hand, the outputs' store path only depends on the derivation's name, data, and the `method` of the outputs' specs.
The rest of the derivation is ignored for the purpose of computing the output path.
> **History Note**
>
> Fixed content-addressing is especially important both today and historically as the *only* form of content-addressing that is stabilized.
> This is why the rationale above contrasts it with [input addressing].
## (Floating) Content-Addressing {#floating}
> **Warning**
> This is part of an [experimental feature](@docroot@/development/experimental-features.md).
>
> To use this type of output addressing, you must enable the
> For example, in [nix.conf](@docroot@/command-ref/conf-file.md) you could add:
>
> ```
> extra-experimental-features = ca-derivations
> ```
With this experimemental feature enabled, derivation outputs can also be content-addressed *without* fixing in the output spec what the outputs' content address must be.
### Purity
Because the derivation output is not fixed (just like with [input addressing]), the [builder] is not given any impure capabilities [^purity].
> **Configuration note**
>
> Strictly speaking, the extent to which sandboxing and deprivilaging is possible varies with the environment Nix is running in.
> Nix's configuration settings indicate what level of sandboxing is required or enabled.
> Builds of derivations will fail if they request an absence of sandboxing which is not allowed.
> Builds of derivations will also fail if the level of sandboxing specified in the configure exceeds what is possible in the given environment.
>
> (The "environment", in this case, consists of attributes such as the Operating System Nix runs atop, along with the operating-system-specific privileges that Nix has been granted.
> Because of how conventional operating systems like macos, Linux, etc. work, granting builders *fewer* privileges may ironically require that Nix be run with *more* privileges.)
That said, derivations producing floating content-addressed outputs may declare their builders as impure (like the builders of derivations producing fixed outputs).
This is provisionally supported as part of the [`impure-derivations`][xp-feature-impure-derivations] experimental feature.
### Compatibility negotiation
Any derivation producing a floating content-addressed output implicitly requires the `ca-derivations` [system feature](@docroot@/command-ref/conf-file.md#conf-system-features).
This prevents scheduling the building of the derivation on a machine without the experimental feature enabled.
Even once the experimental feature is stabilized, this is still useful in order to be allow using remote builder running odler versions of Nix, or alternative implementations that do not support floating content addressing.
### Determinism
In the earlier [discussion of how self-references are handled when content-addressing store objects](@docroot@/store/store-object/content-address.html#self-references), it was pointed out that methods of producing store objects ought to be deterministic regardless of the choice of provisional store path.
For store objects produced by manually inserting into the store to create a store object, the "method of production" is an informally concept --- formally, Nix has no idea where the store object came from, and content-addressing is crucial in order to ensure that the derivation is *intrinsically* tamper-proof.
But for store objects produced by derivation, the "method is quite formal" --- the whole point of derivations is to be a formal notion of building, after all.
In this case, we can elevate this informal property to a formal one.
A *deterministic* content-addressing derivation should produce outputs with the same content addresses:
1. Every time the builder is run
This is because either the builder is completely sandboxed, or because all any remaining impurities that leak inside the build sandbox are ignored by the builder and do not influence its behavior.
2. Regardless of the choice of any provisional outputs paths
Provisional store paths must be chosen for any output that has a self-reference.
The choice of provisional store path can be thought of as an impurity, since it is an arbitrary choice.
If provisional outputs paths are deterministically chosen, we are in the first branch of part (1).
The builder the data it produces based on it in arbitrary ways, but this gets us closer to [input addressing].
Deterministically choosing the provisional path may be considered "complete sandboxing" by removing an impurity, but this is unsatisfactory
<!--
TODO
(Both these points will be expanded-upon below.)
-->
If provisional outputs paths are randomly chosen, we are in the second branch of part (1).
The builder *must* not let the random input affect the final outputs it produces, and multiple builds may be performed and the compared in order to ensure that this is in fact the case.
### Floating versus Fixed
While the distinction between content- and input-addressing is one of *mechanism*, the distinction between fixed and floating content addressing is more one of *policy*.
A fixed output that passes its content address check is just like a floating output.
It is only in the potential for that check to fail that they are different.
> **Design Note**
>
> In a future world where floating content-addressing is also stable, we in principle no longer need separate [fixed](#fixed) content-addressing.
> Instead, we could always use floating content-addressing, and separately assert the precise value content address of a given store object to be used as an input (of another derivation).
> A stand-alone assertion object of this sort is not yet implemented, but its possible creation is tracked in [Issue #11955](https://github.com/NixOS/nix/issues/11955).
>
> In the current version of Nix, fixed outputs which fail their hash check are still registered as valid store objects, just not registered as outputs of the derivation which produced them.
> This is an optimization that means if the wrong output hash is specified in a derivation, and then the derivation is recreated with the right output hash, derivation does not need to be rebuilt --- avoiding downloading potentially large amounts of data twice.
> This optimisation prefigures the design above:
> If the output hash assertion was removed outside the derivation itself, Nix could additionally not only register that outputted store object like today, but could also make note that derivation did in fact successfully download some data.
For example, for the "fetch URL" example above, making such a note is tantamount to recording what data is available at the time of download at the given URL.
> It would only be when Nix subsequently tries to build something with that (refining our example) downloaded source code that Nix would be forced to check the output hash assertion, preventing it from e.g. building compromised malware.
>
> Recapping, Nix would
>
> 1. successfully download data
> 2. insert that data into the store
> 3. associate (presumably with some sort of expiration policy) the downloaded data with the derivation that downloaded it
>
> But only use the downloaded store object in subsequent derivations that depended upon the assertion if the assertion passed.
>
> This possible future extension is included to illustrate this distinction:
As stated on the [main pages on derivations](../index.md#store-derivation),
a derivation produces [store objects](@docroot@/store/store-object.md), which are known as the *outputs* of the derivation.
Indeed, the entire point of derivations is to produce these outputs, and to reliably and reproducibly produce these derivations each time the derivation is run.
One of the parts of a derivation is its *outputs specification*, which specifies certain information about the outputs the derivation produces when run.
The outputs specification is a map, from names to specifications for individual outputs.
## Output Names {#outputs}
Output names can be any string which is also a valid [store path](@docroot@/store/store-path.md) name.
The name mapped to each output specification is not actually the name of the output.
In the general case, the output store object has name `derivationName + "-" + outputSpecName`, not any other metadata about it.
However, an output spec named "out" describes and output store object whose name is just the derivation name.
> **Example**
>
> A derivation is named `hello`, and has two outputs, `out`, and `dev`
>
> - The derivation's path will be: `/nix/store/<hash>-hello.drv`.
>
> - The store path of `out` will be: `/nix/store/<hash>-hello`.
>
> - The store path of `dev` will be: `/nix/store/<hash>-hello-dev`.
The outputs are the derivations are the [store objects](@docroot@/store/store-object.md) it is obligated to produce.
> **Note**
>
> The formal terminology here is somewhat at odds with everyday communication in the Nix community today.
> "output" in casual usage tends to refer to either to the actual output store object, or the notional output spec, depending on context.
>
> For example "hello's `dev` output" means the store object referred to by the store path `/nix/store/<hash>-hello-dev`.
> It is unusual to call this the "`hello-dev` output", even though `hello-dev` is the actual name of that store object.
## Types of output addressing
The main information contained in an output specification is how the derivation output is addressed.
In particular, the specification decides:
- whether the output is [content-addressed](./content-address.md) or [input-addressed](./input-address.md)
- if the content is content-addressed, how is it content addressed
- if the content is content-addressed, [what is its content address](./content-address.md#fixed-content-addressing) (and thus what is its [store path])
## Types of derivations
The sections on each type of derivation output addressing ended up discussing other attributes of the derivation besides its outputs, such as purity, scheduling, determinism, etc.
This is no concidence; for the type of a derivation is in fact one-for-one with the type of its outputs:
- A derivation that produces *xyz-addressed* outputs is an *xyz-addressing* derivations.
The rules for this are fairly concise:
- All the outputs must be of the same type / use the same addressing
- The derivation must have at least one output
- Additionally, if the outputs are fixed content-addressed, there must be exactly one output, whose specification is mapped from the name `out`.
(The name `out` is special, according to the rules described above.
Having only one output and calling its specification `out` means the single output is effectively anonymous; the store path just has the derivation name.)
(This is an arbitrary restriction that could be lifted.)
- The output is either *fixed* or *floating*, indicating whether the store path is known prior to building it.
- With fixed content-addressing it is fixed.
> A *fixed content-addressing* derivation is also called a *fixed-output derivation*, since that is the only currently-implemented form of fixed-output addressing
- With floating content-addressing or input-addressing it is floating.
> Thus, historically with Nix, with no experimental features enabled, *all* outputs are fixed.
- The derivation may be *pure* or *impure*, indicating what read access to the outside world the [builder](../index.md#builder) has.
- An input-addressing derivation *must* be pure.
> If it is impure, we would have a large problem, because an input-addressed derivation always produces outputs with the same paths.
- A content-addressing derivation may be pure or impure
- If it is impure, it may be fixed (typical), or it may be floating if the additional [`impure-derivations`][xp-feature-impure-derivations] experimental feature is enabled.
- If it is pure, it must be floating.
- Pure, fixed content-addressing derivations are not supported
> There is no use for this forth combination.
> The sole purpose of an output's store path being fixed is to support the derivation being impure.
"Input addressing" means the address the store object by the *way it was made* rather than *what it is*.
That is to say, an input-addressed output's store path is a function not of the output itself, but of the derivation that produced it.
Even if two store paths have the same contents, if they are produced in different ways, and one is input-addressed, then they will have different store paths, and thus guaranteed to not be the same store object.
<!---
### Modulo fixed-output derivations
**TODO hash derivation modulo.**
So how do we compute the hash part of the output path of a derivation?
This is done by the function `hashDrv`, shown in Figure 5.10.
It distinguishes between two cases.
If the derivation is a fixed-output derivation, then it computes a hash over just the `outputHash` attributes.
If the derivation is not a fixed-output derivation, we replace each element in the derivation’s inputDrvs with the result of a call to `hashDrv` for that element.
(The derivation at each store path in `inputDrvs` is converted from its on-disk ATerm representation back to a `StoreDrv` by the function `parseDrv`.) In essence, `hashDrv` partitions store derivations into equivalence classes, and for hashing purpose it replaces each store path in a derivation graph with its equivalence class.
The recursion in Figure 5.10 is inefficient:
it will call itself once for each path by which a subderivation can be reached, i.e., `O(V k)` times for a derivation graph with `V` derivations and with out-degree of at most `k`.
In the actual implementation, memoisation is used to reduce this to `O(V + E)` complexity for a graph with E edges.
@@ -46,7 +46,7 @@ be many different serialisations.
For these reasons, Nix has its very own archive format—the Nix Archive (NAR) format,
which is carefully designed to avoid the problems described above.
The exact specification of the Nix Archive format is in `protocols/nix-archive.md`
The exact specification of the Nix Archive format is in [specified here](../../protocols/nix-archive.md).
## Content addressing File System Objects beyond a single serialisation pass
@@ -80,6 +80,7 @@ Thus, Git can encode some, but not all of Nix's "File System Objects", and this
In the future, we may support a Git-like hash for such file system objects, or we may adopt another Merkle DAG format which is capable of representing all Nix file system objects.
@@ -4,7 +4,64 @@ A Nix store is a collection of *store objects* with *references* between them.
A store object consists of
- A [file system object](./file-system-object.md) as data
- A set of [store paths](./store-path.md) as references to other store objects
- A set of [store paths](./store-path.md) as references to store objects
### References
Store objects can refer to both other store objects and themselves.
References from a store object to itself are called *self-references*.
Store objects and their references form a directed graph, where the store objects are the vertices, and the references are the edges.
In particular, the edge corresponding to a reference is from the store object that contains the reference, and to the store object that the store path (which is the reference) refers to.
References other than a self-reference must not form a cycle.
The graph of references excluding self-references thus forms a [directed acyclic graph].
We can take the [transitive closure] of the references graph, which any pair of store objects have an edge not if there is a single reference from the first to the second, but a path of one or more references from the first to the second.
The *requisites* of a store object are all store objects reachable by paths of references which start with given store object's references.
One can also combine both concepts: taking the transitive closure of the transposed references graph.
The *referrers closure* of a store object are the store objects that can reach the given store object via paths of references.
> **Note**
>
> Care must be taken to distinguish between the intrinsic and extrinsic properties of store objects.
> We can create graphs from the store objects in a store, but the contents of the store is not, in general fixed, and may instead change over time.
>
> - The references of a store object --- the set of store paths called the references --- is a field of a store object, and thus intrinsic by definition.
Regardless of what store contains the store object in question, and what else that store may or may not contain, the references are the same.
>
> - The requisites of a store object are almost intrinsic --- some store paths due not precisely refer to a unique single store object.
> Exactly what store object is being referenced, and what in turn *its* references are, depends on the store in question.
> Different stores that disagree.
>
> - The referrers of a store object are completely extrinsic, and depends solely on the store which contains that store object, not the store object itself.
> Other store objects which refer to the store object in question may be added or removed from the store.
### Immutability
Store objects are [immutable](https://en.wikipedia.org/wiki/Immutable_object):
Once created, they do not change until they are deleted.
Once created, they do not change nor can any store object they reference be changed.
> **Note**
>
> Stores which support atomically deleting multiple store objects allow more flexibility while still upholding this property.
### Closure property
A store can only contain a store object if it also contains all the store objects it refers to.
> **Note**
>
> The "closure property" isn't meant to prohibit, for example, [lazy loading](https://en.wikipedia.org/wiki/Lazy_loading) of store objects.
> However, the "closure property" and immutability in conjunction imply that any such lazy loading ought to be deterministic.
@@ -24,13 +24,17 @@ For the full specification of the algorithms involved, see the [specification of
### File System Objects
With all currentlysupported store object contentaddressing methods, the file system object is always [content-addressed][fso-ca] first, and then that hash is incorporated into content address computation for the store object.
With all currently-supported store object content-addressing methods, the file system object is always [content-addressed][fso-ca] first, and then that hash is incorporated into content address computation for the store object.
### References
#### References to other store objects
With all currently supported store object content addressing methods,
other objects are referred to by their regular (string-encoded-) [store paths][Store Path].
#### Self-references
Self-references however cannot be referred to by their path, because we are in the midst of describing how to compute that path!
> The alternative would require finding as hash function fixed point, i.e. the solution to an equation in the form
@@ -40,7 +44,28 @@ Self-references however cannot be referred to by their path, because we are in t
> which is computationally infeasible.
> As far as we know, this is equivalent to finding a hash collision.
Instead we just have a "has selfreference" boolean, which will end up affecting the digest.
Instead we have a "has self-reference" boolean, which ends up affecting the digest:
In all currently-supported store object content-addressing methods, when hashing the file system object data, any occurrence of store object's own store path in the digested data is replaced with a [sentinel value](https://en.wikipedia.org/wiki/Sentinel_value).
The hashes of these modified input streams are used instead.
When validating the content address of a store object after the fact, the above process works as written.
However, when first creating the store object we don't know the store object's store path, as explained just above.
We therefore, strictly speaking, do not know what value we will be replacing with the sentinel value in the inputs to hash functions.
What instead happens is that the provisional store object --- the data from which we wish to create a store object --- is paired with a provisional "scratch" store path (that presumably was chosen when the data was created).
That provisional store path is instead what is replaced with the sentinel value, rather than the final store object which we do not yet know.
> **Design note**
>
> It is an informal property of content-addressed store objects that the choice of provisional store path should not matter.
> In other words, if a provisional store object is prepared in the same way except for the choice of provision store path, the provisional data need not be identical.
> But, after the sentinel value is substituted in place of each provisional store object's provision store path, the final so-normalized data *should* be identical.
>
> If, conversely, the data after this normalization process is still different, we'll compute a different content-address.
> The method of preparing the provisional self-referenced data has *failed* to be deterministic in the sense of not *leaking* the choice of provisional store path --- a choice which is supposed to be arbitrary --- into the final store object.
>
> This property is informal because at this stage, we are just described store objects, which have no formal notion of their origin.
> Without such a formal notion, there is nothing to formally accuse of being insufficiently deterministic.
> Where we cover [derivations](@docroot@/store/derivation/index.md), we will have a chance to make this a formal property, not of content-addressed store objects themselves, but of derivations that *produce* content-addressed store objects.
### Name and Store Directory
@@ -63,7 +88,7 @@ References are not supported: store objects with flat hashing *and* references c
This also uses the corresponding [Flat](../file-system-object/content-address.md#serial-flat) method of file system object content addressing.
References to other store objects are supported, but selfreferences are not.
References to other store objects are supported, but self-references are not.
This is the only store-object content-addressing method that is not named identically with a corresponding file system object method.
It is somewhat obscure, mainly used for "drv files"
@@ -74,7 +99,7 @@ Prefer another method if possible.
This uses the corresponding [Nix Archive](../file-system-object/content-address.md#serial-nix-archive) method of file system object content addressing.
References (to other store objects and selfreferences alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise.
References (to other store objects and self-references alike) are supported so long as the hash algorithm is SHA-256, but not (neither kind) otherwise.
- Discussion meeting: Wednesday 21:00-22:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
1. Triage issues and pull requests from the [No Status](#no-status) column (30 min)
2. Discuss issues and pull requests from the [To discuss](#to-discuss) column (30 min).
@@ -46,7 +46,7 @@ The team meets twice a week (times are denoted in the [Europe/Amsterdam](https:/
- mark it as draft if it is blocked on the contributor
- escalate it back to the team by moving it to To discuss, and leaving a comment as to why the issue needs to be discussed again.
- Work meeting: [Mondays 14:00-16:00 Europe/Amsterdam](https://www.google.com/calendar/event?eid=Ym52NDdzYnRic2NzcDcybjZiNDhpNzhpa3NfMjAyNDA1MTNUMTIwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
- Work meeting: Mondays 14:00-16:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
1. Code review on pull requests from [In review](#in-review).
Release tags are signed by members of the [Nix maintainer team](https://nixos.org/community/teams/nix/) as part of the [release process](../release-process.md). This directory contains the public GPG keys used for signing.
TODO: This script requires the right AWS credentials. Document.
TODO: This script currently requires a
`/home/eelco/Dev/nix-pristine`.
* Trigger the [`upload-release.yml` workflow](https://github.com/NixOS/nix/actions/workflows/upload-release.yml) via `workflow_dispatch` trigger. At the top click `Run workflow` -> select the current release branch from `Use workflow from` -> fill in `Hydra evaluation ID` with `<EVAL-ID>` value from previous steps -> click `Run workflow`. Wait for the run to be approved by `NixOS/nix-team` (or bypass checks if warranted). Wait for the workflow to succeed.
Omit `IS_LATEST=1` when creating a point release that is not on the
most recent stable branch. This prevents `nixos.org` to going back
to an older release.
* Trigger the [`upload-release.yml` workflow](https://github.com/NixOS/nix/actions/workflows/upload-release.yml) via `workflow_dispatch` trigger. At the top click `Run workflow` -> select the current release branch from `Use workflow from` -> fill in `Hydra evaluation ID` with `<EVAL-ID>` value from previous steps -> click `Run workflow`. Wait for the run to be approved by `NixOS/nix-team` (or bypass checks if warranted). Wait for the workflow to succeed.
* Bump the version number of the release branch as above (e.g. to
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
