Fix intermittent SIGSEGV (exit code 139) on macOS when running
nix-shell and shebang tests inside the nix sandbox.
The foo, bar, and ruby test scripts were created without shebangs,
which causes intermittent crashes when executed via command
substitution on macOS. Adding proper shebangs resolves the flakiness.
Potentially closes: #13106
(cherry picked from commit 7b3d7eb634)
Fixes#14628
- Remove mdbook-linkcheck dependency and configuration (was blocking
upgrades to mdbook 0.5.0+, configured with warning-policy = "ignore"
due to false positives, and redundant with lychee-based link checking)
- Update substitute.py and anchors.jq to handle 'items' (mdbook 0.5.x)
in addition to 'sections' (mdbook 0.4.x), as per mdbook 0.5.0
changelog: "Book::sections was renamed to Book::items"
https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#05-migration-guide
(cherry picked from commit d950f884c2)
This gets us back to pre-2.23 behavior of this primop.
Done by inlining the code of `nix::dirOf` from 2.2-maintenance.
(cherry picked from commit a33fccf55a)
These will change in the next commit to fix the silent regression from 2.23
in the handling of multiple subsequent path separators.
(cherry picked from commit 86f090837b)
On macOS, poll() is fundamentally broken for HUP detection. It loses event
subscriptions when EVFILT_READ fires without matching the requested events
in the pollfd. This causes daemon processes to linger after client disconnect.
This commit replaces poll() with kqueue on macOS, which is what poll()
uses internally but without the bugs. The kqueue implementation uses
EVFILT_READ which works for both sockets and pipes, avoiding EVFILT_SOCK
which only works for sockets.
On Linux and other platforms, we continue using poll() with the standard
POSIX behavior where POLLHUP is always reported regardless of requested events.
Based on work from the Lix project (https://git.lix.systems/lix-project/lix)
commit 69ba3c92db3ecca468bcd5ff7849fa8e8e0fc6c0
Fixes: https://github.com/NixOS/nix/issues/13847
Related: https://git.lix.systems/lix-project/lix/issues/729
Apple bugs: rdar://37537852 (poll), FB17447257 (poll)
Co-authored-by: Jade Lovelace <jadel@mercury.com>
(cherry picked from commit 1286d5db78)
This would print erroneous and misleading diagnostics like:
> error (ignored): error: '--arg' and '--argstr' are incompatible with flakes
When run with --expr/--file. Since this installable is used to get the
bash package it doesn't make sense to check this.
(cherry picked from commit b6f98b52a4)
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
→ 'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26)
This update contains d1266642a8722f2a05e311fa151c1413d2b9653c, which
is necessary for the TOML timestamps to get tested via nixpkgsLibTests job.
(cherry picked from commit 625477a7df)
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].
1. Added more regression tests for timestamp formats.
Special attention needs to be paid to the precision
of the subsecond range for local-time. Prior versions select the closest
(upwards) multiple of 3 with a hard cap of 9 digits.
2. Normalize local datetime and offset datetime to always
use the uppercase separator `T`. This is actually the issue
surfaced in [2]. This canonicalization is basically a requirement
by (a certain reading) of rfc3339 section 5.6 [3].
3. If using toml11 >= 4.0 also keep the old behavior wrt
to the number of digits used for subsecond part of the local-time.
Newer versions cap it at 6 digits unconditionally.
[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339
(cherry picked from commit dc769d72cb)
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.
(cherry picked from commit 7ed0229d1a)
Nixpkgs bump in the flake bumps clang-format, so we now have a diff
compared to what was used to format the codebase previously. We should
have made sure that the nixpkgs input is the same in the first place.
About time we upgraded our nixpkgs flake input. Ideally
we'd have automation to do this.
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/adaa24fbf46737f3f1b5497bf64bae750f82942e?narHash=sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY%3D' (2025-05-13)
→ 'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
(cherry picked from commit fc33681583)
Running parallel nix in nix can lead to multiple instances trying to
create the state directories and failing on the `createSymlink` step,
because the link already exists.
`replaceSymlink` is already idempotent, so let's use that.
Resolves#2706
(cherry picked from commit d64c922164)
This overload isn't actually necessary anywhere and
doesn't make much sense. The pointers to `Value`s are
themselves const, but the `Value`s are mutable.
A non-const member function implies that the object itself
can be modified but this doesn't make much sense considering
the return type: `Value * const * `, which is a pointer
to a constant array of pointers to mutable values.
(cherry picked from commit 7b46eb9958)
These tests have been collected from nixpkgs f870c6ccc8951fc48aeb293cf3e98ade6ac42668
usage of builtins.match for x86_64-linux eval system. At most 2 matching and
non-matching cases are included for each encountered regex. This should
hopefully add more confidence when possibly trying to switch the regex implementation
in the future.
(cherry picked from commit d555d6b404)
Allows to copy the archive to a remote host and not get
error: cannot add path '/nix/store/01x2k4nlxcpyd85nnr0b9gm89rm8ff4x-source' because it lacks a signature by a trusted key
(cherry picked from commit 80a4293486)
...and also NIX_STATE_HOME in nix-profile.fish. This is directly
translated from the bash scripts and makes the fish scripts equivalent
in functionality to the bash scripts.
Note that nix-profile.fish checks for NIX_STATE_HOME and
nix-profile-daemon.fish does not, so the two scripts are no longer
identical.
(cherry picked from commit 751f50f4ad)
the second equivalence, using a if-else expression, aligns much closer to how most humans think about implication, adding it might help some people :)
(cherry picked from commit 51151c2c28)
In rendered form:
```diff
-Add references to the manual using ..
+Add references to the manual using [links like this](@docroot@/example.md)
```
(cherry picked from commit 147a34c573)
I can't find a good way to benchmark in isolation from the
git cache, but common sense dictates that creating (and destroying)
a 131KiB std::vector for each regular file from the archive imposes
quite a significant overhead regardless of the IO bound git cache.
AFAICT there is no reason to keep a copy of the data since
it always gets fed into the sink and there are no coroutines/threads
in sight.
(cherry picked from commit 7628155d2b)
Prior patches in 54dc5314e8
and 6db6190002 fixed the default
system double for i686 and ppc/ppc64. This also patch also covers
32 bit arm and mips. ARM cpu names are taken from host_machine.cpu()
for a lack of a better option, but host_machine.cpu_family() is
preferred, since that is supposed to be somewhat standard for cross
files. Endianness is handled correctly by looking at host_machine.endian().
This also updates the documentation to be up to date to how system cpu
is translated from the host_machine specification.
(cherry picked from commit 60d124b36e)
Before this change, if you were cross compiling Nix, then the nix-manual
subproject would never get built. In some situations, it makes sense to
not build the nix-manual subproject when cross compiling. For example,
if the build system is x86_64 and the host system is riscv64, then it
makes sense to not build the manual. Building the manual requires
executing certain build artifacts, and you can’t run x86_64 executables
on riscv64 systems.
That being said, there are some situations where it does make sense to
build the nix-manual subproject when cross compiling. For example, if
the build system is x86_64 and the host system is i686, then it doesn’t
make sense to not build the manual. You can run i686 executables on
x86_64 systems just fine.
This change makes it so that the nix-manual subproject will sometimes
get built when cross compiling. Specifically, the nix-manual subproject
will get built as long as the doc-gen option is enabled and the build
system is capable of running host binaries.
---
The main motivation behind this change is to fix this Nixpkgs issue [1].
Building pkgs.nixStatic counts as cross compiling Nix, and
pkgs.nixStatic is supposed to produce a man output. Building
pkgs.nixStatic currently fails because it isn’t actually producing a man
output. That issue will go away once this commit gets backported to Nix
2.28.x.
[1]: <https://github.com/NixOS/nixpkgs/issues/426410>
(cherry picked from commit 5e407e6abb)
CI on release branches should be stable, otherwise backporting
might become flaky and unreliable. Dogfooding only really makes
sense for CI on master branch, where failures are not as tedious
to work around.
I've missed this while reviewing 6db6190002.
I only built big endian ppc64, so that didn't occur to me.
From meson manual:
> Those porting from autotools should note that Meson does not add
> endianness to the name of the cpu_family. For example, autotools will
> call little endian PPC64 "ppc64le", Meson will not, you must also check
> the .endian() value of the machine for this information.
This code should handle that correctly.
(cherry picked from commit ebd311b7b7)
* It is tough to contribute to a project that doesn't use a formatter,
* It is extra hard to contribute to a project which has configured the formatter, but ignores it for some files
* Code formatting makes it harder to hide obscure / weird bugs by accident or on purpose,
Let's rip the bandaid off?
Note that PRs currently in flight should be able to be merged relatively easily by applying `clang-format` to their tip prior to merge.
Co-authored-by: Graham Christensen <graham@grahamc.com>
Previously, `nix help shell` failed with "Nix has no subcommand 'shell'" despite `nix shell --help`
working correctly. This happened because the `shell` command is actually an alias for `env shell`,
and the help system wasn't resolving aliases when looking up documentation.
This patch modifies the `showHelp` function to check for and resolve aliases before generating the
manpage name, ensuring that shorthand commands like `shell` get proper help documentation.
Closes: #13431
(cherry picked from commit d678b071d6)
Judging by the comment for `makeEmptySourceAccessor` the prefix has
to be empty:
> Return a source accessor that contains only an empty root directory.
Fixes#13295.
(cherry picked from commit fba1bb0c13)
c39cc00404 has added assertions for
all Value accesses and the following case has started failing with
an `unreachable`:
(/tmp/fun.nix):
```nix
{a}: a
```
```
$ nix eval --impure --expr 'import /tmp/fun.nix {a="a";b="b";}'
```
This would crash:
```
terminating due to unexpected unrecoverable internal error: Unexpected condition in getStorage at ../include/nix/expr/value.hh:844
```
This is not a regression, but rather surfaces an existing problem, which previously
was left undiagnosed. In the case of an import `fun` is the `import` primOp, so that read is invalid
and previously this resulted in an access into an inactive union member, which is UB.
The correct thing to use is `vCur`. Identical problem also affected the case of a missing argument.
Add previously failing test cases to the functional/lang test suite.
Fixes#13448.
(cherry picked from commit 6e78cc90d3)
Ensure relative path inputs are relative to the parent node's _actual_
`outPath`, instead of the subtly different `sourceInfo.outPath`.
Additionally, non-flake inputs now also have a `sourceInfo` attribute.
This fixes the relationship between `self.outPath` and
`self.sourceInfo.outPath` in some edge cases.
Fixes#13164
(cherry picked from commit 46beb9af76)
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:
nix-functional-tests> grep: repl_output: No such file or directory
nix-functional-tests> +(repl.sh:174) cat repl_output
This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.
(cherry picked from commit 9eb46e9cc0)
Squashed commit of the following:
commit 04fff3a637d455cbb1d75937a235950e43008db9
Author: Eelco Dolstra <edolstra@gmail.com>
Date: Thu Jun 12 12:30:32 2025 +0200
Chown structured attr files safely
commit 5417ad445e414c649d0cfc71a05661c7bf8f3ef5
Author: Eelco Dolstra <edolstra@gmail.com>
Date: Thu Jun 12 12:14:04 2025 +0200
Replace 'bool sync' with an enum for clarity
And drop writeFileAndSync().
commit 7ae0141f328d8e8e1094be24665789c05f974ba6
Author: Eelco Dolstra <edolstra@gmail.com>
Date: Thu Jun 12 11:35:28 2025 +0200
Drop guessOrInventPathFromFD()
No need to do hacky stuff like that when we already know the original path.
commit 45b05098bd019da7c57cd4227a89bfd0fa65bb08
Author: Eelco Dolstra <edolstra@gmail.com>
Date: Thu Jun 12 11:15:58 2025 +0200
Tweak comment
commit 0af15b31209d1b7ec8addfae9a1a6b60d8f35848
Author: Raito Bezarius <raito@lix.systems>
Date: Thu Mar 27 12:22:26 2025 +0100
libstore: ensure that temporary directory is always 0o000 before deletion
In the case the deletion fails, we should ensure that the temporary
directory cannot be used for nefarious purposes.
Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit 2c20fa37b15cfa03ac6a1a6a47cdb2ed66c0827e
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 12:42:55 2025 +0100
libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds
When calling `_deletePath` with a parent file descriptor, `openat` is
made effective by using relative paths to the directory file descriptor.
To avoid the problem, the signature is changed to resist misuse with an
assert in the prologue of the function.
Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit d3c370bbcae48bb825ce19fd0f73bb4eefd2c9ea
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 01:07:47 2025 +0100
libstore: ensure that `passAsFile` is created in the original temp dir
This ensures that `passAsFile` data is created inside the expected
temporary build directory by `openat()` from the parent directory file
descriptor.
This avoids a TOCTOU which is part of the attack chain of CVE-????.
Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit 45d3598724f932d024ef6bc2ffb00c1bb90e6018
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 01:06:03 2025 +0100
libutil: writeFile variant for file descriptors
`writeFile` lose its `sync` boolean flag to make things simpler.
A new `writeFileAndSync` function is created and all call sites are
converted to it.
Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit 732bd9b98cabf4aaf95a01fd318923de303f9996
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 01:05:34 2025 +0100
libstore: chown to builder variant for file descriptors
We use it immediately for the build temporary directory.
Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit 962c65f8dcd5570dd92c72370a862c7b38942e0d
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 01:04:59 2025 +0100
libstore: open build directory as a dirfd as well
We now keep around a proper AutoCloseFD around the temporary directory
which we plan to use for openat operations and avoiding the build
directory being swapped out while we are doing something else.
Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
Signed-off-by: Raito Bezarius <raito@lix.systems>
commit c9b42462b75b5a37ee6564c2b53cff186c8323da
Author: Raito Bezarius <raito@lix.systems>
Date: Wed Mar 26 01:04:12 2025 +0100
libutil: guess or invent a path from file descriptors
This is useful for certain error recovery paths (no pun intended) that
does not thread through the original path name.
Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
Signed-off-by: Raito Bezarius <raito@lix.systems>
Since the autotools-based build system has been removed
and meson already generates compile database there's no
need to have it in the devshell.
(cherry picked from commit 67535263a5)
When a PUT is redirected, some of the data can be sent by curl before headers are read. This means the subsequent PUT operation needs to seek back to origin.
(cherry picked from commit 90deb665eb)
The current example relies upon [nixfmt's deprecated tree traversal
behavior](https://github.com/NixOS/nixfmt/pull/240). The simplest
alternative is the new `nixfmt-tree` wrapper for `nixfmt`/`treefmt`.
(cherry picked from commit 6f71d8a9c2)
Before the change `nix` was stripping warning flags
reported by `gcc-14` too eagerly:
$ nix build -f. texinfo4
error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
last 25 log lines:
> 1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
> | ~~~~~~~~^~~~
> window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type []
> 1887 | &replen);
> | ^~~~~~~
> | |
> | int *
After the change the compiler flag remains:
$ ~/patched.nix build -f. texinfo4
error: builder for '/nix/store/i9948l91s3df44ip5jlpp6imbrcs646x-texinfo-4.13a.drv' failed with exit code 2;
last 25 log lines:
> 1495 | info_tag (mbi_iterator_t iter, int handle, size_t *plen)
> | ~~~~~~~~^~~~
> window.c:1887:39: error: passing argument 4 of 'printed_representation' from incompatible pointer type [-Wincompatible-pointer-types]
> 1887 | &replen);
> | ^~~~~~~
> | |
> | int *
Note the difference in flag rendering around the warning.
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda has a
good sumamry of why it happens. Befomre the change `nix` was handling
just one form or URL separator:
$ printf '\e]8;;http://example.com\e\\This is a link\e]8;;\e\\\n'
Now it also handled another for (used by gcc-14`):
printf '\e]8;;http://example.com\aThis is a link\e]8;;\a\n'
While at it fixed accumulation of trailing escape `\e\\` symbol.
(cherry picked from commit e322b714dc)
Once again found by an automated migration to `std::format`.
I've tested that boost::format works fine with `std::string_view`
arguments.
(cherry picked from commit 9fff868e39)
It seems that the intention was to format a number in base 8 (as
suggested by the %o format specifier), but `perms` is a `std::string`
and not a number. Looks like `rawMode` is the correct thing to use here.
(cherry picked from commit 1b5c8aac12)
... by moving our stuff out of the way from upstream's
`nixComponents` and `nixDependencies` attrsets.
(I prefer not to use overlays, but let's make it work this way
first)
(cherry picked from commit b257ea94e3)
We had fields set to the same values before in our test data. This is
not a problem per-se, but does mean we wouldn't catch certain mixups.
Now, the fields are set to distinct values (where possible), which makes
the test more robust.
(cherry picked from commit a0b2b75f59)
Now, both the unit and functional tests relating to derivation options
are tested both ways -- with input addressing and content-addressing
derivations.
(cherry picked from commit 307dbe9914)
This requires that we refer to the `sourceInfo` instead of the
`result`. However, `sourceInfo` does not create a chain of basedir
resolution, so we add that back with `flakeDir`.
(cherry picked from commit 2109a5a206)
Comments go to the end of the line, not merely the next ; *or* \n. Fix
by splitting on `;` *within* lines, and test.
(cherry picked from commit f8b13cce19)
nix> meson.build:216: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:222: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:235: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:236: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
nix> meson.build:242: WARNING: Project targets '>= 1.1' but uses feature introduced in '1.4.0': fs.name with build_tgt, custom_tgt, and custom_idx.
(cherry picked from commit 14a829acbb)
Fail when a command fails.
Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.
(cherry picked from commit da36c34db7)
Revert most of "Hack together a fix for the public headers"
- The `libmain` change is kept, and one more libmain change is made.
(Need to update Meson and Nix per the package alike).
- The S3 situation is fixed in a different way: the variable is public
now, used in the header, and fixed accordingly.
- Fix TODO for `HAVE_EMBEDDED_SANDBOX_SHELL`
This reverts commit 2b51250534.
(cherry picked from commit 3294b22a68)
Previously, when lock file contained invalid JSON nix reported a parser
error without specifying the file it came from.
This change adds flake.lock file path to the error message to avoid
confusion.
(cherry picked from commit e3873aa1a0)
we don't seem to use libgit2 for fetching via ssh, hence it shouldn't
matter if it's using libssh or the ssh binary.
(cherry picked from commit 0b61b758fb)
Some of the maintainer attribute names got changed in nixos-unstable
(e.g. "edolstra" is now "eelco") but we want this flake to work on
nixos-24.11. So just get rid of them.
(cherry picked from commit 93d8f62057)
For example, instead of doing
#include "nix/store-config.hh"
#include "nix/derived-path.hh"
Now do
#include "nix/store/config.hh"
#include "nix/store/derived-path.hh"
This was originally planned in the issue, and also recent requested by
Eelco.
Most of the change is purely mechanical. There is just one small
additional issue. See how, in the example above, we took this
opportunity to also turn `<comp>-config.hh` into `<comp>/config.hh`.
Well, there was already a `nix/util/config.{cc,hh}`. Even though there
is not a public configuration header for libutil (which also would be
called `nix/util/config.{cc,hh}`) that's still confusing, To avoid any
such confusion, we renamed that to `nix/util/configuration.{cc,hh}`.
Finally, note that the libflake headers already did this, so we didn't
need to do anything to them. We wouldn't want to mistakenly get
`nix/flake/flake/flake.hh`!
Progress on #7876
(cherry picked from commit cc24766fa6)
This includes the logic that disables checks on cross appropriately.
Co-authored-by: Peder Bergebakken Sundt <pbsds@hotmail.com>
(cherry picked from commit 27d71b21fc)
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.
(cherry picked from commit 7e540059a3)
There are two big changes:
1. Public and private config is now separated. Configuration variables
that are only used internally do not go in a header which is
installed.
(Additionally, libutil has a unix-specific private config header,
which should only be used in unix-specific code. This keeps things a
bit more organized, in a purely private implementation-internal way.)
2. Secondly, there is no more `-include`. There are very few config
items that need to be publically exposed, so now it is feasible to
just make the headers that need them just including the (public)
configuration header.
And there are also a few more small cleanups on top of those:
- The configuration files have better names.
- The few CPP variables that remain exposed in the public headers are
now also renamed to always start with `NIX_`. This ensures they should
not conflict with variables defined elsewhere.
- We now always use `#if` and not `#ifdef`/`#ifndef` for our
configuration variables, which helps avoid bugs by requiring that
variables must be defined in all cases.
(cherry picked from commit c204e307ac)
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:
/home/thomas/.nix-profile/bin
/nix/var/nix/profiles/default/bin
Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.
(cherry picked from commit 2b4e3fa144)
See comments on the script; this is supposed to avoid breaking muscle
memory without complicating the build system (which proved harder than I
thought too) or not doing the header hygiene change at all.
link-headers: use pathlib consistenly and fix type errors
(cherry picked from commit c6a176be62)
The short answer for why we need to do this is so we can consistently do
`#include "nix/..."`. Without this change, there are ways to still make
that work, but they are hacky, and they have downsides such as making it
harder to make sure headers from the wrong Nix library (e..g.
`libnixexpr` headers in `libnixutil`) aren't being used.
The C API alraedy used `nix_api_*`, so its headers are *not* put in
subdirectories accordingly.
Progress on #7876
We resisted doing this for a while because it would be annoying to not
have the header source file pairs close by / easy to change file
path/name from one to the other. But I am ameliorating that with
symlinks in the next commit.
(cherry picked from commit f3e1c47f47)
- Since it's now private, give it a rename. Note that I want to switch the
word order on the public ones too.
- Since it is only needed by two files, just include there rather than
the nasty blanket-forced thing.
(cherry picked from commit 326548bae5)
This fixes a few of the property tests, now that the property tests
are actually generating arbitrary data - some of that data now
requiring experimental features to function properly.
(cherry picked from commit c82ef825d4)
Here we're switching to combinators instead of dereference operator.
It turns out the dereference operator was being executed upon test
setup, meaning that we were only using a only single value for each of
the executions of the property tests! Really not good.
And on Windows, we instead get:
operator* is not allowed in this context
https://github.com/emil-e/rapidcheck/blob/ff6af6fc683159deb51c543b065eba14dfcf329b/src/gen/detail/GenerationHandler.cpp#L16C31-L16C71
Now a few of the property tests fail, because we're generating cases
which haven't been exercised before.
(cherry picked from commit 9a04f1e732)
Without this :u, :sh and :i repl commands fail with:
> Cannot run 'nix-shell'/`nix-env` because no method of calling the Nix
> CLI was provided. This is a configuration problem pertaining to how
> this program was built.
Remove the default ctor argument as it evidently makes catching
refactoring bugs much harder. `NixRepl` implementation lives completely
in `repl.cc`, so we can be as explicit as necessary.
(cherry picked from commit 44055dc09d)
1. Fix confusing wording that might imply unnecessary double-hashing.
2. Add references to specifics of base-32 encoding.
3. Fix incorrect description that sha256 hash of `fingerprint` is
truncated. "Truncated" is actual wording used in Nix theses, but it has
unusual meaning, that is better conveyed by word "compressed", which is
used by the reference C++ implementation.
4. Clarify details of base16 encoding.
(cherry picked from commit a0facb2aba)
Doing this makes catching non-obvious bugs easier. GHA CI workload is
already a concern and there isn't much benefit in running the tests with
and without sanitizers at the same time, so UBSAN is enabled for default
checks.
This change doesn't affect production builds in any way, but is rather a
step in the direction of improving automated testing during development.
Relates to #10969.
(cherry picked from commit 874587516c)
This is supposed to firstly improve the docs as they are, and secondly
hint at how the core conceptual information ought to be moved to the
store derivation section of the manual.
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
(cherry picked from commit 637aa0944d)
- Some headers were completely redundant and have been removed.
- Other headers have been turned private.
- Unnecessary meson.build code has been removed.
- libutil-tests now has a private config header, where previously
it had none. This removes the need to expose a package version
macro publicly.
(cherry picked from commit b86a76044e)
This makes it behave the same as nix-daemon. Opening the store in the
parent can cause a SIGBUS in libsqlite in the child:
#0 0x00007f141cf6f789 in __memset_avx2_unaligned_erms () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
#1 0x00007f141c322fe8 in walIndexAppend () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#2 0x00007f141c3711a2 in pagerWalFrames () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#3 0x00007f141c38317e in sqlite3PagerCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#4 0x00007f141c383555 in sqlite3BtreeCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#5 0x00007f141c384797 in sqlite3VdbeHalt () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#6 0x00007f141c3b8f60 in sqlite3VdbeExec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#7 0x00007f141c3bbfef in sqlite3_step () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#8 0x00007f141c3bd0e5 in sqlite3_exec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
#9 0x00007f141da140e0 in nix::SQLiteTxn::commit() () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
#10 0x00007f141d9ce69c in nix::LocalStore::registerValidPaths(std::map<nix::StorePath, nix::ValidPathInfo, std::less<nix::StorePath>, std::allocator<std::pair<nix::StorePath const, nix::ValidPathInfo> > > const&)::{lambda()#1}::operator()() const () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
(cherry picked from commit 9590167290)
I encountered this with a misconfigured libutil. I doubt that a
non-lutimes config is viable, because tests were failing.
(cherry picked from commit 1cffcd91a9)
Otherwise without the change the test fails on nix-2.26 as:
error: derivation contains an illegal reference specifier 'dev'
Note: the error message does not match intended change.
(cherry picked from commit 1e7c7244cf)
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.
Today's `nixpkgs` example:
Before the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation contains an illegal reference specifier 'man'
After the change:
$ nix build --no-link -f. postgresql_14
...
error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])
(cherry picked from commit bbdc3197a9)
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.
(cherry picked from commit 49f486d8e0)
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.
It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.
Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843
Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.
(cherry picked from commit 9b3352c3c8)
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.
We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.
Co-authored-by: Mic92 <joerg@thalheim.io>
Fixes https://github.com/NixOS/nix/issues/8946
See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.
(cherry picked from commit 1c636284a3)
We want the $doc, $man outputs to be symlinks pointing to nix-manual and
nix-manual.man. Creating the directories first makes the `ln` command
produce symlink $doc/${nix-manual} instead.
```
$file /nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
/nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man:
symbolic link to /nix/store/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
```
This is the reason `nix-env --help` is once again broken on 2.26/2.27/master
after 4108529.
(cherry picked from commit 0ddfbc5939)
It was first introduced in https://git.lix.systems/lix-project/lix/commit/19e0ce2c03d8e0baa16998b086665664c420c1df
In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.
Sample output:
Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]: 0# 0x000000000076876A in nix
1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
10# 0x000000000087A563 in nix
11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
13# 0x00000000006F4DF5 in nix
Co-authored-by: eldritch horrors <pennae@lix.systems>
(cherry picked from commit 163f94412a)
We now see exception beeing thrown when remote building in master
because of writing to a non-blocking file descriptor from our json logger.
> #0 0x00007f2ea97aea9c in __pthread_kill_implementation () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #1 0x00007f2ea975c576 in raise () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #2 0x00007f2ea9744935 in abort () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #3 0x00007f2ea99e8c2b in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #4 0x00007f2ea99f820a in __cxxabiv1::__terminate(void (*)()) () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #5 0x00007f2ea99f8275 in std::terminate() () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #6 0x00007f2ea99f84c7 in __cxa_throw () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #7 0x00007f2eaa5035c2 in nix::writeFull (fd=2, s=..., allowInterrupts=true) at ../unix/file-descriptor.cc:43
> #8 0x00007f2eaa5633c4 in nix::JSONLogger::write (this=this@entry=0x249a7d40, json=...) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
> #9 0x00007f2eaa5658d7 in nix::JSONLogger::logEI (this=<optimized out>, ei=...) at ../logging.cc:242
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> #11 nix::handleExceptions (programName="nix", fun=...) at ../shared.cc:343
> #12 0x0000000000465b1f in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/allocator.h:163
> (gdb) frame 10
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> 108 logEI(ei);
So far only drainFD sets the non-blocking flag on a "readable" file descriptor,
while this is a "writeable" file descriptor.
It's not clear to me yet, why we see logs after that point, but it's
also not that bad to handle EAGAIN in read/write functions after all.
(cherry picked from commit 2790f5f9ae)
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.
While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.
As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.
Some ad-hoc benchmarking:
After this commit:
```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
Time (mean ± σ): 784.2 ms ± 7.1 ms [User: 561.4 ms, System: 147.7 ms]
Range (min … max): 773.5 ms … 792.6 ms 10 runs
```
On master 3604c7c51:
```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
Time (mean ± σ): 22.914 s ± 0.178 s [User: 18.524 s, System: 4.151 s]
Range (min … max): 22.738 s … 23.290 s 10 runs
```
(cherry picked from commit adbd08399c)
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.
(cherry picked from commit a53b184e63)
The resume call would get some non-flushed(?) data.
Extending the pause to include the newline makes the complete flush
part of the pause.
(cherry picked from commit 880489051a)
This should fix a few packaging regressions.
`dev` also includes a merged `includes/`, which may be helpful until
inter-component includes are fixed properly.
(cherry picked from commit 41085295ab)
The bug reappeared after all, and the fix introduced a different bug. We
want to release 2.27 imminently so there is no time to do a proper fix,
which appears to require a larger reworking. Hopefully we will have it
for 2.28, however.
This reverts commit c98525235f.
It's not very clear what the ownership model is here, but one thing
is certain: `.up` can't be destroyed before the StaticEnv that refers
to it is.
Changing a non-owning pointer to taking shared ownership of the parent
`StaticEnv` prevents the `.up` from being freed.
I'm not a huge fan of the inverted ownership, where child `StaticEnv`
takes a refcount of the parent, but this seems like the least intrusive
way to fix the use-after-free.
This shouldn't cause any shared_ptr cycles to appear (hopefully).
When #9863 converted the `Nix::Store` free functions into member functions, the
implicit `this` argument was not accounted for when iterating over the variable
number of arguments in some functions.
Seems like this got dropped at some point during meson migration, so
put it back in the build system.
Drop all tests for `parseGitUrl`, since that function doesn't exist
and migrating doesn't look sensible because git-lfs stuff seems to use
`ParsedURL`.
Note that in pure mode, we don't need to use the union FS even when
using a chroot store, since the user shouldn't have access to the
physical /nix/store.
This makes `nix.version` quicker to evaluate, which should speed up
package listing operations.
If you want an accurate count, use `lib.optionals` in your override
instead of `null` values.
By appending a colon to MANPATH NIX_MAN_DIR gets prepended to the
final MANPATH before default search paths.
This makes man still consider default search paths, but prefers
NIX_MAN_DIR (if it exists).
It still makes sense to point NIX_MAN_DIR to a correct location
by moving man pages build from nix-manual.man to nix-cli.man, but
this should fix most common use-cases where nix is installed globally.
... as intended.
Requirements:
- don't build fresh libraries for each git commit
- have git commit in the CLI
Bug:
- echo ${version} went into the wrong file => use the fact that it's
a symlink, not just for reading but also for writing.
Logging to another Logger was kind of nonsensical - it was really just
an easy way to get it to write its output to stderr, but that only
works if the underlying logger writes to stderr.
This change is needed to make it easy to log JSON output somewhere
else (like a file or socket).
It is unused in Nix currently, but will be used in Hydra. This reflects
what Hydra does in https://github.com/NixOS/hydra/pull/1387.
We may probably to use it more widely for better SSH store performance,
but this needs to be subject to more testing before we do that.
This is a first step towards PR #10760, and the issues it addresses.
See the Doxygen for details.
Thanks to these changes, we are able to drastically restrict how the
rest of the code-base uses `ParseDerivation`.
Co-Authored-By: HaeNoe <git@haenoe.party>
It's best we teach users that the "foo" derivation is less than pure in the sense that it cannot be built just on any system, in particular that builders cannot be selected arbitrarily but based on their system-features. The `"recursive-nix"` system-feature is automatically defined by `--extra-experimental-features recursive-nix`
I do not believe there is any problem with computing
`hashDerivationModulo` the normal way with impure derivations.
Conversely, the way this used to work is very suspicious because two
almost-equal derivations that only differ in depending on different
impure derivations could have the same drv hash modulo. That is very
suspicious because there is no reason to think those two different
impure derivations will end up producing the same content-addressed
data!
Co-authored-by: Alain Zscheile <zseri.devel@ytrizja.de>
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.
"content-address*ed*" derivation is misleading because all derivations
are *themselves* content-addressed. What may or may not be
content-addressed is not derivation itself, but the *output* of the
derivation.
The outputs are not *part* of the derivation (for then the derivation
wouldn't be complete before we built it) but rather separate entities
produced by the derivation.
"content-adddress*ed*" is not correctly because it can only describe
what the derivation *is*, and that is not what we are trying to do.
"content-address*ing*" is correct because it describes what the
derivation *does* --- it produces content-addressed data.
This is a big step documenting the store layer on its own, separately from the evaluator (and `builtins.derivation`).
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.
It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.
`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
This uses the single-threaded C-based routines from libblake3.
This is not optimal performance-wise but should be a good starting point
for nix compatibility with BLAKE3 hashing until a more performant
implementation based on the multi-threaded BLAKE3 routines
(written in Rust) can be developed.
This fixes dynamic derivations, reverting #9081.
I believe that this time around, #9052 is fixed. When I first rebased
this, tests were failing (which wasn't the case before). The cause of
those test failures were due to the crude job in which the outer goal
tried to exit with the inner goal's status.
Now, that error handling has been reworked to be more faithful. The exit
exit status and exception of the inner goal is returned by the outer
goal. The exception was what was causing the test failures, but I
believe it was not having the right error code (there is more than one
for failure) that caused #9081.
The only cost of doing things the "right way" was that I had to
introduce a hacky `preserveException` boolean. I don't like this, but,
then again, none of us like anything about how the scheduler works.
Issue #11927 is still there to clean everything up, subsuming the need
for any `preserveException` because I doubt we will be fishing
information out of state machines like this at all.
This reverts commit 8440afbed7.
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
Using `set --local` is better than using `set`/`set --erase`. `--local`
will preserve any existing `NIX_LINK` value. And the local variable is
automatically removed for any execution path.
This allows a flake to specify that it needs Git submodules to be
enabled (or disabled, if we ever change the default) on the top-level
flake. This requires the input to be refetched, but since the first
fetch is lazy, this shouldn't be expensive.
Currently the only attribute allowed by `inputs.self` is `submodules`,
but more can be added in the future (e.g. a `lazy` attribute to opt in
to lazy tree behaviour).
Fixes#5312, #9842.
It seems reasonable to add the `share` folder from the user profile into
`$XDG_DATA_DIRS` both for daemon and profile execution. Nix could add
package shared files into this folder regardless of how the nix daemon
itself is running.
If we previously fetched by revision, the output of "git ls-remote"
won't start with the expected line like
ref: refs/heads/master HEAD
but will be something like
5c4410e3b9891c05ab40d723de78c6f0be45ad30 refs/heads/5c4410e3b9891c05ab40d723de78c6f0be45ad30
This then causes Nix to treat that revision as a refname, which then
leads to warnings like
warning: could not update cached head '5c4410e3b9891c05ab40d723de78c6f0be45ad30' for 'file:///tmp/repo'
This causes Git to create a local ref named refs/head/<rev>, e.g.
$ git -C ~/.cache/nix/gitv3/11irpim06vj4h6c0w8yls6kx4hvl0qd0gr1fvk47n76g6wf1s1vk ls-remote --symref .
5c4410e3b9891c05ab40d723de78c6f0be45ad30 refs/heads/5c4410e3b9891c05ab40d723de78c6f0be45ad30
7f6bde8a20de4cccc2256f088bc5af9dbe38881d refs/heads/7f6bde8a20de4cccc2256f088bc5af9dbe38881d
which confuses readHead(), leading to errors like
fatal: Refusing to point HEAD outside of refs/
warning: could not update cached head 'd275d93aa0bb8a004939b2f1e87f559f989453be' for 'file:///tmp/repo'
Git interprets them as part of the file name, so passing parameters
like 'rev' breaks. Only relevant for testing (when _NIX_FORCE_HTTP is
set) and local bare repos.
Fixes messages like 'copying /tmp/repo/tmp/repo to the store'. The
PosixSourceAccessor already sets the prefix. Setting the prefix twice
shouldn't be a problem, but GitRepoImpl::getAccessor() returns a
wrapped accessor so it's not actually idempotent.
It seems that `meson test --print-errorlogs` only captures stderr,
so this makes it forward the logs as intended.
We might want to redirect stdout in our common setup script instead.
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.
Not sure what the intent was expecting help.sh to fail in the main suite, but it caused `meson test` to fail inside a `nix develop` shell:
$ meson test help --print-errorlogs
ninja: Entering directory `/home/eelco/Dev/nix-master/build'
1/1 nix-functional-tests:main / help UNEXPECTEDPASS 4.02s
I started getting these warnings `warning: download buffer is full; consider increasing the 'download-buffer-size' setting` but the documentation does not make it obvious what unit of measurement it accepts.
1. Fix this eval error:
https://hydra.nixos.org/jobset/nix/master#tabs-errors
The dev package output (actually a separate derivation) needs to skip
this for cross just as the main package output does.
2. Deduplicate libs attrset and list.
3. Move `nix-functional-tests` to `checkInputs`.
With the Meson build system, we no longer need a `check` vs
`install-check` distinction, so it is simpler to just keeep
everything in one place.
This label will be useful for constructing queries to find backportable PRs.
Specifically, those should omit both automatic backports and
"backports reviewed" PRs.
This allows RemoteStore::addMultipleToStore() to free the Source
objects early (and in particular the associated sinkToSource()
buffers). This should fix#7359. For example, memory consumption of
nix copy --derivation --to ssh-ng://localhost?remote-store=/tmp/nix --derivation --no-check-sigs \
/nix/store/4p9xmfgnvclqpii8pxqcwcvl9bxqy2xf-nixos-system-...drv
went from 353 MB to 74 MB.
Fixes
$ nix copy --derivation --to /tmp/nix /nix/store/...
error: cannot enqueue a work item while the thread pool is shutting down
The ThreadPoolShutDown exception was hiding the reason for the thread
pool shut down, e.g.
error: cannot add path '/nix/store/03sl46khd8gmjpsad7223m32ma965vy9-fix-static.patch' because it lacks a signature by a trusted key
In order for the script not be sourced multiple times by the same shell
instance, `__ETC_PROFILE_NIX_SOURCED` needs to be set with a `--global`
flag.
Both files are almost identical. And style differences make it harder
to see what is actually different and keep them in sync, when it is
required.
`nix-profile.fish` and part of `nix-profile-daemon.fish` use 4 space
indentation. Which is also the indentation that the fish shell
documentation is using.
Reformatting a chunk of `nix-profile-daemon.fish` from 2 space
indentation to 4 space indentation for consistency.
- Multiple choices of stdenv are handled more consistently, especially for the dev
shells which were previously not done correctly.
- Some stray nix code was moving into the `packaging` directory
nix-env can read priorities from a derivations meta attributes, but this
only works when installing a nix expression.
nix-env can also install bare store paths, however meta attributes are
not readable in that case. This means that a store path can not be
installed with a specific priority.
Some cases where it is advantageous to install a store path: a remote
host following a `nix copy`, or any time you want to save some
evaluation time and happen to already know the store path.
This PR addresses this shortcoming by adding a --priority flag to
nix-env --install.
Since ff8e2fe84e, 'path:' URLs on the
CLI are interpreted as relative to the current directory of the user,
not the path of the flake we're overriding.
During garbage collection we cache several things -- a set of known-dead
paths, a set of known-alive paths, and a map of paths to their derivers.
Currently they use STL maps and sets, which are ordered structures that
typically are backed by binary trees. Since we are putting pseudorandom
paths into these and looking them up by exact key, we don't need the
ordering, and we're paying a nontrivial cost per insertion.
The existing maps require O(n log n) memory and have O(log n) insertion
and lookup time.
We could instead use unordered maps, which are typically backed by
hashmaps. These require O(n) memory and have O(1) insertion and lookup
time.
On my system this appears to result in a dramatic speedup -- prior to
this patch I was able to delete 400k paths out of 9.5 million over the
course of 34.5 hours. After this patch the same result took 89 minutes.
This result should NOT be taken at face value because the two runs
aren't really comparable; in particular the first started when I had 9.5
million store paths and the seconcd started with 7.8 million, so we are
deleting a different set of paths starting from a much cleaner
filesystem. But I do think it's indicative.
Related: https://github.com/NixOS/nix/issues/9581
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.
Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.
It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.
By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.
[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]
Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
This allows writing lock files with dirty inputs, so long as they have
a NAR hash. (Currently they always have a NAR hash, but with lazy
trees that may not always be the case.)
Generally dirty locks are bad for reproducibility (we can detect if
the dirty input has changed, but we have no way to fetch it except
substitution). Hence we don't allow them by default.
Fixes#11181.
Commands like `nix flake metadata '.?submodules=1'` ignored the query
part of the URL, while `nix build '.?submodules=1#foo'` did work
correctly because of the presence of the fragment part.
The current instructions for building the Nix manual include a command that doesn't work as described. Specifically:
```
nix build .#nix^doc
```
Running this command results in the error:
```
error: derivation '/nix/store/hddqxzfqgx2fhj8q66ss3idym7pk7aj1-nix-2.26.0pre20250107_383ab87.drv' does not have wanted outputs 'doc'
```
However, this command works if you specify the Nix version explicitly, such as:
```
nix build nix/2.24.11#nix^doc
```
Additionally, these commands are run within the Nix root directory.
However, the nix build .#nix^doc command does work when run from the nixpkgs directory and generates the NixOS manual.
I'm not sure if I'm missing something. Is the `nix^doc` supposed to be added somehow to flake outputs?
The incremental build section does not work since as make has been decommissioned in favor of Meson. Should this be simply deleted?
I'd messed up a rebase in my previous iteration, causing `weakly_canonical` to reappear,
but not trigger a test failure.
These two functions behave similarly when the argument is a path that points to a broken
symlink. `weakly_canonical` would not resolve it because the target doesn't exist, and
`makeParentCanonical` would not resolve it, because it never resolves the final path
element.
This new test case now also tests a valid symlink, "differentiating" the two.
The experimental `nix eval` command already supports a `--raw` flag.
This commit implements the same flag for the stable nix-instantiate command.
Until now instructions and scripts that didn't want to rely on experimental
features had to use workarounds such as:
nix-instantiate --eval <something> | tr -d \"
(which also undesirably also removes double quotation marks within the string), or
nix-instantiate --eval <something> | jq -j
(which undesirably depends on another package).
Co-authored-by: Silvan Mosberger <silvan.mosberger@tweag.io>
Upstream change
https://github.com/kristapsdz/lowdown/commit/bab1d75079ac3866fc14e1fa4825ae64a94a7c0e
moved a few fields from `lowdown_opts` toa new `lowdown_opts_term`
struct. As a result the build started failing as:
nix-cmd> [2/17] Compiling C++ object libnixcmd.so.p/markdown.cc.o
nix-cmd> FAILED: libnixcmd.so.p/markdown.cc.o
nix-cmd> g++ -Ilibnixcmd.so.p -I. -I.. -I/nix/store/b0bnrk5lacxbpgxgnc28r8q3wcazrgxj-nix-util-2.26.0pre-dev/include/nix -I/nix/store/cxnynq9ykyj4xxv6wf6dw7r0aw5x6n9k-libarchive-3.7.7-dev/include -I/nix/store/bfgjwkcb8snkizx578rzdahi75m8zyh4-nlohmann_json-3.11.3/include -I/nix/store/3sx8bq3sip6j2nv1m5xx4gbdp33v7iy6-nix-store-2.26.0pre-dev/include/nix -I/nix/store/sih2dgqzvsbv7p510lkfmas7s7wbsl4j-nix-fetchers-2.26.0pre-dev/include/nix -I/nix/store/68p8s20fsiiakj7nys7grbaixfnhsdzs-nix-expr-2.26.0pre-dev/include/nix -I/nix/store/gw7wknhzhfzzj9zww2kyi5xrzgf1ndki-boehm-gc-8.2.8-dev/include -I/nix/store/3jwb9j4vnsk5saq3wfyyp9il3mhs41l9-nix-flake-2.26.0pre-dev/include/nix -I/nix/store/8nwjvmq7m48v8g646jrxkikv6x47bc3m-nix-main-2.26.0pre-dev/include/nix -I/nix/store/rb0hzsw5wc1a7daizhpj824mbxlvijrq-lowdown-1.4.0-dev/include -I/nix/store/m388ywpk53fsp8r98brfd7nf1f5sskv0-editline-1.17.1-dev/include -fdiagnostics-color=always -D_GLIBCXX_ASSERTIONS=1 -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++2a -include config-util.hh -include config-store.hh -include config-expr.hh -include config-main.hh -include config-cmd.hh -Wdeprecated-copy -Werror=suggest-override -Werror=switch -Werror=switch-enum -Werror=unused-result -Wignored-qualifiers -Wimplicit-fallthrough -Wno-deprecated-declarations -O3 -fPIC -pthread -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -MD -MQ libnixcmd.so.p/markdown.cc.o -MF libnixcmd.so.p/markdown.cc.o.d -o libnixcmd.so.p/markdown.cc.o -c ../markdown.cc
nix-cmd> ../markdown.cc: In function 'std::string nix::doRenderMarkdownToTerminal(std::string_view)':
nix-cmd> ../markdown.cc:28:5: error: 'lowdown_opts' has no non-static data member named 'cols'
nix-cmd> 28 | };
nix-cmd> | ^
The change adds version-based conditional to support both pre-1.4 and
1.4 forms of the initialization.
Closes: https://github.com/NixOS/nix/issues/12113
The primitive `readFileType p` has a list of acceptable types, and so does `readDir path`
This edit makes the formatting of the list consistent between themselves, and other parts of the documentation.
Paths are already quoted:
error:
… while fetching the input 'path:/nix/store/rs2s2ca7xs87v82aps54m1p3sqrfz6c8-source'
error: chmod '"/nix/store/rs2s2ca7xs87v82aps54m1p3sqrfz6c8-source"': Read-only file system
src/nix/flake.md describes the format of flake.lock files. Before this
change, it said that the original field was “The original input
specification from `flake.lock`[…]” The original input specification is
in flake.nix, not flake.lock.
When resolving indirect flake references like `nixpkgs` in `flake.nix`
files, Nix will no longer use the system and user flake registries. It
will only use the global flake registry and overrides given on the
command line via `--override-flake`.
Just now there is a dependency on cachix, which means we cannot test
the installer in CI if forks do not have the necessary secrets set up.
We replace this with a simple http server that serves the installer and
can be both used in CI and locally.
Commit cfe66dbec updated `nix upgrade-nix` to use
`ExecutablePath::load().find`, which broke the logic for finding the
profile associated with the nix executable. The error looks something
like:
```
$ sudo -i nix upgrade-nix --debug
found Nix in '"/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin/nix"'
found profile '/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin'
error: directory '"/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin/nix"' does not appear to be part of a Nix profile
```
This seems to happen for two reasons:
1. The original PATH search resulted in a directory, but `find` returns
the path to the executable. Fixed by getting the path's parent.
2. The profile symlink cannot be found because
`ExecutablePath::load().find` canonicalizes the executable path. I
updated find to normalize the path instead, which seems more in line
with how other programs resolve paths. I'm not sure if this affects
other callers though.
I manually tested this on macOS and Linux, and it seemed to fix
upgrading from 2.25.2 to 2.25.3.
- This speeds up macOS builds from 30 minutes to 11 minutes (3x faster).
- Also improve error reporting e.g. printing out what actually failed to build.
- As a result we also no longer need swap.
As far as I can tell, there's no real reason either of these need to
be 664. I'm willing to bet they were just a typo that has lasted for
7 years. While this shouldn't change anything, this is, IMHO, more
correct, so let's stop perpetuating the wrong mode!
We're not testing against these versions anymore.
If we bring that back (I would support that), we could do so in a clean
way, by making sure that the packaging we test against has a proper version
attribute.
Fix a footgun. In my case, I had a couple of build ("output")
directories sitting around.
rm -rf build-*
Was confused for a bit why a meson.build file was missing.
Probably also helps with autocompletion.
I tried meson-build-support first, but I had to add something like
a nix- prefix, in order to make meson happy. They've reserved the
meson- prefix.
I think I have failed to read the very long version-garbage-like
string for the second time now, leaving me oblivious to the crucial
info that a test failure happens in the context of an older daemon.
Before this change, expressions like:
with import <nixpkgs> {};
runCommand "foo" {} ''
echo '@nix {}' >&$NIX_LOG_FD
''
would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.
Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.
Fixes#544.
Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
(cherry picked from commit e55cd3beea710db727fd966f265a1b715b7285f3)
The Determinate Nix Installer has set nosuid and noatime in https://github.com/DeterminateSystems/nix-installer/pull/1338, and figured this perf and security improvement is worthy of upstreaming.
The /nix volume shouldn't have setuid binaries anyway, and filesystems seem to generally be noatime on macOS.
Further, the garbage collector doesn't use atime.
Instead of the unhelpful
warning: 'https://cache.flakehub.com' does not appear to be a binary cache
you now get
warning: unable to download 'https://cache.flakehub.com/nix-cache-info': HTTP error 401
response body:
{"code":401,"error":"Unauthorized","message":"Unauthorized."}
We *could* use a "native" manual instead - ie reusing a native
`nixpkgsFor.${buildPlatform}`, but this works, and also
works for possible cases where we have a custom or patched build tool.
Without the change `meson setup` fails on `Gentoo or Debian as those
don't use multicall binary:
$ meson setup ..
...
Executing subproject nix-functional-tests
...
../src/nix-functional-tests/meson.build:24:14: ERROR: Program 'coreutils' not found or not executable
The change always uses `ls` to look `coreutils` up.
Closes: https://github.com/NixOS/nix/issues/11975
Upstream `bzip2` does not provide `pkg-config` files. As a result an
attempt to build `nix` on some distributions like Gentoo failos the
configure as:
$ meson setup ..
...
Executing subproject perl
...
perl| Run-time dependency bzip2 found: NO (tried pkgconfig and cmake)
../src/perl/meson.build:68:12: ERROR: Dependency "bzip2" not found, tried pkgconfig and cmake
The change falls back to `bz2` library for such cases.
This prevents any potential cases of deletion through base pointer and its
non-virtual dtor, which might leak memory. Also gets rid of the warning:
/nix/store/fg7ass3a5m5pgl26qzfdniicbwbgzccy-gcc-13.2.0/include/c++/13.2.0/bits/stl_construct.h:88:2: warning: destructor called on non-final 'nix::flake::Settings' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
88 | __location->~_Tp();
....
../src/libflake-c/nix_api_flake.cc:10:30: note: in instantiation of function template specialization 'nix::make_ref<nix::flake::Settings>' requested here
10 | auto settings = nix::make_ref<nix::flake::Settings>();
This gets rid of unnecessary copies in range-based-for loops and
local variables, when they are used solely as `const &`.
Also added a fixme comment about a suspicious move out of const,
which might not be intended.
This shuts up a 300-line warning that includes
/nix/store/fg7ass3a5m5pgl26qzfdniicbwbgzccy-gcc-13.2.0/include/c++/13.2.0/bits/stl_tree.h:182:25: warning: ‘*(std::_Rb_tree_header*)((char*)&<unnamed> + offsetof(nix::value_type, nix::DerivedPath::<unnamed>.std::variant<nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Variant_base<nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Move_assign_base<false, nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Copy_assign_base<false, nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Move_ctor_base<false, nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Copy_ctor_base<false, nix::DerivedPathOpaque, nix::DerivedPathBuilt>::<unnamed>.std::__detail::__variant::_Variant_storage<false, nix::DerivedPathOpaque, nix::DerivedPathBuilt>::_M_u) + 24).std::_Rb_tree_header::_M_header.std::_Rb_tree_node_base::_M_parent’ may be used uninitialized [-Wmaybe-uninitialized]
182 | if (__x._M_header._M_parent != nullptr)
| ~~~~~~~~~~~~~~^~~~~~~~~
Since lib{expr,store,util}-test-support subprojects define nix_api_* helpers
for testing nix c bindings, they need to publicly depend on -c counterparts.
This makes their headers self-sufficient and does not rely on the -tests to add
necessary dependencies.
Get rid of this fixme. This does not appear to be used anywhere in
the nix codebase itself. Not sure why the comment mentioned C++20 erase
member function with predicate, but iterator-based algorithms are also fine.
Looks like some cruft has been left over from previous refactorings.
This removes dead variables, which should not have side effects in their
constructors. In cases where the variable initialization has a purpose
[[maybe_unused]] is inserted to silence compiler warnings.
* doc: Clarify that nix-shell still uses shell from host environment
* doc: Fix NIX_BUILD_SHELL description
* doc: Add anchor and link to NIX_BUILD_SHELL
* doc: Add example of default shell trickiness
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
This reduces the amount of boilerplate. More importantly, it provides
a place to add compiler flags (such as -O3) without having to add it
to every subproject (and the risk of forgetting to include it).
Fixes
nix: ../src/libexpr/primops/fetchTree.cc:37: void nix::emitTreeAttrs(EvalState&, const StorePath&, const fetchers::Input&, Value&, bool, bool): Assertion `narHash' failed.
on a lock file with an input that doesn't have a narHash. This can
happen when using a lock file created by the lazy-trees branch.
Cherry-picked from lazy-trees.
When diagnosing infinite recursion references to nullptr `Env` can be formed.
This happens only with `ExprBlackHole` is evaluated, which always leads to
`InfiniteRecursionError`.
UBSAN log for one such case:
```
../src/libexpr/eval-inline.hh:94:31: runtime error: reference binding to null pointer of type 'Env'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/libexpr/eval-inline.hh:94:31 in
```
First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.
This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.
Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.
This is the first part of rewriteDerivation() factored out into its
own method. It's not used anywhere else at the moment, but it's useful
on lazy-trees for rewriting virtual paths.
It's not so common knowledge that forges also expose pull requests as
git refs. But it's actually a cool way of quickly testing someones
contribution, so I found it worth specifically mentioning it.
It seems that I copied the expression for baseDir thoughtlessly and
did not come back to it.
- `baseDir` was only used in the `fromArgs` branch.
- `fromArgs` is true when `packages` is true.
This interferes with the progress bar, resulting in output like
evaluating derivation 'git+file:///home/eelco/Dev/nix-master#packages.x86_64-linux.default'/nix/store/zz8v96j5md952x0mxfix12xqnvq5qv5x-nix-2.26.0pre20241114_a95f6ea.drv
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:
../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'
This was first tagged as 2.15.0, 1½ years ago; plenty of time for
everyone to catch up.
By now, the warning is causing more confusion than that it is helpful,
because passing a `.drv` or `drvPath` has legitimate use cases.
The API docs build is extremely noisy (#11841) and probably not many
people care about it anyway. Also, they get rebuild on *every* ninja
invocation which is generally a waste of time.
Of course, you can still build the docs via `nix build
.#nix-{internal,external}-api-docs`, which is pretty fast.
The new package output attributes are somewhat experimental, and
provided for compatibility most of all.
We'll see how well this goes before the changes proposed in
https://github.com/NixOS/nix/issues/6507
Add options uid, gid, uname, and gname to docker.nix.
Setting these to e.g. 1000, 1000, "user", "user" will build an image
which runs and allows using Nix as that user.
In these trivial cases the final vector size (or lower bound on the size) is known,
so we can avoid some vector reallocations. This is not very important, but is just
good practice and general hygiene.
This is good practice to avoid pessimisations.
Left comments for the reasoning why ctors should be noexcept.
There are some tricky cases where we intentionally want throwing move ctors/assignments.
But those cases should really be reviewed, since some of those can be replaced
with more idiomatic copy/move-and-swap.
`auto &&` and `T &&` are forwarding references and can be
either lvalue or rvalue references. Moving from universal references
is incorrect and should not be done.
Moving from integral or floating-point values is pointless and just
worsens debug performance.
Naming class member variables the same as constructor arguments is a very
slippery slope because of how member variable names get resolved. Compiler
is not very helpful here and we need static analysis to forbid this kind of
stuff.
The following example illustrates the cause quite well:
```cpp
struct B {
B(int) {}
};
struct A {
A(int b): b([&](){
return b;
static_assert(std::is_same_v<decltype(b), int>);
}()) {
static_assert(std::is_same_v<decltype(b), int>);
}
void member() {
static_assert(std::is_same_v<decltype(b), B>);
}
B b;
};
int main() {
A(1).member();
}
```
From N4861 6.5.1 Unqualified name lookup:
> In all the cases listed in [basic.lookup.unqual], the scopes are searched
> for a declaration in the order listed in each of the respective categories;
> name lookup ends as soon as a declaration is found for the name.
> If no declaration is found, the program is ill-formed.
In the affected code there was a use-after-move for all accesses in the constructor
body, but this UB wasn't triggered.
These types of errors are trivial to catch via clang-tidy's [clang-analyzer-cplusplus.Move].
This was broken since a03bb4455c because
Nix 2.18 does not support broken $SHELL settings. So don't try a
broken $SHELL on old Nix versions. (It's a mystery though why
tests.remoteBuilds_local_nix_2_13 and tests.remoteBuilds_local_nix_2_3
didn't fail...)
https://hydra.nixos.org/build/277366807
This may occur when stderr is a tty but stdin is empty.
E.g.
$ nix build </dev/null
error: unexpected EOF reading a line
These stdio handles are how some non-interactive sandboxes behave,
including the Nix build sandbox and Hercules CI Effects.
Unfortunately `StringSource` class is very easy was very easy to misuse
because the ctor took a plain `std::string_view` which has a bad habit
of being implicitly convertible from an rvalue `std::string`. This lead
to unintentional use-after-free bugs.
This patch makes `StringSource` much harder to misuse by disabling the ctor
from a `std::string &&` (but `const std::string &` is ok).
Fix affected tests from libstore-tests.
Reformat those tests with clangd's range formatting since the diff is tiny
and it seems appropriate.
It had gotten rather big. Hopefully we'll eventually have some generic
infra for a "multi-package dev shell" and not need so much code for
this, but until then it's better in a separate file.
operators are an everyday thing in the Nix language, and this page will
hopefully be consulted by many users.
string contexts are quite exotic, and not linking to the detailed
explanation will require readers to figure out manually what this is
about, or worse, skim over and run into problems later.
* doc/manual: Add 'Debugging Nix' section
This commit adds a new 'Debugging Nix' section to the Nix manual. It provides instructions on how to build Nix with debug symbols and how to debug the Nix binary using debuggers like `lldb`.
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
We now just check that the fetcher doesn't change any attributes in
the input, and return all the original attributes (i.e. discarding any
new attributes and keeping any attributes that the fetcher didn't
keep).
This fixes the error
'{"__final":true,"lastModified":1686592866,"narHash":"sha256-riGg89eWhXJcPNrQGcSwTEEm7CGxWC06oSX44hajeMw","owner":"nixos","repo":"nixpkgs","rev":"0eeebd64de89e4163f4d3cf34ffe925a5cf67a05","type":"github"}' resulted in different input
'{"__final":true,"lastModified":1686592866,"narHash":"sha256-riGg89eWhXJcPNrQGcSwTEEm7CGxWC06oSX44hajeMw=","owner":"nixos","repo":"nixpkgs","rev":"0eeebd64de89e4163f4d3cf34ffe925a5cf67a05","type":"github"}'
in flake-regressions/tests/nix-community/patsh/0.2.1 (note the lack of
a trailing '=' in the NAR hash in the lock file).
OpenBSD doesn't support `lutimes`, but does support `utimensat` which
subsumes it. In fact, all the BSDs, Linux, and newer macOS all support
it. So lets make this our first choice for the implementation.
In addition, let's get rid of the `lutimes` `ENOSYS` special case. The
Linux manpage says
> ENOSYS
>
> The kernel does not support this call; Linux 2.6.22 or later is
> required.
which I think is the origin of this check, but that's a very old version
of Linux at this point. The code can be simplified a lot of we drop
support for it here (as we've done elsewhere, anyways).
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
Backward-compatible schema changes (e.g. those that add tables or
nullable columns) now no longer need a change to the global schema
file (/nix/var/nix/db/schema). Thus, old Nix versions can continue to
access the database.
This is especially useful for schema changes required by experimental
features. In particular, it replaces the ad-hoc handling of the schema
changes for CA derivations (i.e. the file /nix/var/nix/db/ca-schema).
Schema versions 8 and 10 could have been handled by this mechanism in
a backward-compatible way as well.
OpenBSD dynamic libraries never link to libc directly.
Instead, they have undefined symbols for all libc functions they use
that ld.so resolves to the libc referred to in the main executable.
Thus, disallowing undefined symbols will always fail
the default int64_t max was still overflowing for me, when this was dumped as json (noticed during building the manual).
So making 0, the default and define it as "no warnings" fixes the situtation.
Also it's much more human-readable in documentation.
This works because the `builder` and `args` variables are only used
in the non-builtin code path.
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
tests/functional/help.sh calls nix-* commands with option --help
if nix is built without documentation the option --help throws an error
because the man page it wants to display is missing
This overall seems like insecure tmp file handling to me. Because other
users could replace files in /tmp with a symlink and make the nix-shell
override other files.
fixes https://github.com/NixOS/nix/issues/11470
Introduced in 8f6b347abd without explanation.
Throwing anything that's not that is a programming mistake that we don't want
to ignore silently. A crash would be ok, because that means we/they can fix
the offending throw.
Otherwise, if checkInterrupt() in any of the supported store operations
would catch onto a user interrupt, the exception would bubble to the thread
start and be handled by std::terminate(): a crash.
We haven't added the narHash attribute yet at that point. And if the
caller uses getAccesor() instead of fetchToStore() (e.g. in `nix
registry pin`), the narHash attribute will never be added. This could
lead to a mismatch.
The ability to substitute inputs was removed in #10612 because it was
broken: with user-specified inputs containing a `narHash` attribute,
substitution resulted in an input that lacked the attributes returned
by the real fetcher (such as `lastModified`).
To fix this, we introduce a new input attribute `final`. If `final =
true`, fetching the input cannot add or change any attributes.
We only attempt to substitute inputs that have `final = true`. This is
implied by lock file entries; we only write a lock file if all its
entries are "final".
The user can specified `final = true` in `fetchTree`, in which case it
is their responsibility to ensure that all attributes returned by the
fetcher are included in the `fetchTree` call. For example,
nix eval --impure --expr 'builtins.fetchTree { type = "github"; owner = "NixOS"; repo = "patchelf"; final = true; narHash = "sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM="; }'
succeeds in a store path with the specified NAR hash exists or is
substitutable, but fails with
error: fetching final input '{"final":true,"narHash":"sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM=","owner":"NixOS","repo":"patchelf","type":"github"}' resulted in different input '{"final":true,"lastModified":1718457448,"narHash":"sha256-FSoxTcRZMGHNJh8dNtKOkcUtjhmhU6yQXcZZfUPLhQM=","owner":"NixOS","repo":"patchelf","rev":"a0f54334df36770b335c051e540ba40afcbf8378","type":"github"}'
... and remove a few unused arguments.
This adds pkg-config to a two or three packages that don't use it,
but we shouldn't let that bother us. It's like our personal stdenv.
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)
Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory. The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.
This allows `nix copy` to atomically copy a store path and point a
profile to it, without the risk that the store path might be GC'ed in
between. This is useful for instance when deploying a new NixOS system
profile from a remote store.
This caused nlohmann/json.hpp to leak into a lot of compilation units,
which is slow (when not using precompiled headers).
Cuts build time from 46m24s to 42m5s (real time with -j24: 2m42s to
2m24s).
These versions are more than 3 years old and were very early in the
existence of CA derivations support (which was and is experimental),
so they're unlikely to still exist in the real world. So let's get rid
of support for them.
The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.
A test added recently checks that when trying to deserialize a NAR with
two files that Unicode-normalize to the same result either succeeds on
Linux, or fails with an "already exists" error on Darwin. However,
failing with an "already exists" error can in fact also happen on Linux,
when using ZFS with the proper utf8 and Unicode normalization options
set.
This commit fixes the issue by not assuming the behavior from the
current system, but just by blindly checking that either one of the two
aforementioned possibilities happen, whether on Darwin or on Linux.
Additionally, we check that the Unicode normalization behaviour of
nix-store is the same as the host file system.
This leads to confusion about what the command does.
E.g. https://github.com/NixOS/nix/issues/9359
- Move the description up
- Remove details about the individual formatters
This patch has been manually adapted from
https://github.com/lix-project/lix/commit/14dc84ed03f1b7e5a41bb6fdce00916faab32b60
Tested with:
$ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 16:57:50 after 1s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1
Now returns:
nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 17:05:48 after 0s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv':
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
This method is marked as `noexcept`, but `enqueueFileTransfer()` can
throw `Interrupted` if the user has hit Ctrl-C or if the `ThreadPool`
that the thread is a part of is shutting down.
When working on speeding up the CI,
I triggered a race condition in the creation of the tarball cache.
This code now instead will ensure that half-initialized repositories
are no longer visible to any other nix process.
This is the error message that I got before:
error: opening Git repository '"/Users/runner/.cache/nix/tarball-cache"': could not find repository at '/Users/runner/.cache/nix/tarball-cache'
* docs: specify that flake.lock files are JSON
Recently, I decided that I was going to write some code that would parse
flake.lock files. I went to the Nix Reference Manual in order to look up
information on the format of flake.lock files, and I realized that a key
detail was missing from the Nix Reference Manual: it never says that
flake.lock files are JSON files. This commit fixes that issue.
This commit makes sure to specify that flake.lock files are encoded in
UTF-8. Confusingly, there’s multiple different JSON standards. Neither
ECMA-404, 2nd Edition [1] nor ISO/IEC 21778:2017 [2] mention UTF-8. RFC
8259 requires UTF-8, but only sometimes [3]. I chose to explicitly
specify that flake.lock files are UTF-8 in order to avoid any possible
ambiguities from the JSON standards.
[1]: <https://ecma-international.org/publications-and-standards/standards/ecma-404>
[2]: <https://www.iso.org/standard/71616.html>
[3]: <https://www.rfc-editor.org/rfc/rfc8259.html#section-8.1>
Most of the time people run single tests for debugging reason,
so it's a sane default to have them see all the console output.
This commit still retains the section about running tests directly with
meson, because in some debugging cases it's just nice to have less
abstractions i.e. when using strace.
Since #8766, invalid base64 is rendered in errors, but we don't actually
want to show this in the case of an invalid private keys.
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
As a prelude to making "or" work like a normal variable, emit a warning
any time the "fn or" production is used in a context that will change
how it is parsed when that production is refactored.
In detail: in the future, OR_KW will be moved to expr_simple, and the
cursed ExprCall production that is currently part of the expr_select
nonterminal will be generated "normally" in expr_app instead. Any
productions that accept an expr_select will be affected, except for the
expr_app nonterminal itself (because, while expr_app has a production
accepting a bare expr_select, its other production will continue to
accept "fn or" expressions). So all we need to do is emit an appropriate
warning when an expr_simple representing a cursed ExprCall is accepted
in one of those productions without first going through expr_app.
As the warning message describes, users can suppress the warning by
wrapping their problematic "fn or" expressions in parentheses. For
example, "f g or" can be made future-proof by rewriting it as
"f (g or)"; similarly "[ x y or ]" can be rewritten as "[ x (y or) ]",
etc. The parentheses preserve the current grouping behavior, as in the
future "f g or" will be parsed as "(f g) or", just like
"f g anything-else" is grouped. (Mechanically, this suppresses the
warning because the problem ExprCalls go through the
"expr_app : expr_select" production, which resets the cursed status on
the ExprCall.)
My SNAFU was that I assumed that all the `Value *`s we put in
`attrsSeen` are already reachable (which they are), but I forgot about
the `elems` pointer in `ListBuilder`.
Fixes#11547.
Subflakes are flakes in the same tree, accessed in flake inputs via
relative paths (e.g. `inputs.foo.url = "path:./subdir"`). Previously
these didn't work very well because they would be separately copied to
the store, which is inefficient and makes references to parent
directories tricky or impossible. Furthermore, they had their own NAR
hash in the lock file, which is superfluous since the parent is
already locked.
Now subflakes are accessed via the accessor of the calling flake. This
avoids the unnecessary copy and makes it possible for subflakes to
depend on flakes in a parent directory (so long as they're in the same
tree).
Lock file nodes for relative flake inputs now have a new `parent` field:
{
"locked": {
"path": "./subdir",
"type": "path"
},
"original": {
"path": "./subdir",
"type": "path"
},
"parent": [
"foo",
"bar"
]
}
which denotes that `./subdir` is to be interpreted relative to the
directory of the `bar` input of the `foo` input of the root flake.
Extracted from the lazy-trees branch.
- run:nix --experimental-features 'nix-command flakes' flake show --all-systems --json
# Steps to test CI automation in your own fork.
# Cachix:
# 1. Sign-up for https://www.cachix.org/
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
# Dockerhub:
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents:none
name:Check Cachix and Docker secrets present for installer tests
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents:none
name:Check Docker secrets present for installer tests
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_UNIT_TESTS" == "yes"],
[AC_MSG_ERROR([Cannot enable unit tests when building overall is disabled. Please do not pass '--enable-unit-tests' or do not pass '--disable-build'.])])
AC_ARG_ENABLE(functional-tests, AS_HELP_STRING([--disable-functional-tests],[Do not build the tests]),
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_DOC_GEN" == "yes"],
[AC_MSG_ERROR([Cannot enable generated docs when building overall is disabled. Please do not pass '--enable-doc-gen' or do not pass '--disable-build'.])])
AS_IF(
[test "$ENABLE_FUNCTIONAL_TESTS" == "yes" || test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(jq, jq)])
AS_IF([test "$ENABLE_BUILD" == "yes"],[
# Look for boost, a required dependency.
# Note that AX_BOOST_BASE only exports *CPP* BOOST_CPPFLAGS, no CXX flags,
# and CPPFLAGS are not passed to the C++ compiler automatically.
# Thus we append the returned CPPFLAGS to the CXXFLAGS here.
AS_HELP_STRING([--disable-cpuid], [Do not determine microarchitecture levels with libcpuid (relevant to x86_64 only)]))
if test "x$enable_cpuid" != "xno"; then
PKG_CHECK_MODULES([LIBCPUID], [libcpuid],
[CXXFLAGS="$LIBCPUID_CFLAGS $CXXFLAGS"
have_libcpuid=1
AC_DEFINE([HAVE_LIBCPUID], [1], [Use libcpuid])]
)
fi
fi
AC_SUBST(HAVE_LIBCPUID, [$have_libcpuid])
# Look for libseccomp, required for Linux sandboxing.
case "$host_os" in
linux*)
AC_ARG_ENABLE([seccomp-sandboxing],
AS_HELP_STRING([--disable-seccomp-sandboxing],[Don't build support for seccomp sandboxing (only recommended if your arch doesn't support libseccomp yet!)
]))
if test "x$enable_seccomp_sandboxing" != "xno"; then
synopsis:Use envvars NIX_CACHE_HOME, NIX_CONFIG_HOME, NIX_DATA_HOME, NIX_STATE_HOME if defined
prs:[11351]
---
Added new environment variables:
-`NIX_CACHE_HOME`
-`NIX_CONFIG_HOME`
-`NIX_DATA_HOME`
-`NIX_STATE_HOME`
Each, if defined, takes precedence over the corresponding [XDG environment variable](@docroot@/command-ref/env-common.md#xdg-base-directories).
This provides more fine-grained control over where Nix looks for files, and allows to have a stand-alone Nix environment, which only uses files in a specific directory, and doesn't interfere with the user environment.
synopsis:Define integer overflow in the Nix language as an error
issues:[10968]
prs:[11188]
---
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *usually* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it appears safe to turn into an error.
Some other overflows were fixed:
-`builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
-`nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
$ nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
The `build-hook` setting's default is less useful when using `libnixstore` as a library
prs:
- 11178
---
*This is an obscure issue that only affects usage of the `libnixstore` library outside of the Nix executable.*
As part the ongoing [rewrite of the build system](https://github.com/NixOS/nix/issues/2503) to use [Meson](https://mesonbuild.com/), we are also switching to packaging individual Nix components separately (and building them in separate derivations).
This means that when building `libnixstore` we do not know where the Nix binaries will be installed --- `libnixstore` doesn't know about downstream consumers like the Nix binaries at all.
*This is also unrelated to the _`post`_-`build-hook`*, which is often used for pushing to a cache.*
This has a small adverse affect on remote building --- the `build-remote` executable that is specified from the [`build-hook`](@docroot@/command-ref/conf-file.md#conf-build-hook) setting will not be gotten from the (presumed) installation location, but instead looked up on the `PATH`.
This means that other applications linking `libnixstore` that wish to use remote building must arrange for the `nix` command to be on the PATH (or manually overriding `build-hook`) in order for that to work.
Long term we don't envision this being a downside, because we plan to [get rid of `build-remote` and the build hook setting entirely](https://github.com/NixOS/nix/issues/1221).
There is simply no need to add a second layer of remote-procedure-calling when we want to connect to a remote builder.
The build hook protocol did in principle support custom ways of remote building, but that can also be accomplished with a custom service for the ssh or daemon/ssh-ng protocols, or with a custom [store type](@docroot@/store/types/index.md) i.e. `Store` subclass. <!-- we normally don't mention classes, but consider that this release note is about a library use case -->
The Perl bindings no longer expose `getBinDir` either, since they libraries those bindings wrap no longer know the location of installed binaries as described above.
synopsis:wrap filesystem exceptions more correctly
issues:[]
prs:[11378]
---
With the switch to `std::filesystem` in different places, Nix started to throw `std::filesystem::filesystem_error` in many places instead of its own exceptions.
This lead to no longer generating error traces, for example when listing a non-existing directory, and can also lead to crashes inside the Nix REPL.
This version catches these types of exception correctly and wrap them into Nix's own exeception type.
Nix now has a setting `fsync-store-paths` that ensures that new store paths are durably written to disk before they are registered as "valid" in Nix's database. This can prevent Nix store corruption if the system crashes or there is a power loss. This setting defaults to `false`.
synopsis:Show package descriptions with `nix flake show`
issues:[10977]
prs:[10980]
---
`nix flake show` will now display a package's `meta.description` if it exists. If the description does not fit in the terminal it will be truncated to fit the terminal width. If the size of the terminal width is unknown the description will be capped at 80 characters.
Nix will no longer attempt to substitute the source code of flakes from a binary cache. This functionality was broken because it could lead to different evaluation results depending on whether the flake was available in the binary cache, or even depending on whether the flake was already in the local store.
A local Nix installation can forward Nix builds to other machines,
this allows multiple builds to be performed in parallel.
Remote builds also allow Nix to perform multi-platform builds in a
semi-transparent way. For example, if you perform a build for a
`x86_64-darwin` on an `i686-linux` machine, Nix can automatically
forward the build to a `x86_64-darwin` machine, if one is available.
## Requirements
For a local machine to forward a build to a remote machine, the remote machine must:
- Have Nix installed
- Be running an SSH server, e.g. `sshd`
- Be accessible via SSH from the local machine over the network
- Have the local machine's public SSH key in `/etc/ssh/authorized_keys.d/<username>`
- Have the username of the SSH user in the `trusted-users` setting in `nix.conf`
## Testing
To test connecting to a remote [Nix instance] (in this case `mac`), run:
```console
nix store info --store ssh://username@mac
```
To specify an SSH identity file as part of the remote store URI add a
query parameter, e.g.
```console
nix store info --store ssh://username@mac?ssh-key=/home/alice/my-key
```
Since builds should be non-interactive, the key should not have a
passphrase. Alternatively, you can load identities ahead of time into
`ssh-agent` or `gpg-agent`.
In a multi-user installation (default), builds are executed by the Nix
Daemon. The Nix Daemon cannot prompt for a passphrase via the terminal
or `ssh-agent`, so the SSH key must not have a passphrase.
In addition, the Nix Daemon's user (typically root) needs to have SSH
access to the remote builder.
Access can be verified by running `sudo su`, and then validating SSH
access, e.g. by running `ssh mac`. SSH identity files for root users
are usually stored in `/root/.ssh/` (Linux) or `/var/root/.ssh` (MacOS).
If you get the error
```console
bash: nix: command not found
error: cannot connect to 'mac'
```
then you need to ensure that the `PATH` of non-interactive login shells
contains Nix.
The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file.
For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine:
@@ -53,6 +53,11 @@ This command has the following operations:
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
> **Note**
>
> Downloaded channel contents are cached.
> Use `--tarball-ttl` or the [`tarball-ttl` configuration option](@docroot@/command-ref/conf-file.md#conf-tarball-ttl) to change the validity period of cached downloads.
-`--list-generations`
Prints a list of all the current existing generations for the
@@ -36,7 +36,7 @@ Instead, it looks in a few locations, and acts on all profiles it finds there:
>
> Not stable; subject to change
>
> Do not rely on this functionality; it just exists for migration purposes and is may change in the future.
> Do not rely on this functionality; it just exists for migration purposes and may change in the future.
> These deprecated paths remain a private implementation detail of Nix.
`$NIX_STATE_DIR/profiles` and `$NIX_STATE_DIR/profiles/per-user`.
@@ -62,6 +62,15 @@ These options are for deleting old [profiles] prior to deleting unreachable [sto
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
@@ -62,7 +62,7 @@ These pages can be viewed offline:
Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of *arguments* that specify the packages on which to operate.
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-names):
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name):
-`name`: Everything up to but not including the first dash (`-`) that is *not* followed by a letter.
-`version`: The rest, excluding the separating dash.
@@ -21,11 +22,11 @@ It is based on the current generation of the active [profile](@docroot@/command-
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
- By default, *args* is a set of names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
@@ -61,11 +62,15 @@ The arguments *args* map to store paths in a number of possible ways:
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If `--priority` *priority* is given, the priority of the derivations being installed is set to *priority*.
This can be used to override the priority of the derivations being installed.
This is useful if *args* are [store paths], which don't have any priority information.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] that point to [store derivations][store derivation], then those store derivations are [realised], and the resulting output paths are installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
- If *args* are [store paths] that do not point to store derivations, then these are [realised] and installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [store derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Example:
@@ -117,6 +122,8 @@ The arguments *args* map to store paths in a number of possible ways:
Loaded 100 of 1766 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.