Compare commits

..
1050 Commits
Author SHA1 Message Date
Jörg Thalheim c55a7e1423 Merge pull request #13744 from NixOS/mergify/bp/2.24-maintenance/pr-13741
libexpr: Canonicalize TOML timestamps for toml11 > 4.0 (backport #13741)
2025-08-29 07:36:10 +02:00
Sergei Zimmerman ae47fd6886 libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339

(cherry picked from commit dc769d72cb)
2025-08-13 23:13:08 +03:00
Sergei Zimmerman b78b79349e libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.

(cherry picked from commit df4e55ffc1)
2025-08-13 22:39:58 +03:00
Sergei Zimmerman 79b932277a libexpr: Format primops/fromTOML.cc 2025-08-13 22:37:03 +03:00
Sergei Zimmerman d52210fae3 tests/functional/lang: Add more tests for TOML timestamps
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.

(cherry picked from commit 7ed0229d1a)
2025-08-13 19:22:48 +00:00
Eelco Dolstra c2914c0d58 Bump version 2025-06-24 19:04:04 +02:00
Eelco Dolstra 148585470c Merge remote-tracking branch 'cve/fod-cves-2.24' into 2.24-maintenance 2025-06-24 16:08:50 +02:00
mergify[bot] ed56b2741d Merge pull request #13389 from NixOS/mergify/bp/2.24-maintenance/pr-13348
Make the repl test more robust (backport #13348)
2025-06-22 19:34:41 +00:00
Eelco Dolstra bcffa0ce59 Make the repl test more robust
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:

  nix-functional-tests> grep: repl_output: No such file or directory
  nix-functional-tests> +(repl.sh:174) cat repl_output

This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.

(cherry picked from commit 9eb46e9cc0)
2025-06-22 19:08:35 +00:00
Eelco Dolstra b0fab9f90b Fixes for GHSA-g948-229j-48j3
Squashed commit of the following:

commit 04fff3a637d455cbb1d75937a235950e43008db9
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:30:32 2025 +0200

    Chown structured attr files safely

commit 5417ad445e414c649d0cfc71a05661c7bf8f3ef5
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:14:04 2025 +0200

    Replace 'bool sync' with an enum for clarity

    And drop writeFileAndSync().

commit 7ae0141f328d8e8e1094be24665789c05f974ba6
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:35:28 2025 +0200

    Drop guessOrInventPathFromFD()

    No need to do hacky stuff like that when we already know the original path.

commit 45b05098bd019da7c57cd4227a89bfd0fa65bb08
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:15:58 2025 +0200

    Tweak comment

commit 0af15b31209d1b7ec8addfae9a1a6b60d8f35848
Author: Raito Bezarius <raito@lix.systems>
Date:   Thu Mar 27 12:22:26 2025 +0100

    libstore: ensure that temporary directory is always 0o000 before deletion

    In the case the deletion fails, we should ensure that the temporary
    directory cannot be used for nefarious purposes.

    Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 2c20fa37b15cfa03ac6a1a6a47cdb2ed66c0827e
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 12:42:55 2025 +0100

    libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds

    When calling `_deletePath` with a parent file descriptor, `openat` is
    made effective by using relative paths to the directory file descriptor.

    To avoid the problem, the signature is changed to resist misuse with an
    assert in the prologue of the function.

    Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit d3c370bbcae48bb825ce19fd0f73bb4eefd2c9ea
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:07:47 2025 +0100

    libstore: ensure that `passAsFile` is created in the original temp dir

    This ensures that `passAsFile` data is created inside the expected
    temporary build directory by `openat()` from the parent directory file
    descriptor.

    This avoids a TOCTOU which is part of the attack chain of CVE-????.

    Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 45d3598724f932d024ef6bc2ffb00c1bb90e6018
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:06:03 2025 +0100

    libutil: writeFile variant for file descriptors

    `writeFile` lose its `sync` boolean flag to make things simpler.

    A new `writeFileAndSync` function is created and all call sites are
    converted to it.

    Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 732bd9b98cabf4aaf95a01fd318923de303f9996
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:05:34 2025 +0100

    libstore: chown to builder variant for file descriptors

    We use it immediately for the build temporary directory.

    Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 962c65f8dcd5570dd92c72370a862c7b38942e0d
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:59 2025 +0100

    libstore: open build directory as a dirfd as well

    We now keep around a proper AutoCloseFD around the temporary directory
    which we plan to use for openat operations and avoiding the build
    directory being swapped out while we are doing something else.

    Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit c9b42462b75b5a37ee6564c2b53cff186c8323da
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:12 2025 +0100

    libutil: guess or invent a path from file descriptors

    This is useful for certain error recovery paths (no pun intended) that
    does not thread through the original path name.

    Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
    Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 16:55:43 +02:00
Jörg Thalheim 7d15dbf9d5 Merge pull request #12801 from pascalj/2.24-maintenance
Exclude 'dir' from the FlakeRef's URL
2025-05-18 23:35:53 +02:00
Pascal Jungblut fa2a3c6de8 Merge branch '2.24-maintenance' into 2.24-maintenance 2025-05-14 22:58:17 +02:00
mergify[bot] 3e2c0680b6 Merge pull request #13091 from NixOS/mergify/bp/2.24-maintenance/pr-13086
libutil: Fix invalid boost format string in infinite symlink recursion error (backport #13086)
2025-04-25 10:11:28 +00:00
Sergei Zimmerman 28c2c78716 libutil: Fix invalid boost format string in infinite symlink recursion error
Found while working on an automated migration to `std::format`.

(cherry picked from commit bfb357c40b)
2025-04-25 09:20:56 +00:00
Pascal Jungblut 72bf563191 Exclude 'dir' from the FlakeRef's URL
This fixes an issue where nix would try to check out invalid URLs,
because it would pass 'dir' to the HTTP endpoint.

For later versions this was fixed in
b2be6fed86. This is a backport of just the
relevant part.

See #12417
2025-04-15 13:34:48 +02:00
mergify[bot] 57190f9a39 Merge pull request #12988 from NixOS/mergify/bp/2.24-maintenance/pr-12986
Fix another machine config parsing bug (backport #12986)
2025-04-09 21:41:41 +00:00
mergify[bot] 89ea1a8758 Merge pull request #12987 from NixOS/mergify/bp/2.24-maintenance/pr-12984
Fix `;` and `#` bug in machine file parsing (backport #12984)
2025-04-09 20:47:38 +00:00
John Ericson 142f94b7e1 Fix another machine config parsing bug
We were ignorning the result of `trim`, and after my last change we were
also trimmming too early.

(cherry picked from commit b74b0f4e1c)
2025-04-09 16:20:07 -04:00
John Ericson 0c6a207916 Fix ; and # bug in machine file parsing
Comments go to the end of the line, not merely the next ; *or* \n. Fix
by splitting on `;` *within* lines, and test.

(cherry picked from commit f8b13cce19)
2025-04-09 20:06:25 +00:00
Eelco Dolstra a568447137 Merge pull request #12952 from NixOS/mergify/bp/2.24-maintenance/pr-12951
tests/functional/repl: fix race condition (backport #12951)
2025-04-07 19:51:27 +02:00
Jörg Thalheim bc88d09a08 tests/functional/repl: fix race condition
the sleep 1 is not enough in some circumstances. Switching to a fifo
helps.

(cherry picked from commit 1de951d31d)
2025-04-07 12:19:00 +00:00
Eelco Dolstra ccea5cce42 Bump version 2025-04-07 11:28:46 +02:00
mergify[bot] e949cdbc05 Merge pull request #12946 from NixOS/mergify/bp/2.24-maintenance/pr-12944
maintainers/release-notes: Let it fail (backport #12944)
2025-04-07 09:11:03 +00:00
Robert Hensing de18f5ac2d maintainers/release-notes: Let it fail
Fail when a command fails.

Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.

(cherry picked from commit da36c34db7)
2025-04-07 08:52:30 +00:00
Jörg Thalheim adbf23e818 Merge pull request #12922 from NixOS/mergify/bp/2.24-maintenance/pr-12911
create cache entry for paths already in the nix store (backport #12911)
2025-04-04 16:32:47 +02:00
Jörg Thalheim 2a4e2ecd89 create cache entry for paths already in the nix store
This allows path:/nix/store/* paths to not be copied twice to the nix
store.

(cherry picked from commit 61c6210dbf)
2025-04-04 16:01:17 +02:00
mergify[bot] b47bd02a6a Merge pull request #12854 from NixOS/mergify/bp/2.24-maintenance/pr-12765
Add various clang tidy fixes (backport #12765)
2025-04-01 14:35:45 +00:00
Jörg Thalheim 8abb67b842 libstore/daemon: make sure monitor is not considered "unused"
(cherry picked from commit 5c3682d7a1)
2025-04-01 13:37:38 +00:00
Jörg Thalheim 97f9f8e909 test/ca-fd-leak: fix clang-tidy lints
(cherry picked from commit b050db951b)
2025-04-01 13:37:37 +00:00
Jörg Thalheim 7ef04ba6dd git/getStringUntil: fix uninitialized stack variable
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.

(cherry picked from commit 7e540059a3)
2025-04-01 13:37:37 +00:00
Jörg Thalheim bc6e78402c libstore/local-store: fix linting warning about unused variable
(cherry picked from commit 05082ea1c5)
2025-04-01 13:37:37 +00:00
mergify[bot] 4687dc99f7 Merge pull request #12840 from NixOS/mergify/bp/2.24-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:47:08 +00:00
Thomas Miedema 8d99b6578a nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:47:25 +00:00
John Ericson e1dad7daa5 Merge pull request #12795 from NixOS/mergify/bp/2.24-maintenance/pr-12794
Fix windows build (backport #12794)
2025-03-28 21:54:39 -04:00
John Ericson d92b509bda Fix windows build
PR #12767 accidentally broke it.

(cherry picked from commit 99041b4d84)
2025-03-28 21:07:08 -04:00
mergify[bot] 8adc1f3011 Merge pull request #12774 from NixOS/mergify/bp/2.24-maintenance/pr-12767
use createDirs consistently everywhere (backport #12767)
2025-03-28 16:48:21 +00:00
Jörg Thalheim 076fa01a84 use createDirs consistently everywhere
(cherry picked from commit a8217f2642)
2025-03-28 15:54:56 +00:00
Eelco Dolstra 61f8fd3e9c Bump version 2025-03-24 19:35:50 +01:00
Eelco Dolstra 5ce8d922fd Merge pull request #12737 from NixOS/mergify/bp/2.24-maintenance/pr-12736
`MonitorFdHup::~MonitorFdHup`: use proper close method instead of lib… (backport #12736)
2025-03-24 14:00:41 +01:00
Jörg Thalheim 333479e6be MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)
2025-03-24 12:16:00 +00:00
John Ericson 71ab003a0c Merge pull request #12731 from NixOS/mergify/bp/2.24-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-23 20:39:08 -04:00
John Ericson 3fccc71492 MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-23 23:54:09 +00:00
Jade Lovelace 729d18fd4c daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-23 23:54:08 +00:00
Félix Baylac Jacqué 699f27c4ea MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-23 23:54:08 +00:00
John Ericson 2d792a1867 MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-23 23:54:08 +00:00
John Ericson da213e8ee5 MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-23 23:54:08 +00:00
Jörg Thalheim 787a14b173 MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-23 23:54:07 +00:00
John Ericson 01aace706a monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-23 23:54:07 +00:00
mergify[bot] 74a0ab3eea Merge pull request #12683 from NixOS/mergify/bp/2.24-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 21:07:12 +00:00
Pierre-Etienne Meunier 589da451e4 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:20:48 +00:00
mergify[bot] 56e113d67a Merge pull request #12653 from NixOS/mergify/bp/2.24-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-14 14:11:27 +00:00
Sergei Zimmerman aaf86cc0d5 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 6faf66d2f7 libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 3ab83f507c libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-14 13:22:43 +00:00
Sergei Zimmerman 0033cf4270 {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:43 +00:00
mergify[bot] 79828c12e0 Merge pull request #12535 from NixOS/mergify/bp/2.24-maintenance/pr-12391
Only try to chmod /nix/var/nix/profiles/per-user when necessary (backport #12391)
2025-02-20 16:55:38 +00:00
Sandro Jäckel 2a795bb793 Only try to chmod /nix/var/nix/profiles/per-user when necessary
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit dcbf4dcc09)
2025-02-20 16:11:26 +00:00
mergify[bot] 9238c892d3 Merge pull request #12528 from NixOS/mergify/bp/2.24-maintenance/pr-12439
Set FD_CLOEXEC on sockets created by curl (backport #12439)
2025-02-19 20:30:16 +00:00
MaxHearnden 5862686ce2 Set FD_CLOEXEC on sockets created by curl
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

(cherry picked from commit 12d2527276)
2025-02-19 19:47:40 +00:00
John Ericson feadb3178d Merge pull request #12508 from NixOS/mergify/bp/2.24-maintenance/pr-12499
startDaemon(): Detect if the daemon crashes before creating the socket (backport #12499)
2025-02-18 12:21:39 -05:00
Eelco Dolstra be35515ef7 startDaemon(): Detect if the daemon crashes before creating the socket
This avoids timeouts like those seen in
https://github.com/NixOS/nix/actions/runs/13376958708/job/37358120348?pr=6962.

(cherry picked from commit 11c42cb2e1)
2025-02-18 16:38:55 +00:00
Eelco Dolstra 71f93012fd Merge pull request #12487 from NixOS/mergify/bp/2.24-maintenance/pr-12481
resolveLookupPathPath(): Fix caching of negative lookups (backport #12481)
2025-02-18 17:27:52 +01:00
Eelco Dolstra b706642133 resolveLookupPathPath(): Fix caching of negative lookups
This avoids spamming in case the missing search path entry does not
exist (#12480).

(cherry picked from commit df08e1e204)
2025-02-18 16:58:07 +01:00
Eelco Dolstra 658392e029 getDefaultNixPath(): Don't add symlinks if the target doesn't exist
(cherry picked from commit 8ac49ea5de)
2025-02-17 17:08:45 +00:00
mergify[bot] d3a52c2ef4 Merge pull request #12451 from NixOS/mergify/bp/2.24-maintenance/pr-12448
copyPathToStore(): Preserve symlinks (backport #12448)
2025-02-11 22:21:12 +00:00
Eelco Dolstra 3d6a9c7410 copyPathToStore(): Preserve symlinks
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

(cherry picked from commit 26b87e78b5)

# Conflicts:
#	tests/functional/meson.build
2025-02-11 21:51:20 +00:00
mergify[bot] 0f5f24223d Merge pull request #12434 from NixOS/mergify/bp/2.24-maintenance/pr-12373
pre-commit/check-merge-conflicts-2: fix use outside dev shell (backport #12373)
2025-02-10 14:37:20 +00:00
Robert Hensing fcea626e49 test: Fix shellcheck by giving git-hashing scripts shebangs
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.

It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
2025-02-10 15:09:36 +01:00
Robert Hensing c6e09073ea pre-commit/check-merge-conflicts-2: fix use outside dev shell
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.

`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
2025-02-10 15:08:54 +01:00
mergify[bot] b9e6e00fe1 Merge pull request #12357 from NixOS/mergify/bp/2.24-maintenance/pr-12356
Improve "illegal path references in fixed output derivation" error (backport #12356)
2025-01-28 09:37:45 +00:00
mergify[bot] 5fb6551217 Merge pull request #12365 from NixOS/mergify/bp/2.24-maintenance/pr-12362
refactor: Extract EvalState::realiseString (backport #12362)
2025-01-28 08:51:24 +00:00
Robert Hensing 0f19bfe811 Fix conflict 2025-01-28 09:20:56 +01:00
Robert Hensing 865239c732 refactor: Extract EvalState::realiseString
(cherry picked from commit 7465fbe926)

# Conflicts:
#	src/libexpr/primops.cc
2025-01-27 15:50:52 +00:00
Ben Millwood 9c5482167f Improve "illegal path references in fixed output derivation" error
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.

(cherry picked from commit 4e5d1b281e)
2025-01-26 19:38:38 +00:00
Robert Hensing a28791a91e Merge pull request #12352 from NixOS/nixfmt-on-2.24
Apply nixfmt (backport 2.24)
2025-01-24 22:01:53 +01:00
Robert Hensing ae0bb7d71c test: Fix shifted source positions after formatting 2025-01-24 21:32:07 +01:00
Robert Hensing 2f1b70a529 Format .nix files
This does not include any automation for the release branch, but
is based on the configuration of https://github.com/NixOS/nix/pull/12349

    pre-commit run -a nixfmt-rfc-style
2025-01-24 20:30:07 +01:00
Eelco Dolstra 42b22fe3de Actually bump the version
Fixes 532092a812.
2025-01-24 16:47:16 +01:00
mergify[bot] 62e7144ba7 Merge pull request #12344 from NixOS/mergify/bp/2.24-maintenance/pr-12336
libstore: Fix progress bars (backport #12336)
2025-01-24 11:52:17 +00:00
Philipp Otterbein a106e33bab libstore: fix progress bars
(cherry picked from commit be97dc1efc)
2025-01-24 11:10:16 +00:00
Eelco Dolstra 7616bae342 Merge pull request #12332 from NixOS/mergify/bp/2.24-maintenance/pr-12331
GitRepo::fetch(): Ignore $GIT_DIR (backport #12331)
2025-01-22 22:49:18 +01:00
Eelco Dolstra 1a402db046 GitRepo::fetch(): Ignore $GIT_DIR
Fixes #12325.

(cherry picked from commit 41983dba8f)
2025-01-22 22:10:09 +01:00
Eelco Dolstra 4511c47ab7 GitRepo::fetch(): Cleanup
(cherry picked from commit bd10b859f7)

# Conflicts:
#	src/libfetchers/git-utils.cc
2025-01-22 22:10:09 +01:00
Eelco Dolstra 532092a812 Bump version 2025-01-22 11:20:50 +01:00
Robert Hensing f22359ba1a Merge pull request #12287 from bryango/2.24-maintenance
Backport git+file:./ fixes to 2.24 (#12107 + #12277)
2025-01-21 22:36:06 +01:00
mergify[bot] f366beaa78 Merge pull request #12318 from NixOS/mergify/bp/2.24-maintenance/pr-12315
tests: Wait for network *online* targets and/or multi-user targets (more) (backport #12315)
2025-01-20 18:29:53 +00:00
Robert Hensing 011084b127 tests: Wait for network *online* targets
More prevalent than I thought in 15073e86a

See also https://github.com/NixOS/nix/actions/runs/12872412321/job/35887830320?pr=12310
which is a failed github-flakes test without "Network is Online"

(cherry picked from commit 6f0bdd9ae7)

# Conflicts:
#	tests/nixos/nix-docker.nix
2025-01-20 18:16:42 +00:00
Eelco Dolstra d8872fce2c Merge pull request #12311 from NixOS/mergify/bp/2.24-maintenance/pr-12302
tests: Wait for network *online* and multi-user targets (backport #12302)
2025-01-20 18:27:50 +01:00
Robert Hensing 12c90a0ec0 tests: Wait for network *online* and multi-user targets
This should help prevent some test stalls.

By default, multi-user.target does not imply that the network is fully up.

(cherry picked from commit 15073e86a8)
2025-01-20 16:40:04 +00:00
Eelco Dolstra 296769742b Merge pull request #12307 from NixOS/mergify/bp/2.24-maintenance/pr-12294
processGraph(): Don't throw ThreadPoolShutDown if there is an exception (backport #12294)
2025-01-20 17:20:40 +01:00
Eelco Dolstra d31be32c4b Merge pull request #12299 from NixOS/mergify/bp/2.24-maintenance/pr-12296
addMultipleToStore(): Move pathsToCopy (backport #12296)
2025-01-20 16:25:17 +01:00
Eelco Dolstra a1b4bc7292 processGraph(): Don't throw ThreadPoolShutDown if there is an exception
Fixes

  $ nix copy --derivation --to /tmp/nix /nix/store/...
  error: cannot enqueue a work item while the thread pool is shutting down

The ThreadPoolShutDown exception was hiding the reason for the thread
pool shut down, e.g.

  error: cannot add path '/nix/store/03sl46khd8gmjpsad7223m32ma965vy9-fix-static.patch' because it lacks a signature by a trusted key

(cherry picked from commit a8c69cc907)
2025-01-20 15:24:44 +00:00
Eelco Dolstra ff91a777a6 Add comment
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 2669e4ac4f)
2025-01-20 14:17:51 +00:00
Eelco Dolstra b4418c5c1d addMultipleToStore(): Move pathsToCopy
This allows RemoteStore::addMultipleToStore() to free the Source
objects early (and in particular the associated sinkToSource()
buffers). This should fix #7359. For example, memory consumption of

  nix copy --derivation --to ssh-ng://localhost?remote-store=/tmp/nix --derivation --no-check-sigs \
    /nix/store/4p9xmfgnvclqpii8pxqcwcvl9bxqy2xf-nixos-system-...drv

went from 353 MB to 74 MB.

(cherry picked from commit cc838e8181)
2025-01-20 14:17:51 +00:00
Eelco Dolstra 876d724061 Add link to tracking issue
(cherry picked from commit 3197c19a31)
2025-01-19 15:41:06 +08:00
Eelco Dolstra f27e263f89 Warn against the use of relative 'git+file:' flake inputs
(cherry picked from commit 12e14956e2)
2025-01-19 15:38:05 +08:00
Eelco Dolstra d944cb7f58 Use isAbsolute()
(cherry picked from commit ff9d886f3c)
2025-01-19 14:58:36 +08:00
Robert Hensing b2acf4ab97 Clarify cd call in tests/functional/flakes/flakes.sh
(cherry picked from commit d9a50c0af2)
2025-01-19 14:56:13 +08:00
Bryan Lai d4f0e8f4e3 tests/flake-in-submodule: git+file:./* input
(cherry picked from commit 37ac18d1d9)
2025-01-19 14:56:11 +08:00
Bryan Lai 84e3f4ad79 tests/flakes: check git+file:./${submodule} protocol
Relative, local git repo used to work (for submodules), but it
fails after 3e0129ce3b.

This commit adds a test to prevent such failure in the future.

(cherry picked from commit 9d088fa502)
2025-01-19 14:55:51 +08:00
Bryan Lai 6e44bc2a89 fetchers/git: make path absolute for local repo
(cherry picked from commit 96bd9bad2f)
2025-01-19 14:55:49 +08:00
mergify[bot] 2009cc137a Merge pull request #12278 from NixOS/mergify/bp/2.24-maintenance/pr-12258
sequoia-nixbld-user-migration: nail down PATH (backport #12258)
2025-01-16 16:49:44 +00:00
Travis A. Everett 0cefe354c8 sequoia-nixbld-user-migration: nail down PATH
Fixes a user report of trouble with toybox grep and avoids
potential of same basic issue with other utils.

(cherry picked from commit 6a874c2865)

# Conflicts:
#	scripts/sequoia-nixbld-user-migration.sh
2025-01-16 16:43:56 +00:00
mergify[bot] c6f504d827 Merge pull request #12239 from NixOS/mergify/bp/2.24-maintenance/pr-12237
Reject merge conflicts (backport #12237)
2025-01-15 19:40:36 +00:00
mergify[bot] 6e718c7c5c Merge pull request #12259 from NixOS/mergify/bp/2.24-maintenance/pr-11922
gc: resume GC after a pathinuse error (backport #11922)
2025-01-15 16:42:29 +00:00
Picnoir f68cdf1801 gc: resume GC after a pathinuse error
First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.

This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.

Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.

(cherry picked from commit ced8d311a5)
2025-01-15 15:59:24 +00:00
Robert Hensing 1422832f7c Resolve conflict
Conflict introduced in 9ee4a85a99
2025-01-15 07:10:34 +01:00
Jörg Thalheim 7012e1ff8c Merge pull request #12208 from NixOS/mergify/bp/2.24-maintenance/pr-11305
[2.24] Document function application operator (backport #11305)
2025-01-13 09:56:16 +01:00
Jörg Thalheim e81be91738 Merge pull request #12242 from NixOS/mergify/bp/2.24-maintenance/pr-12235
libutil: thread-pool: ensure threads finished on error (backport #12235)
2025-01-13 09:49:15 +01:00
Robert Hensing 9cc5786201 doc: Edit language/operators
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 06b18cff20)
2025-01-13 09:07:22 +01:00
Robert Hensing acd43b2239 doc: Document function application operator
(cherry picked from commit e225b63062)
2025-01-13 09:07:22 +01:00
Dominique Martinet 3cd8c1e53d libutil: thread-pool: ensure threads finished on error
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.

Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.

It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.

By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.

[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]

Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
(cherry picked from commit afac093b34)
2025-01-13 08:03:02 +00:00
Robert Hensing 10438e6cfd Reject merge conflicts
They're usually found by other checks, but docs would remain
susceptible.

(cherry picked from commit 29a1a21ce4)
2025-01-12 21:22:43 +00:00
mergify[bot] c803fe0785 Merge pull request #12197 from NixOS/mergify/bp/2.24-maintenance/pr-11541
nix-profile.sh.in: fix envvar condition (backport #11541)
2025-01-12 13:09:04 +00:00
mergify[bot] 9a85784be2 Merge pull request #12209 from NixOS/mergify/bp/2.24-maintenance/pr-11294
parser-state: fix attribute merging (backport #11294)
2025-01-12 12:36:06 +00:00
Robert Hensing a69d32d194 Resolve conflict
Some of the code before https://github.com/NixOS/nix/pull/11351 did not
have the problem. What remains wasn't all that problematic, but is good
to have anyway.
2025-01-12 13:15:03 +01:00
Ryan Hendrickson e0e50fb01f parser-state: fix attribute merging
(cherry picked from commit 8034589d7e)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson 6ae5aa7fa0 parser.y: move attr doc setting into addAttr
(cherry picked from commit b0a8430e85)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson cd169f14f0 parser.y: use names where I'll be refactoring
(cherry picked from commit 429a197d24)
2025-01-12 13:07:28 +01:00
Jörg Thalheim e3715053b1 Merge pull request #12183 from NixOS/mergify/bp/2.24-maintenance/pr-12051
Skip tests on systems with restricted usernamespaces (backport #12051)
2025-01-11 21:42:36 +01:00
Jörg Thalheim ada649ad7f functional-tests: skip tests if the kernel restricts unprivileged user namespaces
Update tests/functional/common/functions.sh

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: bryango <bryango@users.noreply.github.com>
(cherry picked from commit da7f7ba810)
2025-01-11 20:59:50 +01:00
mergify[bot] e1e219a299 Merge pull request #12211 from NixOS/mergify/bp/2.24-maintenance/pr-11240
fix: bash mangles flake ref completion (backport #11240)
2025-01-10 23:43:24 +00:00
mergify[bot] a658bafce7 Merge pull request #12207 from NixOS/mergify/bp/2.24-maintenance/pr-11343
withFramedSink(): Don't use a thread to monitor the other side (backport #11343)
2025-01-10 23:43:21 +00:00
mergify[bot] e76757083a Merge pull request #12206 from NixOS/mergify/bp/2.24-maintenance/pr-11381
Fix few duplicate word occurrences (backport #11381)
2025-01-10 23:43:18 +00:00
mergify[bot] c9dd4a7c28 Merge pull request #12205 from NixOS/mergify/bp/2.24-maintenance/pr-11407
add removal of ~root/.cache/nix to uninstall instructions (backport #11407)
2025-01-10 23:43:16 +00:00
mergify[bot] 01cb05a265 Merge pull request #12203 from NixOS/mergify/bp/2.24-maintenance/pr-11424
Pull fut.get() out of the lock (backport #11424)
2025-01-10 22:12:00 +00:00
mergify[bot] 9328fc7529 Merge pull request #12204 from NixOS/mergify/bp/2.24-maintenance/pr-11371
Fix diskutil deleteVolume command in Uninstall guide (backport #11371)
2025-01-10 22:11:57 +00:00
mergify[bot] 92e5f64068 Merge pull request #12202 from NixOS/mergify/bp/2.24-maintenance/pr-11514
nix derivation add: Remove reference to "installable" (backport #11514)
2025-01-10 22:11:54 +00:00
mergify[bot] 283249a972 Merge pull request #12201 from NixOS/mergify/bp/2.24-maintenance/pr-11611
doc, src: Fix various spelling typos (backport #11611)
2025-01-10 22:11:52 +00:00
mergify[bot] 4d2cc07b48 Merge pull request #12200 from NixOS/mergify/bp/2.24-maintenance/pr-11613
doc/nix fmt: Do not document behavior we do not control (backport #11613)
2025-01-10 22:11:49 +00:00
mergify[bot] 5c0f2b744a Merge pull request #12199 from NixOS/mergify/bp/2.24-maintenance/pr-11526
docs: Provide an example of updating multiple inputs (backport #11526)
2025-01-10 20:51:42 +00:00
mergify[bot] 0e95d8b467 Merge pull request #12198 from NixOS/mergify/bp/2.24-maintenance/pr-11530
add description + example for nix flake archive (backport #11530)
2025-01-10 20:51:38 +00:00
mergify[bot] 53bc8b543c Merge pull request #12196 from NixOS/mergify/bp/2.24-maintenance/pr-11621
Improve documentation of `nix flake lock` (backport #11621)
2025-01-10 20:51:35 +00:00
mergify[bot] 86da8bbbf3 Merge pull request #12194 from NixOS/mergify/bp/2.24-maintenance/pr-11684
libstore: Make our sandbox pivot_root directory accessible to ourself (backport #11684)
2025-01-10 20:51:33 +00:00
mergify[bot] fcdca4f8d1 Merge pull request #12195 from NixOS/mergify/bp/2.24-maintenance/pr-11629
doc/manual: Fix some broken fragments (backport #11629)
2025-01-10 20:51:30 +00:00
mergify[bot] f393260a5d Merge pull request #12192 from NixOS/mergify/bp/2.24-maintenance/pr-11717
docs: clarify syntax for escaping dollar curlies (backport #11717)
2025-01-10 19:35:02 +00:00
mergify[bot] d3cb938e07 Merge pull request #12191 from NixOS/mergify/bp/2.24-maintenance/pr-11723
doc: Explain why tryEval does not return the message (backport #11723)
2025-01-10 19:34:59 +00:00
mergify[bot] 51e41edbef Merge pull request #12190 from NixOS/mergify/bp/2.24-maintenance/pr-11813
fix(libstore-tests): remove use-after-free bug for `StringSource` (backport #11813)
2025-01-10 19:34:56 +00:00
mergify[bot] daf526f19d Merge pull request #12188 from NixOS/mergify/bp/2.24-maintenance/pr-11853
fix(libmain/common-args): do not exceed maximum allowed verbosity (backport #11853)
2025-01-10 19:34:54 +00:00
mergify[bot] d6217bc691 Merge pull request #12187 from NixOS/mergify/bp/2.24-maintenance/pr-11981
docker: Fix command "nix profile install", Don't require --impure (fix for 2.25) (backport #11981)
2025-01-10 19:34:51 +00:00
mergify[bot] 22c24a87a1 Merge pull request #12176 from NixOS/mergify/bp/2.24-maintenance/pr-12114
fix documentation of substring (backport #12114)
2025-01-10 17:21:50 +00:00
Eelco Dolstra f2bb7e043b Merge pull request #12185 from NixOS/mergify/bp/2.24-maintenance/pr-12016
Disable suid and atime on the /nix mount point on Darwin (backport #12016)
2025-01-10 15:16:44 +01:00
Eelco Dolstra 9a7196ce62 Merge pull request #12180 from NixOS/mergify/bp/2.24-maintenance/pr-12091
libstore: fixup unformatted uri when S3 getObject fails (backport #12091)
2025-01-10 15:11:08 +01:00
Eelco Dolstra 7a2b1cbd71 Merge pull request #12173 from NixOS/mergify/bp/2.24-maintenance/pr-12127
toJSON: re-throw serialization exception (backport #12127)
2025-01-10 13:32:42 +01:00
mergify[bot] 534473e90b Merge pull request #12170 from NixOS/mergify/bp/2.24-maintenance/pr-12155
Document `--max-freed` for `nix-collect-garbage` (backport #12155)
2025-01-10 11:05:01 +00:00
0x5a4 ce5c0c64af fix: bash mangles flake ref completion
(cherry picked from commit 088fa815d3)
2025-01-10 10:42:07 +00:00
Eelco Dolstra 931eb85f50 Add FIXME
(cherry picked from commit fac756fed4)
2025-01-10 10:36:06 +00:00
Eelco Dolstra a9020404cf select() -> poll() for Windows compat
(cherry picked from commit 270c8469d7)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 77a71c518f Update src/libutil/serialise.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit ce7cf4a2d3)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 36f3fb72e9 withFramedSink(): Don't use a thread to monitor the other side
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.

  nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298

would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.

So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).

This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).

(cherry picked from commit 39daa4a0d3)
2025-01-10 10:36:05 +00:00
Jan Hrcek 5099768970 Fix few duplicate word occurrences (#11381)
(cherry picked from commit cb4b9be458)
2025-01-10 10:29:59 +00:00
Jeremy Kerfs 6f5684f6d1 add removal of ~root/.cache/nix to uninstall instructions (#11407)
(cherry picked from commit 4decd6f8b3)
2025-01-10 10:29:55 +00:00
Seggy Umboh d6488efeae Fix diskutil deleteVolume command in Uninstall guide (#11371)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 8e63dc4455)
2025-01-10 10:28:54 +00:00
Jeremy Kolb bc8611a29b Remote nullptr
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 8152c5c828)
2025-01-10 10:26:53 +00:00
Jeremy Kolb 1a21eb43af Pull fut.get() out of the lock
This is https://gerrit.lix.systems/c/lix/+/1462 by @jade_

see: https://git.lix.systems/lix-project/lix/issues/366
see: https://gerrit.lix.systems/c/lix/+/1462
(cherry picked from commit 46f6505865)
2025-01-10 10:26:53 +00:00
Eelco Dolstra bcf34e89c0 nix derivation add: Remove reference to "installable"
It doesn't operate on an installable, so don't mention it.

(cherry picked from commit 65f138f669)
2025-01-10 10:23:49 +00:00
Jordan Justen 0060a4ba5b doc, src: Fix various spelling typos
Signed-off-by: Jordan Justen <jljusten@gmail.com>
(cherry picked from commit 3b6cf350e7)
2025-01-10 10:22:51 +00:00
Robert Hensing 24e63b14e5 doc/nix fmt: Do not document behavior we do not control
This leads to confusion about what the command does.
E.g. https://github.com/NixOS/nix/issues/9359

- Move the description up

- Remove details about the individual formatters

(cherry picked from commit 8c4842de8c)
2025-01-10 10:22:50 +00:00
quatquatt 93e8e5b347 docs: Provide an example of updating multiple inputs with nix flake update
docs: Provide an example of updating multiple inputs with `nix flake update`
(cherry picked from commit 9e335ee3b9)
2025-01-10 10:22:45 +00:00
Jörg Thalheim 792099081c add description + example for nix flake archive
Update src/nix/flake-archive.md

Update src/nix/flake-archive.md

(cherry picked from commit f0a4f19087)
2025-01-10 10:21:45 +00:00
Noam Yorav-Raphael 971748ace9 nix-profile.sh.in: fix envvar condition
(cherry picked from commit 97fffd8765)

# Conflicts:
#	scripts/nix-profile.sh.in
2025-01-10 10:20:46 +00:00
Kevin Cox 0e421e7a35 Improve documentation of nix flake lock
The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.

(cherry picked from commit d2f627dda9)
2025-01-10 10:18:03 +00:00
Robert Hensing 9ee4a85a99 doc/manual: Fix some broken fragments
(cherry picked from commit 071ddbed57)

# Conflicts:
#	doc/manual/src/language/operators.md
2025-01-10 10:17:08 +00:00
Geoffrey Thomas 24542a7b96 libstore: Make our sandbox pivot_root directory accessible to ourself
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)

Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory.  The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.

(cherry picked from commit 5a794d9366)
2025-01-10 10:16:07 +00:00
Eman Resu 6304fa5cd1 docs: clarify syntax for escaping dollar curlies
(cherry picked from commit defff01a51)

# Conflicts:
#	doc/manual/source/language/string-literals.md
2025-01-10 10:14:55 +00:00
Robert Hensing 9c54fac5ff doc: Explain why tryEval does not return the message
(cherry picked from commit 90d257b771)
2025-01-10 10:12:37 +00:00
Sergei Zimmerman e863e6ab83 fix(libstore-tests): remove use-after-free bug for StringSource
Unfortunately `StringSource` class is very easy was very easy to misuse
because the ctor took a plain `std::string_view` which has a bad habit
of being implicitly convertible from an rvalue `std::string`. This lead
to unintentional use-after-free bugs.

This patch makes `StringSource` much harder to misuse by disabling the ctor
from a `std::string &&` (but `const std::string &` is ok).

Fix affected tests from libstore-tests.
Reformat those tests with clangd's range formatting since the diff is tiny
and it seems appropriate.

(cherry picked from commit 5bc8957c73)
2025-01-10 10:08:26 +00:00
Sergei Zimmerman dea80c4fec fix(libmain/common-args): do not exceed maximum allowed verbosity
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:

../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'

(cherry picked from commit b9f8c4af40)
2025-01-10 10:04:39 +00:00
h0nIg 26866eef0c docker: Fix command "nix profile install", Don't require --impure
(cherry picked from commit 61950c3375)
2025-01-10 09:58:36 +00:00
Graham Christensen 56332fda1c Disable suid and atime on the /nix mount point on Darwin
The Determinate Nix Installer has set nosuid and noatime in https://github.com/DeterminateSystems/nix-installer/pull/1338, and figured this perf and security improvement is worthy of upstreaming.

The /nix volume shouldn't have setuid binaries anyway, and filesystems seem to generally be noatime on macOS.
Further, the garbage collector doesn't use atime.

(cherry picked from commit 4137ead7a1)
2025-01-10 09:56:35 +00:00
Cole Helbling 4a7c7f12f5 libstore: fixup unformatted uri when S3 getObject fails
(cherry picked from commit b978fa8450)
2025-01-10 09:48:23 +00:00
Cole Helbling fed54f3550 tests/nixos/s3-binary-cache-store: disable default substituter so it runs faster
Since networking is disabled in these VMs, trying to talk to the default
cache.nixos.org slows the test down (since it can't resolve it).

(cherry picked from commit f0c1262d23)
2025-01-10 09:48:23 +00:00
Cole Helbling 6c1dfab347 tests/nixos/s3-binary-cache-store: test that "object does not exist" error message is properly formatted
(cherry picked from commit 535724fd79)
2025-01-10 09:48:22 +00:00
Philipp Otterbein 4b4b6377b2 fix documentation of substring
(cherry picked from commit 1e2cace5f1)
2025-01-10 09:44:14 +00:00
Philipp Otterbein 9936cb21ab toJSON: re-throw serialization exception
(cherry picked from commit 4a2310a3a0)
2025-01-10 09:42:14 +00:00
Domagoj Mišković 7c8276b799 Document --max-freed for nix-collect-garbage (#12155)
* Update nix-collect-garbage.md

Referencing issue at: https://github.com/NixOS/nix/issues/12132

Copied the description of `--max-freed` option from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44

(cherry picked from commit 5230d3ecc4)
2025-01-10 09:34:09 +00:00
mergify[bot] e0c8b0fc4f Merge pull request #12150 from NixOS/mergify/bp/2.24-maintenance/pr-12143
mergify: fix installer test name (backport #12143)
2025-01-07 19:42:11 +00:00
Jörg Thalheim 55dc2a9c9a mergify: fix installer test name
(cherry picked from commit 438a20427f)
2025-01-07 19:18:05 +00:00
Jörg Thalheim 112d0a7d85 Merge pull request #12144 from NixOS/mergify/bp/2.24-maintenance/pr-12046
CLI symlink fixes (backport #12046)
2025-01-07 08:06:58 +01:00
Jörg Thalheim b59b317e47 Merge pull request #12136 from NixOS/mergify/bp/2.24-maintenance/pr-11876
Filter OSC 8 correctly (backport #11876)
2025-01-07 07:22:37 +01:00
Jörg Thalheim 1eda38420f Merge pull request #11743 from NixOS/mergify/bp/2.24-maintenance/pr-11729
nix/tests: run test help.sh only if nix is built with documentation (backport #11729)
2025-01-07 07:21:51 +01:00
Robert Hensing caeec0404e test: Avoid regressing accidental use of weakly_canonical instead of makeParentCanonical
I'd messed up a rebase in my previous iteration, causing `weakly_canonical` to reappear,
but not trigger a test failure.

These two functions behave similarly when the argument is a path that points to a broken
symlink. `weakly_canonical` would not resolve it because the target doesn't exist, and
`makeParentCanonical` would not resolve it, because it never resolves the final path
element.
This new test case now also tests a valid symlink, "differentiating" the two.

(cherry picked from commit 4c74d679b6)
2025-01-07 07:20:55 +01:00
Jörg Thalheim f145dde10b makeParentCanonical: test case where parent is empty
(cherry picked from commit 5a5a86949a)

# Conflicts:
#	src/libutil-tests/file-system.cc
2025-01-07 07:20:55 +01:00
Robert Hensing 73dd3db039 test: Add hydraJobs.tests.functional_symlinked-home
(cherry picked from commit 628c11d237)
2025-01-07 07:20:55 +01:00
Robert Hensing e31d071793 fix: Resolve CLI parent symlinks before adding to store
Fixes https://github.com/NixOS/nix/issues/11941

(cherry picked from commit ddbbf53767)
2025-01-07 07:20:55 +01:00
Robert Hensing 42104d650a refactor: Don't re-construct SourcePath unnecessarily
(cherry picked from commit c0b64f3377)
2025-01-07 07:20:55 +01:00
Robert Hensing 129f8aaab2 doc: Document nix-store --add-fixed symlink behavior
Tested with

    nix run nix/2.3-maintenance#nix-store -- --add some_symlink
    nix run nix/2.3-maintenance#nix-store -- --add-fixed sha256 --recursive some_symlink

(cherry picked from commit 91e91f62fa)
2025-01-07 07:20:55 +01:00
Robert Hensing b9c30a9c87 fix: Handle symlinks and FIFOs in nix hash where possible
Fixes https://github.com/NixOS/nix/issues/11756
Fixes https://github.com/NixOS/nix/issues/11681

(cherry picked from commit 36563c69a4)
2025-01-07 07:20:55 +01:00
Robert Hensing fa3fd41063 Add makeParentCanonical()
(cherry picked from commit 69853c067c)
2025-01-07 07:20:55 +01:00
Marian Hammer e79d66ceef nix/tests: run test help.sh only if nix is built with documentation
tests/functional/help.sh calls nix-* commands with option --help
if nix is built without documentation the option --help throws an error
because the man page it wants to display is missing

(cherry picked from commit 85b0cd320a)
2025-01-07 05:55:13 +00:00
Jack Wilsdon e144f61ec0 Filter OSC 8 correctly
This allows Nix to use lowdown 1.2.0 which outputs OSC-8 links.

(cherry picked from commit 1301f8434d)
2025-01-07 05:33:19 +00:00
Jörg Thalheim 2c89c38fa1 Merge pull request #12074 from NixOS/mergify/bp/2.24-maintenance/pr-12059
installer: make sure we can always test the installer in ci and locally (backport #12059)
2025-01-07 00:27:16 +01:00
Robert Hensing 1a710e54dc .github/ci: Use fixed names
This lets us update "runs-on" without creating a mismatch with the
required checks that are configured for the repo in the github ui.
2025-01-07 00:04:32 +01:00
NAHO 365e0a9ab1 ci: lock macOS runner to macos-14 2025-01-07 00:04:32 +01:00
NAHO b3ea1e9b19 ci: update Ubuntu runner to ubuntu-24.04
Link: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
NAHO c78b818a0c ci: lock Ubuntu runner to ubuntu-22.04
Lock the Ubuntu runner to ubuntu-22.04 to avoid accidental updates [1]
and increase reproducibility.

[1]: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
Jörg Thalheim 3109250faf mergify: updated required tests
(cherry picked from commit df3eb4f3f6)
2025-01-07 00:04:32 +01:00
Jörg Thalheim 2fbd0a943c installer: make sure we can always test the installer in ci and locally
Just now there is a dependency on cachix, which means we cannot test
the installer in CI if forks do not have the necessary secrets set up.
We replace this with a simple http server that serves the installer and
can be both used in CI and locally.

(cherry picked from commit 6162e1220e)
2025-01-07 00:04:32 +01:00
mergify[bot] bda59dee55 Merge pull request #12077 from NixOS/mergify/bp/2.24-maintenance/pr-12045
EvalState::realiseContext(): Allow access to the entire closure (backport #12045)
2024-12-31 15:33:23 +00:00
Robert Hensing 000db53f8f tests/functional: Add $config_nix 2024-12-31 15:14:56 +01:00
mergify[bot] e40d0352b2 Merge pull request #12118 from NixOS/mergify/bp/2.24-maintenance/pr-12115
libcmd: update to support lowdown-1.4 API (backport #12115)
2024-12-30 20:26:40 +00:00
Sergei Trofimovich c530b13321 libcmd: update to support lowdown-1.4 API
Upstream change
https://github.com/kristapsdz/lowdown/commit/bab1d75079ac3866fc14e1fa4825ae64a94a7c0e
moved a few fields from `lowdown_opts` toa  new `lowdown_opts_term`
struct. As a result the build started failing as:

    nix-cmd> [2/17] Compiling C++ object libnixcmd.so.p/markdown.cc.o
    nix-cmd> FAILED: libnixcmd.so.p/markdown.cc.o
    nix-cmd> g++ -Ilibnixcmd.so.p -I. -I.. -I/nix/store/b0bnrk5lacxbpgxgnc28r8q3wcazrgxj-nix-util-2.26.0pre-dev/include/nix -I/nix/store/cxnynq9ykyj4xxv6wf6dw7r0aw5x6n9k-libarchive-3.7.7-dev/include -I/nix/store/bfgjwkcb8snkizx578rzdahi75m8zyh4-nlohmann_json-3.11.3/include -I/nix/store/3sx8bq3sip6j2nv1m5xx4gbdp33v7iy6-nix-store-2.26.0pre-dev/include/nix -I/nix/store/sih2dgqzvsbv7p510lkfmas7s7wbsl4j-nix-fetchers-2.26.0pre-dev/include/nix -I/nix/store/68p8s20fsiiakj7nys7grbaixfnhsdzs-nix-expr-2.26.0pre-dev/include/nix -I/nix/store/gw7wknhzhfzzj9zww2kyi5xrzgf1ndki-boehm-gc-8.2.8-dev/include -I/nix/store/3jwb9j4vnsk5saq3wfyyp9il3mhs41l9-nix-flake-2.26.0pre-dev/include/nix -I/nix/store/8nwjvmq7m48v8g646jrxkikv6x47bc3m-nix-main-2.26.0pre-dev/include/nix -I/nix/store/rb0hzsw5wc1a7daizhpj824mbxlvijrq-lowdown-1.4.0-dev/include -I/nix/store/m388ywpk53fsp8r98brfd7nf1f5sskv0-editline-1.17.1-dev/include -fdiagnostics-color=always -D_GLIBCXX_ASSERTIONS=1 -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++2a -include config-util.hh -include config-store.hh -include config-expr.hh -include config-main.hh -include config-cmd.hh -Wdeprecated-copy -Werror=suggest-override -Werror=switch -Werror=switch-enum -Werror=unused-result -Wignored-qualifiers -Wimplicit-fallthrough -Wno-deprecated-declarations -O3 -fPIC -pthread -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -MD -MQ libnixcmd.so.p/markdown.cc.o -MF libnixcmd.so.p/markdown.cc.o.d -o libnixcmd.so.p/markdown.cc.o -c ../markdown.cc
    nix-cmd> ../markdown.cc: In function 'std::string nix::doRenderMarkdownToTerminal(std::string_view)':
    nix-cmd> ../markdown.cc:28:5: error: 'lowdown_opts' has no non-static data member named 'cols'
    nix-cmd>    28 |     };
    nix-cmd>       |     ^

The change adds version-based conditional to support both pre-1.4 and
1.4 forms of the initialization.

Closes: https://github.com/NixOS/nix/issues/12113
(cherry picked from commit edbfe863ce)
2024-12-30 19:16:43 +00:00
Eelco Dolstra a924db7d0b Resolve merge conflict 2024-12-16 18:01:52 +01:00
Eelco Dolstra f986f7e89b EvalState::realiseContext(): Allow access to the entire closure
Fixes #11030.

(cherry picked from commit 08361f031d)

# Conflicts:
#	src/libexpr/eval.cc
#	tests/functional/import-from-derivation.nix
#	tests/functional/import-from-derivation.sh
2024-12-16 15:36:40 +00:00
Jörg Thalheim 6a791e946f Merge pull request #11904 from NixOS/mergify/bp/2.24-maintenance/pr-11902
Fix issue #11892 (backport #11902)
2024-12-16 13:22:30 +01:00
mergify[bot] aa22f98b25 Merge pull request #12054 from NixOS/mergify/bp/2.24-maintenance/pr-12050
speed up ci by splitting off eval and build + fix ci with restricted namespaces (backport #12050)
2024-12-14 15:46:25 +00:00
Jörg Thalheim b3dc855569 ci: disable apparmor restrictions
For our tests we need to map the root user for some tests.
However ubuntu no longer allows this by default:
https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces

(cherry picked from commit 9f2b9256b6)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 7bafbfd50e binary-tarball: create tarball without listing files
This is producing quite a bit of log output in CI.

(cherry picked from commit 31b3a34e5b)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 578f185daf speed up ci by splitting off eval and build
- This speeds up macOS builds from 30 minutes to 11 minutes (3x faster).
- Also improve error reporting e.g. printing out what actually failed to build.
- As a result we also no longer need swap.

(cherry picked from commit 691b1ea237)
2024-12-14 15:24:48 +00:00
Eelco Dolstra 628961d136 Bump version 2024-12-10 17:11:49 +01:00
Jörg Thalheim a4f978bd9b Merge pull request #12034 from NixOS/mergify/bp/2.24-maintenance/pr-11921
Make @nix json structured build log parsing warn instead of fail (backport #11921)
2024-12-10 05:37:28 +01:00
Robert Hensing 180031fb5a tests: Make unusual logging test conditional on fixed daemon version
(cherry picked from commit c783cd22ac)
2024-12-10 00:12:19 +01:00
Robert Hensing 29d91db99c Remove redundant warning: prefix from structured build log warning
(cherry picked from commit f3c722cab2)
2024-12-09 21:40:12 +00:00
Robert Hensing 6b2722fc84 test: Move unusual-logging to run only in logging test case
(cherry picked from commit 1421420e86)
2024-12-09 21:40:07 +00:00
Robert Hensing c951bb0689 Push log source description out of libutil and report build hook @nix warning correctly
(cherry picked from commit 03d4bfd852)
2024-12-09 21:40:04 +00:00
Robert Hensing 60eadfac7f tests/functional/dependencies.nix: Check that we tolerate syntax and type errors
(cherry picked from commit 1485937b89)
2024-12-09 21:40:00 +00:00
Linus Heckemann 9fed865d1d libutil: handle json builder log messages with unexpected format
Before this change, expressions like:

with import <nixpkgs> {};
runCommand "foo" {} ''
  echo '@nix {}' >&$NIX_LOG_FD
''

would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.

Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.

Fixes #544.

Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
(cherry picked from commit e55cd3beea710db727fd966f265a1b715b7285f3)
(cherry picked from commit ee03fd478e)
2024-12-09 21:39:58 +00:00
Robert Hensing 3cd4e9de1e tests/functional/dependencies.nix: Refactor, replace arcane let
(cherry picked from commit e82ff51726)
2024-12-09 21:39:55 +00:00
Eelco Dolstra 0f8fef49c3 Merge pull request #12017 from NixOS/mergify/bp/2.24-maintenance/pr-12013
nix hash convert: Support SRI hashes that lack trailing '=' characters (backport #12013)
2024-12-09 13:14:14 +01:00
Eelco Dolstra bd1b11ec66 Resolve conflict 2024-12-06 12:13:12 +01:00
Eelco Dolstra 6a8e9736e0 nix hash: Don't print 'nix hash' deprecation message
Fixes #11997.

(cherry picked from commit 408c2faf93)

# Conflicts:
#	src/nix/hash.cc
2024-12-05 22:30:24 +00:00
Eelco Dolstra dd79a490bf nix hash convert: Don't fail on uppercase base-16 hashes
(cherry picked from commit 33b645cedf)
2024-12-05 22:30:24 +00:00
Eelco Dolstra 00d0e63c61 nix hash convert: Support SRI hashes that lack trailing '=' characters
Fixes #11996.

(cherry picked from commit 52f1cd0595)
2024-12-05 22:30:24 +00:00
Jörg Thalheim fc11659148 Merge pull request #11932 from NixOS/mergify/bp/2.24-maintenance/pr-11915
Bump fetcher cache version (backport #11915)
2024-11-23 09:31:55 +01:00
Robert HensingandEelco Dolstra bf67741ff1 Bump fetcher cache version
We're getting more reports in https://github.com/NixOS/nix/issues/10985

It appears that something hasn't gone right process-wise.
I find this mistake not to be worth investigating, but rather something
to pay attention to going forward.

Let's nip this in the bud.

Closes https://github.com/NixOS/nix/issues/10985

(cherry picked from commit e948c8e033)

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-11-23 09:05:52 +01:00
Robert Hensing 403f388fdb Fix issue 11892
It seems that I copied the expression for baseDir thoughtlessly and
did not come back to it.

- `baseDir` was only used in the `fromArgs` branch.
- `fromArgs` is true when `packages` is true.

(cherry picked from commit c4b95dbdd1)
2024-11-18 21:21:13 +00:00
Eelco Dolstra 8e8a572d37 Merge pull request #11889 from NixOS/mergify/bp/2.24-maintenance/pr-11884
nix path-info: Don't write to std::cout directly (backport #11884)
2024-11-15 16:29:29 +01:00
Eelco Dolstra 3bfc35ba74 nix path-info: Don't write to std::cout directly
This interferes with the progress bar, resulting in output like

  evaluating derivation 'git+file:///home/eelco/Dev/nix-master#packages.x86_64-linux.default'/nix/store/zz8v96j5md952x0mxfix12xqnvq5qv5x-nix-2.26.0pre20241114_a95f6ea.drv

(cherry picked from commit 33a0fa882f)
2024-11-15 14:08:22 +00:00
Eelco Dolstra aaeaa4e133 Merge pull request #11879 from NixOS/mergify/bp/2.24-maintenance/pr-11878
Remove warning that interpretation of .drv has changed (backport #11878)
2024-11-13 18:23:23 +01:00
Robert Hensing 22e1613814 Remove warning that interpretation of .drv has changed
This was first tagged as 2.15.0, 1½ years ago; plenty of time for
everyone to catch up.

By now, the warning is causing more confusion than that it is helpful,
because passing a `.drv` or `drvPath` has legitimate use cases.

(cherry picked from commit f534a7a524)
2024-11-13 16:12:58 +00:00
Jörg Thalheim ef21dfa221 Merge pull request #11838 from NixOS/mergify/bp/2.24-maintenance/pr-11837
fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug (backport #11837)
2024-11-08 16:02:45 +01:00
Sergei Zimmerman c556c205a9 fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug
Naming class member variables the same as constructor arguments is a very
slippery slope because of how member variable names get resolved. Compiler
is not very helpful here and we need static analysis to forbid this kind of
stuff.

The following example illustrates the cause quite well:

```cpp

struct B {
    B(int) {}
};

struct A {
    A(int b): b([&](){
        return b;
        static_assert(std::is_same_v<decltype(b), int>);
    }()) {
       static_assert(std::is_same_v<decltype(b), int>);
    }
    void member() {
        static_assert(std::is_same_v<decltype(b), B>);
    }
    B b;
};

int main() {
    A(1).member();
}
```

From N4861 6.5.1 Unqualified name lookup:

> In all the cases listed in [basic.lookup.unqual], the scopes are searched
> for a declaration in the order listed in each of the respective categories;
> name lookup ends as soon as a declaration is found for the name.
> If no declaration is found, the program is ill-formed.

In the affected code there was a use-after-move for all accesses in the constructor
body, but this UB wasn't triggered.

These types of errors are trivial to catch via clang-tidy's [clang-analyzer-cplusplus.Move].

(cherry picked from commit 3e0129ce3b)
2024-11-08 13:58:17 +00:00
Jörg Thalheim 6e095ddef9 Merge pull request #11823 from NixOS/mergify/bp/2.24-maintenance/pr-11821
initRepoAtomically: Catch directory_not_empty (backport #11821)
2024-11-07 08:03:52 +01:00
Robert Hensing c3e2419e73 initRepoAtomically: Catch directory_not_empty
(cherry picked from commit 388271e8ec)
2024-11-07 06:23:15 +00:00
Eelco Dolstra 960c288fa0 Merge pull request #11777 from NixOS/mergify/bp/2.24-maintenance/pr-11759
fix: make sure directory exists before using `ln` (backport #11759)
2024-11-01 16:56:25 +01:00
Valentin Gagarin d14b46f9ba Merge pull request #11783 from NixOS/mergify/bp/2.24-maintenance/pr-11775
doc/rl-2.19: add entry for always-allow-substitutes option (backport #11775)
2024-11-01 13:12:58 +01:00
Bjørn Forsman c9325fe343 doc/rl-2.19: add entry for always-allow-substitutes option (#11775)
* doc/rl-2.19: add entry for always-allow-substitutes option

Fixes https://github.com/NixOS/nix/issues/9427.

(cherry picked from commit 020dbac0e0)
2024-11-01 10:57:56 +00:00
Pol Dellaiera 2a2d484986 fix: make sure directory exists before using ln
(cherry picked from commit 5f71ebb956)
2024-11-01 10:19:42 +00:00
Eelco Dolstra 9728a4ac94 maintainers/upload-release.pl: Delete temporary directories when we're done
(cherry picked from commit 02bb633a58)
2024-10-31 12:12:17 +01:00
Eelco Dolstra 75b00e52ac 'build' is now 'build.nix'
(cherry picked from commit 617e711820)
2024-10-31 12:12:12 +01:00
Eelco Dolstra 6044907c13 Bump version 2024-10-31 12:11:43 +01:00
Eelco Dolstra 597fcc98e1 Merge remote-tracking branch 'nix-ghsa-wf4c-57rh-9pjg/advisory-fix-1-2.24' into 2.24-maintenance 2024-10-30 21:44:39 +01:00
Eelco Dolstra 82abed901f Merge pull request #11769 from DeterminateSystems/fix-release-notes
Fix 2.24 release notes
2024-10-30 16:12:43 +01:00
Eelco Dolstra f9180f12c4 release notes: 2.24.9 2024-10-30 15:32:37 +01:00
John Ericson baa7565710 Merge pull request #11763 from NixOS/mergify/bp/2.24-maintenance/pr-11751
Add support for `utimensat` as an alternative to `lutimes` (backport #11751)
2024-10-28 15:27:39 +01:00
John Ericson 7e6e75fd48 Merge pull request #11762 from NixOS/mergify/bp/2.24-maintenance/pr-11750
Fix building on OpenBSD (backport #11750)
2024-10-28 15:27:23 +01:00
Artemis Tosini 803943fce4 Add support for utimensat as an alternative to lutimes
OpenBSD doesn't support `lutimes`, but does support `utimensat` which
subsumes it. In fact, all the BSDs, Linux, and newer macOS all support
it. So lets make this our first choice for the implementation.

In addition, let's get rid of the `lutimes` `ENOSYS` special case. The
Linux manpage says

> ENOSYS
>
> The kernel does not support this call; Linux 2.6.22 or later is
> required.

which I think is the origin of this check, but that's a very old version
of Linux at this point. The code can be simplified a lot of we drop
support for it here (as we've done elsewhere, anyways).

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
(cherry picked from commit d023202811)
2024-10-28 10:27:39 +00:00
Artemis Tosini 0ae90918db package.nix: Disable GC on OpenBSD
Nix fails to build on OpenBSD with a linking error due to a non-found
symbol in boehm-gc. Just disable the GC until we can find a proper
workaround.

(cherry picked from commit fecc1ca205)
2024-10-28 10:27:37 +00:00
Artemis Tosini d6ece7e94a Fix OpenBSD build with Makefiles
OpenBSD dynamic libraries never link to libc directly.
Instead, they have undefined symbols for all libc functions they use
that ld.so resolves to the libc referred to in the main executable.

Thus, disallowing undefined symbols will always fail

(cherry picked from commit c49bff2434)
2024-10-28 10:27:37 +00:00
Robert Hensing 50f83e4bbd local-derivation-goal: Move builder preparation to non-builtin code path 2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt 047ee50db2 local-derivation-goal: Refactor
This works because the `builder` and `args` variables are only used
in the non-builtin code path.

Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt ae7a2ea741 local-derivation-goal: Print sandbox error detail on darwin
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert Hensing f8a1a149c7 packaging: Add darwin -lsandbox in meson 2024-10-22 17:56:06 +02:00
Puck Meerburg 170242cf0c fix: Run all derivation builders inside the sandbox on macOS 2024-10-22 17:56:06 +02:00
Eelco Dolstra 7718688f52 Merge pull request #11730 from NixOS/mergify/bp/2.24-maintenance/pr-11713
fix env-vars beeing written to `/tmp` (backport #11713)
2024-10-22 15:01:56 +02:00
Jörg Thalheim 0e9b04a66e fix env-vars beeing written to /tmp
This overall seems like insecure tmp file handling to me. Because other
users could replace files in /tmp with a symlink and make the nix-shell
override other files.

fixes https://github.com/NixOS/nix/issues/11470

(cherry picked from commit 2105574702)
2024-10-22 12:13:47 +00:00
Eelco Dolstra 411ec33db3 Merge pull request #11692 from NixOS/mergify/bp/2.24-maintenance/pr-11690
Handle tarballs where directory entries are not contiguous (backport #11690)
2024-10-14 16:01:43 +02:00
Eelco Dolstra 31df105f45 Merge pull request #11691 from NixOS/mergify/bp/2.24-maintenance/pr-11677
builtins.fetchurl: Fix segfault on s3:// URLs (backport #11677)
2024-10-14 15:23:49 +02:00
Eelco Dolstra 57ace600af Add a test
(cherry picked from commit a7b9877da9)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 9da1300617 Handle tarballs where directory entries are not contiguous
I.e. when not all entries underneath a directory X follow eachother,
but there is some entry Y that isn't a child of X in between.

Fixes #11656.

(cherry picked from commit 4012954b59)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 1294442c6c Add assert
(cherry picked from commit d2f4d07619)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 339236d32e Make S3 downloads slightly more interruptable
(cherry picked from commit d38f62f64d)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 4912a9e7fd builtins.fetchurl: Fix segfault on s3:// URLs
Also, add an activity to show that we're downloading an s3:// file.

Fixes #11674.

(cherry picked from commit 0500fba56a)
2024-10-14 14:44:28 +02:00
Eelco Dolstra d80bf54e3b Add a VM test for S3BinaryCacheStore
Fixes #11238.

(cherry picked from commit 2950f9e18a)
2024-10-14 14:44:28 +02:00
Robert Hensing f1dc3b7d55 Merge pull request #11649 from NixOS/mergify/bp/2.24-maintenance/pr-11610
fix passing CA files into builtins:fetchurl sandbox (backport #11610)
2024-10-13 12:43:33 +02:00
Jörg Thalheim 5f1b132187 tests/nixos/fetchurl: drop unused variables
(cherry picked from commit 410853ddcf)
2024-10-07 12:45:04 +00:00
Puck Meerburg 742eb0f815 fix passing CA files into builtins:fetchurl sandbox
This patch has been manually adapted from
https://github.com/lix-project/lix/commit/14dc84ed03f1b7e5a41bb6fdce00916faab32b60

Tested with:

$ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 16:57:50 after 1s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1

Now returns:

nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 17:05:48 after 0s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

(cherry picked from commit c1ecf0bee9)
2024-10-07 12:45:03 +00:00
Robert Hensing 13e200df45 Merge pull request #11601 from Mic92/git-cache-backport
[2.24] create git caches atomically
2024-09-30 13:11:22 +02:00
Eelco Dolstra a1d841bf2c Bump version 2024-09-28 00:05:03 +02:00
Eelco Dolstra 048cfe51c9 Merge pull request #11604 from NixOS/mergify/bp/2.24-maintenance/pr-11600
HttpBinaryCacheStore::getFile(): Fix uncaught exception (backport #11600)
2024-09-27 13:26:21 +02:00
Eelco Dolstra 15a2b49115 HttpBinaryCacheStore::getFile(): Fix uncaught exception
This method is marked as `noexcept`, but `enqueueFileTransfer()` can
throw `Interrupted` if the user has hit Ctrl-C or if the `ThreadPool`
that the thread is a part of is shutting down.

(cherry picked from commit 4566854981)
2024-09-27 10:38:03 +00:00
Jörg Thalheim 34fd00accc create git caches atomically
When working on speeding up the CI,
I triggered a race condition in the creation of the tarball cache.
This code now instead will ensure that half-initialized repositories
are no longer visible to any other nix process.

This is the error message that I got before:

error: opening Git repository '"/Users/runner/.cache/nix/tarball-cache"': could not find repository at '/Users/runner/.cache/nix/tarball-cache'
(cherry picked from commit 12d5b2cfa1)
2024-09-27 10:06:58 +02:00
Eelco Dolstra b23812a59c Bump version 2024-09-26 03:25:40 +02:00
Eelco Dolstra 618a0cc987 Merge pull request #11592 from NixOS/mergify/bp/2.24-maintenance/pr-11585
builtin:fetchurl: Enable TLS verification (backport #11585)
2024-09-26 01:04:39 +02:00
Eelco Dolstra ba81598017 Resolve conflict 2024-09-26 00:17:03 +02:00
Eelco Dolstra e87be60055 Typo
(cherry picked from commit ef8987955b)
2024-09-26 00:16:17 +02:00
Eelco Dolstra 345a264a39 Add release note
(cherry picked from commit 7b39cd631e)
2024-09-25 21:55:36 +00:00
Eelco Dolstra ee6a5faf4b Add a test for builtin:fetchurl cert verification
(cherry picked from commit f2f47fa725)

# Conflicts:
#	tests/nixos/default.nix
2024-09-25 21:55:36 +00:00
Eelco Dolstra d4824c8ff7 builtin:fetchurl: Enable TLS verification
This is better for privacy and to avoid leaking netrc credentials in a
MITM attack, but also the assumption that we check the hash no longer
holds in some cases (in particular for impure derivations).

Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038.

(cherry picked from commit c04bc17a5a)
2024-09-25 21:55:36 +00:00
Eelco Dolstra b4fcd27590 Merge pull request #11578 from Mic92/key-backport
[2.24-maintainence] Ensure error messages don't leak private key
2024-09-24 13:45:43 +02:00
John Ericson 082f6bb35d Ensure error messages don't leak private key
Since #8766, invalid base64 is rendered in errors, but we don't actually
want to show this in the case of an invalid private keys.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 2b6b03d8df)
2024-09-24 06:39:03 +02:00
John Ericson 1e03ea386b Revert "base64Decode: clearer error message when an invalid character is detected"
We have a safer way of doing this.

This reverts commit dc3ccf02bf.

(cherry picked from commit d0c351bf43)
2024-09-24 06:31:50 +02:00
John Ericson b523e4de34 Merge pull request #11571 from NixOS/mergify/bp/2.24-maintenance/pr-11390
Don't refer to public keys as secret keys in error (backport #11390)
2024-09-23 18:50:03 -04:00
Alyssa Ross 563dedcf64 Don't refer to public keys as secret keys in error
This constructor is used for public keys as well.

(cherry picked from commit 9cc550d652)
2024-09-23 22:00:10 +00:00
Eelco Dolstra a7fdef6858 Bump version 2024-09-20 01:19:15 +02:00
Eelco Dolstra b5154deba3 Merge pull request #11553 from NixOS/mergify/bp/2.24-maintenance/pr-11548
Fix missing GC root in zipAttrsWith (backport #11548)
2024-09-19 22:09:56 +02:00
Eelco Dolstra ecd83dc155 Use HAVE_BOEHMGC
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 4449b0da74)
2024-09-19 19:04:17 +00:00
Eelco Dolstra 5b5e1920eb Fix missing GC root in zipAttrsWith
My SNAFU was that I assumed that all the `Value *`s we put in
`attrsSeen` are already reachable (which they are), but I forgot about
the `elems` pointer in `ListBuilder`.

Fixes #11547.

(cherry picked from commit 0c2fdd2f3c)
2024-09-19 19:04:17 +00:00
Valentin Gagarin fc1d6b2f03 Merge pull request #11521 from NixOS/mergify/bp/2.24-maintenance/pr-8766
base64Decode: clearer error message when an invalid character is detected (backport #8766)
2024-09-19 11:29:21 +02:00
Brian McGee 9941f620c4 base64Decode: clearer error message when an invalid character is detected
Output the offending string in its entirety to provide context.

Closes #8479

(cherry picked from commit dc3ccf02bf)
2024-09-17 14:03:19 +00:00
Valentin Gagarin 5b2a8c223e Merge pull request #11497 from Mic92/mergify/bp/2.24-maintenance/pr-11378 2024-09-17 07:09:32 +02:00
mergify[bot]andTravis A. Everett 1b076b4f84 doc: add admonitions for macOS 15 Sequoia update (#11487) (#11509)
The impending release of macOS 15 Sequoia will break many existing nix
installs on macOS, which may lead to an increased number of people who
are looking to try to reinstall Nix without noticing the open/pinned
issue (#10892) that explains the problem and outlines how to migrate
existing installs.

These admonitions are a short-term measure until we are over the hump
and support volumes dwindle.

(cherry picked from commit 48477d4a3e)

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2024-09-16 16:03:05 +02:00
Robert Hensing d9ef3dd012 Merge pull request #11484 from NixOS/mergify/bp/2.24-maintenance/pr-11473
Fix making the build directory kept by `keep-failed` readable (backport #11473)
2024-09-16 12:46:23 +02:00
Robert Hensing f9714bac34 Merge pull request #11456 from NixOS/mergify/bp/2.24-maintenance/pr-11321
replace backport github action with mergify (backport #11321)
2024-09-16 12:41:14 +02:00
Jörg Thalheim 684a690480 update filesystem-errors changelog to 2.24 release 2024-09-13 14:20:34 +02:00
John Ericson 4354d90384 tweak unpack channel built-in, std::filesystem::path for tarball
(cherry picked from commit 193dc49097)
2024-09-13 14:11:36 +02:00
Jörg Thalheim 60001b1936 add release notes for filesystem fixes
Update doc/manual/rl-next/filesystem-errors.md

Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit 04ce0e648a)
2024-09-13 14:11:36 +02:00
Jörg Thalheim c84fc0120f builtins.unpackChannel: wrap filesystem errors and sanitize channelName
Otherwise these errors are not caught correctly

(cherry picked from commit 70c52d72f4)
2024-09-13 14:11:36 +02:00
Jörg Thalheim cd97688bce builtins.readDir: fix nix error trace on filesystem errors
Before:

nix-env % ./src/nix/nix eval --impure --expr 'let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f' --show-trace
error: filesystem error: directory iterator cannot open directory: No such file or directory [/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo]

After:

error:
       … while calling the 'readDir' builtin
         at «string»:1:9:
            1| let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f
             |         ^

       error: reading directory '/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo': No such file or directory

(cherry picked from commit 22ba4dc78d)
2024-09-13 11:56:41 +00:00
Eelco Dolstra 07909de6ed Merge pull request #11493 from NixOS/mergify/bp/2.24-maintenance/pr-11423
Git fetcher: Ignore .gitmodules entries that are not submodules (backport #11423)
2024-09-12 20:30:45 +02:00
Eelco Dolstra 751907dc8a Git fetcher: Ignore .gitmodules entries that are not submodules
Fixes #10739.

(cherry picked from commit 9d24080090)
2024-09-12 16:16:36 +00:00
Eelco Dolstra d9dd6c62d6 Merge pull request #11486 from NixOS/mergify/bp/2.24-maintenance/pr-11466
Git fetcher: Don't update mtime of ref file if fetching by rev (backport #11466)
2024-09-11 19:59:10 +02:00
Eelco Dolstra 97c5ac5752 Git fetcher: Don't update mtime of ref file if fetching by rev
This fixes the warning

  $ nix eval --store /tmp/nix --expr 'builtins.fetchTree { type = "git"; url = "https://github.com/DeterminateSystems/attic"; ref = "fixups-for-magic-nix-cache"; rev = "635753a2069d4b8228e846dc5c09ad361c75cd1a"; }'
  warning: could not update mtime for file '/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache': error: changing modification time of '"/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache"': No such file or directory

When we're fetching by rev, that file doesn't necessarily exist, and we
don't care about it anyway.

(cherry picked from commit b80b091bac)
2024-09-11 15:37:36 +00:00
Artturin 40461a8e0e Fix making the build directory kept by keep-failed readable
Caused by 1d3696f0fb

Without this fix the kept build directory is readable only by root

```
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5
drwx------ root root 60 B Wed Sep 11 00:09:48 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/

$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build
drwxr-xr-x nixbld1 nixbld 80 B Wed Sep 11 00:09:58 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build/
```

(cherry picked from commit ebebe626ff)
2024-09-11 12:56:18 +00:00
Eelco Dolstra 0f825b38f4 Bump version 2024-09-10 13:45:04 +02:00
Eelco Dolstra eb11c14998 Merge commit from fork
Fix unsafe NAR unpacking (2.24)
2024-09-10 12:42:55 +02:00
Eelco Dolstra a6ad5565ef Improve use-case-hack description slightly
(cherry picked from commit 5ca2f58798)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 2e1cb495c1 Typo
(cherry picked from commit 4cfa59fdb3)
2024-09-10 09:51:50 +02:00
Eelco Dolstra e25410c788 Test that deserializing regular files / symlinks is exclusive
(cherry picked from commit 52ba3cc5ea)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 25510ba66f RestoreSink::createDirectory(): Use append()
On macOS, `mkdir("x/')` behaves differently than `mkdir("x")` if `x` is
a dangling symlink (the formed succeed while the latter fails). So make
sure we always strip the trailing slash.

(cherry picked from commit 9fcb588dd8)
2024-09-10 09:51:50 +02:00
Eelco Dolstra d9b60b3902 Fix test on macOS
(cherry picked from commit 21dcbd7e83)
2024-09-10 09:51:50 +02:00
Eelco Dolstra a041688133 Test that deserializing NARs with names with equal Unicode normal forms fails on macOS
The test is based on the one by @puckipedia but with the file names
swapped to make them sorted.

(cherry picked from commit 7a765a6aaf)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 1288970496 Detect NAR directory entries that collide with another path after case-hacking
The test was made by @puckipedia.

(cherry picked from commit 3557587381)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 0cfc9bf133 More tests
(cherry picked from commit 77c090cdbd)
2024-09-10 09:51:50 +02:00
Eelco Dolstra f160d3ac68 Test that nix-store --restore fails if the output already exists
This restores the behaviour from before the std::filesystem
refactorings.

(cherry picked from commit da1ad28912)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 6187ee468f Add test case for NARs with duplicate directory entries
This test was made by @puckipedia.

(cherry picked from commit 83d5b32803)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 12fa019ae5 NAR parser: Fix check for duplicate / incorrectly sorted entries
"prevName" was always empty because it was declared in the wrong scope.

(cherry picked from commit 495d32e1b8)
2024-09-10 09:51:50 +02:00
Jörg Thalheim f0cffa7300 replace backport github action with mergify
The current backport action cannot automerge because
the github action bot does not trigger github CI actions.
Mergify instead does not have this limitation and can also
use a merge queue.

On top we have now a declarative configuration to allow
contributers to add new tests to required without having access
to the github org.

An example pull request and backport can be seen here:

https://github.com/Mic92/nix-1/pull/4

and here:

https://github.com/Mic92/nix-1/pull/5

To complete the setup the mergify app must be enabled for this repository.
It's already installed in the nixos organization for nixos-hardware and
other repositories.

(cherry picked from commit 80f20fa4cb)
2024-09-09 16:12:24 +00:00
Robert Hensing ae486b2910 Merge pull request #11446 from NixOS/mergify/bp/2.24-maintenance/pr-11285
fetchers::downloadTarball(): Return a cacheable accessor (backport #11285)
2024-09-09 18:05:32 +02:00
Eelco Dolstra 437f7a0042 fetchers::downloadTarball(): Return a cacheable accessor
downloadTarball() is used by `-I foo=<url>` etc. fetchToStore() needs
the accessor to have a fingerprint to enable caching.

Fixes #11271.

(cherry picked from commit 9f6ee93f48)
2024-09-09 12:46:07 +00:00
Eelco Dolstra b53b07b30b Merge pull request #11421 from NixOS/mergify/bp/2.24-maintenance/pr-10919
install-darwin: fix _nixbld uids for macOS sequoia (backport #10919)
2024-09-04 18:55:18 +02:00
Emily 8d0414d682 install-darwin: increment base UID by 1 (#15)
(cherry picked from commit 11cf29b15c)
2024-09-03 23:59:16 +00:00
Travis A. Everett c5a0e624d9 install-darwin: move nixbld gid to match first UID
(cherry picked from commit 75567423fb)
2024-09-03 23:59:15 +00:00
Travis A. Everett 0679505d8c install-darwin: fix _nixbld uids for macOS sequoia
Starting in macOS 15 Sequoia, macOS daemon UIDs are encroaching on our
default UIDs of 301-332. This commit relocates our range up to avoid
clashing with the current UIDs of 301-304 and buy us a little time
while still leaving headroom for people installing more than 32 users.

(cherry picked from commit df36ff0d1e)
2024-09-03 23:59:15 +00:00
Eelco Dolstra ea7abb58b5 Bump version 2024-09-03 16:51:36 +02:00
Eelco Dolstra f1ab41b2bc Merge pull request #11408 from NixOS/mergify/bp/2.24-maintenance/pr-11402
Respect max-substitution-jobs again (backport #11402)
2024-09-03 14:23:49 +02:00
Eelco Dolstra c21f664e82 "unsigned" -> size_t
Slight cleanup.

(cherry picked from commit b7acd1c414)
2024-09-03 11:36:40 +00:00
Eelco Dolstra 0c25bea7cc Respect max-substitution-jobs again
This broke in #11005. Any number of PathSubstitutionGoals would
be woken up by a single build slot becoming available. If there
are a lot of substitution goals active, this could lead to us
running out of file descriptors (especially on macOS where the
default limit is 256).

(cherry picked from commit a33cb8af56)
2024-09-03 11:36:39 +00:00
Eelco Dolstra 9d8669b14a Bump version 2024-08-23 16:15:11 +02:00
Eelco Dolstra cb0439f0c2 Merge pull request #11360 from DeterminateSystems/backport-11152
[Backport 2.24-maintenance] addToStore(): Evaluate on the main stack
2024-08-23 15:08:56 +02:00
Eelco Dolstra 30a57328d2 Backport https://github.com/NixOS/nix/pull/11152 2024-08-23 13:15:30 +02:00
Robert Hensing 79a43160ca Merge pull request #11334 from NixOS/backport-11329-to-2.24-maintenance
[Backport 2.24-maintenance] fix: check to see if there are any lines before
2024-08-19 16:38:44 +02:00
Tom Bereknyei 7befd60c01 fix: check to see if there are any lines before
(cherry picked from commit 59db8fd62b)
2024-08-19 13:40:38 +00:00
Eelco Dolstra 90fb4e8890 Bump version 2024-08-19 12:46:17 +02:00
tomberek 3ac5d736e2 Merge pull request #11319 from NixOS/backport-11270-to-2.24-maintenance
[Backport 2.24-maintenance] libstore: fix port binding in __darwinAllowLocalNetworking sandbox
2024-08-17 03:17:29 -04:00
Andrew Marshall 4e707b8e57 libstore: fix port binding in __darwinAllowLocalNetworking sandbox
In d60c3f7f7c, this was changed to close a
hole in the sandbox. Unfortunately, this was too restrictive such that it
made local port binding fail, thus making derivations that needed
`__darwinAllowLocalNetworking` gain nearly nothing, and thus largely
fail (as the primary use for it is to enable port binding).

This unfortunately does mean that a sandboxed build process can, in
coordination with an actor outside the sandbox, escape the sandbox by
binding a port and connecting to it externally to send data. I do not
see a way around this with my experimentation and understanding of the
(quite undocumented) macOS sandbox profile API. Notably it seems not
possible to use the sandbox to do any of:

- Restrict the remote IP of inbound network requests
- Restrict the address being bound to

As such, the `(local ip "*:*")` here appears to be functionally no
different than `(local ip "localhost:*")` (however it *should* be
different than removing the filter entirely, as that would make it also
apply to non-IP networking). Doing `(allow network-inbound (require-all
(local ip "localhost:*") (remote ip "localhost:*")))` causes listening
to fail.

Note that `network-inbound` implies `network-bind`.

(cherry picked from commit 00f6db36fd)
2024-08-17 03:17:45 +00:00
Eelco Dolstra 20cae372f4 Merge pull request #11314 from cole-h/backport-2.24-update-nixpkgs-input-fix-darwin
[2.24] Update nixpkgs input to fix darwin ccache evaluation, have CI check that all outputs on all systems evaluate
2024-08-16 20:48:49 +02:00
Cole Helbling d550139191 ci: check that all outputs for all systems can evaluate
(cherry picked from commit aa3d35c1f4)
2024-08-16 11:46:29 -07:00
Cole Helbling 5b62a1dbd6 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/c3d4ac725177c030b1e289015989da2ad9d56af0?narHash=sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz%2BNG82pbdg%3D' (2024-08-15)

(cherry picked from commit 8866d2cd83)
2024-08-16 11:46:24 -07:00
Eelco Dolstra 450252c92c Bump version 2024-08-08 17:21:00 +02:00
Eelco Dolstra 4036c3aafb Bump version 2024-08-08 15:02:48 +02:00
Robert Hensing 935bf1157d Merge pull request #11267 from NixOS/backport-11244-to-2.24-maintenance
[Backport 2.24-maintenance] allow to c api with older c versions
2024-08-08 01:43:24 +02:00
Jörg Thalheim b1941c9f8a allow to c api with older c versions
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix

(cherry picked from commit 739418504c)
2024-08-07 23:12:17 +00:00
Eelco Dolstra 40832b0a95 Merge pull request #11262 from NixOS/backport-11257-to-2.24-maintenance
[Backport 2.24-maintenance] PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
2024-08-06 11:20:31 +02:00
Eelco Dolstra fa78d7f72f PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
It is not an error if queryPathInfo() indicates that a path does not
exist in the substituter.

Fixes #11198. This was broken in 846869da0e.

(cherry picked from commit 0a00bd07b2)
2024-08-06 08:33:46 +00:00
Eelco Dolstra 2382a52c84 Merge pull request #11239 from NixOS/backport-11237-to-2.24-maintenance
[Backport 2.24-maintenance] Fix the S3 store
2024-08-01 17:56:18 +02:00
Eelco Dolstra fe6a7c805c Fix the S3 store
It was failing with:

   error: AWS error fetching 'nix-cache-info': The specified bucket does not exist

because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.

(cherry picked from commit 9b5b7b7963)
2024-08-01 15:46:45 +00:00
Eelco Dolstra 0a167ffd1f Bump version 2024-08-01 10:41:11 +02:00
Eelco Dolstra 206e32e2d7 Mark release 2024-07-31 23:37:43 +02:00
Eelco Dolstra b24757f08a Merge pull request #11231 from DeterminateSystems/release-notes
2.24 release notes
2024-07-31 23:24:39 +02:00
Eelco Dolstra 794a50065b base32 -> nix32 2024-07-31 22:33:41 +02:00
Qyriad cb5a5dd4f3 docs: clarify how ^ works for -E/-f installables
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.

Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
2024-07-31 22:27:58 +02:00
Qyriad 8ff169715d docs: clarify how the different kinds of installables are selected
Change-Id: I146736bb97ebe035e04be69ce9fb60a557e38c6c
2024-07-31 22:27:58 +02:00
Eelco Dolstra f136ec5290 Add contributors 2024-07-31 22:16:44 +02:00
Eelco Dolstra 22ad0e653f Edit release notes 2024-07-31 22:14:27 +02:00
John Ericson 733c816d34 Small windows cross fixes (#11230) 2024-07-31 20:04:18 +00:00
Eelco Dolstra c952d933e5 release notes: 2.24.0 2024-07-31 21:57:31 +02:00
Valentin Gagarin db5bacb637 reword documentation on nix-path config option (#7772)
* docs: unify documentation on search paths

- put all the information on search path semantics into `builtins.findFile`
- put all the information on determining the value of `builtins.nixPath` into the
  `nix-path` setting

  maybe `builtins.nixPath` is a better place for this, but those bits
  can still be moved around now that it's all next to each other.
- link to the syntax page for lookup paths from all places that are
  concerned with it
- add or clarify examples
- add a test verifying a claim from documentation
2024-07-31 21:41:26 +02:00
Eelco Dolstra e8b37e9e70 Merge pull request #11220 from NixOS/release-credits
Credit all contributors in release notes
2024-07-31 21:12:03 +02:00
Eelco Dolstra ed0934b884 Merge pull request #11140 from DeterminateSystems/protocol-features
WorkerProto: Support fine-grained protocol feature negotiation
2024-07-31 17:47:38 +02:00
Eelco Dolstra 69fb2848a5 Merge pull request #11170 from NixOS/release-notes-2.24
Release notes for 2.24
2024-07-31 17:06:49 +02:00
tomberek 4c007bf88f Merge pull request #11216 from tomberek/tomberek.add_flake_headers
fix: add flake headers
2024-07-30 21:19:14 -04:00
Tom Bereknyei ef80217448 fix: add flake headers 2024-07-30 13:19:55 -04:00
Robert Hensing f011cfd28d maintainers/release-*: Add mode line
This lets various tools figure out the language more easily.
2024-07-30 17:54:37 +02:00
Corbin SimpsonandValentin Gagarin ee86e7f361 doc/command-ref/nix-shell: Shebangs can occur anywhere (#11202)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-30 12:51:47 +00:00
Eelco Dolstra c77b671a66 Merge pull request #11221 from pinotree/hurd-fixes
Some fixes for GNU/Hurd
2024-07-30 13:59:19 +02:00
Pino Toscano a1ccf60613 tests: define fallback PATH_MAX
Few filesystem-related tests rely on PATH_MAX for buffers, and PATH_MAX
is optional in POSIX (and not available on the Hurd). To make them build
and pass, provide a fallback definition of PATH_MAX in case not
available.

Ideally speaking, the tests ought to not unconditionally rely on
PATH_MAX, do alternative strategies (e.g. dynamically allocate buffers,
expand them as needed, etc); OTOH this is test code, so it would be more
work that what it would be worth, so IMHO the define fallback is good
enough.
2024-07-30 05:34:34 +02:00
Pino Toscano 7442f4a161 libutil: use /proc/self/exe on Hurd as well
Rely on the Linux-compatible procfs available on the Hurd to get the
path of the current executable.
2024-07-30 05:31:42 +02:00
Pino Toscano d7f46cf28e makefiles: recognize GNU/Hurd
Set HOST_HURD & HOST_UNIX for GNU/Hurd in the makefile-based build
system; the latter variable is important as it will include all the
commit Unix bits.
2024-07-30 05:29:32 +02:00
John Ericson 2b78561335 Merge pull request #11219 from obsidiansystems/better-warning-solution
Make sure we use `-isystem` with Meson on some deps
2024-07-29 17:57:36 -05:00
Robert Hensing 850e4779d6 release-credits: Remove bots 2024-07-30 00:15:49 +02:00
Robert Hensing f380becffa Credit all contributors in release notes 2024-07-29 23:58:38 +02:00
John Ericson 12717325cc Make sure we use -isystem with Meson on some deps
Otherwise we get warnings on external code.
2024-07-29 13:06:26 -04:00
Eelco Dolstra 0b96c586e0 Merge pull request #11195 from DeterminateSystems/tarball-roots
Improve handling of tarballs that don't consist of a single top-level directory
2024-07-29 16:58:59 +02:00
Eelco Dolstra 84243027ec Merge pull request #11127 from NixOS/issue-10635-c-api-error-enum
C API: Make nix_err an enum
2024-07-29 16:00:58 +02:00
Eelco Dolstra f9d55b4d51 Merge pull request #11191 from DeterminateSystems/hash-symbol
Use std::unordered_map for ValueMap
2024-07-29 15:30:37 +02:00
Eelco DolstraandRobert Hensing a3171cec54 Update src/libfetchers/git-utils.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-29 15:12:01 +02:00
Eelco Dolstra e8bf2e74a5 Add release note 2024-07-29 15:09:06 +02:00
Eelco Dolstra 71865dee2d Fix fetchTarball docs 2024-07-29 15:04:55 +02:00
Robert Hensing 6e3bba5e26 Merge pull request #11171 from DeterminateSystems/speed-up-tarball-downloads
Increase download buffer size and improve tarball import logging
2024-07-29 15:02:35 +02:00
Eelco Dolstra 7c18b4d060 Don't dereference top-level regular files
Since this yielded an empty directory as far back as Nix 2.3, we don't
really need special handling for executables vs non-executables.
2024-07-29 14:34:02 +02:00
Eelco Dolstra e0012b97ab Split tarball-specific logic from GitFileSystemObjectSink 2024-07-29 14:26:25 +02:00
Eelco Dolstra 3c0963487e Merge pull request #11196 from NixOS/rename-lock-read
Rename SyncBase::read() -> readLock()
2024-07-29 14:10:46 +02:00
Eelco Dolstra 836d24d6e8 Merge pull request #11209 from pinotree/libutil-current-process-includes
libutil: fix/improve includes in current-process.cc
2024-07-29 14:01:02 +02:00
Eelco Dolstra 9e2bed7827 Merge pull request #11206 from tie/getxattr-enotsup
libstore: return ENOTSUP for getxattr functions
2024-07-29 14:00:36 +02:00
Eelco Dolstra 3faa77bb82 Merge pull request #11200 from NixOS/buildNoTests-no-unit-tests
buildNoTests: Disable unit tests
2024-07-29 13:57:23 +02:00
Eelco Dolstra 673f2dcadb Merge pull request #11203 from pinotree/libutil-ctor-remove-template-id
libutil: remove template id from constructors
2024-07-29 13:56:55 +02:00
Pino Toscano c34077578e libutil: fix/improve includes in current-process.cc
- move <sys/resource.h> from a __linux__ block to a !_WIN32 block: this
  matches what the actual code does, using getrlimit() & setrlimit() in
  !_WIN32 blocks
- drop <sys/mount.h>, which is not portable, and it is not used
2024-07-28 17:33:24 +02:00
Pino Toscano 96e06b2b06 libutil: remove template id from constructors
This is not allowed in C++20, and GCC 14 warns about it:

../src/libutil/ref.hh:26:20: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   26 |     explicit ref<T>(const std::shared_ptr<T> & p)
      |                    ^
../src/libutil/ref.hh:26:20: note: remove the '< >'
../src/libutil/ref.hh:33:21: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   33 |     explicit ref<T>(T * p)
      |                     ^
../src/libutil/ref.hh:33:21: note: remove the '< >'
2024-07-28 16:35:09 +02:00
Robert Hensing 0e151bcbf0 Merge pull request #11204 from pinotree/libcmd-editline-helpers
libcmd: do not compile editline helpers when building w/ readline
2024-07-28 16:10:32 +02:00
Valentin Gagarin 933f2c086a docs: fix link to building instructions (#11207) 2024-07-28 13:34:48 +00:00
Ivan Trubach 1b47748e5a libstore: return ENOTSUP for getxattr functions
This change updates the seccomp profile to return ENOTSUP for getxattr
functions family. This reflects the behavior of filesystems that don’t
support extended attributes (or have an option to disable them), e.g.
ext2.

The current behavior is confusing for some programs because we can read
extended attributes, but only get to know that they are not supported
when setting them. In addition to that, ACLs on Linux are implemented
via extended attributes internally and if we don’t return ENOTSUP, acl
library converts file mode to ACL.
https://git.savannah.nongnu.org/cgit/acl.git/tree/libacl/acl_get_file.c?id=d9bb1759d4dad2f28a6dcc8c1742ff75d16dd10d#n69
2024-07-28 13:28:52 +03:00
Pino Toscano e0198c513a libcmd: do not compile editline helpers when building w/ readline
The internal "completionCallback" and "listPossibleCallback" helpers
are used only when building with editline; hence, do not build then
when using readline, matching their usage in
"ReadlineLikeInteracter::init()".
2024-07-28 11:40:16 +02:00
Robert Hensing aa2b1d10e2 Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md 2024-07-27 14:58:57 +02:00
Robert Hensing 9f1e73ed37 Merge pull request #11199 from NixOS/troubleshoot-remote-build-tests
Troubleshoot remote build tests
2024-07-27 14:25:10 +02:00
Robert Hensing cc5b8cdc85 buildNoTests: Disable unit tests
This seems to have been the intent all along.

The odd combination of unit tests, but no functional tests caused a
build error where some data for the unit test was source-filtered out.
Apparently. It's unclear to me why that happened, so I'm proposing this
alternate "fix" to get the buildNoTests to pass.

It would be nice to test more configurations, but this mode of building
is on the way out anyway, so let's just make it pass and see what
configurations make sense to test as part of the meson migration.
2024-07-27 13:42:03 +02:00
Robert Hensing 7c5a0b06a4 tests/nixos/remote-builds: Wait for multi-user
This should make the test more robust, considering the strange hang
in https://hydra.nixos.org/build/267517233/nixlog/8

`builder` seems to have reached `multi-user.target` before the
SSH connection was established, but this seems to be coincidental.
This does tell us that enforcing this has a minimal cost in terms
of runtime.

Waiting for `multi-user.target` on the client is honestly paranoid,
but flaky tests are very bad for productivity.
2024-07-27 13:08:30 +02:00
Robert Hensing f4464873f5 tests/nixos/remote-builds: Print hello world to stderr
Trying to learn more about enigmatic spurious hang at
https://hydra.nixos.org/build/267517233/nixlog/8
- builder1 seems to have started properly
- ssh connection and session are established
- ssh client doesn't exit or client.succeed does not return
  for some reason.

Seeing the stdout on the console might give a tiny bit more info.
2024-07-27 13:01:56 +02:00
Robert Hensing 04c20dc0c0 Merge pull request #11197 from NixOS/aws-sdk-cpp-like-nixpkgs
dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
2024-07-27 03:30:39 +02:00
Robert Hensing 17b5d40445 package.nix: Empty build inputs if not doBuild 2024-07-27 02:39:55 +02:00
Robert Hensing 22f943bb1f dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.

Changes:

- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)

  > nix: disable big-parallel for aws-sdk-cpp

  > aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
  > which does not justify the requirement on big parallel.

  > Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.

  > I can finally build nix without requiring a big-parallel machine.

- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)

  > nix: use [ ] instead null to empty requiredSystemFeatures

  > fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
2024-07-27 02:16:05 +02:00
Robert Hensing 6af40f488a Rename SyncBase::read() -> readLock()
Make it explicit so it's clear what it's about when I and other
contributors read its call sites.
2024-07-27 01:39:13 +02:00
Robert Hensing 95845d92f7 Merge pull request #11192 from DeterminateSystems/store-sharedsync
Store: Use SharedSync
2024-07-27 01:32:13 +02:00
Robert Hensing 861bd102a6 Merge pull request #11167 from NixOS/repl-test-rejiggle
Fix repl test for `buildReadlineNoMarkdown`
2024-07-27 00:55:57 +02:00
Robert Hensing 88e8c9017a Merge pull request #11187 from Mic92/diff-closure-fix
diff-closures: fix a use after free
2024-07-27 00:52:38 +02:00
Eelco Dolstra c1f7ba7a98 Merge pull request #11190 from DeterminateSystems/unnecessary-eval-string
nix repl: Remove unnecessary call to evalString
2024-07-26 22:27:51 +02:00
Eelco Dolstra 5e83c0427f Fix test 2024-07-26 20:46:07 +02:00
Eelco Dolstra b88950ec77 Update fetchTree docs 2024-07-26 20:34:04 +02:00
Eelco Dolstra 06b686b62d Handle tarballs that don't consist of a single top-level directory
Fixes #4785 (top-level directories are no longer merged into one).

Fixes #10983 (top-level non-directories are no longer discarded).
2024-07-26 20:24:58 +02:00
Eelco Dolstra d9ba2a1634 Fix error message 2024-07-26 19:06:49 +02:00
Eelco Dolstra ea46264bd3 Store: Use SharedSync for state 2024-07-26 16:14:03 +02:00
Eelco Dolstra ce663d75e3 LRUCache: Mark size() as const 2024-07-26 16:13:00 +02:00
Eelco Dolstra 6d843ce9fe Provide std::hash<Symbol> 2024-07-26 16:06:09 +02:00
Eelco Dolstra 2141a52ca3 nix repl: Remove unnecessary call to evalString
This crashes with the multithreaded evaluator, which checks against
attempts to finish an already finished value.
2024-07-26 15:40:32 +02:00
Robert Hensing c274e005b6 Merge pull request #11186 from cole-h/fixup-docs-build
Fix reference to experimental features docs
2024-07-25 21:47:18 +02:00
Jade Lovelace 07aeedd37e diff-closures: remove gratuitous copy
This was done originally because std::smatch does not accept `const char
*` as iterators. However, this was because we should have been using
std::cmatch instead.

(cherry picked from commit https://git.lix.systems/lix-project/lix/commit/12a5838d11f790d36e2ac626e8916a2c19bcb80b)
2024-07-25 21:41:31 +02:00
Jade Lovelace 492715c0bb diff-closures: fix a use after free
Found by looking for interesting asan reports from the test suite.

What happened here is that name got overwritten, but it was what
actually held the backing memory for the thing it got overwritten by,
which was a by-reference value coming out of std::regex.

Due to absurd reasons I cannot seem to use a string_view iterator here,
so I just copy the string with a longer lifetime instead. idk lol

==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0
READ of size 3 at 0x503000014c61 thread T0
    0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c)
    1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33
    <...>
    7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6
    8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16
    9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26
    10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65)
freed by thread T0 here:
    0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470)
    <...>
    6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9
    7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13
    8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    <...>

previously allocated by thread T0 here:
    0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38)
    <...>
    9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4
    10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21
    11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

(cherry-picked from https://git.lix.systems/lix-project/lix/commit/b9b1bbd22fc3b34a4d8e2090f0d033f742c32ee0)
2024-07-25 21:41:31 +02:00
Cole Helbling 90459e60dc Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 21:41:31 +02:00
Cole Helbling 6e178cd899 Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 11:38:45 -07:00
Robert Hensing 6ec123ad6b Merge pull request #11131 from rhendric/rhendric/pipe-operators
libexpr: experimental pipe operators
2024-07-25 16:58:43 +02:00
Robert Hensing 76f04b4146 Merge pull request #11111 from NixOS/grep-safety-AGAIN
Grep newline safety
2024-07-25 16:30:33 +02:00
Eelco Dolstra 499a9b5f89 Merge pull request #11182 from Mic92/less-alloc
build-remote: only allocate storeUri once
2024-07-25 16:23:06 +02:00
Robert Hensing 55a654abfd Make panic() and unreachable() robust
Plus one or two tweaks.
2024-07-25 15:50:01 +02:00
Robert HensingandJörg Thalheim f0fe1d880d Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2024-07-25 15:39:15 +02:00
Robert HensingandJohn Ericson baa28159d3 Update tests/functional/test-infra.sh
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-25 15:38:02 +02:00
Jörg Thalheim 9b5ce9acc2 build-remote: only allocate storeUri once
also it's probably not much overhead compared to the networking stuff
it, but it's less code at least.
2024-07-25 14:04:03 +02:00
Eelco Dolstra 99c20d6624 Merge pull request #11179 from obsidiansystems/misc-fix
Misc fixes
2024-07-25 14:01:54 +02:00
John Ericson ca1e1e983f Merge pull request #11180 from Mic92/override-warnings
add werror=suggest-override
2024-07-25 03:00:50 -04:00
John Ericson 36252fed6b Merge pull request #11176 from obsidiansystems/flake-test-group
Put flake functional tests in their own group
2024-07-25 02:41:11 -04:00
Jörg Thalheim 63e50a4b56 add werror=suggest-override
Improves code readability by making overrides explicit.
Inspired by lix code-base
2024-07-25 07:41:12 +02:00
Jörg Thalheim 2c07ea8abb build-utils-meson: remove oudated meson build comment 2024-07-25 07:40:36 +02:00
Jörg Thalheim 6c38bc0952 {src/perl,build-utils-meson/diagnostics}: sort cflags
This makes them easier to copy between places.
2024-07-25 07:38:00 +02:00
Robert Hensing 6662c9012d Merge pull request #11177 from obsidiansystems/skip-exit-code
Change skipped test error code from 99 to 77
2024-07-25 06:59:18 +02:00
John Ericson 90f7f2139e Merge pull request #11115 from NixOS/doc-derivation
Document builtins.derivation
2024-07-25 00:21:53 -04:00
John Ericson 1ae5738317 Fix some warnings
I think they came from the last Nixpkgs bump.
2024-07-25 00:02:43 -04:00
Robert Hensing b711fcbef9 rl-next: Drop zzz-other. Number soup. 2024-07-25 06:00:59 +02:00
John Ericson e92dd06a7b build-remote: Cope with long store URLs by falling back on hashing
I hit this in the Meson port of the functional tests, because the use of
standalone build directories.
2024-07-25 00:00:52 -04:00
Robert Hensing 7275d68d3b rl-next: Add top 10 by +1 reactions on PRs
We should use a metric that weighs the related issues.
Counterbalancing time doesn't make much sense to me.
If it's around for longer, the fix will be relevant to more people.
2024-07-25 05:57:53 +02:00
Robert Hensing 8a7e31362a rl-next: Add credit 2024-07-25 05:57:06 +02:00
John Ericson 3b49f7a143 Deduplicate our many package.nix a bit (#11175)
- They should all be built in parallel

- They should all use strict deps by default
2024-07-25 03:12:39 +00:00
Valentin Gagarin a2fed6db9e manual: Contributing -> Development, Hacking -> Building (#9014)
* manual: Contributing -> Development, Hacking -> Building

what's currently called "hacking" are really instructions for setting up
a development environment and compiling from source. we have
a contribution guide in the repo (which rightly focuses on GitHub
workflows), and the material in the manual is more about working
on the code itself.

since we'd otherwise have three headings that amount to "Building Nix",
this change also moves the "classic Nix" instructions to the top.

we may want to reorganise this in the future, and bring
contributor-oriented information closer to the code, but for now let's
stick to more accurate names to ease navigation.
2024-07-25 02:53:06 +00:00
John Ericson dcbe2453f5 Change skipped test error code from 99 to 77
Meson uses a venerable GNU convention described in
https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
in which:

> When no test protocol is in use, an exit status of 0 from a test
> script will denote a success, an exit status of 77 a skipped test, an
> exit status of 99 a hard error, and any other exit status will denote
> a failure.

77 is thus what we want, not 99.
2024-07-24 22:36:43 -04:00
John Ericson f4915af71c Put flake functional tests in their own group
This is a nice thing to have, and it made it easier to work on the
Meson-ifcation of these functional tests too.
2024-07-24 22:31:37 -04:00
Valentin GagarinandRyan Hendrickson dba1142c01 docs: add identifiers (#11174)
* docs: add identifiers

* clarify attribute set notation and add examples

* add definition of names

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-25 03:45:34 +02:00
e062021314 fix NIX_PATH for real (#11079)
* fix NIX_PATH overriding

- test restricted evaluation
- test precedence for setting the search path

Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-07-24 21:17:15 +00:00
Eelco Dolstra 01839b525c Show when we're unpacking an archive into the Git cache
This happens in parallel with the download (which starts later), so
you only see this message when the download has finished but the
import hasn't.
2024-07-24 20:22:26 +02:00
Eelco Dolstra f6a9a71b38 Warn if the download buffer is full 2024-07-24 20:14:31 +02:00
Eelco Dolstra 8ffea0a018 Add 'download-buffer-size' setting
We are piping curl downloads into `unpackTarfileToSink()`, but the
latter is typically slower than the former if you're on a fast
connection. So the download could appear unnecessarily slow. (There is
even a risk that if the Git import is *really* slow for whatever
reason, the TCP connection could time out.)

So let's make the download buffer bigger by default - 64 MiB is big
enough for the Nixpkgs tarball. Perhaps in the future, we could have
an unlimited buffer that spills data to disk beyond a certain
threshold, but that's probably overkill.
2024-07-24 20:10:45 +02:00
Eelco Dolstra caf4e98f0c Log download durations 2024-07-24 20:10:41 +02:00
Robert Hensing 4bfc96f376 Fix and update release notes 2024-07-24 19:36:49 +02:00
Robert Hensing 459ee00563 Render the release notes when building the manual from dev shell 2024-07-24 19:36:49 +02:00
Ryan Hendrickson e086d5d899 libexpr: experimental pipe operators 2024-07-24 13:17:28 -04:00
Robert Hensing 3172e88af5 Make abort() call sites log first 2024-07-24 16:52:04 +02:00
Eelco Dolstra 874c1bdbbf Merge pull request #11169 from NixOS/flake-update
Flake update
2024-07-24 16:52:03 +02:00
Eelco Dolstra 3be7c0037e WorkerProto: Support fine-grained protocol feature negotiation
Currently, the worker protocol has a version number that we increment
whenever we change something in the protocol. However, this can cause
a collision between Nix PRs / forks that make protocol changes
(e.g. PR #9857 increments the version, which could collide with
another PR). So instead, the client and daemon now exchange a set of
protocol features (such as `auth-forwarding`). They will use the
intersection of the sets of features, i.e. the features they both
support.

Note that protocol features are completely distinct from
`ExperimentalFeature`s.
2024-07-24 16:23:37 +02:00
Eelco Dolstra b13ba7490c Merge pull request #11162 from NixOS/rename-stack-limit
Rename osStackLow -> osStackLimit
2024-07-24 16:18:26 +02:00
Robert Hensing 171ef75218 Rename pre-commit-hooks -> git-hooks-nix
Following the upstream rename
2024-07-24 15:55:57 +02:00
Robert Hensing c316f1557d flake: Switch to nixos-24.05 channel
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
  → 'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
2024-07-24 15:31:30 +02:00
Robert Hensing f9a23c8d21 flake.nix: Remove outdated comment 2024-07-24 15:30:28 +02:00
Robert Hensing 650f189402 flake.nix: Pin libgit2 to a release 2024-07-24 15:24:10 +02:00
Robert Hensing a16df88252 flake.lock: Update
Flake lock file updates:

• Updated input 'flake-compat':
    'github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9?narHash=sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm%2B504Ch3sNKLd8%3D' (2023-01-17)
  → 'github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33?narHash=sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U%3D' (2023-10-04)
• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/9126214d0a59633752a136528f5f3b9aa8565b7d?narHash=sha256-sB4SWl2lX95bExY2gMFG5HIzvva5AVMJd4Igm%2BGpZNw%3D' (2024-04-01)
  → 'github:hercules-ci/flake-parts/9227223f6d922fee3c7b190b2cc238a99527bbb7?narHash=sha256-pQMhCCHyQGRzdfAkdJ4cIWiw%2BJNuWsTX7f0ZYSyz0VY%3D' (2024-07-03)
• Updated input 'libgit2':
    'github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D' (2023-10-18)
  → 'github:libgit2/libgit2/503b66cf00ad7dca940148529f60b1a409ccc462?narHash=sha256-tDUQi%2Bs8sxJ30SmUH7Ln9WmDz5jGatlgKumjwi7KnCo%3D' (2024-07-17)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/be3ca229c85e978880babdeda9748b14e6aa008f?narHash=sha256-L61BXz7n/yNzOeZ3FqlnUmxj4145JOVeq9fvQTQzbNM%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
• Updated input 'pre-commit-hooks':
    'github:cachix/pre-commit-hooks.nix/40e6053ecb65fcbf12863338a6dcefb3f55f1bf8?narHash=sha256-nMirxrGteNAl9sWiOhoN5tIHyjBbVi5e2tgZUgZlK3Y%3D' (2024-04-12)
  → 'github:cachix/pre-commit-hooks.nix/f451c19376071a90d8c58ab1a953c6e9840527fd?narHash=sha256-6FPUl7HVtvRHCCBQne7Ylp4p%2BdpP3P/OYuzjztZ4s70%3D' (2024-07-15)
• Removed input 'pre-commit-hooks/flake-utils'
2024-07-24 15:18:50 +02:00
Robert Hensing 68693276f9 Update fixupBoehmStackPointer doc
lo might have made sense in the bdwgc code, maybe?, but not here.
2024-07-24 13:02:49 +02:00
Robert Hensing 907b0a371a Rename osStackLow -> osStackLimit
This is in accordance with ARM's naming convention.
"Low" is confusing, because it could refer to either the cold end
of the stack as an abstract data type, or a low address.
These are different places, because the stack grows down through
the address space.
2024-07-24 13:02:49 +02:00
Robert Hensing e48e0cbab0 markdown.hh: Improve includes 2024-07-24 12:54:40 +02:00
Robert Hensing 0bd2d36375 Document renderMarkdownToTerminal 2024-07-24 12:53:37 +02:00
Robert Hensing 7d4d34a27d eval-gc.cc: Fix warning 2024-07-24 12:48:38 +02:00
Robert Hensing 6e680a6644 tests/functional/repl: Improve failure reporting 2024-07-24 12:48:38 +02:00
Robert Hensing c4ae9bb45b tests/functional/repl: Normalize final prompt 2024-07-24 12:48:38 +02:00
Robert Hensing ca2cc26e12 tests/functional/repl: Improve precision and readability
... as well as match buildReadlineNoMarkdown.

Unfortunately it doesn't support long inputs or multiline inputs
for now.
This needs to make better use of the interacter interface.
2024-07-24 12:48:38 +02:00
Robert Hensing 712ce2feac ReadlineLikeInteracter::getLine: Add _NIX_TEST_REPL_ECHO env var
... for testing
2024-07-24 12:48:38 +02:00
Robert Hensing 97b0114ab8 renderMarkdownToTerminal: Add _NIX_TEST_RAW_MARKDOWN env var
For testing only.
2024-07-24 12:48:38 +02:00
Robert Hensing 2b4e3f04a4 markdown.cc: Format
Slightly custom because the automated formatting messes up the
braced initializer with named fields.
2024-07-24 01:02:48 +02:00
Eelco Dolstra fb450de20e Merge pull request #11163 from NixOS/rejiggle-getGCCyles-buildNoGc
Rejiggle getGCCycles() for buildNoGc
2024-07-23 17:17:47 +02:00
Robert Hensing 5d6bc484be Hide getGCCycles when we have no GC
Alternatively, we could make it return 0, but we don't need it in
the first place because the caller exists conditionally too.
2024-07-23 16:24:43 +02:00
Robert Hensing eb89e50cbb Rejiggle getGCCycles() for buildNoGc 2024-07-23 16:21:43 +02:00
John Ericson e20b0a5dda Merge pull request #11159 from DeterminateSystems/framedsource-terminate
~FramedSource(): Don't throw an exception if the remote has disconnected
2024-07-23 09:19:39 -04:00
Eelco Dolstra 498eed0a25 ~FramedSource(): Don't throw an exception if the remote has disconnected
This would cause the daemon to crash with a call to terminate().
2024-07-23 10:56:29 +02:00
Robert Hensingandtomberek 4457cebe05 Update comment in tests//vars-and-functions.sh
Co-authored-by: tomberek <tomberek@users.noreply.github.com>
2024-07-23 10:24:18 +02:00
Eelco Dolstra babfd0cd99 Merge pull request #11150 from hercules-ci/issue-11141-broken-sp-corrector
Fix issue #11141 broken stack pointer corrector
2024-07-22 19:59:46 +02:00
John Ericson 046cc8cbb3 Merge pull request #11156 from obsidiansystems/fix-bsd-pthread
Fix some BSD builds missing pthread functions
2024-07-22 13:58:41 -04:00
John Ericson 00fd180a79 Merge pull request #11153 from nix-windows/wine-ci
Run unit tests with wine
2024-07-22 13:35:32 -04:00
John Ericson d39bbcabb9 Fix some BSD builds missing pthread functions
In addition to adding the missing thread deps in the last commit, we
also appear to need to skip `-Wl,--as-needed` flags that Meson wants to
use, but doesn't work with our *BSD toolchains.

See https://github.com/mesonbuild/meson/issues/3593
2024-07-22 13:10:03 -04:00
John Ericson da625b8817 Merge pull request #11155 from obsidiansystems/build-system-fixes
Two Build system fixes
2024-07-22 12:16:56 -04:00
John Ericson d3cee8160c Add missing threads deps 2024-07-22 11:46:54 -04:00
Robert Hensing f2e0cecf34 tests/functional/lang-gc: Disable for now 2024-07-22 17:46:08 +02:00
Eelco Dolstra fe158e3e92 Merge pull request #10603 from edolstra/flake-regressions
Run the flake regressions test suite
2024-07-22 17:45:27 +02:00
Eelco Dolstra 9b9fc405d2 Merge pull request #11154 from DeterminateSystems/ping-store
nix ping-store: Redirect to 'nix store info'
2024-07-22 17:38:51 +02:00
Eelco Dolstra d08bb025e1 Merge pull request #11043 from hercules-ci/assert-eq
`assert`: Report why values aren't equal
2024-07-22 17:34:28 +02:00
John Ericson d7024ac9b7 Add S3 opt dep to Meson, and simplify build
Numeric version macros are now defined upstream, so we don't need roll
our own.
2024-07-22 11:11:38 -04:00
John Ericson 823baa25f3 Meson build: libstore check for statvfs 2024-07-22 11:09:53 -04:00
Eelco Dolstra 4878c31815 nix ping-store: Redirect to 'nix store info'
This avoids the double warning

   warning: 'ping-store' is a deprecated alias for 'store ping'
   warning: 'nix store ping' is a deprecated alias for 'nix store info'
2024-07-22 16:39:02 +02:00
John Ericson eea63d5f99 Run unit tests with wine
Fix #10547

The tests do not all run successfully, but that is a separate issue.
2024-07-22 10:31:26 -04:00
John Ericson dc6dbbc1a5 Test exes do not need dev outputs 2024-07-22 10:26:05 -04:00
John Ericson 112373c03c flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/88269ab3044128b7c2f4c7d68448b2fb50456870' (2024-06-03)
  → 'github:NixOS/nixpkgs/be3ca229c85e978880babdeda9748b14e6aa008f' (2024-07-21)
2024-07-22 10:26:05 -04:00
Eelco Dolstra f343364918 Merge remote-tracking branch 'origin/master' into flake-regressions 2024-07-22 16:03:45 +02:00
Eelco Dolstra 40f80e1b5c Merge pull request #11142 from detroyejr/fix-alias-flags
Allow flag aliases
2024-07-22 15:53:41 +02:00
Eelco Dolstra 56a2f7ab1d Merge pull request #11102 from fzakaria/issue-10532
doc: Add comment for fetchurl for name & url
2024-07-22 15:49:41 +02:00
Eelco Dolstra 3ef72040bf Merge pull request #11149 from hercules-ci/gc-stats
libexpr: Track and show GC time and cycle number
2024-07-22 15:39:50 +02:00
Robert Hensing 380becf0db Fix #11141 broken sp corrector 2024-07-22 14:52:08 +02:00
Robert Hensing b16861d82e libexpr: Track and show GC time and cycle number 2024-07-22 12:56:01 +02:00
Robert Hensing c00e396494 maintainers/README: Update Monday meeting time (#11147) 2024-07-22 12:05:50 +02:00
0ec5e3a1bc Progress on Wine CI support, MinGW dev shell with Meson (#10975)
* Only build perl subproject on Linux

* Fix various Windows regressions

* Don't put the emulator hook in test builds

  We run the tests in a separate derivation. Only need it for the dev shell.

* Fix native dev shells

* Fix cross dev shells we don't know how to emulate

Co-authored-by: PoweredByPie <poweredbypie@users.noreply.github.com>
Co-authored-by: Joachim Schiele <js@lastlog.de>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-21 22:03:04 +00:00
Robert Hensing 56757e15cf Merge pull request #11123 from tweag/fix-non-det-parser-printing
Fix non-deterministic inherit printing
2024-07-20 19:12:48 +02:00
Robert Hensing 584f8cb065 Merge pull request #11120 from tweag/early-string-cutoff
parser: Remove empty multiline string parts earlier
2024-07-20 19:10:50 +02:00
detroyejr d54dfbf879 addFlag: test that alias flags are allowed 2024-07-19 15:56:54 -04:00
detroyejr 74dccef004 addFlag: use aliases 2024-07-19 15:05:00 -04:00
Eelco Dolstra c4213f0e6c Merge pull request #11125 from DeterminateSystems/basic-connection
Factor out commonality between WorkerProto::Basic{Client,Server}Connection
2024-07-19 14:26:04 +02:00
Eelco Dolstra 58a79b6943 performOp(): Take a WorkerProto::BasicServerConnection 2024-07-19 13:35:46 +02:00
John Ericson d629b27e3c Merge pull request #11138 from emilazy/push-znxusvxozllv
libmain: add missing header include
2024-07-18 21:46:47 -04:00
Silvan MosbergerandRobert Hensing 0c91bb97e5 parser: Remove empty multiline string parts earlier
Makes parsing more consistent and is a super minor optimisation

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:43:44 +02:00
Silvan MosbergerandRobert Hensing a6dccae223 Fix non-deterministic parser printing
In _very_ rare cases (I had about 7 cases out of 32200 files!),
the order of how inherit-from bindings are printed when using
`nix-instantiate --parse` gets messed up.

The cause of this seems to be because the std::map the bindings are
placed in is keyed on a _pointer_, which then uses an
[implementation-defined strict total order](https://en.cppreference.com/w/cpp/language/operator_comparison#Pointer_total_order).

The fix here is to key the bindings on their displacement instead,
which maintains the same order as they appear in the file.

Unfortunately I wasn't able to make a reproducible test for this in the
source, there's something about the local environment that makes it
unreproducible for me.

However I was able to make a reproducible test in a Nix build on a Nix
version from a very recent master:

    nix build github:infinisil/non-det-nix-parsing-repro

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:42:00 +02:00
Emily 31e151386b libmain: add missing header include 2024-07-18 23:26:22 +01:00
Eelco Dolstra fa7aa0389a FdSource: Fix operator =
This wasn't moving the underlying buffer, so if the buffer was
non-empty, it could lose data.
2024-07-18 16:10:48 +02:00
Eelco Dolstra d231d802f5 Typo 2024-07-18 16:10:48 +02:00
Eelco Dolstra c1d5cf6f34 Factor out commonality between WorkerProto::Basic{Client,Server}Connection
This also renames clientVersion and daemonVersion to the more correct
protoVersion (since it's the version agreed to by both sides).
2024-07-18 16:10:48 +02:00
Robert Hensing 8ce4287409 Merge pull request #11086 from kognise/eval-cache-fixes
Eval cache: fix cache regressions
2024-07-18 14:57:07 +02:00
John Ericson 606ace41d2 Merge pull request #11112 from obsidiansystems/move-schemes-set
Move `uriSchemes` to `*StoreConfig`
2024-07-18 00:39:07 -04:00
John Ericson 2aa9cf34dd Move uriSchemes to *StoreConfig
It is a property of the configuration of a store --- how a store URL is
parsed into a store config, not a store itself.

Progress towards #10766
2024-07-17 23:48:19 -04:00
Farid ZakariaandJohn Ericson 57399bfc0e Refactor unix domain socket store config (#11109)
Following what is outlined in #10766 refactor the uds-remote-store such
that the member variables (state) don't live in the store itself but in
the config object.

Additionally, the config object includes a new necessary constructor
that takes a scheme & authority.

Tests are commented out because of linking errors with the current config system.
When there is a new config system we can reenable them.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-17 23:32:27 -04:00
Eelco Dolstra 17051ca80a Merge pull request #11061 from rhendric/rhendric/reference-manual
docs: fill out language/types.md#type-path
2024-07-17 21:59:05 +02:00
Eelco Dolstra 1e1a8e8ad1 Merge pull request #11058 from hercules-ci/more-nix-shell
Make `#!nix-shell` arguments and options relative to script
2024-07-17 21:52:34 +02:00
John Ericson a915862450 Merge pull request #11126 from DeterminateSystems/dont-send-version-twice
BasicClientConnection::handshake(): Don't send our version twice
2024-07-17 15:50:17 -04:00
Eelco Dolstra 6867cb1096 Merge pull request #11103 from fzakaria/issue-10795
lint: fix shellcheck for misc/systemv/nix-daemon
2024-07-17 21:46:17 +02:00
Eelco Dolstra b23da1ceca Merge pull request #11100 from NixOS/pretty-print-idempotent
Pretty print idempotently
2024-07-17 21:35:27 +02:00
Eelco Dolstra 05751de32b Merge pull request #10931 from hercules-ci/test-run-and-shell-envs
Test the `run` and `shell` envs for stray variables
2024-07-17 21:21:14 +02:00
RTUnrealandValentin Gagarin f0a1c130a1 doc: add example usage for Gitea in tarball fetcher (#11116)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-17 19:08:33 +00:00
Robert Hensing 83d585b423 C API: Make nix_err an enum
This generally gives a better experience with bindings generators,
possibly other tooling.

A possible risk is that some generators may not represent unknown
codes correctly.
Rust bindgen by default generates suitable code:
  * a type alias nix_err = c_int
  * individual constants for the known enum values
It does _not_ generate a closed type that can only hold the values
that were known at code generation time.

If this proves to be a problem, we could instead split the type:

`typedef int nix_err;` for return values
`enum nix_known_err` for code generation.

This would complicate the interface, so let's not do it unless it
is shown to be needed.
2024-07-17 16:54:30 +02:00
Eelco Dolstra 87f8ff23fe BasicClientConnection::handshake(): Don't send our version twice
This was accidentally introduced
in f71b4da0b3.  We didn't notice this
because the version got interpreted by the daemon as the obsolete "CPU
affinity will follow" field, and being non-zero, it would then read
another integer for the ignored CPU affinity.
2024-07-17 16:51:53 +02:00
Eelco Dolstra 621c23bbea Merge pull request #11113 from NixOS/doc-comment-unordered-map
Doc comments: use std::unordered_map
2024-07-17 16:50:48 +02:00
Las Safin 464e5925cb Avoid accessing uninitialized settings in own init (#11117)
The default value for the setting was evaluated by
calling a method on the object _being currently constructed_,
so we were using it before all fields were initialized.

This has been fixed by making the called method static,
and not using the previously used fields at all.

But functionality hasn't changed!
The fields were usually always zero (by chance?) anyway,
meaning the conditional path was always taken.

Thus the current logic has been kept, the code simplified,
and UB removed.

This was found with the helper of UBSan.
2024-07-17 14:10:01 +02:00
Robert Hensing da3eff60bc printList: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing a0635a80b2 printAttrs: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing ece334b532 tests/functional/repl: Characterize side effecting print behavior
Reported on matrix by aleksana:
https://matrix.to/#/!VRULIdgoKmKPzJZzjj:nixos.org/$7wZp5lUDTd-_u6MYo8kWWcysjtqTiQqP8dLI0RDNVVM?via=nixos.org&via=matrix.org&via=nixos.dev
2024-07-17 13:42:05 +02:00
Robert Hensing f5ebaea277 Simplify PosIdx::hash()
In C++ we don't need to salt the hash.
2024-07-17 13:31:31 +02:00
Robert Hensing b230c01f73 Merge pull request #11014 from obsidiansystems/plugins-libmain
Move plugins infra to `libnixmain`
2024-07-17 09:42:09 +02:00
Silvan Mosberger 9fae50ed4b Add parser test for indented strings
So that in the next commit we can see what changes about this test
2024-07-17 02:43:47 +02:00
John Ericson 463256b9e8 Merge pull request #11122 from L-as/fix-ub
Fix ub
2024-07-16 18:51:17 -04:00
Las Safin a1f3f103bc Check if drv is initialized in DerivationGoal::waiteeDone
It might not be set, in which case we shouldn't do anything.
Surprisingly, this somehow did not cause segfaults before?

Caught by UBSan.
2024-07-16 22:01:39 +00:00
Las Safin 5b6a21acc5 Avoid casting function pointer in libutil test support
Casting function pointers seems to be almost always UB.
See https://stackoverflow.com/questions/559581/casting-a-function-pointer-to-another-type

Fixed by doing the casting of `void*` to `std::string*` inside the function instead.

Caught by UBSan.
2024-07-16 22:01:34 +00:00
Robert Hensing 8f22245fdf Merge pull request #11119 from philiptaron/attrbute
builtins.genericClosure: fix documentation typo
2024-07-16 23:54:59 +02:00
Philip Taron 0a1a116f4b builtins.genericClosure: fix documentation typo 2024-07-16 13:51:52 -07:00
Robert Hensing 7dce074634 tests/functional/lang: Avoid /usr/bin/env for sandbox 2024-07-16 22:43:56 +02:00
Robert Hensing d0e9878389 Remove unused boost include and split out std-hash.hh
Splitting it out immediately answers questions like [this],
without increasing the number of compilation units.

I did consider using boost::hash_combine instead, but it doesn't seem
to be quite as capable, accepting only two arguments.

[this]: https://github.com/NixOS/nix/pull/11113#discussion_r1679991573
2024-07-16 22:31:25 +02:00
Robert Hensing f9a1d6b018 tests/functional/lang: Add post processing and remove certain line numbers 2024-07-16 17:36:30 +02:00
Eelco Dolstra 9300f855fc Merge pull request #11114 from DeterminateSystems/fix-warning
Pos::getSnippetUpTo(): Fix warning
2024-07-16 17:24:43 +02:00
Robert Hensing 74698d54c8 Document builtins.derivation 2024-07-16 17:20:09 +02:00
Robert Hensing 64b46000ad Add std::hash<PosIdx> 2024-07-16 16:46:41 +02:00
Eelco Dolstra 3d8fa9f668 Pos::getSnippetUpTo(): Fix warning 2024-07-16 16:34:13 +02:00
Robert HensingandEelco Dolstra 6c9d62dceb Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-16 15:19:39 +02:00
Eelco Dolstra 9c6678da0e Merge pull request #11092 from DeterminateSystems/hash-SourcePath
Use std::unordered_map for the EvalState caches
2024-07-16 11:06:43 +02:00
John Ericson 4bbadba83a Merge pull request #11108 from obsidiansystems/remote-store-constructors
Ensure we can construct remote store configs in isolation
2024-07-15 22:51:22 -04:00
Robert Hensing 41a03738d6 tests/functional: Also keep plain grep calls safe from newlines 2024-07-16 01:54:12 +02:00
Robert Hensing 644b97ce25 tests/functional: Make our grep* helpers reject newlines in the query
Newlines behave like *OR*; not "and then".
2024-07-16 01:41:22 +02:00
Robert Hensing f2df3f0c6c tests/vars-and-functions: Add callerPrefix helper 2024-07-16 01:40:33 +02:00
Robert Hensing 783a8341ee tests/functional: Support negative codes in expect, expectStderr 2024-07-16 01:32:54 +02:00
Robert Hensing 03326d606f Merge pull request #11110 from GoldsteinE/fix-repl-tests
tests/functional/repl.sh: fail test on wrong stdout
2024-07-16 00:23:42 +02:00
John Ericson 808082ea03 Ensure we can construct remote store configs in isolation
Progress towards #10766

I thought that #10768 achieved, but when I went to use this stuff (in
Hydra), turns out it did not. (Those `using FooConfig;` lines were not
working --- they are so finicky!) This PR gets the job done, and adds
some trivial unit tests to make sure I did what I intended.

I had to add add a header to expose `SSHStoreConfig`, after which the
preexisting `ssh-store-config.*` were very confusingly named files, so I
renamed them to `common-ssh-store-config.hh` to match the type defined
therein.
2024-07-15 17:32:49 -04:00
John Ericson 0feeab755a Move plugins infra to libnixmain
They are not actually part of the store layer, but instead part of the
Nix executable infra (libraries don't need plugins, executables do).

This is part of a larger project of moving all of our legacy settings
infra to libmain, and having the underlying libraries just have plain
configuration structs detached from any settings infra / UI layer.

Progress on #5638
2024-07-15 17:26:03 -04:00
John Ericson 1a273a623f Inline settings.pluginFiles.name
In theory the warning is more noisy now, but in practice this will not
happen unless the client is older than 2.14 (highly unlikely).
2024-07-15 16:50:57 -04:00
Las Safin 846869da0e Make goals use C++20 coroutines (#11005)
undefined
2024-07-15 16:49:15 -04:00
Goldstein 1bec90e3c4 tests/functional/repl.sh: fail test on wrong stdout
Previous test implementation assumed that grep supports newlines
in patterns. It doesn't, so tests spuriously passed, even though
some tests outputs were broken.

This patches output (and expected output) before grepping,
so there're no newlines in pattern.
2024-07-15 23:11:26 +03:00
Robert Hensing c6b5503190 Merge pull request #11072 from NixOS/doc-comments
Doc comments
2024-07-15 21:00:42 +02:00
Robert Hensing 61a4d3d45c getSnippetUpTo: Return optional
This makes it possible to certain discern failures from empty
snippets, which I think is an ok review comment.

Maybe it should do so for swapped column indexes too, but I'm not
sure.

I don't think it matters in the grand scheme. We don't even have
a real use case for `nullopt` now anyway.

Since we don't have a use case, I'm not applying this logic to
higher level functions yet.
2024-07-15 20:10:45 +02:00
Robert Hensing 03d33703ef Revert "Use HintFmt for doc comments"
Unfortunately these don't render correctly, because they go into the
markdown renderer, instead of the terminal.

```
nix-repl> :doc lib.version
Attribute '[35;1mversion[0m'

    … defined at [35;1m/home/user/h/nixpkgs/lib/default.nix:73:40[0m
```

We could switch that to go direct to the terminal, but then we should
do the same for the primops, to get a consistent look.

Reverting for now.

This reverts commit 3413e0338cbee1c7734d5cb614b5325e51815cde.
2024-07-15 19:56:40 +02:00
Robert Hensing ce31a0457f Use HintFmt for doc comments 2024-07-15 19:56:40 +02:00
Robert Hensing 6a125e65d0 Revert "Doc comments: use std::unordered_map"
hash<SourcePath> isn't implemented yet, and I can't cherry-pick
a bug-free commit yet.

This reverts commit 95529f31e3bbda99111c5ce98a33484dc6e7a462.
2024-07-15 19:56:40 +02:00
Robert Hensing ac89df815d libcmd/repl.cc: Explain evalString call and defend 2024-07-15 19:56:40 +02:00
Robert HensingandEelco Dolstra 21817473e8 Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 19:56:40 +02:00
Robert Hensing 131b6ccc71 nixexpr.hh: Avoid the warning and pragmas 2024-07-15 19:56:40 +02:00
Robert Hensing 6bbd493d49 libcmd/repl-interacter: INT_MAX -> numeric_limits 2024-07-15 19:56:40 +02:00
Robert Hensing 8a855296f5 tests/function/repl: Characterise the missing doc comment behavior 2024-07-15 19:56:40 +02:00
Robert Hensing 71cb8bf509 libexpr: Rename "column" fields to offset
... because that's what they are.
2024-07-15 19:56:40 +02:00
Robert Hensing 77e9f9ee82 libexpr: Get rid of unused line tracking fields 2024-07-15 19:56:40 +02:00
Robert Hensing f9243eca75 tests/functional/repl.sh: Work around GHA failure 2024-07-15 19:56:40 +02:00
Robert Hensing cef11b23e8 Add missing .sh in _NIX_TEST_ACCEPT=1 message 2024-07-15 19:56:40 +02:00
Robert Hensing d4f576b0b2 nix repl: Render docs for attributes 2024-07-15 19:56:40 +02:00
Robert Hensing 491b9cf415 Refactor: extract DocComment::getInnerText(PosTable) 2024-07-15 19:56:40 +02:00
Robert Hensing e68234c4f9 libexpr: Rearrange lexer files so that yylex_init_extra can be found 2024-07-15 19:56:40 +02:00
Robert Hensing 7fae378835 Track doc comments and render them in :doc 2024-07-15 19:56:40 +02:00
Robert Hensing e5af7cbeb9 libutil: Add Pos::getSnippetUpTo(Pos) 2024-07-15 18:41:11 +02:00
Robert Hensing 63f520fd00 doc/testing: Typo 2024-07-15 18:41:11 +02:00
Farid ZakariaandEelco Dolstra 945fff5674 Apply suggestions from code review
Add @edolstra suggestion fixes.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 09:12:56 -07:00
Eelco Dolstra 550b3479cf Include the accessor in the SourcePath hash 2024-07-15 15:46:30 +02:00
Farid Zakaria 104aba0fad Remove nix-daemon from exclusion 2024-07-14 19:57:55 -07:00
Farid Zakaria bc801e2c59 lint: fix shellcheck for misc/systemv/nix-daemon
Got shellcheck passing for misc/systemv/nix-daemon

Not sure how to test this since it's not running on my NixOS machine and
I see no references to it in the directory otherwise.

See #10795
2024-07-14 19:56:03 -07:00
Farid Zakaria a4ce96e5f1 doc: Add comment for fetchurl for name & url
fetchurl can be given a name and url aside from just the url.
Giving a name can be useful if the url has invalid characters such as
tilde for the store.
2024-07-14 19:07:18 -07:00
Robert Hensing 9d7397c4ba Merge pull request #11093 from NixOS/fix-concatStringsSep
Fix C++ `concatStringsSep`
2024-07-14 13:11:29 +02:00
Robert Hensing 7e604f716c concatStrings: Give compiler access to definition for inlining
... at call sites that are may be in the hot path.

I do not know how clever the compiler gets at these sites.
My primary concern is to not regress performance and I am confident
that this achieves it the easy way.
2024-07-14 12:20:45 +02:00
Robert Hensing 97e01107ec dropEmptyInitThenConcatStringsSep -> concatStringSep: empty separator
When the separator is empty, no difference is observable.

Note that concatStringsSep has centralized definitions. This adds the
required definitions. Alternatively, `strings-inline.hh` could be
included at call sites.
2024-07-14 12:10:39 +02:00
Robert Hensing d40fdb5711 dropEmptyInitThenConcatStringsSep: Update doc and deprecate 2024-07-14 11:50:20 +02:00
Robert Hensing 1c97718146 dropEmptyInitThenConcatStringsSep: Allow it to drop items again
It's usually harmless, if it occurs at all.
2024-07-13 03:06:24 +02:00
Robert Hensing 6b2c277c36 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are not empty
... but if they are, I'd like to see at least a hint of it so that
I'd know to fix it.
2024-07-13 03:06:24 +02:00
Robert Hensing 76b2d5ef3d dropEmptyInitThenConcatStringsSep -> concatStringSep: PATH handling
It's still wrong, but one step closer to correct.
Not that anyone should use "" or "." in their PATH, but that is not
for us to intervene.
2024-07-13 03:06:24 +02:00
Robert Hensing 9ca42d5da2 dropEmptyInitThenConcatStringsSep -> concatStringSep: setting value was already harmed
Considering that `value` was probably parsed with tokenizeString
prior, it's unlikely to contain empty strings, and we have no
reason to remove them either.
2024-07-13 03:06:24 +02:00
Robert Hensing 4029426ca8 dropEmptyInitThenConcatStringsSep -> concatStringSep: tokens from tokenizeString are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0fe3525223 illegal configuration line -> syntax error in configuration line
The law has nothing to do with this, although I do feel like a
badass when I mess with the config.
I'm a conf artist.
2024-07-13 03:06:24 +02:00
Robert Hensing cf3c5cd189 dropEmptyInitThenConcatStringsSep -> concatStringSep: showVersions version is not empty 2024-07-13 03:06:24 +02:00
Robert Hensing d9043021df dropEmptyInitThenConcatStringsSep -> concatStringSep: break nix help "" "" "" build
Garbage in, error out. Experimental CLI. Zero derivations given.
2024-07-13 03:06:24 +02:00
Robert Hensing 062672b022 dropEmptyInitThenConcatStringsSep -> concatStringSep: CLI commands are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0480bfe50b dropEmptyInitThenConcatStringsSep -> concatStringSep: do not drop attributes with empty names
Empty attributes are probably not well supported, but the least we
could do is leave a hint.
Attribute path rendering and parsing should be done according to
Nix expression syntax in my opinion.
2024-07-13 03:06:24 +02:00
Robert Hensing 4b34feb4c2 dropEmptyInitThenConcatStringsSep -> concatStringSep: system string should not be empty 2024-07-13 03:06:24 +02:00
Robert Hensing 837c3612d4 dropEmptyInitThenConcatStringsSep -> concatStringSep: escaped shell args are never empty 2024-07-13 03:06:24 +02:00
Robert Hensing 3b77f13451 dropEmptyInitThenConcatStringsSep -> concatStringSep: experimental features do not render as empty strings 2024-07-13 03:06:24 +02:00
Robert Hensing e64643bf63 dropEmptyInitThenConcatStringsSep -> concatStringSep: feature should not be empty
(System) features are unlikely to be empty strings, but when they
come in through structuredAttrs, they probably can.
I don't think this means we should drop them, but most likely they
will be dropped after this because next time they'll be parsed with
tokenizeString.

TODO: We should forbid empty features.
2024-07-13 03:06:24 +02:00
Robert Hensing f1966e22d9 dropEmptyInitThenConcatStringsSep -> concatStringSep: store paths are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 49d100ba8b dropEmptyInitThenConcatStringsSep -> concatStringSep: output name empty not feasible
I don't think it's completely impossible, but I can't construct
one easily as derivationStrict seems to (re)tokenize the outputs
attribute, dropping the empty output.

It's not a scenario we have to account for here.
2024-07-13 03:06:24 +02:00
Robert Hensing d3e49ac881 dropEmptyInitThenConcatStringsSep -> concatStringSep: shortRefs are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 608a425550 dropEmptyInitThenConcatStringsSep -> concatStringSep: diag 2024-07-13 03:06:24 +02:00
Robert Hensing 75dde71ff9 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are non-empty
The sigs field is produced by tokenizeStrings, which does not return
empty strings.
2024-07-13 03:06:24 +02:00
Robert Hensing 3f37785afd NIX_REMOTE_SYSTEMS: actually support multiple :-separated entries
Bug not reported in 6 years, but here you go.

Also it is safe to switch to normal concatStringsSep behavior
because tokenizeString does not produce empty items.
2024-07-13 03:06:24 +02:00
Robert Hensing 39878c8979 dropEmptyInitThenConcatStringsSep -> concatStringSep: preserve empty attr
The empty attribute name should not be dropped from attribute paths.
Rendering attribute paths with concatStringsSep is lossy and wrong,
but this is just a first improvement while dealing with the
dropEmptyInitThenConcatStringsSep problem.
2024-07-13 03:06:24 +02:00
Robert Hensing ea966a70fc dropEmptyInitThenConcatStringsSep -> concatStringSep: diagnostics and docs
These are non-critical, so their behavior is ok to change.
Dropping empty items is not needed and usually not expected.
2024-07-13 03:06:24 +02:00
Robert Hensing a681d354e7 Add fresh concatStringsSep without bug
The buggy version was previously renamed to
dropEmptyInitThenConcatStringsSep
2024-07-13 03:06:24 +02:00
Robert Hensing 79eb0adf9d dropEmptyInitThenConcatStringSep: Check that we don't drop...
... initial empty strings.

The tests pass, which is encouraging.
2024-07-13 03:05:50 +02:00
Robert Hensing 1a8defd06f Refactor: rename C++ concatStringsSep -> dropEmptyInitThenConcatStringsSep 2024-07-13 03:05:50 +02:00
tomberek b1effc9649 Merge pull request #11080 from NixOS/fix-build-x86_64-darwin-minSDK
packaging: Set darwinMinVersion to fix x86_64-darwin build
2024-07-12 20:50:50 -04:00
John Ericson db3e99d9d4 Merge pull request #10838 from obsidiansystems/spaceship-no-comparator
Remove `comparator.hh` and switch to `<=>` in a bunch of places
2024-07-12 15:43:17 -04:00
John Ericson bc83b9dc1f Remove comparator.hh and switch to <=> in a bunch of places
Known behavior changes:

- `MemorySourceAccessor`'s comparison operators no longer forget to
  compare the `SourceAccessor` base class.

Progress on #10832

What remains for that issue is hopefully much easier!
2024-07-12 14:54:18 -04:00
Lexi Mattick 6c4470ec2a Clean up cache for all commands 2024-07-12 11:54:12 -07:00
Lexi Mattick e764ed31f6 Eval cache: fix cache regressions
- Fix eval cache not being persisted in `nix develop` (since #10570)
- Don't attempt to commit cache transaction if there is no active transaction, which will spew errors in edge cases
- Drive-by: trivial typo fix
2024-07-12 09:45:35 -07:00
Eelco Dolstra 2a95a2d780 Merge pull request #11091 from NixOS/darwin-no-map-spaceship
Remove unused `operator<=>`'s that darwin can't generate
2024-07-12 18:40:20 +02:00
Eelco Dolstra cdc23b67a6 Provide std::hash<SourcePath> 2024-07-12 17:54:27 +02:00
Robert Hensing 6c5d2a1506 Merge pull request #11090 from amarshall/fix-stackoverflow-build
Fix stackoverflow during doc generation
2024-07-12 17:51:29 +02:00
Robert Hensing 11a6db5993 Remove unused operator<=>'s that darwin can't generate
It was complaining *a lot*, with dozens of MB of logs.
2024-07-12 17:37:27 +02:00
Andrew Marshall 51a12b38bd Fix stackoverflow during doc generation
On some systems, previous usage of `match` may cause a stackoverflow
(presumably due to the large size of the match result). Avoid this by
(ab)using `replaceStrings` to test for containment without using
regexes, thereby avoiding the issue. The causal configuration seems to
be the stack size hard limit, which e.g. Amazon Linux sets, whereas most
Linux distros leave unlimited.

Match the fn name to similar fn in nixpkgs.lib, but different
implementation that does not use `match`. This impl gives perhaps
unexpected results when the needle is `""`, but the scope of this is
narrow and that case is a bit odd anyway.

This makes for some duplication-of-work as we do a different
`replaceStrings` if this one is true, but this only runs during doc
generation at build time so has no runtime impact.

See https://github.com/NixOS/nix/issues/11085 for details.
2024-07-12 10:59:40 -04:00
John Ericson 337a5a23b7 Merge pull request #11089 from NixOS/warnings-includes
Fix warnings and optimize includes
2024-07-12 10:29:26 -04:00
John Ericson dfb169ca68 Merge pull request #11007 from obsidiansystems/push-down-fetcher-flake-settings
No global settings in `libnixfetchers` and `libnixflake`
2024-07-12 09:41:10 -04:00
Robert Hensing 27eaeebc41 nar-accessor.cc: Silence unused variable warning 2024-07-12 15:38:17 +02:00
Robert Hensing 8df041cbc6 Solve unused header warnings reported by clangd 2024-07-12 15:37:54 +02:00
John Ericson 3fc77f281e No global settings in libnixfetchers and libnixflake
Progress on #5638

There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.

Continuing process pioneered in
52bfccf8d8.
2024-07-12 08:50:28 -04:00
Robert Hensing f070d68c32 Add BaseError assignment operators
The move assignment was implicitly generated and used in

    src/libstore/build/goal.cc:90:22:
       90 |             this->ex = std::move(*ex);

Clang warns about this generated method being deprecated, so making
them explicit fixes the warning.
2024-07-12 14:25:16 +02:00
Eelco Dolstra b57c361097 Merge pull request #11082 from DeterminateSystems/symbol-table-string-view
SymbolStr: Remove std::string conversion
2024-07-12 13:39:43 +02:00
Eelco Dolstra 61080554ab SymbolStr: Remove std::string conversion
This refactoring allows the symbol table to be stored as something
other than std::strings.
2024-07-11 17:43:10 +02:00
Eelco Dolstra 0363dbf2b9 Merge pull request #11081 from DeterminateSystems/remove-dead-code
Remove unused InstallableFlake::getFlakeOutputs()
2024-07-11 17:11:28 +02:00
Eelco Dolstra 87323a5689 Remove unused InstallableFlake::getFlakeOutputs() 2024-07-11 16:21:27 +02:00
Robert HensingandEmily 0395ff9bd3 packaging: Set darwinMinVersion to fix x86_64-darwin build
Ported from https://github.com/NixOS/nixpkgs/pull/326172

Co-authored-by: Emily <vcs@emily.moe>
2024-07-11 15:04:38 +02:00
Robert Hensing bb312a7174 Edit CompatibilitySettings 2024-07-11 13:59:49 +02:00
Robert Hensing 6f5f741157 doc/rl-next/shebang-relative: Update with example 2024-07-11 13:52:03 +02:00
John Ericson 426e2af6f7 Merge pull request #11077 from hercules-ci/support-hardlinks-in-tarballs
Support hardlinks in tarballs
2024-07-11 07:10:25 -04:00
Robert Hensing 56bf39e905 eqValues/assertEqValues: Clean up assertions
It's still paranoid, and probably a waste of words, but at least
now it's consistent and readily identifyable from a log.
2024-07-11 12:19:29 +02:00
Robert Hensing 4fd8f19ecf Fix build to use CanonPath in new FSO sinks 2024-07-11 12:14:48 +02:00
Robert Hensing 86420753ec Merge remote-tracking branch 'upstream/master' into support-hardlinks-in-tarballs 2024-07-11 11:43:02 +02:00
Robert Hensing 61577402ba Add EvalErrorBuilder::panic()
An nicer alternative to printError + abort, or assert(false /* foo */)
2024-07-11 11:35:58 +02:00
Robert Hensing c4e3e2dc27 Soft-deprecate the compatibility settings 2024-07-10 16:24:31 +02:00
Robert Hensing d942d505ca Merge remote-tracking branch 'upstream/master' into more-nix-shell 2024-07-10 16:19:18 +02:00
Robert Hensing 13522229a9 assertEqValues: clarify potential bug error message 2024-07-10 16:08:10 +02:00
Eelco Dolstra 142e566adb Merge pull request #11071 from DeterminateSystems/remove-stray-line
Remove reference to check-hydra-status
2024-07-09 17:33:39 +02:00
Eelco Dolstra 4c788504fa Remove reference to check-hydra-status 2024-07-09 16:44:01 +02:00
John Ericson 3755493642 Merge pull request #11060 from obsidiansystems/meson-nix
Package the Nix CLI with Meson
2024-07-09 09:49:03 -04:00
Robert Hensing d69fcbc9d7 Merge pull request #11065 from DeterminateSystems/remove-hydra-status-check
Remove the Hydra status check workflow
2024-07-09 10:28:48 +02:00
John EricsonandQyriad 7a6269ba7b Package the Nix CLI with Meson
Co-Authored-By: Qyriad <qyriad@qyriad.me>
2024-07-08 17:49:44 -04:00
John Ericson f1d88e0f27 Merge pull request #11064 from obsidiansystems/header-generater
Use a meson "generator" to deduplicate `.gen.hh` creation
2024-07-08 12:03:59 -04:00
John Ericson 1439de874f Merge pull request #11063 from obsidiansystems/config-expr
Forgot to include `config-expr.hh` in two places
2024-07-08 11:27:57 -04:00
John Ericson 6e5cec292b Use a meson "generator" to deduplicate .gen.hh creation 2024-07-08 11:13:11 -04:00
John Ericson c5284a84f3 Forgot to include config-expr.hh in some places 2024-07-08 10:37:34 -04:00
Eelco Dolstra ab0b54b905 Remove the Hydra status check workflow
I'm sick of receiving an email about this every 30 minutes.
2024-07-08 14:39:33 +02:00
Robert Hensing cfe3ee3de8 nix-shell: look up shell.nix when argument is a directory (#11057)
* Refactor: rename runEnv -> isNixShell

* Refactor: rename left -> remainingArgs

* nix-build.cc: Refactor: extract baseDir variable

* nix-build.cc: Refactor: extract sourcePath, resolvedPath variables

* nix-shell: Look for shell.nix when directory is specified

* Add legacy setting: nix-shell-always-looks-for-shell-nix

* rl-next: Add note about shell.nix lookups

* tests/functional/shell.nix: Implement runHook for dummy stdenv
2024-07-08 14:36:36 +02:00
Ryan Hendrickson 48804cffbf docs: fill out language/types.md#type-path 2024-07-08 00:41:19 -04:00
John Ericson d8850618b6 Merge pull request #11059 from rhendric/rhendric/reference-manual
docs: merge builtin-constants into builtins
2024-07-07 21:47:33 -04:00
Ryan Hendrickson 95890b3e1d docs: merge builtin-constants into builtins 2024-07-07 15:57:23 -04:00
Robert Hensing a9592077fb Merge pull request #11055 from NixOS/packaging-for-nixpkgs
Indirections for packaging meson-based granular build for Nixpkgs
2024-07-07 21:33:01 +02:00
Robert Hensing 193dd5d934 Fixup: add missing test file 2024-07-07 14:49:52 +02:00
Robert Hensing 3e424b1040 Merge branch 'nix-shell-lookup-shell-nix' into more-nix-shell 2024-07-07 14:16:15 +02:00
Robert Hensing e1106b45a3 tests/functional/nix-shell.sh: Fix Polo test for VM test
It is unclear to me why this worked when not in a VM test, but the
explanation would be in the part of nix-shell we're getting rid of
with the devShell attribute.
2024-07-07 13:03:19 +02:00
Robert Hensing 0f8a655023 tests/functional/shell.nix: Implement runHook for dummy stdenv 2024-07-07 13:02:21 +02:00
Robert Hensing fd4b17ab2c Merge pull request #11056 from romain-neil/s3-allow-use-of-proxy
Configure aws s3 lib to use system defined proxy, if existent
2024-07-07 02:47:24 +02:00
Robert Hensing c4a20a4101 rl-next: Add note about shell.nix lookups 2024-07-07 01:22:00 +02:00
Robert Hensing 2f1fada76b Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-07 01:22:00 +02:00
Robert Hensing 73602a7c6f nix-shell: Look for shell.nix when directory is specified 2024-07-07 01:22:00 +02:00
Robert Hensing 63262e78c7 Add opt-out: nix-shell-shebang-arguments-relative-to-script 2024-07-07 00:55:33 +02:00
Robert Hensing 4c59d6e9f5 Merge branch 'nix-shell-lookup-shell-nix' into more-nix-shell 2024-07-07 00:27:07 +02:00
Robert Hensing 6959ac157b rl-next: Add note about shell.nix lookups 2024-07-07 00:25:56 +02:00
Robert Hensing 6c6d5263e2 Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-07 00:25:56 +02:00
Robert Hensing f5b59fbc64 Fix and extend nix-shell baseDir test 2024-07-07 00:23:22 +02:00
Robert Hensing d5854f33e2 rl-next: Typo 2024-07-07 00:18:26 +02:00
Robert Hensing 8838f5c746 Merge remote-tracking branch 'matthewbauer/nix-shell-relative-shebang' into more-nix-shell 2024-07-07 00:18:03 +02:00
Robert Hensing afbe7c3d08 rl-next: Enter PR 2024-07-06 23:15:01 +02:00
John Ericson 41b6c735eb Merge pull request #11054 from NixOS/meson-fixes
Meson fixes
2024-07-06 17:11:11 -04:00
Robert Hensing b865625a8e nix-shell: Look for shell.nix when directory is specified 2024-07-06 23:05:34 +02:00
Robert Hensing a22f8b5276 rl-next: Add note about shell.nix lookups 2024-07-06 23:05:34 +02:00
Robert Hensing 32fb127b9c Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-06 23:05:34 +02:00
Romain NEIL 514062c227 feat: configure aws s3 lib to use system defined proxy, if existent 2024-07-06 21:46:58 +02:00
Robert Hensing 76245ffbeb nix-build.cc: Refactor: extract sourcePath, resolvedPath variables 2024-07-06 20:55:27 +02:00
Robert Hensing e9479b272f nix-build.cc: Refactor: extract baseDir variable 2024-07-06 20:51:45 +02:00
Robert Hensing 5c367ece89 Refactor: rename left -> remainingArgs 2024-07-06 20:03:30 +02:00
Robert Hensing 13181356fc Refactor: rename runEnv -> isNixShell 2024-07-06 20:01:46 +02:00
Robert Hensing bea54d116e Add resolvePath, filesetToSource indirections for Nixpkgs 2024-07-06 19:49:55 +02:00
Robert Hensing da4c55995b ci.yml: Build non unit-tested components in meson_build 2024-07-06 19:15:53 +02:00
Robert Hensing 0729f0a113 packaging: Pass version directly 2024-07-06 17:52:57 +02:00
Robert Hensing efd5f50f5e nix-perl: Add deps, use mkMesonDerivation 2024-07-06 17:52:57 +02:00
Robert Hensing 4c014e238b nix-main: Add openssl 2024-07-06 17:52:57 +02:00
Robert Hensing 4d0c55ae55 api docs: Use mkMesonDerivation 2024-07-06 17:52:57 +02:00
Robert Hensing b7e5446b81 flake.nix: Remove unused binding 2024-07-06 17:52:57 +02:00
Robert Hensing 896eb7a44b Merge pull request #11034 from obsidiansystems/meson-nix
Package libnixmain and libnixcmd with Meson
2024-07-06 01:26:59 +02:00
John EricsonandRobert Hensing 3acf3fc746 Package libnixmain and libnixcmd with Meson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-05 16:40:55 -04:00
Robert Hensing 0b901e10ee Merge pull request #11050 from hercules-ci/issue-10677
Explain when `man` is missing
2024-07-05 22:25:38 +02:00
John Ericson ff9b6d0e1f Merge pull request #11037 from fricklerhandwerk/document-config-parsing
use self-descriptive name for config file parser, document
2024-07-05 15:21:26 -04:00
Eelco Dolstra d5461b9009 Merge pull request #11051 from Mic92/fix-prefetch
src/nix/prefetch: fix prefetch containing current directory instead o…
2024-07-05 20:33:05 +02:00
Robert Hensing ddff76f667 Merge pull request #10973 from NixOS/meson-libexpr
Meson build for libexpr libflake, external C API, unit tests
2024-07-05 20:27:12 +02:00
Jörg ThalheimandEelco Dolstra 05381c0b30 Update src/nix/prefetch.cc
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-05 19:45:03 +02:00
Jörg Thalheim 8cea1fbd97 src/nix/prefetch: fix prefetch containing current directory instead of tarball
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
2024-07-05 19:18:49 +02:00
Robert Hensing 6ef00a503a Explain when man is missing
Have you seen this man?

Fixes #10677
2024-07-05 19:18:23 +02:00
Eelco Dolstra 8f280d72ff Merge pull request #11019 from DeterminateSystems/fix-failed-to-open-archive
Tarball fetcher: Fix handling of cached tarballs
2024-07-05 17:10:02 +02:00
Robert Hensing a476383f46 Merge pull request #11031 from emilazy/push-xsrvoyspsvqx
libstore: fix sandboxed builds on macOS
2024-07-05 17:08:39 +02:00
Robert Hensing d63bd8295e assert: Report why values aren't equal 2024-07-05 16:43:48 +02:00
Eelco Dolstra 98bef7c38e Merge pull request #11035 from siddhantk232/refactor
Factor duplicate code into util function `append`
2024-07-05 16:34:23 +02:00
Eelco Dolstra 61e1880847 Merge pull request #11041 from hercules-ci/trace-nix-env-attribute-names
getDerivations: add attributes to trace
2024-07-05 16:32:27 +02:00
Eelco Dolstra e1b6b3ce27 Merge pull request #11020 from DeterminateSystems/fix-tarball-caching
Tarball fetcher: Fix fetchToStore() and eval caching
2024-07-05 16:30:12 +02:00
Eelco Dolstra e7e070d36b Document 2024-07-05 16:29:16 +02:00
Robert Hensing 09763c7cad getDerivations: add attributes to trace
This improves the error message of nix-env -qa, among others, which
is crucial for understanding some ofborg eval error reports, such as
https://gist.github.com/GrahamcOfBorg/89101ca9c2c855d288178f1d3c78efef

After this change, it will report the same trace, but also start with

```
error:
       … while evaluating the attribute 'devShellTools'

       … while evaluating the attribute 'nixos'

       … while evaluating the attribute 'docker-tools-nix-shell'

       … while evaluating the attribute 'aarch64-darwin'

       … from call site
         at /home/user/h/nixpkgs/outpaths.nix:48:6:
           47|   tweak = lib.mapAttrs
           48|     (name: val:
             |      ^
           49|       if name == "recurseForDerivations" then true

<same>
```
2024-07-05 15:30:07 +02:00
John EricsonandRobert Hensing e4056b9afd Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 17:48:27 -04:00
Emily af2e1142b1 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
2024-07-04 16:28:37 +01:00
Emily 76e4adfaac libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
2024-07-04 16:22:02 +01:00
Valentin GagarinandRobert Hensing c66079f1e8 use self-descriptive name for config file parser, document
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 10:36:48 +02:00
siddhantCodes 976c05879f factor duplicate code into util function append 2024-07-04 11:09:23 +05:30
Eelco Dolstra 5b4102c3b2 Tarball fetcher: Include revCount/lastModified in the fingerprint
This can influence the evaluation result so they should be included in
the fingerprint.
2024-07-03 22:05:45 +02:00
John Ericson 509be0e77a Merge pull request #11022 from obsidiansystems/fix-openbsd-socket-peercred
Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
2024-07-03 11:56:39 -04:00
kn 10ccdb7a41 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).
2024-07-03 11:16:39 -04:00
John Ericson a09360400b Ident some CPP in nix daemon
Makes it easier for me to read.
2024-07-03 11:15:56 -04:00
Eelco Dolstra 1ff186fc6e nix flake metadata: Show flake fingerprint
This is useful for testing/debugging and maybe for sharing eval caches
(since it tells you what file in ~/.cache/nix/eval-cache-v5 to copy).
2024-07-03 17:00:30 +02:00
Eelco Dolstra 9d95c228ee Tarball fetcher: Fix fetchToStore() and eval caching 2024-07-03 16:50:45 +02:00
John Ericson 30de61f16d Merge pull request #11018 from siddhantk232/canonpath-fs-sink
Use `CanonPath` in `fs-sink.hh`
2024-07-03 10:36:18 -04:00
Eelco Dolstra 8bdd0ecd80 Add a test 2024-07-03 15:57:05 +02:00
Eelco Dolstra 79ed3df8f8 Tarball fetcher: Fix handling of cached tarballs
Fixes a regression introduced in
5a9e1c0d20 where downloading a cached
file causes the error "Failed to open archive (Unrecognized archive
format)".
2024-07-03 14:14:20 +02:00
siddhantCodes 2cf24a2df0 fix tests and minor changes
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
2024-07-03 17:43:55 +05:30
Valentin GagarinandRyan Hendrickson a713476790 docs: split types from syntax (#11013)
move together all syntactic and semantic information into one
page, and add a page on data types, which in turn links to the syntax and
semantics.

also split out the note on scoping rules into its own page.

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-03 09:03:41 +02:00
John Ericson b0bc2a97bf Put unit tests back in old build system for now 2024-07-02 09:26:22 -04:00
John Ericson 451f8a8c19 Put back files for now
We'll revert this sometime later
2024-07-02 09:26:22 -04:00
John Ericson 11946817f0 fileset for store unit test data 2024-07-02 09:26:22 -04:00
John Ericson 4d6bc61b8d Fix things 2024-07-02 09:26:22 -04:00
John Ericson 4727d5c3c5 Fix format blacklist 2024-07-02 09:26:22 -04:00
John Ericson 11dab30be9 Update docs on the unit tests 2024-07-02 09:26:22 -04:00
John Ericson 224c6c3256 Fix test symlinks 2024-07-02 09:26:22 -04:00
John Ericson 496b4a9cd2 Move around unit test dirs to match new names 2024-07-02 09:26:22 -04:00
John Ericson 3ad39d2afb Fix library name 2024-07-02 09:26:22 -04:00
John Ericson 513f6b9718 meson: Prelink links to avoid missing C++ initializers
This is the same as what the old build system did in
7eca8a16ea, done for the same reasons.
2024-07-02 09:26:22 -04:00
John Ericson 912c517bc0 Fix build of unit tests 2024-07-02 09:26:22 -04:00
John Ericson f7ce10dbc1 Fix static build 2024-07-02 09:26:22 -04:00
John Ericson 874ff000d4 Fix format 2024-07-02 09:26:22 -04:00
John Ericson 7312d13acc Keep another test dir 2024-07-02 09:26:22 -04:00
John Ericson 17c843c5c5 Fix more issues 2024-07-02 09:26:22 -04:00
John Ericson 479befa76d More fixes 2024-07-02 09:26:22 -04:00
John Ericson 5ba9f6cec6 Fix typo 2024-07-02 09:26:22 -04:00
John Ericson 6a0582d9fd Rename file to avoid reserved name 2024-07-02 09:26:22 -04:00
John Ericson 79e0ef88bf Include missing components 2024-07-02 09:26:22 -04:00
John Ericson 2c184f694b Ensure we have data dir for libexpr unit tests 2024-07-02 09:26:22 -04:00
John Ericson 46ec69a483 Everything builds in the dev shell now 2024-07-02 09:26:22 -04:00
John Ericson 429d6ae2b5 Add missing package.nix 2024-07-02 09:26:22 -04:00
John Ericson 92d3a06b25 Remove overrides of removed flags since unit tests broken out 2024-07-02 09:26:22 -04:00
John Ericson 0b539dea4a Improve boost hacks 2024-07-02 09:26:22 -04:00
John Ericson 8399bd6b8f Dedup 2024-07-02 09:26:21 -04:00
John Ericson 8198888bc4 More dedup 2024-07-02 09:23:25 -04:00
John Ericson d6f57f3260 More dedup 2024-07-02 09:23:25 -04:00
John Ericson c88f83b471 More dedup 2024-07-02 09:23:25 -04:00
John Ericson 4609ab318c Fix internal API docs 2024-07-02 09:23:25 -04:00
John Ericson d902481a36 Better org 2024-07-02 09:23:25 -04:00
John Ericson a81e319528 Deduplicating 2024-07-02 09:23:24 -04:00
John Ericson 17a8c2bfce Unit tests and external libraries 2024-07-02 09:23:24 -04:00
John Ericson 4fa8068b78 Mesonify other external API 2024-07-02 09:23:24 -04:00
John Ericson 31257009e1 Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
John Ericson fbdc554908 Fix Nix shell for building Perl too 2024-07-02 09:23:24 -04:00
John Ericson f809edba4f Merge pull request #11011 from Mic92/compiler-checks
enable -Werror=unused-result
2024-07-02 09:23:03 -04:00
Jörg Thalheim 101915c9b7 enable -Werror=unused-result
Inspired by
https://git.lix.systems/lix-project/lix/commit/010ff57ebb40f1a9aaff99867d2886f0e59f774a

From the original PR:

> We do not have any of these warnings appearing at the moment, but
> it seems like a good idea to enable [[nodiscard]] checking anyway.
> Once we start introducing more functions with must-use conditions we will
> need such checking, and the rust stdlib has proven them very useful.
2024-07-02 08:46:06 +02:00
Robert Hensing 80d67d603b Merge pull request #11009 from NixOS/installer-tests
Installer tests
2024-07-02 00:45:12 +02:00
Robert Hensing df3e92ff96 installerScriptForGHA: aarch64-darwin
GitHub Actions seems to have magically switched architectures
without changing their identifiers.
See https://github.com/actions/runner-images/blob/2813ee66cbe85b31a8322ff8967148548b1a5db9/README.md#available-images
Maybe they have more complete documentation elsewhere, but it
seems to be incapable of selecting a runner based on architecture.
2024-07-01 20:40:53 +02:00
Eelco Dolstra b01b60f08c Merge pull request #11006 from DeterminateSystems/fix-eval-cache-test
tests/functional/flakes/eval-cache.sh: Don't write a result symlink in the wrong location
2024-07-01 19:47:54 +02:00
Eelco Dolstra 6600b1c7e0 tests/functional/flakes/eval-cache.sh: Don't write a result symlink in the wrong location 2024-07-01 19:10:41 +02:00
Robert Hensing f0329568b5 GitFileSystemObjectSink: catch an overflow 2024-07-01 17:29:07 +02:00
Robert Hensing a409c1a882 Start unit testing GitFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Robert Hensing 1fac22b16e GitFileSystemObjectSink: Add path context to some messages 2024-07-01 17:29:07 +02:00
Robert Hensing 992912f3b4 test-support: Add TracingFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Eelco Dolstra 10c9764c27 Merge pull request #10992 from hercules-ci/issue-10947-dont-cache-disallowed-ifd
Fix #10947; don't cache disallowed IFD
2024-07-01 11:20:27 +02:00
John Ericson 10ceb7d173 Merge pull request #10995 from NixOS/meson-remove-symlink-hack
Remove symlink hack from meson packaging
2024-06-30 17:35:46 -04:00
Robert Hensing 93b50857ed packaging: Restore .version value altering behavior 2024-06-30 19:44:36 +02:00
Robert Hensing 7dd938b228 libutil/package.nix: Remove .version symlink replacement solution 2024-06-30 19:44:04 +02:00
Robert Hensing e084316130 Add mkMesonPackage for local meson packages
This helper makes it easy to use filesets that include files from
parent directories, which we'll need more of in
https://github.com/NixOS/nix/pull/10973
2024-06-30 19:42:19 +02:00
Sergei TrofimovichandJohn Ericson 5a16bf86c5 doc: fix directory definition in nix-archive.md (#10997)
* doc: fix `directory` definition in nix-archive.md

Before the change the document implied that directory of a single entry
contained entry:

    "type" "directory" "type" directory" "entry" ...

After the change document should expand into:

   "type" "directory" "entry" ...

Co-authored-by: John Ericson <git@JohnEricson.me>
2024-06-30 16:22:04 +00:00
Robert Hensing c74b5aa3e0 Merge pull request #10999 from NixOS/ci-cachix-verbose
ci.yml: Put installer cachix in verbose mode
2024-06-30 17:45:22 +02:00
Robert Hensing 39154ed9be ci.yml: Put installer cachix in verbose mode
Maybe it prints something, as a workaround for https://github.com/cachix/cachix-action/issues/153,
which might explain our flaky installer_test.
2024-06-30 17:10:34 +02:00
John Ericson dc538ad2f7 Merge pull request #10994 from hercules-ci/fix-value-print-elided
Fix underflow in `printAttrs`, `printList`
2024-06-30 10:11:47 -04:00
siddhantCodes 72bb530141 use CanonPath in fs-sink and its derivatives 2024-06-30 19:03:15 +05:30
Robert Hensing b2c7f09b0a Fix underflow in Printer::printList
Analogous to 9b88bf8adf / three commits back
2024-06-29 14:10:58 +02:00
Robert Hensing bfc5416240 Refactor: rename ValuePrinter::totalListItemsPrinted 2024-06-29 14:02:28 +02:00
Robert Hensing ce1dc87711 Refactor: rename ValuePrinter::totalAttrsPrinted
Make it more distinct from the attrs printed of any specific
attrset.
2024-06-29 14:01:15 +02:00
Robert Hensing 9b88bf8adf Fix underflow in Printer::printAttrs
The code that counts the number of elided attrs incorrectly used the
per-printer "global" attribute counter instead of a counter that
was relevant only to the current attribute set.

This bug flew under the radar because often the attribute sets aren't
nested, not big enough, or we wouldn't pay attention to the numbers.
I've noticed the issue because the difference underflowed.

Although this behavior is tested by the functional test
lang/eval-fail-bad-string-interpolation-4.nix, the underflow slipped
through review. A simpler reproducer would be as follows, but I
haven't added it to the test suite to keep it simple and marginally
faster.

```
$ nix run nix/2.23.1 -- eval --expr '"" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)'
error:
       … while evaluating a path segment
         at «string»:1:6:
            1| "" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)
             |      ^

       error: cannot coerce a set to a string: { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; «4294967289 attributes elided» }; }
```
2024-06-29 13:53:05 +02:00
Robert Hensing fd94b74ee5 Fix #10947; don't cache disallowed IFD 2024-06-29 13:19:04 +02:00
Robert Hensing 32e6cc64b5 Merge pull request #10988 from cole-h/restore-summary-rename
Restore commit-lock-file-summary rename for consistency
2024-06-29 11:51:50 +02:00
Cole Helbling d906d42330 Restore commit-lock-file-summary rename for consistency
It was originally renamed in https://github.com/NixOS/nix/pull/10691,
but https://github.com/NixOS/nix/pull/9063 accidentally removed the new
name and alias.
2024-06-28 14:53:40 -07:00
Cole Helbling 9e9730ef0f Test that commit-lock-file-summary and its alias work 2024-06-28 14:53:40 -07:00
John Ericson 02963f712e Merge pull request #10987 from tweag/remove-thufschmitt-from-maintainers
maintainers: Drop thufschmitt
2024-06-28 16:19:14 -04:00
Théophane Hufschmitt 26089183e6 maintainers: Drop thufschmitt
https://github.com/NixOS/nixos-homepage/pull/1490
2024-06-28 15:56:53 +02:00
Valentin Gagarin b44909ac22 add many more examples on escaping in strings (#10974) 2024-06-27 10:58:59 +02:00
3b388f6629 string interpolation escape example (#10966)
* string interpolation escape example

Make it easier to find the documentation, and the example might be enough for most cases.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-27 08:30:21 +00:00
John Ericson aff0fbc764 Merge pull request #10972 from NixOS/no-vendor-toml11
Stop vendoring toml11
2024-06-27 00:10:09 -04:00
John Ericson b87e048b3b Override toml11 so it evaluates on Windows too 2024-06-26 23:30:38 -04:00
Winter 149d8eb8aa Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes https://git.lix.systems/lix-project/lix/issues/164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
(cherry picked from commit 80405d06264f0de1c16ee2646388ab501df20628)
2024-06-26 22:27:13 -04:00
John Ericson f002f85861 Avoid libmain header in libexpr
We just don't need it!
2024-06-26 22:26:45 -04:00
John Ericson 7e66f0d91c Merge pull request #9063 from obsidiansystems/libfetchers-prep
Introduce `libnixflake`
2024-06-26 20:32:46 -04:00
John Ericson ed129267dc Merge pull request #10971 from NixOS/user-sandbox-escape-post
Slight fixes to get CI passing again
2024-06-26 20:28:56 -04:00
John Ericson 7181d1f4a1 Reformat
Factored out code is now elegible for formatting.
2024-06-26 19:56:21 -04:00
John EricsonandRobert Hensing 0084a486cc Split out a new libnixflake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John EricsonandRobert Hensing 52730d38e2 Factor out flake:... lookup path from evaluator
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John Ericson 88f9d8ccb1 Don't format the just-added test .c file
On one hand, new things should be formatted. On the other, we just
bacported this file to many prior branches, and if we need to make
changes to it and backport them also, formatting the file on master but
not the release branches would cause issues.
2024-06-26 19:55:06 -04:00
John Ericson 32e67eba8b Remove invalid release notes YAML field
There is no PR for this, since it was an embargoed fix before
disclosure.
2024-06-26 19:35:57 -04:00
tomberek 2dd7f8f42d Merge pull request from GHSA-q82p-44mg-mgh5
Fix user sandbox escape
2024-06-26 18:49:21 -04:00
John Ericson 8a420162ab Merge branch 'master' into fix-sandbox-escape 2024-06-26 18:11:39 -04:00
John Ericson 5e4e3345d4 Merge pull request #10963 from hercules-ci/build-with-nixpkgs-scope
Build with nixpkgs scope
2024-06-26 11:08:45 -04:00
Robert Hensing 8eb804b0e6 Merge pull request #10967 from NixOS/ci-meson-darwin
ci.yml: Build meson on darwin
2024-06-26 11:41:31 +02:00
Robert Hensing 9f8e387c3f ci.yml: Build meson on darwin
We're building a bit of Darwin meson indirectly through `checks`,
but it'd be annoying to encounter broken un-`check`-ed stuff
during the porting process, so let's just do the right thing now.
2024-06-26 11:03:47 +02:00
Robert Hensing e644289161 Remove unused boehmgc patch 2024-06-26 10:54:17 +02:00
Robert Hensing 985c211061 flake.nix: Move {in,ex}ternal-api-docs into nixComponents scope 2024-06-26 10:48:45 +02:00
Robert Hensing 409eded541 flake.nix: Move dependencies scope to packaging/dependencies.nix 2024-06-26 10:41:56 +02:00
Robert Hensing 65802da98d Move maintainers/hydra.nix -> packaging/hydra.nix 2024-06-26 04:24:50 +02:00
Robert Hensing c24dbf1457 components.nix: Simplify 2024-06-26 04:17:30 +02:00
Robert Hensing 25dc12aab1 components.nix: Extract dependency scope
This avoids polluting nixComponents with things that aren't our
components.
Fixes the extraction of passthru tests, which failed for boehmgc
which had many irrelevant ones anyway.
2024-06-26 04:11:20 +02:00
Robert Hensing ebf77c79ae flake.nix: Use Nixpkgs convention for package variants 2024-06-26 04:11:20 +02:00
Robert Hensing 74b9b77c9f components.nix: Simplify with scope 2024-06-26 04:11:20 +02:00
Robert Hensing 85de5a60c7 Use lib instead of explicit fileset passing 2024-06-26 04:11:20 +02:00
Robert Hensing d40c59ed19 flake.nix: Use the nixComponents scope instead of bare pkgs packages
... which aren't around anymore.
2024-06-26 04:11:18 +02:00
Robert Hensing a14faa869d flake.nix: Use a Nixpkgs scope for components 2024-06-26 04:07:49 +02:00
John Ericson a92f3071bb Merge pull request #10962 from hercules-ci/tidy-packages-matrix
flake.nix: Tidy `packages` build matrix code
2024-06-25 22:06:33 -04:00
John Ericson 3f191cc0e7 Merge pull request #10961 from hercules-ci/tidy
Tidy
2024-06-25 22:04:32 -04:00
Robert Hensing e28e6b96bb flake.nix: Tidy packages build matrix code
flatMapAttrs is easier to read because it introduces the values
before using them, kind of like a `let` bindings with multiple
values.

The repeated comments remind the reader of the purpose of the
innermost attrsets, which is actually very simple.
Knowing that they go right into the result should help a lot
with building a mental model for this pattern.
2024-06-26 02:03:21 +02:00
Robert Hensing 6fe8fb967a libstore/worker.hh: Document Worker 2024-06-26 01:43:31 +02:00
Robert Hensing fd0b376c79 libstore/worker.cc: Remove outdated comment
It was added above this conditional

    Worker::Worker(LocalStore & store)
        : store(store)
    {
        /* Debugging: prevent recursive workers. */
        if (working) abort();
        working = true;

However, `working` has since been removed.

Source: https://github.com/NixOS/nix/blame/7f8e805c8ef2d7728648553de6b762964730a09a/src/libstore/build.cc#L2617
2024-06-26 01:43:31 +02:00
Robert Hensing 052f1320dd Merge pull request #10960 from rhendric/rhendric/reference-manual
docs: internal documentation touchup
2024-06-26 01:16:45 +02:00
John Ericson 65d7c80365 Merge pull request #10955 from NixOS/meson-nix-util-c
Build nix-util-c with meson + unit tests
2024-06-25 19:06:06 -04:00
Ryan Hendrickson fba81cf74b docs: internal documentation touchup
Make two comments more accurate for the next reader.
2024-06-25 18:44:56 -04:00
Robert Hensing ae3304bde9 Test static build of nix-util on non-darwin 2024-06-25 21:35:23 +02:00
Robert Hensing 1eaddb209d TMP: disable static meson build on darwin 2024-06-25 21:35:23 +02:00
Robert Hensing 6a28566db6 refact: concatMapAttrs -> flatMapAttrs
This should be slightly easier to read.
We could apply this to all concatMapAttrs calls.
2024-06-25 21:35:23 +02:00
Robert Hensing ac89828b5a Build nix-util-c with meson and unit test 2024-06-25 21:35:23 +02:00
John Ericson ccb679ee01 Merge pull request #10941 from hercules-ci/check-derivation-name
Improve error messages for invalid derivation names
2024-06-25 14:11:34 -04:00
Robert Hensing 7df9d6da65 Improve error messages for invalid derivation names 2024-06-25 19:41:29 +02:00
John Ericson 5f4f789144 Merge pull request #10954 from NixOS/ci-meson
ci.yml: Add meson_build
2024-06-25 09:02:33 -04:00
John Ericson d02d38f16d Merge pull request #7048 from puffnfresh/lutimes
Guard uses of lutimes, for portability
2024-06-25 06:06:45 -04:00
Robert Hensing 0674be8d49 nix-util: Fix build 2024-06-25 10:26:57 +02:00
Robert Hensing 1801119e29 ci.yml: Add meson_build
Restore meson CI after https://github.com/NixOS/nix/pull/10929
2024-06-25 10:15:28 +02:00
Brian McKenna 5be44d235a Guard uses of lutimes, for portability 2024-06-24 17:35:34 -04:00
John Ericson 05580a373f Fix error in the no-GC build 2024-06-24 17:18:16 -04:00
John Ericson 5c497a992b Merge pull request #10922 from hercules-ci/functional-tests-on-nixos
Run the functional tests in a NixOS environment
2024-06-24 13:36:13 -04:00
Robert Hensing 445a4a0298 ci.yml: Add swap and monitor it 2024-06-24 19:05:30 +02:00
Robert Hensing c66f1e7660 Merge pull request #10913 from NixOS/no-global-eval-settings-in-libexpr
No global eval settings in `libnixexpr`
2024-06-24 18:52:19 +02:00
John Ericson fda4c78921 Merge pull request #10951 from obsidiansystems/load-just-one-config
Small global config refactors
2024-06-24 12:38:04 -04:00
John Ericson 52bfccf8d8 No global eval settings in libnixexpr
Progress on #5638

There is still a global eval settings, but it pushed down into
`libnixcmd`, which is a lot less bad a place for this sort of thing.
2024-06-24 12:15:16 -04:00
Robert Hensing 5a7ccd6580 tests/functional: Print all args of fail() 2024-06-24 18:11:58 +02:00
Robert Hensing d4ca634508 tests/functional: Differentiate die and fail 2024-06-24 18:11:10 +02:00
John Ericson cb0c868da4 Allow loading config files into other config objects
This gives us some hope of moving away from global variables.
2024-06-24 12:07:56 -04:00
John Ericson b46e13840b Format config-global.{cc,hh}
Since the code is factored out, it is no longer avoding the formatter.
2024-06-24 12:07:56 -04:00
Robert Hensing 602c444411 Merge remote-tracking branch 'upstream/master' into functional-tests-on-nixos 2024-06-24 18:07:21 +02:00
John Ericson 1620ad4587 Split out GlobalConfig into its own header
This makes it easier to understand the reach of global variables /
global state in the config system.
2024-06-24 11:36:21 -04:00
Robert Hensing ce4bcef830 Merge pull request #10745 from obsidiansystems/ca-cleanup
Cleanup `ContentAddressMethod` to match docs
2024-06-24 17:25:45 +02:00
John EricsonandRobert Hensing b51e161af5 Cleanup ContentAddressMethod to match docs
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-24 10:24:06 -04:00
John Ericson 64e599ebe1 Rename Recursive -> NixArchive
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
2024-06-24 10:24:06 -04:00
Eelco Dolstra 903acc7c0f Merge pull request #10873 from siddhantk232/rm-createdirs
use `std::filesystem::create_directories` for createDirs
2024-06-24 14:54:37 +02:00
Eelco Dolstra bc21c54565 Merge pull request #10943 from pineapplehunter/master
Accept response from gitlab api with more than one entry in json
2024-06-24 14:23:47 +02:00
Robert Hensing 6f64154eea Merge pull request #10884 from tomberek/tomberek.warn_structuredAttrs_advanced
fix: warn and document when advanced attributes will have no impact d…
2024-06-24 07:56:26 +02:00
John Ericson 927b719bce Merge pull request #10874 from haenoe/derivation-tests
Tests for derivation "advanced attrs"
2024-06-23 22:09:10 -04:00
HaeNoeandJohn Ericson 7fb14201af Unit test for derivation "advanced attrs"
This tests the parser and JSON format using the DRV files from the tests
added in the previous commit.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
HaeNoeandJohn Ericson 9f9984e4d0 Functional test for derivation "advanced attrs"
This tests the Nix language side of things.

We are purposely skipping most of `common.sh` because it is overkill for
this test: we don't want to have an "overfit" test environment.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
John Ericson 490ca93cf8 Factor out a bit more language testings infra
Will be used in a second test after `lang.sh`.
2024-06-23 15:33:45 -04:00
John Ericson df068734ac Merge pull request #10769 from poweredbypie/mingw-spawn
Implement runProgram for Windows
2024-06-23 14:12:36 -04:00
Shogo Takata 0468061dd2 accept response from gitlab with more than one entry 2024-06-23 00:52:19 +09:00
Eelco Dolstra d54590fdf3 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.
2024-06-21 17:06:19 +02:00
Eelco Dolstra 58b7b3fd15 Formatting 2024-06-21 17:06:19 +02:00
Eelco Dolstra ede95b1fc1 Put the chroot inside a directory that isn't group/world-accessible
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).

To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt d99c868b04 Add a release note for the build-dir hardening 2024-06-21 17:06:19 +02:00
Théophane Hufschmitt 1d3696f0fb Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt 717f3eea39 Add a test for the user sandboxing 2024-06-21 17:06:18 +02:00
Eelco Dolstra d4a70b67a0 Move flake-regressions repos to the NixOS org 2024-06-21 15:38:03 +02:00
Eelco Dolstra 6f3d2daee6 Fix spellcheck 2024-06-21 15:37:08 +02:00
Eelco Dolstra 0eec60968a flake-regressions.sh: Make the sort order deterministic 2024-06-21 15:37:08 +02:00
Eelco Dolstra 36cc8d5f4b Run the flake-regressions test suite 2024-06-21 15:37:08 +02:00
Robert Hensing d9684664c8 Revert "tests/functional/common/init.sh: Use parentheses around negation"
ShellCheck doesn't want us to add extra parentheses for show.

This reverts commit 7c9f3eeef8.
2024-06-20 22:31:32 +02:00
siddhantCodes 85b7989764 fix: handle errors in nix::createDirs
the `std::filesystem::create_directories` can fail due to insufficient
permissions. We convert this error into a `SysError` and catch it
wherever required.
2024-06-20 19:53:25 +05:30
siddhantCodes 857e380c7d Merge branch 'rm-createdirs' of github.com:siddhantk232/nix into rm-createdirs 2024-06-20 18:47:51 +05:30
Robert HensingandValentin Gagarin dcee46a0ef Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-20 14:54:11 +02:00
Robert HensingandEelco Dolstra 7c9f3eeef8 tests/functional/common/init.sh: Use parentheses around negation
roberth: Not strictly necessary, but probably a good habit

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-20 14:54:11 +02:00
Robert Hensing 648302b833 tests/functional: Enable more tests in NixOS VM 2024-06-20 14:54:11 +02:00
Robert Hensing f0abe4d8f0 ci: Build tests.functional_user for PRs 2024-06-20 14:54:11 +02:00
Robert Hensing fca160fbcd doc/contributing/testing: Describe functional VM tests and quickBuild 2024-06-20 14:54:11 +02:00
Robert Hensing d208e9dd9f tests: Add quickBuild to all VM tests 2024-06-20 14:54:11 +02:00
Robert Hensing 8557d79650 tests/functional: Skip tests that don't work in NixOS environment yet 2024-06-20 14:54:11 +02:00
Robert Hensing 211aec473e tests/functional/timeout.sh: Find missing test case
This reproduces an instance of
https://github.com/NixOS/nix/issues/4813
2024-06-20 14:54:11 +02:00
Robert Hensing 439022c5ac tests: Add hydraJobs.tests.functional_* 2024-06-20 14:54:11 +02:00
Robert Hensing dc720f89f2 flake.nix: Factor pkgs.nix_noTests out of buildNoTests
This is useful when iterating on the functional tests when trying
to run them in a VM test, for example.
2024-06-20 14:54:10 +02:00
Valentin Gagarin 1c131ec2b7 Port C API docs to Meson (#10936)
* Port C API docs to Meson

* don't cross-compile the docs
2024-06-19 22:43:54 +02:00
John Ericson 0c6029669d Merge pull request #10935 from fricklerhandwerk/cli-docs-formatting
use separate paragraphs inside list items
2024-06-18 15:24:44 -04:00
John Ericson 613d598daa Merge pull request #10934 from Artoria2e5/patch-1
optimize-store.cc: Update macos exclusion comments
2024-06-18 15:15:45 -04:00
Valentin Gagarin b975151c09 dedent lists
this indentation is unnecessary and probably an artefact from the
migration off XML.
2024-06-18 11:26:09 +02:00
PoweredByPie 8b81d083a7 Remove lookupPathForProgram and implement initial runProgram test
Apparently, CreateProcessW already searches path, so manual path search
isn't really necessary.
2024-06-18 01:01:52 -07:00
PoweredByPie fcb92b4fa4 Fix DWORD vs. int comparison warning 2024-06-17 22:14:38 -07:00
Mingye Wang ff1fc780d2 optimize-store.cc: Update macos exclusion comments
#2230 broadened the scope of macOS hardlink exclusion but did not change the comments. This was a little confusing for me, so I figured the comments should be updated.
2024-06-18 12:05:59 +08:00
PoweredByPie 4f6e3b9402 Implement tests for lookupPathForProgram and fix bugs caught by tests 2024-06-17 18:46:08 -07:00
PoweredByPie d7537f6955 Implement initial spawn tests (just testing windowsEscape for now) 2024-06-17 14:58:17 -07:00
PoweredByPie 4662e7d856 Implement windowsEscape 2024-06-17 14:57:57 -07:00
John Ericson daf1b6b23a Merge pull request #10933 from NixOS/meson-libfetchers
Meson for libfetchers
2024-06-17 17:56:11 -04:00
Tom Bereknyei 706edf26eb build: meson for libfetchers 2024-06-17 17:25:56 -04:00
John Ericson 93218dc62a Merge pull request #10930 from fricklerhandwerk/meson-docs
migrate internal API docs to Meson
2024-06-17 16:25:03 -04:00
PoweredByPie b11cf8166f Format runProgram declaration 2024-06-17 13:12:28 -07:00
Valentin Gagarin 6e34c68327 Convert the internal API doc build to Meson 2024-06-17 15:51:58 -04:00
John Ericson 69d404edad Merge pull request #10914 from NixOS/combo-shell-perl
Integrate perl with the other meson builds
2024-06-17 15:15:05 -04:00
John Ericson a83d95e26e Integrate perl with the other meson builds
One big dev shell!
2024-06-17 14:48:20 -04:00
PoweredByPie a58ca342ca Initial runProgram implementation for Windows
This is incomplete; proper shell escaping needs to be done
2024-06-17 11:13:22 -07:00
John Ericson a1bb668ccb Merge remote-tracking branch 'upstream/master' into rm-createdirs 2024-06-17 12:57:54 -04:00
Valentin Gagarin 4f340213bb use separate paragraphs inside list items 2024-06-17 17:55:29 +02:00
Robert Hensing 316b58dd5f tests/shell.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
Robert Hensing 68b8a28bc4 tests/run.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
John Ericson e0b4691754 Merge pull request #10929 from NixOS/avoid-building-too-many-jobs-at-once
Avoid building too many jobs at once
2024-06-17 09:53:43 -04:00
John Ericson c9cdc2423a Temporarily remove the Meson builds from packages in the flake
This will avoid some out-of-memory issues in GitHub actions that result
from num jobs > 1 and num cores = 4. Once we only have the Meson build
system, this problem should go away, and we can reenable these jobs.
2024-06-17 09:16:18 -04:00
John Ericson 5e806673c3 Make hydraJobs.build include the constituent packages
We were only doing that for the more exotic builds, just forgot.
2024-06-17 08:33:09 -04:00
Robert Hensing e48abec567 Merge pull request #10916 from jmbaur/read-only-no-chown
Don't chown when local-store is read-only
2024-06-17 13:49:19 +02:00
Eelco Dolstra 48d38b32d2 Merge pull request #10918 from andir/restrict-tarfile-formats
Restrict supported tarball formats to actual Tarballs
2024-06-17 13:20:23 +02:00
Robert Hensing 83d1bc95b3 Merge pull request #10925 from hercules-ci/junit-report
`testresults` output
2024-06-17 10:54:29 +02:00
Robert Hensing 5e0e0ec2d3 Merge pull request #10927 from poweredbypie/windows-fix
Fix a compile error on windows
2024-06-17 10:52:09 +02:00
PoweredByPie b0cfac8f93 Fix compile error on windows 2024-06-17 00:03:50 -07:00
Jared Baur de639ceafe Don't chown when local-store is read-only
If the local-store is using the read-only flag, the underlying
filesystem might be read-only, thus an attempt to `chown` would always
fail.
2024-06-16 23:03:33 -07:00
Robert Hensing 2894c1b38e WIP add testresults output 2024-06-16 16:34:54 +02:00
Andreas Rammhold 5a9e1c0d20 Restrict supported tarball formats to actual Tarballs
The documentation is clear about the supported formats (with at least
`builtins.fetchTarball`). The way the code was written previously it
supported all the formats that libarchive supported. That is a
surprisingly large amount of formats that are likely not on the radar
of the Nix developers and users. Before people end up relying on
this (or if they do) it is better to break it now before it becomes a
widespread "feature".

Zip file support has been retained as (at least to my knowledge)
historically that has been used to fetch nixpkgs in some shell
expressions *many* years back.

Fixes https://github.com/NixOS/nix/issues/10917
2024-06-15 14:28:20 +02:00
Robert Hensing 573e385a68 Merge pull request #10907 from hercules-ci/issue-10561
C API: Use opaque struct instead of void for `nix_value`
2024-06-15 10:12:13 +02:00
John Ericson 4728840ca8 Merge pull request #10908 from NixOS/meson-libstore
Build `nix-store` with Meson, adjust Perl bindings deps to integrate with other meson builds
2024-06-14 12:16:14 -04:00
Eelco Dolstra b3bd31c82d Merge pull request #10878 from kirillrdy/kr/sysv
darwin: allow ipc-sysv* in sandbox
2024-06-14 17:11:20 +02:00
+6 81004a05c6 Build nix-store with Meson
Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-by: Eli Schwartz <eschwartz93@gmail.com>
2024-06-14 10:25:14 -04:00
John Ericson ea8e49bea5 Force the cpuid option for libutil rather than relying on detection
This is more robust, and match's Nixpkgs policy to force enable flags
statically by default (a common distro thing).
2024-06-14 10:25:14 -04:00
John Ericson 2f5fdab06c Merge pull request #10905 from obsidiansystems/platform-namespace
Put some file descriptor functions in unix and windows namespaces
2024-06-14 08:45:31 -04:00
Robert Hensing 80f9b04f5e Merge pull request #10903 from DeterminateSystems/fix-stack-overflow
eval-fail-infinite-recursion-lambda: Reduce recursion depth
2024-06-14 14:20:42 +02:00
Hamir Mahal 03883f0d1d fix: copy in install-multi-user.sh (#10902) 2024-06-13 21:13:21 +00:00
Robert Hensing b94e1d6218 C API: Value -> nix_value
See issue https://github.com/NixOS/nix/issues/10434
2024-06-13 18:51:58 +02:00
Robert Hensing 5d8118d9cb C API: Docs 2024-06-13 18:23:21 +02:00
Robert Hensing c50db4e58c C API: Add nix_value_{inc,dec}ref
- Can be implemented more easily by more eval architectures.
- Better types in generated bindings remove some uncertainty and doubt.
2024-06-13 18:21:04 +02:00
Robert Hensing 0b56c98b1c C API: Value -> nix_value 2024-06-13 18:18:36 +02:00
John Ericson c9766c6489 Merge pull request #10904 from obsidiansystems/meson-document
Document the nix-util Meson build system more extensively
2024-06-13 12:15:39 -04:00
John EricsonandValentin Gagarin 7a5ee5d597 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-13 11:42:17 -04:00
John EricsonandRobert Hensing 56f0b5304f Document the nix-util Meson build system more extensively
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-13 11:39:59 -04:00
John Ericson ff87c1a318 Put some file descriptor functions in unix and windows namespaces
It is misleading when platform-specific functions are in the overall
`nix` namespace. More namespaces also makes for nicer doxygen.
2024-06-13 11:18:59 -04:00
Eelco Dolstra 1dc7c8e599 eval-fail-infinite-recursion-lambda: Reduce recursion depth
This prevents the test from failing in environments with a smaller
configured stack size.
2024-06-13 13:55:42 +02:00
Robert Hensing afdd12be5e Merge pull request #10900 from obsidiansystems/hash-order
hash: Compare hash algo second for back compat
2024-06-13 01:35:31 +02:00
John Ericson 96cf6b0f5f Merge pull request #10855 from NixOS/meson-libutil
Build `nix-util` with Meson
2024-06-12 19:19:22 -04:00
John Ericson 25a9894943 hash: Compare hash algo second for back compat
Previously (in cfc18a7739), we forgot to
compare the algo at all. This means we keep the same ordering as before
by making the stuff we always have compared take priority.
2024-06-12 18:35:47 -04:00
John Ericson 0e225596d6 Merge pull request #10857 from SkamDart/shellcheck-auto-fixup
Shellcheck: test scripts
2024-06-12 18:33:10 -04:00
+7 28d2af4ea6 Build nix-util with Meson
The idea is two-fold:

- Replace autotools with Meson

- Build each library in its own derivation

The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.

I tested the Linux native build, and NetBSD and Windows cross builds.

Also do some clean ups of the Flake in the process of supporting new
jobs.

Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 18:31:02 -04:00
John Ericson 33241887d1 More quote coalescing 2024-06-12 17:47:54 -04:00
John Ericson d8ae28617d Try to fix quotes that don't go to end with sed 2024-06-12 17:41:16 -04:00
Cameron Dart 1afac8fbbc remove tests from pre-commit excludes 2024-06-12 17:41:16 -04:00
Cameron Dart 2d467b4731 housekeeping: shellcheck for tests/functional/import-derivation.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 1c93360989 housekeeping: shellcheck for tests/functional/hash-path.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d1c476865a housekeeping: shellcheck for tests/functional/gc-runtime.sh 2024-06-12 17:41:16 -04:00
Cameron Dart cd46ec17f9 housekeeping: shellcheck for tests/functional/function-trace.sh 2024-06-12 17:41:16 -04:00
Cameron Dart b764dd9aa4 housekeeping: shellcheck for tests/functional/flakes/unlocked-override.sh 2024-06-12 17:41:16 -04:00
Cameron Dart e1ce349d05 housekeeping: shellcheck for tests/functional/flakes/search-root.sh 2024-06-12 17:41:16 -04:00
Cameron Dart b9768b4872 housekeeping: shellcheck for tests/functional/flakes/mercurial.sh 2024-06-12 17:41:16 -04:00
Cameron Dart c7b3468968 housekeeping: shellcheck for tests/functional/flakes/inputs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d95adb531e housekeeping: shellcheck for tests/functional/flakes/init.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 71d5baca47 housekeeping: shellcheck for tests/functional/flakes/flake-in-submodule.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 3b853e795b housekeeping: shellcheck for tests/functional/flakes/circular.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ece86b7191 housekeeping: shellcheck for tests/functional/flakes/bundle.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ee57c610ae housekeeping: shellcheck for tests/functional/flakes/build-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d81fd4a1c3 housekeeping: shellcheck for tests/functional/flakes/absolute-attr-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ae6a842c55 housekeeping: shellcheck for tests/functional/filter-source.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 224f5515b9 housekeeping: shellcheck for tests/functional/fetchTree-file.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart f0492a6197 housekeeping: shellcheck for tests/functional/fetchPath.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 823d53c643 housekeeping: shellcheck for tests/functional/experimental-features.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 1c1abefdd2 housekeeping: shellcheck for tests/functional/dyn-drv/text-hashed-output.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 847842c4bb housekeeping: shellcheck for tests/functional/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 58a653dcc6 housekeeping: shellcheck for tests/functional/compression-levels.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 48520cb71e housekeeping: shellcheck for tests/functional/chroot-store.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 63272235e2 housekeeping: shellcheck for tests/functional/case-hacks.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart 4f04006bc1 housekeeping: shellcheck for tests/functional/check-reqs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d7bb5bde48 housekeeping: shellcheck for tests/functional/check-refs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 259b502773 housekeeping: shellcheck for tests/functional/ca/substitute.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 04876c39e4 housekeeping: shellcheck for tests/functional/ca/signatures.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 8f85537629 housekeeping: shellcheck for tests/functional/ca/nix-run.sh 2024-06-12 17:41:16 -04:00
Cameron Dart deacc421eb housekeeping: shellcheck for tests/functional/ca/nix-copy.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 195c0da849 housekeeping: shellcheck for tests/functional/ca/duplicate-realisation-in-closure.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 2dfbba3e5e housekeeping: shellcheck for tests/functional/ca/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 627176fd54 housekeeping: shellcheck for tests/functional/ca/build.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 80c44138cb housekeeping: shellcheck for tests/functional/ca/build-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart c127625095 housekeeping: shellcheck for tests/functional/build-dry.sh 2024-06-12 17:41:16 -04:00
Cameron Dart f615489e0e housekeeping: shellcheck for tests/functional/build-delete.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 7186c68f75 housekeeping: shellcheck for tests/functional/brotli.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:15 -04:00
Cameron DartandSandro aeed835a2e housekeeping: shellcheck for tests/functional/binary-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:08 -04:00
Cameron Dart 4a28ba7877 housekeeping: shellcheck for tests/functional/binary-cache-build-remote.sh 2024-06-12 16:58:25 -04:00
Cameron Dart 7738b295e5 housekeeping: shellcheck for tests/functional/bash-profile.sh 2024-06-12 16:58:25 -04:00
John Ericson 394286cf5e Merge pull request #10896 from obsidiansystems/fix-freebsd-build
Fix FreeBSD build
2024-06-12 16:49:25 -04:00
John Ericson 5b53d8fec3 Factor out GC initialization code
This is not really part of the evaluator: it is just an integration
between Boehm GC and Boost coroutines usable for any purpose. The
evaluator (merely) optionally uses it.
2024-06-12 16:00:03 -04:00
John EricsonandEelco Dolstra 7c2981fc55 Fix FreeBSD build
This restores some CPP'd code that was added in
c18911602e and accidentally lost in
2477e4e3b8.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-12 15:59:54 -04:00
John Ericson e74ce01b7f Fix precompiled headers building with clang
Since 24.05 (I think) we need to pass `-c` or Clang thinks we want to
compile *both* a final executable and precompiled header file, and
complains that we cannot use `-o` with multiple outputs. `-c` seems fine
with GCC too, so I just put it in there conditionally.
2024-06-12 10:51:43 -04:00
Eelco Dolstra 01644744ed Merge pull request #10895 from NixOS/bump-version
Bump version
2024-06-12 15:29:29 +02:00
Eelco Dolstra 258e2a32ec Bump version 2024-06-12 14:57:40 +02:00
Eelco DolstraandRobert Hensing efd4bf6533 Update src/libfetchers/git-utils.cc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 14:41:35 +02:00
Eelco DolstraandRobert Hensing bd37a70d8f Update tests/functional/tarball.sh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-11 19:39:42 +02:00
Valentin Gagarin b130d2f2e3 add more context on the README (#9871)
the thesis is still the defining document with all the motivation and
explanations.
adding it here for greater visibility.

also more emphasis and clarity around the community aspect.
the hydra build job seems a bit arbitrary right there. may be better for
the contributing guide.
2024-06-11 15:52:33 +00:00
Eelco Dolstra 35bdb9cee7 Support hard links in tarballs
Fixes #10395.
2024-06-11 16:05:57 +02:00
siddhantCodes 552a2cee21 Merge branch 'master' of github.com:NixOS/nix into rm-createdirs 2024-06-11 19:16:56 +05:30
d1dd7abbf0 mention the actual meaning of FODs in the glossary (#10888)
* mention the actual meaning of FODs in the glossary

Co-authored-by: Alex Groleau <source@proof.construction>
Co-authored-by: Daniel Baker <daniel.n.baker@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-10 20:50:35 +02:00
Robert Hensing de3fd52a95 Add tests/f/lang/eval-okay-derivation-legacy 2024-06-10 16:31:46 +02:00
Tom Bereknyei 4809e59b7e fix: warn and document when advanced attributes will have no impact due to __structuredAttrs 2024-06-10 09:31:21 -04:00
Robert Hensing e943ee32bd Merge pull request #10883 from tomberek/tomberek.remove_xdg_tmp
fix: remove usage of XDG_RUNTIME_DIR for TMP
2024-06-10 15:08:50 +02:00
Tom Bereknyei 1363f51bcb fix: remove usage of XDG_RUNTIME_DIR for TMP 2024-06-10 08:38:57 -04:00
Eelco Dolstra 12355e7ac5 Merge pull request #10880 from hercules-ci/bdwgc-8.2.6-has-sp-corrector
bdw-gc 8.2.6 has sp corrector
2024-06-10 13:28:05 +02:00
Robert Hensing f91f34aa65 bdwgc 8.2.4 has sp_corrector
> Support client-defined stack pointer adjustment before thread stack push

-- https://github.com/ivmai/bdwgc/releases/tag/v8.2.4

This fixes an inaccuracy in cc6f315252,
in the update to Nixpkgs 24.05 https://github.com/NixOS/nix/pull/10835

After this fixup, the build log won't ask for an upgrade, and we'll be
able to collect when a coroutine exists, e.g. during filterSource.
2024-06-10 12:57:13 +02:00
Robert Hensing 0a09597790 Typo 2024-06-10 12:46:04 +02:00
Robert Hensing 4755e133c4 Fix warning 2024-06-10 12:46:04 +02:00
siddhantCodes 7a21432e77 fix: catch filesystem_error thrown by createDirs 2024-06-10 11:30:39 +05:30
Kirill Radzikhovskyy 372d5a441e darwin: allow ipc-sysv* in sandbox 2024-06-10 11:21:17 +10:00
siddhantCodes 25b0242ca6 std::filesystem::create_directories for createDirs
The implementation of `nix::createDirs` allows it to be a simple wrapper
around `std::filesystem::create_directories` as its return value is not
used anywhere.
2024-06-09 19:49:39 +05:30
Robert Hensing 0ab9369572 Merge pull request #10854 from DeterminateSystems/thread-safe-RegexCache
Make RegexCache thread-safe
2024-06-08 18:07:35 +02:00
Robert Hensing e1b3716d50 Merge pull request #10853 from DeterminateSystems/thread-safe-srcToStore
Make EvalState::srcToStore thread-safe
2024-06-06 19:31:46 +02:00
John Ericson 5ff79efcad Merge pull request #10869 from Mic92/improve-github-actions-doc
Improve GitHub actions doc
2024-06-06 11:10:51 -04:00
John Ericson dd46ed85eb Merge pull request #10868 from DeterminateSystems/shared-sync
Add SharedSync class
2024-06-06 10:11:36 -04:00
Jörg Thalheim e505434332 document how to test github ci fully in your own fork 2024-06-06 12:55:45 +02:00
Jörg Thalheim 7e6a7c9258 make it possible to push to different docker registries in forks 2024-06-06 11:34:30 +02:00
Eelco Dolstra fd9e49480a PosixSourceAccessor: Use SharedSync 2024-06-06 10:44:53 +02:00
Eelco Dolstra 2f39caf180 Sync: Add support for shared locks 2024-06-06 10:44:09 +02:00
John Ericson b0283240a1 Merge pull request #10864 from Mic92/tarball-flakes-test
tests/nixos: make the tarball-flakes test better reflect real use cases
2024-06-05 18:40:16 -04:00
Robert Hensing dadb6f86cf Merge pull request #10865 from Mic92/better-cleanup
libutil: guard Finally against invalid exception throws
2024-06-05 23:59:51 +02:00
John Ericson 1a32490aca Merge pull request #10835 from hercules-ci/update-nixpkgs-24.05
Update to Nixpkgs 24.05
2024-06-05 17:56:16 -04:00
eldritch horrors e291087747 libutil: guard Finally against invalid exception throws
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Source: https://git.lix.systems/lix-project/lix/commit/6c777476c9e97abfc5232f0707985caf6df2baea
2024-06-05 22:45:37 +02:00
Pierre Bourdon 162d94d975 tests/nixos: make the tarball-flakes test better reflect real use cases
In most real world cases, the Link header is set on the redirect, not on
the final file. This regressed in Lix earlier and while new unit tests
were added to cover it, this integration test should probably have also
caught it.

Source: https://git.lix.systems/lix-project/lix/commit/a3256a93759206dc64e597e2ad790ce582e84cf3
2024-06-05 20:14:24 +02:00
Eelco Dolstra edd445f745 Merge pull request #10861 from DeterminateSystems/fix-optional-deref
PackageInfo::queryDrvPath(): Don't dereference an empty optional
2024-06-05 16:48:09 +02:00
Eelco Dolstra 3e72ed9743 Typo 2024-06-05 16:19:01 +02:00
Eelco Dolstra d2eeabf3e6 PackageInfo::queryDrvPath(): Don't dereference an empty optional
Fixes a regression introduced in f923ed6b6a.

https://hydra.nixos.org/build/262267313
2024-06-05 16:17:24 +02:00
Robert Hensing 9d34815cb6 Merge pull request #10858 from elohmeier/master
flake check: Recognize well known homeModule/homeModules attribute
2024-06-05 13:11:15 +02:00
Enno Richter 80ba7778e7 flake check: Recognize well known homeModule/homeModules attributes 2024-06-05 07:38:20 +02:00
Jörg Thalheim 49c6f34911 docs: fixup description of builtins.importNative (#10810)
There was an argument missing and the fact that the imported function is
called.
2024-06-04 19:55:05 +00:00
Eelco Dolstra ef140c25d1 Merge pull request #10852 from siddhantk232/add-interrupts
add call to `checkInterrupt` in a bunch of places
2024-06-04 17:21:03 +02:00
Eelco Dolstra fbbca59453 Make RegexCache thread-safe 2024-06-04 16:59:30 +02:00
Eelco Dolstra 7d295c594e Make EvalState::srcToStore thread-safe 2024-06-04 16:56:06 +02:00
siddhantCodes 8f1a26667e add call to checkInterrupt in a bunch of places
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
2024-06-04 19:35:40 +05:30
John Ericson 4e0d058fc3 eval.cc: Fix for Windows 2024-06-04 10:18:22 +02:00
John Ericson 6588420103 Bring back FreeBSD 2024-06-04 10:17:58 +02:00
Robert Hensing 5d460d563e TMP: Disable tests.setuid.i686-linux
Temporarily(?) blocked on https://github.com/NixOS/nixpkgs/pull/297475#issuecomment-2145589501
2024-06-03 18:47:53 +02:00
Robert Hensing d494ac15e2 Use Nixpkgs changelog-d 2024-06-03 18:47:53 +02:00
Robert Hensing 5ddc11d0eb Update nixpkgs 2024-06-03 18:47:53 +02:00
Robert Hensing 27f880c098 Format after clang-format update 2024-06-03 18:38:56 +02:00
Robert Hensing e6ba450ce2 .clang-format: Remove duplicated key 2024-06-03 18:38:56 +02:00
Robert Hensing 8a510f4ede Add tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing efc2508e8b Refactor hydraJobs.tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing 6558025e77 Fix eval remoteBuilds_*_2_13 2024-06-03 17:34:55 +02:00
Robert Hensing 449e4b9232 Change checkOverrideNixVersion for NixOS 24.05 2024-06-03 17:34:55 +02:00
Robert Hensing 6067525162 hydraJobs.installTests.<s>.againstCurrent{Unstable -> Latest}
Nixpkgs has reshuffled its Nix versions.
2024-06-03 16:37:39 +02:00
Robert Hensing f01f65b615 Fix nixpkgsLibTests 2024-06-03 16:37:39 +02:00
Robert Hensing cc6f315252 nix: Disable GC during coroutine when bdwgc < 8.4
This re-enables support for older bwdgc versions without complicating
the code too much.
Coroutines generally only interfere with GC during source filtering,
so it's not too bad of a regression on older bdwgc.
This seems preferable over conditional compilation to enable the patch
etc; we've already spent a lot of complexity budget on this GC-coroutine
interaction...
2024-06-03 16:37:39 +02:00
Robert Hensing b311f51f84 boehmgc-nix: Remove released traceable_allocator patch 2024-06-03 16:37:39 +02:00
Robert Hensing 2477e4e3b8 libexpr: Use GC_set_sp_corrector instead of patch
Manually tested by printing to stderr in both branches (sp in os
stack, or not), and triggering a GC in a filterSource function,
e.g.:

    let
      generateTree = n: if n == 0 then "ha" else { left = generateTree (n - 1); right = generateTree (n - 1); };
    in
      builtins.deepSeq (generateTree 18) ...

Note that the darwin still uses the strategy of disabling GC, despite
having an implementation that compiles. The proper solution will be
enabled and tested later.
2024-06-03 16:37:39 +02:00
Robert Hensing 2edcdf8508 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e' (2024-02-28)
  → 'github:NixOS/nixpkgs/88dca77be222aedd1f47d2cf0942dffefee76216' (2024-06-03)
2024-06-03 16:37:39 +02:00
Robert Hensing 8df206be54 Update nixpkgs ref to 24.05 2024-06-03 16:37:39 +02:00
Tom Bereknyei f66f498bd4 notes: document change in nix-shell behavior 2023-11-27 21:22:59 -05:00
Tom Bereknyei 9a4641146f tests: ensure nix-shell uses relative paths for expressions 2023-11-27 14:40:38 -05:00
Matthew Bauer 04023360ed Evaluate nix-shell -i args relative to script
When writing a shebang script, you expect your path to be relative to
the script, not the cwd. We previously handled this correctly for
relative file paths, but not for expressions.

This handles both -p & -E args. My understanding is this should be
what we want in any cases I can think of - people run scripts from
many different working directories. @edolstra is there any reason to
handle -p args differently in this case?

Fixes #4232
2023-11-27 14:40:38 -05:00
1151 changed files with 29078 additions and 24811 deletions

No files matched your search

-1
View File
@@ -31,4 +31,3 @@ AlwaysBreakBeforeMultilineStrings: true
IndentPPDirectives: AfterHash
PPIndentWidth: 2
BinPackArguments: false
BinPackParameters: false
+4 -4
View File
@@ -4,20 +4,20 @@
# Top-most EditorConfig file
root = true
# Unix-style newlines with a newline ending every file, utf-8 charset
# Unix-style newlines with a newline ending every file, UTF-8 charset
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8
# Match nix files, set indent to spaces with width of two
# Match Nix files, set indent to spaces with width of two
[*.nix]
indent_style = space
indent_size = 2
# Match c++/shell/perl, set indent to spaces with width of four
[*.{hpp,cc,hh,sh,pl,xs}]
# Match C++/C/shell/Perl, set indent to spaces with width of four
[*.{hpp,cc,hh,c,h,sh,pl,xs}]
indent_style = space
indent_size = 4
+1 -1
View File
@@ -23,4 +23,4 @@ maintainers/*.md @fricklerhandwerk
src/**/*.md @fricklerhandwerk
# Libstore layer
/src/libstore @thufschmitt @ericson2314
/src/libstore @ericson2314
+125 -80
View File
@@ -7,14 +7,28 @@ on:
permissions: read-all
jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
tests:
needs: [check_secrets]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
@@ -23,75 +37,54 @@ jobs:
- uses: cachix/install-nix-action@V27
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
extra_nix_config: |
sandbox = true
max-jobs = 1
- uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'linux'
- run: scripts/build-checks
- run: scripts/prepare-installer-for-github-actions
- name: Upload installer tarball
uses: actions/upload-artifact@v4
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' flake check -L
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v15
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- id: prepare-installer
run: scripts/prepare-installer-for-github-actions
name: installer-${{matrix.os}}
path: out/*
installer_test:
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
needs: [tests]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
- name: Download installer tarball
uses: actions/download-artifact@v4
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
name: installer-${{matrix.os}}
path: out
- name: Serving installer
id: serving_installer
run: ./scripts/serve-installer-for-github-actions
- uses: cachix/install-nix-action@v30
with:
install_url: 'http://localhost:8126/install'
install_options: "--tarball-url-prefix http://localhost:8126/"
- run: sudo apt install fish zsh
if: matrix.os == 'ubuntu-latest'
if: matrix.os == 'linux'
- run: brew install fish
if: matrix.os == 'macos-latest'
if: matrix.os == 'darwin'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
@@ -99,36 +92,54 @@ jobs:
- run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs"
- run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello"
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
docker_push_image:
needs: [check_secrets, tests]
needs: [tests, vm_tests, check_secrets]
permissions:
contents: read
packages: write
if: >-
needs.check_secrets.outputs.docker == 'true' &&
github.event_name == 'push' &&
github.ref_name == 'master' &&
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
github.ref_name == 'master'
runs-on: ubuntu-24.04
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION nixos/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION nixos/nix:master
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:master
# We'll deploy the newly built image to both Docker Hub and Github Container Registry.
#
# Push to Docker Hub first
@@ -137,8 +148,8 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- run: docker push nixos/nix:$NIX_VERSION
- run: docker push nixos/nix:master
- run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
- run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/nix:master
# Push to GitHub Container Registry as well
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
@@ -161,9 +172,43 @@ jobs:
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes .#hydraJobs.tests.functional_user
meson_build:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
# Only meson packages that don't have a tests.run derivation.
# Those that have it are already built and tested as part of nix flake check.
- run: nix build -L .#hydraJobs.build.{nix-cmd,nix-main}.$(nix-instantiate --eval --expr builtins.currentSystem | sed -e 's/"//g')
flake_regressions:
needs: vm_tests
runs-on: ubuntu-24.04
steps:
- name: Checkout nix
uses: actions/checkout@v4
- name: Checkout flake-regressions
uses: actions/checkout@v4
with:
repository: NixOS/flake-regressions
path: flake-regressions
- name: Checkout flake-regressions-data
uses: actions/checkout@v4
with:
repository: NixOS/flake-regressions-data
path: flake-regressions/tests
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build --out-link ./new-nix && PATH=$(pwd)/new-nix/bin:$PATH scripts/flake-regressions.sh
-20
View File
@@ -1,20 +0,0 @@
name: Hydra status
permissions: read-all
on:
schedule:
- cron: "12,42 * * * *"
workflow_dispatch:
jobs:
check_hydra_status:
name: Check Hydra status
if: github.repository_owner == 'NixOS'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: bash scripts/check-hydra-status.sh
+1 -1
View File
@@ -15,7 +15,7 @@ permissions:
jobs:
labels:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/labeler@v5
+5
View File
@@ -11,6 +11,8 @@ perl/Makefile.config
/svn-revision
/libtool
/config/config.*
# Default meson build dir
/build
# /doc/manual/
/doc/manual/*.1
@@ -52,6 +54,9 @@ perl/Makefile.config
# /src/libfetchers
/tests/unit/libfetchers/libnixfetchers-tests
# /src/libflake
/tests/unit/libflake/libnixflake-tests
# /src/libstore/
*.gen.*
/src/libstore/tests
+4 -2
View File
@@ -5,8 +5,10 @@ queue_rules:
- check-success=installer
- check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest)
- check-success=tests (macos-latest)
- check-success=tests (ubuntu-latest)
- check-success=tests on macos
- check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests
merge_method: rebase
batch_size: 5
+2
View File
@@ -1,2 +1,4 @@
external-sources=true
source-path=SCRIPTDIR
# Hack for scripts in e.g. tests/functional/ca
source-path=SCRIPTDIR/..
+1 -1
View File
@@ -1 +1 @@
2.23.4
2.24.16
+4 -4
View File
@@ -41,9 +41,9 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
There are many open pull requests that might already do what you intend to work on.
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
3. Check the [Nix reference manual](https://nix.dev/manual/nix/development/development/building.html) for information on building Nix and running its tests.
For contributions to the command line interface, please check the [CLI guidelines](https://nixos.org/manual/nix/unstable/contributing/cli-guideline.html).
For contributions to the command line interface, please check the [CLI guidelines](https://nix.dev/manual/nix/development/development/cli-guideline.html).
4. Make your change!
@@ -69,7 +69,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
- [ ] API documentation in header files
- [ ] Code and comments are self-explanatory
- [ ] Commit message explains **why** the change was made
- [ ] New feature or incompatible change: [add a release note](https://nixos.org/manual/nix/stable/contributing/hacking#add-a-release-note)
- [ ] New feature or incompatible change: [add a release note](https://nix.dev/manual/nix/development/development/contributing.html#add-a-release-note)
7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams).
@@ -78,7 +78,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
The Nix reference manual is hosted on https://nixos.org/manual/nix.
The underlying source files are located in [`doc/manual/src`](./doc/manual/src).
For small changes you can [use GitHub to edit these files](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)
For larger changes see the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html).
For larger changes see the [Nix reference manual](https://nix.dev/manual/nix/development/development/contributing.html).
## Getting help
+5 -24
View File
@@ -18,6 +18,7 @@ makefiles = \
src/libfetchers/local.mk \
src/libmain/local.mk \
src/libexpr/local.mk \
src/libflake/local.mk \
src/libcmd/local.mk \
src/nix/local.mk \
src/libutil-c/local.mk \
@@ -45,13 +46,15 @@ makefiles += \
tests/unit/libstore-support/local.mk \
tests/unit/libfetchers/local.mk \
tests/unit/libexpr/local.mk \
tests/unit/libexpr-support/local.mk
tests/unit/libexpr-support/local.mk \
tests/unit/libflake/local.mk
endif
ifeq ($(ENABLE_FUNCTIONAL_TESTS), yes)
ifdef HOST_UNIX
makefiles += \
tests/functional/local.mk \
tests/functional/flakes/local.mk \
tests/functional/ca/local.mk \
tests/functional/git-hashing/local.mk \
tests/functional/dyn-drv/local.mk \
@@ -68,14 +71,6 @@ ifeq ($(ENABLE_DOC_GEN), yes)
makefiles-late += doc/manual/local.mk
endif
ifeq ($(ENABLE_INTERNAL_API_DOCS), yes)
makefiles-late += doc/internal-api/local.mk
endif
ifeq ($(ENABLE_EXTERNAL_API_DOCS), yes)
makefiles-late += doc/external-api/local.mk
endif
# Miscellaneous global Flags
OPTIMIZE = 1
@@ -98,7 +93,7 @@ ifdef HOST_WINDOWS
GLOBAL_LDFLAGS += -Wl,--export-all-symbols
endif
GLOBAL_CXXFLAGS += -g -Wall -Wdeprecated-copy -Wignored-qualifiers -Wimplicit-fallthrough -include $(buildprefix)config.h -std=c++2a -I src
GLOBAL_CXXFLAGS += -g -Wall -Wdeprecated-copy -Wignored-qualifiers -Wimplicit-fallthrough -Werror=unused-result -Werror=suggest-override -include $(buildprefix)config.h -std=c++2a -I src
# Include the main lib, causing rules to be defined
@@ -131,17 +126,3 @@ manual-html manpages:
@echo "Generated docs are disabled. Configure without '--disable-doc-gen', or avoid calling 'make manpages' and 'make manual-html'."
@exit 1
endif
ifneq ($(ENABLE_INTERNAL_API_DOCS), yes)
.PHONY: internal-api-html
internal-api-html:
@echo "Internal API docs are disabled. Configure with '--enable-internal-api-docs', or avoid calling 'make internal-api-html'."
@exit 1
endif
ifneq ($(ENABLE_EXTERNAL_API_DOCS), yes)
.PHONY: external-api-html
external-api-html:
@echo "External API docs are disabled. Configure with '--enable-external-api-docs', or avoid calling 'make external-api-html'."
@exit 1
endif
-2
View File
@@ -11,8 +11,6 @@ EDITLINE_LIBS = @EDITLINE_LIBS@
ENABLE_BUILD = @ENABLE_BUILD@
ENABLE_DOC_GEN = @ENABLE_DOC_GEN@
ENABLE_FUNCTIONAL_TESTS = @ENABLE_FUNCTIONAL_TESTS@
ENABLE_INTERNAL_API_DOCS = @ENABLE_INTERNAL_API_DOCS@
ENABLE_EXTERNAL_API_DOCS = @ENABLE_EXTERNAL_API_DOCS@
ENABLE_S3 = @ENABLE_S3@
ENABLE_UNIT_TESTS = @ENABLE_UNIT_TESTS@
GTEST_LIBS = @GTEST_LIBS@
+14 -10
View File
@@ -4,30 +4,34 @@
[![Test](https://github.com/NixOS/nix/workflows/Test/badge.svg)](https://github.com/NixOS/nix/actions)
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
management reliable and reproducible. Please refer to the [Nix manual](https://nix.dev/reference/nix-manual)
for more details.
## Installation and first steps
Visit [nix.dev](https://nix.dev) for [installation instructions](https://nix.dev/tutorials/install-nix) and [beginner tutorials](https://nix.dev/tutorials/first-steps).
Full reference documentation can be found in the [Nix manual](https://nixos.org/nix/manual).
Full reference documentation can be found in the [Nix manual](https://nix.dev/reference/nix-manual).
## Building And Developing
## Building and developing
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
set up a development environment and build Nix from source.
Follow instructions in the Nix reference manual to [set up a development environment and build Nix from source](https://nix.dev/manual/nix/development/building.html).
## Contributing
Check the [contributing guide](./CONTRIBUTING.md) if you want to get involved with developing Nix.
## Additional Resources
## Additional resources
- [Nix manual](https://nixos.org/nix/manual)
- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
- [NixOS Discourse](https://discourse.nixos.org/)
- [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org)
Nix was created by Eelco Dolstra and developed as the subject of his PhD thesis [The Purely Functional Software Deployment Model](https://edolstra.github.io/pubs/phd-thesis.pdf), published 2006.
Today, a world-wide developer community contributes to Nix and the ecosystem that has grown around it.
- [The Nix, Nixpkgs, NixOS Community on nixos.org](https://nixos.org/)
- [Official documentation on nix.dev](https://nix.dev)
- [Nixpkgs](https://github.com/NixOS/nixpkgs) is [the largest, most up-to-date free software repository in the world](https://repology.org/repositories/graphs)
- [NixOS](https://github.com/NixOS/nixpkgs/tree/master/nixos) is a Linux distribution that can be configured fully declaratively
- [Discourse](https://discourse.nixos.org/)
- [Matrix](https://matrix.to/#/#nix:nixos.org)
## License
+36
View File
@@ -0,0 +1,36 @@
# These are private dependencies with pkg-config files. What private
# means is that the dependencies are used by the library but they are
# *not* used (e.g. `#include`-ed) in any installed header file, and only
# in regular source code (`*.cc`) or private, uninstalled headers. They
# are thus part of the *implementation* of the library, but not its
# *interface*.
#
# See `man pkg-config` for some details.
deps_private = [ ]
# These are public dependencies with pkg-config files. Public is the
# opposite of private: these dependencies are used in installed header
# files. They are part of the interface (and implementation) of the
# library.
#
# N.B. This concept is mostly unrelated to our own concept of a public
# (stable) API, for consumption outside of the Nix repository.
# `libnixutil` is an unstable C++ library, whose public interface is
# likewise unstable. `libutilc` conversely is a hopefully-soon stable
# C library, whose public interface --- including public but not private
# dependencies --- will also likewise soon be stable.
#
# N.B. For distributions that care about "ABI" stability and not just
# "API" stability, the private dependencies also matter as they can
# potentially affect the public ABI.
deps_public = [ ]
# These are subproject deps (type == "internal"). They are other
# packages in `/src` in this repo. The private vs public distinction is
# the same as above.
deps_private_subproject = [ ]
deps_public_subproject = [ ]
# These are dependencencies without pkg-config files. Ideally they are
# just private, but they may also be public (e.g. boost).
deps_other = [ ]
+11
View File
@@ -0,0 +1,11 @@
add_project_arguments(
'-Wdeprecated-copy',
'-Werror=suggest-override',
'-Werror=switch',
'-Werror=switch-enum',
'-Werror=unused-result',
'-Wignored-qualifiers',
'-Wimplicit-fallthrough',
'-Wno-deprecated-declarations',
language : 'cpp',
)
@@ -0,0 +1,11 @@
if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
# Windows DLLs are stricter about symbol visibility than Unix shared
# objects --- see https://gcc.gnu.org/wiki/Visibility for details.
# This is a temporary sledgehammer to export everything like on Unix,
# and not detail with this yet.
#
# TODO do not do this, and instead do fine-grained export annotations.
linker_export_flags = ['-Wl,--export-all-symbols']
else
linker_export_flags = []
endif
+30
View File
@@ -0,0 +1,30 @@
requires_private = []
foreach dep : deps_private_subproject
requires_private += dep.name()
endforeach
requires_private += deps_private
requires_public = []
foreach dep : deps_public_subproject
requires_public += dep.name()
endforeach
requires_public += deps_public
import('pkgconfig').generate(
this_library,
filebase : meson.project_name(),
name : 'Nix',
description : 'Nix Package Manager',
subdirs : ['nix'],
extra_cflags : ['-std=c++2a'],
requires : requires_public,
requires_private : requires_private,
libraries_private : libraries_private,
)
meson.override_dependency(meson.project_name(), declare_dependency(
include_directories : include_dirs,
link_with : this_library,
compile_args : ['-std=c++2a'],
dependencies : deps_public_subproject + deps_public,
))
@@ -0,0 +1,7 @@
bash = find_program('bash', native: true)
gen_header = generator(
bash,
arguments : [ '-c', '{ echo \'R"__NIX_STR(\' && cat @INPUT@ && echo \')__NIX_STR"\'; } > "$1"', '_ignored_argv0', '@OUTPUT@' ],
output : '@PLAINNAME@.gen.hh',
)
+19
View File
@@ -0,0 +1,19 @@
foreach maybe_subproject_dep : deps_private_maybe_subproject
if maybe_subproject_dep.type_name() == 'internal'
deps_private_subproject += maybe_subproject_dep
# subproject sadly no good for pkg-config module
deps_other += maybe_subproject_dep
else
deps_private += maybe_subproject_dep
endif
endforeach
foreach maybe_subproject_dep : deps_public_maybe_subproject
if maybe_subproject_dep.type_name() == 'internal'
deps_public_subproject += maybe_subproject_dep
# subproject sadly no good for pkg-config module
deps_other += maybe_subproject_dep
else
deps_public += maybe_subproject_dep
endif
endforeach
+6
View File
@@ -0,0 +1,6 @@
# This is only conditional to work around
# https://github.com/mesonbuild/meson/issues/13293. It should be
# unconditional.
if not (host_machine.system() == 'windows' and cxx.get_id() == 'gcc')
deps_private += dependency('threads')
endif
-186
View File
@@ -1,186 +0,0 @@
{ inputs
, binaryTarball
, forAllCrossSystems
, forAllSystems
, lib
, linux64BitSystems
, nixpkgsFor
, self
}:
let
inherit (inputs) nixpkgs nixpkgs-regression;
inherit (lib) fileset;
installScriptFor = tarballs:
nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix {
inherit tarballs;
};
testNixVersions = pkgs: client: daemon:
pkgs.callPackage ../package.nix {
pname =
"nix-tests"
+ lib.optionalString
(lib.versionAtLeast daemon.version "2.4pre20211005" &&
lib.versionAtLeast client.version "2.4pre20211005")
"-${client.version}-against-${daemon.version}";
inherit fileset;
test-client = client;
test-daemon = daemon;
doBuild = false;
};
in
{
# Binary package for various platforms.
build = forAllSystems (system: self.packages.${system}.nix);
shellInputs = forAllSystems (system: self.devShells.${system}.default.inputDerivation);
buildStatic = lib.genAttrs linux64BitSystems (system: self.packages.${system}.nix-static);
buildCross = forAllCrossSystems (crossSystem:
lib.genAttrs [ "x86_64-linux" ] (system: self.packages.${system}."nix-${crossSystem}"));
buildNoGc = forAllSystems (system:
self.packages.${system}.nix.override { enableGC = false; }
);
buildNoTests = forAllSystems (system:
self.packages.${system}.nix.override {
doCheck = false;
doInstallCheck = false;
installUnitTests = false;
}
);
# Toggles some settings for better coverage. Windows needs these
# library combinations, and Debian build Nix with GNU readline too.
buildReadlineNoMarkdown = forAllSystems (system:
self.packages.${system}.nix.override {
enableMarkdown = false;
readlineFlavor = "readline";
}
);
# Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nix.perl-bindings);
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of
# the installation script.
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (system:
forAllCrossSystems (crossSystem:
binaryTarball
self.packages.${system}."nix-${crossSystem}"
nixpkgsFor.${system}.cross.${crossSystem}));
# The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary
# tarball for the user's system and calls the second half of the
# installation script.
installerScript = installScriptFor [
# Native
self.hydraJobs.binaryTarball."x86_64-linux"
self.hydraJobs.binaryTarball."i686-linux"
self.hydraJobs.binaryTarball."aarch64-linux"
self.hydraJobs.binaryTarball."x86_64-darwin"
self.hydraJobs.binaryTarball."aarch64-darwin"
# Cross
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
installerScriptForGHA = installScriptFor [
# Native
self.hydraJobs.binaryTarball."x86_64-linux"
self.hydraJobs.binaryTarball."aarch64-darwin"
# Cross
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
# docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# Line coverage analysis.
coverage = nixpkgsFor.x86_64-linux.native.nix.override {
pname = "nix-coverage";
withCoverageChecks = true;
};
# API docs for Nix's unstable internal C++ interfaces.
internal-api-docs = nixpkgsFor.x86_64-linux.native.callPackage ../package.nix {
inherit fileset;
doBuild = false;
enableInternalAPIDocs = true;
};
# API docs for Nix's C bindings.
external-api-docs = nixpkgsFor.x86_64-linux.native.callPackage ../package.nix {
inherit fileset;
doBuild = false;
enableExternalAPIDocs = true;
};
# System tests.
tests = import ../tests/nixos { inherit lib nixpkgs nixpkgsFor; } // {
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in
runCommand "eval-nixos" { buildInputs = [ nix ]; }
''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
(
set -x
time nix-env --store dummy:// -f ${nixpkgs-regression} -qaP --drv-path | sort | grep -v nixos-install-tools > packages
[[ $(sha1sum < packages | cut -c1-40) = e01b031fc9785a572a38be6bc473957e3b6faad7 ]]
)
mkdir $out
'';
nixpkgsLibTests =
forAllSystems (system:
import (nixpkgs + "/lib/tests/release.nix")
{
pkgs = nixpkgsFor.${system}.native;
nixVersions = [ self.packages.${system}.nix ];
}
);
};
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native;
nixpkgs = nixpkgs-regression;
};
installTests = forAllSystems (system:
let pkgs = nixpkgsFor.${system}.native; in
pkgs.runCommand "install-tests"
{
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentUnstable =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux"
then testNixVersions pkgs pkgs.nix pkgs.nixUnstable
else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
installerTests = import ../tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball;
inherit nixpkgsFor;
};
}
+21 -24
View File
@@ -93,9 +93,10 @@ AC_LANG_POP(C++)
AC_CHECK_FUNCS([statvfs pipe2])
# Check for lutimes, optionally used for changing the mtime of
# symlinks.
AC_CHECK_FUNCS([lutimes])
# Check for lutimes and utimensat, optionally used for changing the
# mtime of symlinks.
AC_CHECK_DECLS([AT_SYMLINK_NOFOLLOW], [], [], [[#include <fcntl.h>]])
AC_CHECK_FUNCS([lutimes utimensat])
# Check whether the store optimiser can optimise symlinks.
@@ -153,11 +154,6 @@ AC_ARG_ENABLE(unit-tests, AS_HELP_STRING([--disable-unit-tests],[Do not build th
ENABLE_UNIT_TESTS=$enableval, ENABLE_UNIT_TESTS=$ENABLE_BUILD)
AC_SUBST(ENABLE_UNIT_TESTS)
# Build external API docs by default
AC_ARG_ENABLE(external_api_docs, AS_HELP_STRING([--enable-external-api-docs],[Build API docs for Nix's C interface]),
external_api_docs=$enableval, external_api_docs=yes)
AC_SUBST(external_api_docs)
AS_IF(
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_UNIT_TESTS" == "yes"],
[AC_MSG_ERROR([Cannot enable unit tests when building overall is disabled. Please do not pass '--enable-unit-tests' or do not pass '--disable-build'.])])
@@ -175,19 +171,14 @@ AS_IF(
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_DOC_GEN" == "yes"],
[AC_MSG_ERROR([Cannot enable generated docs when building overall is disabled. Please do not pass '--enable-doc-gen' or do not pass '--disable-build'.])])
# Building without API docs is the default as Nix' C++ interfaces are internal and unstable.
AC_ARG_ENABLE(internal-api-docs, AS_HELP_STRING([--enable-internal-api-docs],[Build API docs for Nix's internal unstable C++ interfaces]),
ENABLE_INTERNAL_API_DOCS=$enableval, ENABLE_INTERNAL_API_DOCS=no)
AC_SUBST(ENABLE_INTERNAL_API_DOCS)
AC_ARG_ENABLE(external-api-docs, AS_HELP_STRING([--enable-external-api-docs],[Build API docs for Nix's external unstable C interfaces]),
ENABLE_EXTERNAL_API_DOCS=$enableval, ENABLE_EXTERNAL_API_DOCS=no)
AC_SUBST(ENABLE_EXTERNAL_API_DOCS)
AS_IF(
[test "$ENABLE_FUNCTIONAL_TESTS" == "yes" || test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(jq, jq)])
AS_IF(
[test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(man, man)])
AS_IF([test "$ENABLE_BUILD" == "yes"],[
# Look for boost, a required dependency.
@@ -358,13 +349,6 @@ AC_CHECK_HEADERS([aws/s3/S3Client.h],
AC_SUBST(ENABLE_S3, [$enable_s3])
AC_LANG_POP(C++)
if test -n "$enable_s3"; then
declare -a aws_version_tokens=($(printf '#include <aws/core/VersionConfig.h>\nAWS_SDK_VERSION_STRING' | $CPP $CPPFLAGS - | grep -v '^#.*' | sed 's/"//g' | tr '.' ' '))
AC_DEFINE_UNQUOTED([AWS_VERSION_MAJOR], ${aws_version_tokens@<:@0@:>@}, [Major version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_MINOR], ${aws_version_tokens@<:@1@:>@}, [Minor version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_PATCH], ${aws_version_tokens@<:@2@:>@}, [Patch version of aws-sdk-cpp.])
fi
# Whether to use the Boehm garbage collector.
AC_ARG_ENABLE(gc, AS_HELP_STRING([--enable-gc],[enable garbage collection in the Nix expression evaluator (requires Boehm GC) [default=yes]]),
@@ -373,6 +357,14 @@ if test "$gc" = yes; then
PKG_CHECK_MODULES([BDW_GC], [bdw-gc])
CXXFLAGS="$BDW_GC_CFLAGS $CXXFLAGS"
AC_DEFINE(HAVE_BOEHMGC, 1, [Whether to use the Boehm garbage collector.])
# See `fixupBoehmStackPointer`, for the integration between Boehm GC
# and Boost coroutines.
old_CFLAGS="$CFLAGS"
# Temporary set `-pthread` just for the next check
CFLAGS="$CFLAGS -pthread"
AC_CHECK_FUNCS([pthread_attr_get_np pthread_getattr_np])
CFLAGS="$old_CFLAGS"
fi
AS_IF([test "$ENABLE_UNIT_TESTS" == "yes"],[
@@ -410,6 +402,11 @@ AS_CASE(["$enable_markdown"],
PKG_CHECK_MODULES([LIBGIT2], [libgit2])
# Look for toml11, a required dependency.
AC_LANG_PUSH(C++)
AC_CHECK_HEADER([toml.hpp], [], [AC_MSG_ERROR([toml11 is not found.])])
AC_LANG_POP(C++)
# Setuid installations.
AC_CHECK_FUNCS([setresuid setreuid lchown])
+9 -10
View File
@@ -1,10 +1,9 @@
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix
@@ -1,99 +0,0 @@
diff --git a/darwin_stop_world.c b/darwin_stop_world.c
index 0468aaec..b348d869 100644
--- a/darwin_stop_world.c
+++ b/darwin_stop_world.c
@@ -356,6 +356,7 @@ GC_INNER void GC_push_all_stacks(void)
int nthreads = 0;
word total_size = 0;
mach_msg_type_number_t listcount = (mach_msg_type_number_t)THREAD_TABLE_SZ;
+ size_t stack_limit;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -411,6 +412,19 @@ GC_INNER void GC_push_all_stacks(void)
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
}
if (altstack_lo) {
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ stack_limit = pthread_get_stacksize_np(p->id);
+ if (altstack_lo >= altstack_hi || altstack_lo < altstack_hi - stack_limit) { // sp outside stack
+ altstack_lo = altstack_hi - stack_limit;
+ }
+
total_size += altstack_hi - altstack_lo;
GC_push_all_stack(altstack_lo, altstack_hi);
}
diff --git a/include/gc.h b/include/gc.h
index edab6c22..f2c61282 100644
--- a/include/gc.h
+++ b/include/gc.h
@@ -2172,6 +2172,11 @@ GC_API void GC_CALL GC_win32_free_heap(void);
(*GC_amiga_allocwrapper_do)(a,GC_malloc_atomic_ignore_off_page)
#endif /* _AMIGA && !GC_AMIGA_MAKINGLIB */
+#if !__APPLE__
+/* Patch doesn't work on apple */
+#define NIX_BOEHM_PATCH_VERSION 1
+#endif
+
#ifdef __cplusplus
} /* extern "C" */
#endif
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
index b5d71e62..aed7b0bf 100644
--- a/pthread_stop_world.c
+++ b/pthread_stop_world.c
@@ -768,6 +768,8 @@ STATIC void GC_restart_handler(int sig)
/* world is stopped. Should not fail if it isn't. */
GC_INNER void GC_push_all_stacks(void)
{
+ size_t stack_limit;
+ pthread_attr_t pattr;
GC_bool found_me = FALSE;
size_t nthreads = 0;
int i;
@@ -851,6 +853,37 @@ GC_INNER void GC_push_all_stacks(void)
hi = p->altstack + p->altstack_size;
/* FIXME: Need to scan the normal stack too, but how ? */
/* FIXME: Assume stack grows down */
+ } else {
+#ifdef HAVE_PTHREAD_ATTR_GET_NP
+ if (!pthread_attr_init(&pattr)
+ || !pthread_attr_get_np(p->id, &pattr))
+#else /* HAVE_PTHREAD_GETATTR_NP */
+ if (pthread_getattr_np(p->id, &pattr))
+#endif
+ {
+ ABORT("GC_push_all_stacks: pthread_getattr_np failed!");
+ }
+ if (pthread_attr_getstacksize(&pattr, &stack_limit)) {
+ ABORT("GC_push_all_stacks: pthread_attr_getstacksize failed!");
+ }
+ if (pthread_attr_destroy(&pattr)) {
+ ABORT("GC_push_all_stacks: pthread_attr_destroy failed!");
+ }
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ #ifndef STACK_GROWS_UP
+ if (lo >= hi || lo < hi - stack_limit) { // sp outside stack
+ lo = hi - stack_limit;
+ }
+ #else
+ #error "STACK_GROWS_UP not supported in boost_coroutine2 (as of june 2021), so we don't support it in Nix."
+ #endif
}
GC_push_all_stack_sections(lo, hi, traced_stack_sect);
# ifdef STACK_GROWS_UP
@@ -1,12 +0,0 @@
diff --git a/include/gc_allocator.h b/include/gc_allocator.h
index 597c7f13..587286be 100644
--- a/include/gc_allocator.h
+++ b/include/gc_allocator.h
@@ -312,6 +312,7 @@ public:
template<>
class traceable_allocator<void> {
+public:
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef void* pointer;
-7
View File
@@ -1,7 +0,0 @@
$(docdir)/external-api/html/index.html $(docdir)/external-api/latex: $(d)/doxygen.cfg src/lib*-c/*.h
mkdir -p $(docdir)/external-api
{ cat $< ; echo "OUTPUT_DIRECTORY=$(docdir)/external-api" ; } | doxygen -
# Generate the HTML API docs for Nix's unstable C bindings
.PHONY: external-api-html
external-api-html: $(docdir)/external-api/html/index.html
-7
View File
@@ -1,7 +0,0 @@
$(docdir)/internal-api/html/index.html $(docdir)/internal-api/latex: $(d)/doxygen.cfg src/**/*.hh
mkdir -p $(docdir)/internal-api
{ cat $< ; echo "OUTPUT_DIRECTORY=$(docdir)/internal-api" ; } | doxygen -
# Generate the HTML API docs for Nix's unstable internal interfaces.
.PHONY: internal-api-html
internal-api-html: $(docdir)/internal-api/html/index.html
-31
View File
@@ -1,31 +0,0 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import <nix/utils.nix>) optionalString squash;
in
builtinsInfo:
let
showBuiltin = name: { doc, type, impure-only }:
let
type' = optionalString (type != null) " (${type})";
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name}</code></a>${type'}
</dt>
<dd>
${doc}
${impureNotice}
</dd>
'';
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+22 -4
View File
@@ -5,12 +5,22 @@ in
builtinsInfo:
let
showBuiltin = name: { doc, args, arity, experimental-feature }:
showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
let
type' = optionalString (type != null) " (${type})";
experimentalNotice = optionalString (experimental-feature != null) ''
> **Note**
>
> This function is only available if the [`${experimental-feature}` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) is enabled.
> This function is only available if the [`${experimental-feature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimental-feature}) is enabled.
>
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
@@ -18,18 +28,26 @@ let
> extra-experimental-features = ${experimental-feature}
> ```
'';
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name} ${listArgs args}</code></a>
<a href="#builtins-${name}"><code>${name}${listArgs args}</code></a>${type'}
</dt>
<dd>
${experimentalNotice}
${doc}
${impureNotice}
</dd>
'';
listArgs = args: concatStringsSep " " (map (s: "<var>${s}</var>") args);
listArgs = args: concatStringsSep "" (map (s: " <var>${s}</var>") args);
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+93 -62
View File
@@ -32,13 +32,19 @@ let
commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }:
showCommand =
{
command,
details,
filename,
toplevel,
}:
let
result = ''
> **Warning** \
> This program is
> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
> [**experimental**](@docroot@/development/experimental-features.md#xp-feature-nix-command)
> and its interface is subject to change.
# Name
@@ -56,26 +62,27 @@ let
${maybeOptions}
'';
showSynopsis = command: args:
showSynopsis =
command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in ''
in
''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories);
@@ -99,35 +106,39 @@ let
${allStores}
'';
index = replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
index =
replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview =
let
showEntry = store:
"- [${store.name}](#${store.slug})";
showEntry = store: "- [${store.name}](#${store.slug})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storesList = showStoreDocs {
storeInfo = commandInfo.stores;
inherit inlineHTML;
};
hasInfix =
infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in
optionalString (details ? doc) (
if match ".*@store-types@.*" details.doc != null
then help-stores
else details.doc
# An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc then help-stores else details.doc
);
maybeOptions =
let
allVisibleOptions = filterAttrs
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
in
optionalString (allVisibleOptions != { }) ''
# Options
@@ -139,55 +150,73 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions = inlineHTML: allOptions:
showOptions =
inlineHTML: allOptions:
let
showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
'';
showOption = name: option:
showOption =
name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in result;
categories = mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy
(cmd: cmd.value.category)
(attrsToList allOptions));
in concatStrings (attrValues (mapAttrs showCategory categories));
in squash result;
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories =
mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy (cmd: cmd.value.category) (attrsToList allOptions));
in
concatStrings (attrValues (mapAttrs showCategory categories));
in
squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }:
processCommand =
{
command,
details,
filename,
toplevel,
}:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename toplevel; };
value = showCommand {
inherit
command
details
filename
toplevel
;
};
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand {
command = "nix";
@@ -196,9 +225,11 @@ let
toplevel = commandInfo.args;
};
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
tableOfContents =
let
showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in
concatStringsSep "\n" (map showEntry manpages) + "\n";
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+74 -42
View File
@@ -1,67 +1,99 @@
let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs;
inherit (import <nix/utils.nix>) concatStrings indent optionalString squash;
inherit (builtins)
attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo:
{
prefix,
inlineHTML ? true,
}:
settingsInfo:
let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
let
result = squash ''
- ${item}
- ${item}
${indent " " body}
'';
item = if inlineHTML
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else "`${setting}`";
${indent " " body}
'';
item =
if inlineHTML then
''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
# separate body to cleanly handle indentation
body = ''
${experimentalFeatureNote}
${experimentalFeatureNote}
${description}
${description}
**Default:** ${showDefault documentDefault defaultValue}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
${showAliases aliases}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
showDefault = documentDefault: defaultValue:
showDefault =
documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
"*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else
"`${toString defaultValue}`"
else
"*machine-specific*";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
showAliases =
aliases:
optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result;
in
result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+39 -16
View File
@@ -1,6 +1,20 @@
let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
inherit (import <nix/utils.nix>) optionalString filterAttrs trim squash toLower unique indent;
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
showSettings = import <nix/generate-settings.nix>;
in
@@ -14,7 +28,13 @@ in
let
showStore = { name, slug }: { settings, doc, experimentalFeature }:
showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
let
result = squash ''
# ${name}
@@ -25,17 +45,20 @@ let
## Settings
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings}
${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
>
> This store is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To use this store, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature})
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
@@ -43,15 +66,15 @@ let
> extra-experimental-features = ${experimentalFeature}
> ```
'';
in result;
in
result;
storesList = map
(name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
})
(attrNames storeInfo);
storesList = map (name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
}) (attrNames storeInfo);
in storesList
in
storesList
+16 -10
View File
@@ -1,5 +1,11 @@
let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>;
in
@@ -14,26 +20,26 @@ let
index =
let
showEntry = store:
"- [${store.name}](./${store.filename})";
showEntry = store: "- [${store.name}](./${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings
[ "@store-types@" ] [ index ]
(readFile ./src/store/types/index.md.in);
"index.md" = replaceStrings [ "@store-types@" ] [ index ] (readFile ./src/store/types/index.md.in);
tableOfContents =
let
showEntry = store:
" - [${store.name}](store/types/${store.filename})";
showEntry = store: " - [${store.name}](store/types/${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents;
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
in
storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -2,8 +2,8 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
''
- [`${name}`](@docroot@/contributing/experimental-features.md#xp-feature-${name})
'';
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+2 -1
View File
@@ -2,7 +2,8 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
showExperimentalFeature =
name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
+5 -11
View File
@@ -95,7 +95,7 @@ $(d)/nix-profiles.5: $(d)/src/command-ref/files/profiles.md
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=5 $^.tmp -o $@
@rm $^.tmp
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/development/experimental-feature-descriptions.md
@cp $< $@
@$(call process-includes,$@,$@)
@@ -124,7 +124,7 @@ $(d)/conf-file.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) config show --json --experimental-features nix-command > $@.tmp
@mv $@.tmp $@
$(d)/src/contributing/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(doc_nix)
$(d)/src/development/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(doc_nix)
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-xp-features.nix (builtins.fromJSON (builtins.readFile $<))'
@mv $@.tmp $@
@@ -140,16 +140,10 @@ $(d)/xp-features.json: $(doc_nix)
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(doc_nix)
@cat doc/manual/src/language/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<)).builtins' >> $@.tmp;
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@cat doc/manual/src/language/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtin-constants.md: $(d)/language.json $(d)/generate-builtin-constants.nix $(d)/src/language/builtin-constants-prefix.md $(doc_nix)
@cat doc/manual/src/language/builtin-constants-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtin-constants.nix (builtins.fromJSON (builtins.readFile $<)).constants' >> $@.tmp;
@cat doc/manual/src/language/builtin-constants-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/language.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) __dump-language > $@.tmp
@mv $@.tmp $@
@@ -213,11 +207,11 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
done
@touch $@
# the `! -name 'contributing.md'` filter excludes the one place where
# the `! -name 'documentation.md'` filter excludes the one place where
# `@docroot@` is to be preserved for documenting the mechanism
# FIXME: maybe contributing guides should live right next to the code
# instead of in the manual
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md $(d)/src/release-notes/rl-next.md $(d)/src/figures $(d)/src/favicon.png $(d)/src/favicon.svg
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/development/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/release-notes/rl-next.md $(d)/src/figures $(d)/src/favicon.png $(d)/src/favicon.svg
$(trace-gen) \
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
+18 -9
View File
@@ -1,7 +1,7 @@
// redirect rules for URL fragments (client-side) to prevent link rot.
// this must be done on the client side, as web servers do not see the fragment part of the URL.
// it will only work with JavaScript enabled in the browser, but this is the best we can do here.
// see ./_redirects for path redirects (client-side)
// see src/_redirects for path redirects (server-side)
// redirects are declared as follows:
// each entry has as its key a path matching the requested URL path, relative to the mdBook document root.
@@ -143,7 +143,7 @@ const redirects = {
"opt-timeout": "command-ref/opt-common.html#opt-timeout",
"sec-common-options": "command-ref/opt-common.html",
"ch-utilities": "command-ref/utilities.html",
"chap-hacking": "contributing/hacking.html",
"chap-hacking": "development/building.html",
"adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
@@ -238,12 +238,12 @@ const redirects = {
"attr-system": "language/derivations.html#attr-system",
"ssec-derivation": "language/derivations.html",
"ch-expression-language": "language/index.html",
"sec-constructs": "language/constructs.html",
"sect-let-language": "language/constructs.html#let-language",
"ss-functions": "language/constructs.html#functions",
"sec-constructs": "language/syntax.html",
"sect-let-language": "language/syntax.html#let-expressions",
"ss-functions": "language/syntax.html#functions",
"sec-language-operators": "language/operators.html",
"table-operators": "language/operators.html",
"ssec-values": "language/values.html",
"ssec-values": "language/types.html",
"gloss-closure": "glossary.html#gloss-closure",
"gloss-derivation": "glossary.html#gloss-derivation",
"gloss-deriver": "glossary.html#gloss-deriver",
@@ -335,18 +335,22 @@ const redirects = {
"ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html",
},
"language/values.html": {
"language/types.html": {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"attribute-sets": "#attribute-set",
"type-number": "#type-int",
},
"language/syntax.html": {
"scoping-rules": "scoping.html",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
},
"contributing/hacking.html": {
"development/building.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
"running-tests": "testing.html#running-tests",
@@ -357,7 +361,12 @@ const redirects = {
"installer-tests": "testing.html#installer-tests",
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
"characterization-testing": "#characterisation-testing-unit",
"characterization-testing": "testing.html#characterisation-testing-unit",
"add-a-release-note": "contributing.html#add-a-release-note",
"add-an-entry": "contributing.html#add-an-entry",
"build-process": "contributing.html#build-process",
"reverting": "contributing.html#reverting",
"branches": "contributing.html#branches",
},
"glossary.html": {
"gloss-local-store": "store/types/local-store.html",
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
@@ -1,6 +0,0 @@
---
synopsis: Harden the user sandboxing
significance: significant
---
The build directory has been hardened against interference with the outside world by nesting it inside another directory owned by (and only readable by) the daemon user.
-8
View File
@@ -1,8 +0,0 @@
---
synopsis: "`<nix/fetchurl.nix>` uses TLS verification"
prs: [11585]
---
Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
@@ -0,0 +1,205 @@
# String literals
A *string literal* represents a [string](types.md#type-string) value.
> **Syntax**
>
> *expression* → *string*
>
> *string* → `"` ( *string_char*\* [*interpolation_element*][string interpolation] )* *string_char*\* `"`
>
> *string* → `''` ( *indented_string_char*\* [*interpolation_element*][string interpolation] )* *indented_string_char*\* `''`
>
> *string* → *uri*
>
> *string_char* ~ `[^"$\\]|\$(?!\{)|\\.`
>
> *indented_string_char* ~ `[^$']|\$\$|\$(?!\{)|''[$']|''\\.|'(?!')`
>
> *uri* ~ `[A-Za-z][+\-.0-9A-Za-z]*:[!$%&'*+,\-./0-9:=?@A-Z_a-z~]+`
Strings can be written in three ways.
The most common way is to enclose the string between double quotes, e.g., `"foo bar"`.
Strings can span multiple lines.
The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation].
[string interpolation]: ./string-interpolation.md
The following must be escaped to represent them within a string, by prefixing with a backslash (`\`):
- Double quote (`"`)
> **Example**
>
> ```nix
> "\""
> ```
>
> "\""
- Backslash (`\`)
> **Example**
>
> ```nix
> "\\"
> ```
>
> "\\"
- Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly"
> **Example**
>
> ```nix
> "\${"
> ```
>
> "\${"
The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> "$${"
> ```
>
> "$\${"
String values are output on the terminal with Nix-specific escaping.
Strings written to files will contain the characters encoded by the escaping.
The second way to write string literals is as an *indented string*, which is enclosed between pairs of *double single-quotes* (`''`), like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
> **Note**
>
> Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line.
> **Warning**
>
> Prefixed tab characters are not stripped.
>
> > **Example**
> >
> > The following indented string is prefixed with tabs:
> >
> > <pre><code class="nohighlight">''
> > all:
> > @echo hello
> > ''
> > </code></pre>
> >
> > "\tall:\n\t\t@echo hello\n"
Indented strings support [string interpolation].
The following must be escaped to represent them in an indented string:
- `$` is escaped by prefixing it with two single quotes (`''`)
> **Example**
>
> ```nix
> ''
> ''$
> ''
> ```
>
> "$\n"
- `''` is escaped by prefixing it with one single quote (`'`)
> **Example**
>
> ```nix
> ''
> '''
> ''
> ```
>
> "''\n"
These special characters are escaped as follows:
- Linefeed (`\n`): `''\n`
- Carriage return (`\r`): `''\r`
- Tab (`\t`): `''\t`
`''\` escapes any other character.
A "dollar-curly" (`${`) can be written as follows:
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
> **Note**
>
> This differs from the syntax for escaping a dollar-curly within double quotes (`"\${"`). Be aware of which one is needed at a given moment.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> ''
> $${
> ''
> ```
>
> "$\${\n"
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
+19 -16
View File
@@ -25,17 +25,18 @@
- [Store Types](store/types/index.md)
{{#include ./store/types/SUMMARY.md}}
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [Data Types](language/types.md)
- [String context](language/string-context.md)
- [Syntax and semantics](language/syntax.md)
- [Identifiers](language/identifiers.md)
- [Scoping rules](language/scope.md)
- [String interpolation](language/string-interpolation.md)
- [Lookup path](language/constructs/lookup-path.md)
- [String context](language/string-context.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Built-ins](language/builtins.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Package Management](package-management/index.md)
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
@@ -115,16 +116,18 @@
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
- [C API](c-api.md)
- [Glossary](glossary.md)
- [Contributing](contributing/index.md)
- [Hacking](contributing/hacking.md)
- [Testing](contributing/testing.md)
- [Documentation](contributing/documentation.md)
- [Experimental Features](contributing/experimental-features.md)
- [CLI guideline](contributing/cli-guideline.md)
- [JSON guideline](contributing/json-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Development](development/index.md)
- [Building](development/building.md)
- [Testing](development/testing.md)
- [Documentation](development/documentation.md)
- [CLI guideline](development/cli-guideline.md)
- [JSON guideline](development/json-guideline.md)
- [C++ style guide](development/cxx.md)
- [Experimental Features](development/experimental-features.md)
- [Contributing](development/contributing.md)
- [Releases](release-notes/index.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.24 (2024-07-31)](release-notes/rl-2.24.md)
- [Release 2.23 (2024-06-03)](release-notes/rl-2.23.md)
- [Release 2.22 (2024-04-23)](release-notes/rl-2.22.md)
- [Release 2.21 (2024-03-11)](release-notes/rl-2.21.md)
+13 -2
View File
@@ -1,5 +1,5 @@
# redirect rules for paths (server-side) to prevent link rot.
# see ./redirects.js for redirects based on URL fragments (client-side)
# see ../redirects.js for redirects based on URL fragments (client-side)
#
# concrete user story this supports:
# - user finds URL to the manual for Nix x.y
@@ -20,13 +20,24 @@
/command-ref/command-ref /command-ref 301!
/contributing/contributing /contributing 301!
/contributing/contributing /development 301!
/contributing /development 301!
/contributing/hacking /development/building 301!
/contributing/testing /development/testing 301!
/contributing/documentation /development/documentation 301!
/contributing/experimental-features /development/experimental-features 301!
/contributing/cli-guideline /development/cli-guideline 301!
/contributing/json-guideline /development/json-guideline 301!
/contributing/cxx /development/cxx 301!
/expressions/expression-language /language/ 301!
/expressions/language-constructs /language/constructs 301!
/expressions/language-operators /language/operators 301!
/expressions/language-values /language/values 301!
/expressions/* /language/:splat 301!
/language/values /language/types 301!
/language/constructs /language/syntax 301!
/language/builtin-constants /language/builtins 301!
/installation/installation /installation 301!
+1 -1
View File
@@ -10,7 +10,7 @@ See:
- [Matrix Room *Nix Bindings*](https://matrix.to/#/#nix-bindings:nixos.org) for discussion and questions.
- [Stabilisation Milestone](https://github.com/NixOS/nix/milestone/52)
- [Other C API PRs and issues](https://github.com/NixOS/nix/labels/c%20api)
- [Contributing C API Documentation](contributing/documentation.md#c-api-documentation), including how to build it locally.
- [Contributing C API Documentation](development/documentation.md#c-api-documentation), including how to build it locally.
[Getting Started]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs
[Index]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs/globals.html
+15 -11
View File
@@ -9,22 +9,26 @@ Most Nix commands interpret the following environment variables:
- <span id="env-NIX_PATH">[`NIX_PATH`](#env-NIX_PATH)</span>
A colon-separated list of directories used to look up the location of Nix
expressions using [paths](@docroot@/language/values.md#type-path)
enclosed in angle brackets (i.e., `<path>`),
e.g. `/home/eelco/Dev:/etc/nixos`. It can be extended using the
[`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
A colon-separated list of search path entries used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
This environment variable overrides the value of the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path).
1. `$HOME/.nix-defexpr/channels`
2. `nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
3. `/nix/var/nix/profiles/per-user/root/channels`
It can be extended using the [`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
> **Example**
>
> ```bash
> $ export NIX_PATH=`/home/eelco/Dev:nixos-config=/etc/nixos
> ```
If `NIX_PATH` is set to an empty string, resolving search paths will always fail.
For example, attempting to use `<nixpkgs>` will produce:
error: file 'nixpkgs' was not found in the Nix search path
> **Example**
>
> ```bash
> $ NIX_PATH= nix-instantiate --eval '<nixpkgs>'
> error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
> ```
- <span id="env-NIX_IGNORE_SYMLINK_STORE">[`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE)</span>
@@ -1,6 +1,6 @@
# Experimental Commands
This section lists [experimental commands](@docroot@/contributing/experimental-features.md#xp-feature-nix-command).
This section lists [experimental commands](@docroot@/development/experimental-features.md#xp-feature-nix-command).
> **Warning**
>
+9 -9
View File
@@ -55,20 +55,20 @@ All options not listed here are passed to
[`nix-store --realise`](nix-store/realise.md),
except for `--arg` and `--attr` / `-A` which are passed to [`nix-instantiate`](nix-instantiate.md).
- <span id="opt-no-out-link">[`--no-out-link`](#opt-no-out-link)<span>
- <span id="opt-no-out-link">[`--no-out-link`](#opt-no-out-link)<span>
Do not create a symlink to the output path. Note that as a result
the output does not become a root of the garbage collector, and so
might be deleted by `nix-store --gc`.
Do not create a symlink to the output path. Note that as a result
the output does not become a root of the garbage collector, and so
might be deleted by `nix-store --gc`.
- <span id="opt-dry-run">[`--dry-run`](#opt-dry-run)</span>
- <span id="opt-dry-run">[`--dry-run`](#opt-dry-run)</span>
Show what store paths would be built or downloaded.
Show what store paths would be built or downloaded.
- <span id="opt-out-link">[`--out-link`](#opt-out-link)</span> / `-o` *outlink*
- <span id="opt-out-link">[`--out-link`](#opt-out-link)</span> / `-o` *outlink*
Change the name of the symlink to the output path created from
`result` to *outlink*.
Change the name of the symlink to the output path created from
`result` to *outlink*.
{{#include ./status-build-failure.md}}
+32 -26
View File
@@ -27,40 +27,46 @@ The moving parts of channels are:
This command has the following operations:
- `--add` *url* \[*name*\]\
Add a channel *name* located at *url* to the list of subscribed channels.
If *name* is omitted, default to the last component of *url*, with the suffixes `-stable` or `-unstable` removed.
- `--add` *url* \[*name*\]
> **Note**
>
> `--add` does not automatically perform an update.
> Use `--update` explicitly.
Add a channel *name* located at *url* to the list of subscribed channels.
If *name* is omitted, default to the last component of *url*, with the suffixes `-stable` or `-unstable` removed.
A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
> **Note**
>
> `--add` does not automatically perform an update.
> Use `--update` explicitly.
- `--remove` *name*\
Remove the channel *name* from the list of subscribed channels.
A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
- `--list`\
Print the names and URLs of all subscribed channels on standard output.
- `--remove` *name*
- `--update` \[*names*…\]\
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
Remove the channel *name* from the list of subscribed channels.
- `--list-generations`\
Prints a list of all the current existing generations for the
channel profile.
- `--list`
Works the same way as
```
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
```
Print the names and URLs of all subscribed channels on standard output.
- `--rollback` \[*generation*\]\
Revert channels to the state before the last call to `nix-channel --update`.
Optionally, you can specify a specific channel *generation* number to restore.
- `--update` \[*names*…\]
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
- `--list-generations`
Prints a list of all the current existing generations for the
channel profile.
Works the same way as
```
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
```
- `--rollback` \[*generation*\]
Revert channels to the state before the last call to `nix-channel --update`.
Optionally, you can specify a specific channel *generation* number to restore.
{{#include ./opt-common.md}}
@@ -48,18 +48,29 @@ Instead, it looks in a few locations, and acts on all profiles it finds there:
These options are for deleting old [profiles] prior to deleting unreachable [store objects].
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`\
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`
Delete all old generations of profiles.
This is the equivalent of invoking [`nix-env --delete-generations old`](@docroot@/command-ref/nix-env/delete-generations.md#generations-old) on each found profile.
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*\
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*
Delete all generations of profiles older than the specified amount (except for the generations that were active at that point in time).
*period* is a value such as `30d`, which would mean 30 days.
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
- <span id="opt-max-freed">[`--max-freed`](#opt-max-freed)</span> *bytes*
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
{{#include ./opt-common.md}}
{{#include ./env-common.md}}
+16 -16
View File
@@ -27,38 +27,38 @@ When using public key authentication, you can avoid typing the passphrase with `
# Options
- `--to`
- `--to`
Copy the closure of _paths_ from a Nix store accessible from the local machine to the Nix store on the remote _machine_.
This is the default behavior.
Copy the closure of _paths_ from a Nix store accessible from the local machine to the Nix store on the remote _machine_.
This is the default behavior.
- `--from`
- `--from`
Copy the closure of _paths_ from the Nix store on the remote _machine_ to the local machine's specified Nix store.
Copy the closure of _paths_ from the Nix store on the remote _machine_ to the local machine's specified Nix store.
- `--gzip`
- `--gzip`
Enable compression of the SSH connection.
Enable compression of the SSH connection.
- `--include-outputs`
- `--include-outputs`
Also copy the outputs of [store derivation]s included in the closure.
Also copy the outputs of [store derivation]s included in the closure.
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
- `--use-substitutes` / `-s`
- `--use-substitutes` / `-s`
Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters).
Any store objects that cannot be substituted on the target are still copied normally from the source.
This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast.
Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters).
Any store objects that cannot be substituted on the target are still copied normally from the source.
This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast.
{{#include ./opt-common.md}}
# Environment variables
- `NIX_SSHOPTS`
- `NIX_SSHOPTS`
Additional options to be passed to `ssh` on the command line.
Additional options to be passed to `ssh` on the command line.
{{#include ./env-common.md}}
+1 -1
View File
@@ -62,7 +62,7 @@ These pages can be viewed offline:
Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of *arguments* that specify the packages on which to operate.
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-names):
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name):
- `name`: Everything up to but not including the first dash (`-`) that is *not* followed by a letter.
- `version`: The rest, excluding the separating dash.
@@ -12,7 +12,8 @@ This operation deletes the specified generations of the current profile.
*generations* can be a one of the following:
- <span id="generations-list">[`<number>...`](#generations-list)</span>:\
- <span id="generations-list">[`<number>...`](#generations-list)</span>
A list of generation numbers, each one a separate command-line argument.
Delete exactly the profile generations given by their generation number.
@@ -30,7 +31,8 @@ This operation deletes the specified generations of the current profile.
> Because one can roll back to a previous generation, it is possible to have generations newer than the current one.
> They will also be deleted.
- <span id="generations-time">[`<number>d`](#generations-time)</span>:\
- <span id="generations-time">[`<number>d`](#generations-time)</span>
The last *number* days
*Example*: `30d`
@@ -38,7 +40,8 @@ This operation deletes the specified generations of the current profile.
Delete all generations created more than *number* days ago, except the most recent one of them.
This allows rolling back to generations that were available within the specified period.
- <span id="generations-count">[`+<number>`](#generations-count)</span>:\
- <span id="generations-count">[`+<number>`](#generations-count)</span>
The last *number* generations up to the present
*Example*: `+5`
@@ -1,6 +1,7 @@
# Environment variables
- `NIX_PROFILE`\
- `NIX_PROFILE`
Location of the Nix profile. Defaults to the target of the symlink
`~/.nix-profile`, if it exists, or `/nix/var/nix/profiles/default`
otherwise.
+92 -92
View File
@@ -21,125 +21,125 @@ It is based on the current generation of the active [profile](@docroot@/command-
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
[derivation]: @docroot@/glossary.md#gloss-derivation
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise
[derivation]: @docroot@/glossary.md#gloss-derivation
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
```console
nix-env --query --available --attr-path
```
```console
nix-env --query --available --attr-path
```
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/constructs.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/syntax.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Example:
Example:
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
Installing from this Nix expression will make files from both outputs appear in the current profile.
Installing from this Nix expression will make files from both outputs appear in the current profile.
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
```nix
# example-outputs.nix
import ./example.nix // { meta.outputsToInstall = [ "bar" ]; }
```
```nix
# example-outputs.nix
import ./example.nix // { meta.outputsToInstall = [ "bar" ]; }
```
```console
$ nix-env --install --file example-outputs.nix
installing 'example'
$ ls ~/.nix-profile
bar-file
manifest.nix
```
```console
$ nix-env --install --file example-outputs.nix
installing 'example'
$ ls ~/.nix-profile
bar-file
manifest.nix
```
# Options
- `--prebuilt-only` / `-b`
- `--prebuilt-only` / `-b`
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--preserve-installed` / `-P`
- `--preserve-installed` / `-P`
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
- `--remove-all` / `-r`
- `--remove-all` / `-r`
Remove all previously installed packages first. This is equivalent
to running `nix-env --uninstall '.*'` first, except that everything happens
in a single transaction.
Remove all previously installed packages first. This is equivalent
to running `nix-env --uninstall '.*'` first, except that everything happens
in a single transaction.
{{#include ./opt-common.md}}
@@ -2,34 +2,37 @@
The following options are allowed for all `nix-env` operations, but may not always have an effect.
- `--file` / `-f` *path*\
Specifies the Nix expression (designated below as the *active Nix
expression*) used by the `--install`, `--upgrade`, and `--query
--available` operations to obtain derivations. The default is
`~/.nix-defexpr`.
- `--file` / `-f` *path*
If the argument starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must include a single
top-level directory containing at least a file named `default.nix`.
Specifies the Nix expression (designated below as the *active Nix
expression*) used by the `--install`, `--upgrade`, and `--query
--available` operations to obtain derivations. The default is
`~/.nix-defexpr`.
- `--profile` / `-p` *path*\
Specifies the profile to be used by those operations that operate on
a profile (designated below as the *active profile*). A profile is a
sequence of user environments called *generations*, one of which is
the *current generation*.
If the argument starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must include a single
top-level directory containing at least a file named `default.nix`.
- `--dry-run`\
For the `--install`, `--upgrade`, `--uninstall`,
`--switch-generation`, `--delete-generations` and `--rollback`
operations, this flag will cause `nix-env` to print what *would* be
done if this flag had not been specified, without actually doing it.
- `--profile` / `-p` *path*
`--dry-run` also prints out which paths will be
[substituted](@docroot@/glossary.md) (i.e., downloaded) and which paths
will be built from source (because no substitute is available).
Specifies the profile to be used by those operations that operate on
a profile (designated below as the *active profile*). A profile is a
sequence of user environments called *generations*, one of which is
the *current generation*.
- `--system-filter` *system*\
By default, operations such as `--query
--available` show derivations matching any platform. This option
allows you to use derivations for the specified platform *system*.
- `--dry-run`
For the `--install`, `--upgrade`, `--uninstall`,
`--switch-generation`, `--delete-generations` and `--rollback`
operations, this flag will cause `nix-env` to print what *would* be
done if this flag had not been specified, without actually doing it.
`--dry-run` also prints out which paths will be
[substituted](@docroot@/glossary.md) (i.e., downloaded) and which paths
will be built from source (because no substitute is available).
- `--system-filter` *system*
By default, operations such as `--query --available` show derivations matching any platform. This option
allows you to use derivations for the specified platform *system*.
+36 -18
View File
@@ -35,11 +35,13 @@ The derivations are sorted by their `name` attributes.
The following flags specify the set of things on which the query
operates.
- `--installed`\
- `--installed`
The query operates on the store paths that are installed in the
current generation of the active profile. This is the default.
- `--available`; `-a`\
- `--available` / `-a`
The query operates on the derivations that are available in the
active Nix expression.
@@ -50,24 +52,28 @@ selected derivations. Multiple flags may be specified, in which case the
information is shown in the order given here. Note that the name of the
derivation is shown unless `--no-name` is specified.
- `--xml`\
- `--xml`
Print the result in an XML representation suitable for automatic
processing by other tools. The root element is called `items`, which
contains a `item` element for each available or installed
derivation. The fields discussed below are all stored in attributes
of the `item` elements.
- `--json`\
- `--json`
Print the result in a JSON representation suitable for automatic
processing by other tools.
- `--prebuilt-only` / `-b`\
- `--prebuilt-only` / `-b`
Show only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, this shows all packages that
probably can be installed quickly.
- `--status`; `-s`\
- `--status` / `-s`
Print the *status* of the derivation. The status consists of three
characters. The first is `I` or `-`, indicating whether the
derivation is currently installed in the current generation of the
@@ -78,49 +84,61 @@ derivation is shown unless `--no-name` is specified.
derivation to be built. The third is `S` or `-`, indicating whether
a substitute is available for the derivation.
- `--attr-path`; `-P`\
- `--attr-path` / `-P`
Print the *attribute path* of the derivation, which can be used to
unambiguously select it using the `--attr` option available in
commands that install derivations like `nix-env --install`. This
option only works together with `--available`
- `--no-name`\
- `--no-name`
Suppress printing of the `name` attribute of each derivation.
- `--compare-versions` / `-c`\
- `--compare-versions` / `-c`
Compare installed versions to available versions, or vice versa (if
`--available` is given). This is useful for quickly seeing whether
upgrades for installed packages are available in a Nix expression. A
column is added with the following meaning:
- `<` *version*\
- `<` *version*
A newer version of the package is available or installed.
- `=` *version*\
- `=` *version*
At most the same version of the package is available or
installed.
- `>` *version*\
- `>` *version*
Only older versions of the package are available or installed.
- `- ?`\
- `- ?`
No version of the package is available or installed.
- `--system`\
- `--system`
Print the `system` attribute of the derivation.
- `--drv-path`\
- `--drv-path`
Print the path of the [store derivation](@docroot@/glossary.md#gloss-store-derivation).
- `--out-path`\
- `--out-path`
Print the output path of the derivation.
- `--description`\
- `--description`
Print a short (one-line) description of the derivation, if
available. The description is taken from the `meta.description`
attribute of the derivation.
- `--meta`\
- `--meta`
Print all of the meta-attributes of the derivation. This option is
only available with `--xml` or `--json`.
+16 -16
View File
@@ -13,24 +13,24 @@ to be modified. There are several attributes that can be usefully
modified, because they affect the behaviour of `nix-env` or the user
environment build script:
- `priority` can be changed to resolve filename clashes. The user
environment build script uses the `meta.priority` attribute of
derivations to resolve filename collisions between packages. Lower
priority values denote a higher priority. For instance, the GCC
wrapper package and the Binutils package in Nixpkgs both have a file
`bin/ld`, so previously if you tried to install both you would get a
collision. Now, on the other hand, the GCC wrapper declares a higher
priority than Binutils, so the former’s `bin/ld` is symlinked in the
user environment.
- `priority` can be changed to resolve filename clashes. The user
environment build script uses the `meta.priority` attribute of
derivations to resolve filename collisions between packages. Lower
priority values denote a higher priority. For instance, the GCC
wrapper package and the Binutils package in Nixpkgs both have a file
`bin/ld`, so previously if you tried to install both you would get a
collision. Now, on the other hand, the GCC wrapper declares a higher
priority than Binutils, so the former’s `bin/ld` is symlinked in the
user environment.
- `keep` can be set to `true` to prevent the package from being
upgraded or replaced. This is useful if you want to hang on to an
older version of a package.
- `keep` can be set to `true` to prevent the package from being
upgraded or replaced. This is useful if you want to hang on to an
older version of a package.
- `active` can be set to `false` to “disable” the package. That is, no
symlinks will be generated to the files of the package, but it
remains part of the profile (so it won’t be garbage-collected). It
can be set back to `true` to re-enable the package.
- `active` can be set to `false` to “disable” the package. That is, no
symlinks will be generated to the files of the package, but it
remains part of the profile (so it won’t be garbage-collected). It
can be set back to `true` to re-enable the package.
{{#include ./opt-common.md}}
+37 -31
View File
@@ -28,42 +28,48 @@ version is installed.
# Flags
- `--lt`\
Only upgrade a derivation to newer versions. This is the default.
- `--lt`
- `--leq`\
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same version. Version are not a unique
identification of a derivation, so there may be many derivations
that have the same version. This flag may be useful to force
“synchronisation” between the installed and available derivations.
Only upgrade a derivation to newer versions. This is the default.
- `--eq`\
*Only* “upgrade” to derivations that have the same version. This may
not seem very useful, but it actually is, e.g., when there is a new
release of Nixpkgs and you want to replace installed applications
with the same versions built against newer dependencies (to reduce
the number of dependencies floating around on your system).
- `--leq`
- `--always`\
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same or a lower version. I.e., derivations
may actually be downgraded depending on what is available in the
active Nix expression.
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same version. Version are not a unique
identification of a derivation, so there may be many derivations
that have the same version. This flag may be useful to force
“synchronisation” between the installed and available derivations.
- `--prebuilt-only` / `-b`\
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--eq`
- `--preserve-installed` / `-P`\
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
*Only* “upgrade” to derivations that have the same version. This may
not seem very useful, but it actually is, e.g., when there is a new
release of Nixpkgs and you want to replace installed applications
with the same versions built against newer dependencies (to reduce
the number of dependencies floating around on your system).
- `--always`
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same or a lower version. I.e., derivations
may actually be downgraded depending on what is available in the
active Nix expression.
- `--prebuilt-only` / `-b`
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--preserve-installed` / `-P`
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
{{#include ./opt-common.md}}
+49 -38
View File
@@ -29,54 +29,65 @@ md5sum`.
# Options
- `--flat`\
Print the cryptographic hash of the contents of each regular file *path*.
That is, instead of computing
the hash of the [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) of *path*,
just [directly hash]((@docroot@/store/file-system-object/content-address.md#serial-flat) *path* as is.
This requires *path* to resolve to a regular file rather than directory.
The result is identical to that produced by the GNU commands
`md5sum` and `sha1sum`.
- `--flat`
- `--base16`\
Print the hash in a hexadecimal representation (default).
Print the cryptographic hash of the contents of each regular file *path*.
That is, instead of computing
the hash of the [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) of *path*,
just [directly hash]((@docroot@/store/file-system-object/content-address.md#serial-flat) *path* as is.
This requires *path* to resolve to a regular file rather than directory.
The result is identical to that produced by the GNU commands
`md5sum` and `sha1sum`.
- `--base32`\
Print the hash in a base-32 representation rather than hexadecimal.
This base-32 representation is more compact and can be used in Nix
expressions (such as in calls to `fetchurl`).
- `--base16`
- `--base64`\
Similar to --base32, but print the hash in a base-64 representation,
which is more compact than the base-32 one.
Print the hash in a hexadecimal representation (default).
- `--sri`\
Print the hash in SRI format with base-64 encoding.
The type of hash algorithm will be prepended to the hash string,
followed by a hyphen (-) and the base-64 hash body.
- `--base32`
- `--truncate`\
Truncate hashes longer than 160 bits (such as SHA-256) to 160 bits.
Print the hash in a base-32 representation rather than hexadecimal.
This base-32 representation is more compact and can be used in Nix
expressions (such as in calls to `fetchurl`).
- `--type` *hashAlgo*\
Use the specified cryptographic hash algorithm, which can be one of
`md5`, `sha1`, `sha256`, and `sha512`.
- `--base64`
- `--to-base16`\
Don’t hash anything, but convert the base-32 hash representation
*hash* to hexadecimal.
Similar to --base32, but print the hash in a base-64 representation,
which is more compact than the base-32 one.
- `--to-base32`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-32.
- `--sri`
- `--to-base64`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-64.
Print the hash in SRI format with base-64 encoding.
The type of hash algorithm will be prepended to the hash string,
followed by a hyphen (-) and the base-64 hash body.
- `--to-sri`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to SRI.
- `--truncate`
Truncate hashes longer than 160 bits (such as SHA-256) to 160 bits.
- `--type` *hashAlgo*
Use the specified cryptographic hash algorithm, which can be one of
`md5`, `sha1`, `sha256`, and `sha512`.
- `--to-base16`
Don’t hash anything, but convert the base-32 hash representation
*hash* to hexadecimal.
- `--to-base32`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-32.
- `--to-base64`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-64.
- `--to-sri`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to SRI.
# Examples
+82 -74
View File
@@ -30,89 +30,97 @@ standard input.
# Options
- `--add-root` *path*\
See the [corresponding option](nix-store.md) in `nix-store`.
- `--add-root` *path*
- `--parse`\
Just parse the input files, and print their abstract syntax trees on
standard output as a Nix expression.
See the [corresponding option](nix-store.md) in `nix-store`.
- `--eval`\
Just parse and evaluate the input files, and print the resulting
values on standard output. No instantiation of store derivations
takes place.
- `--parse`
> **Warning**
>
> This option produces output which can be parsed as a Nix expression which
> will produce a different result than the input expression when evaluated.
> For example, these two Nix expressions print the same result despite
> having different meaning:
>
> ```console
> $ nix-instantiate --eval --expr '{ a = {}; }'
> { a = <CODE>; }
> $ nix-instantiate --eval --expr '{ a = <CODE>; }'
> { a = <CODE>; }
> ```
>
> For human-readable output, `nix eval` (experimental) is more informative:
>
> ```console
> $ nix-instantiate --eval --expr 'a: a'
> <LAMBDA>
> $ nix eval --expr 'a: a'
> «lambda @ «string»:1:1»
> ```
>
> For machine-readable output, the `--xml` option produces unambiguous
> output:
>
> ```console
> $ nix-instantiate --eval --xml --expr '{ foo = <CODE>; }'
> <?xml version='1.0' encoding='utf-8'?>
> <expr>
> <attrs>
> <attr column="3" line="1" name="foo">
> <unevaluated />
> </attr>
> </attrs>
> </expr>
> ```
Just parse the input files, and print their abstract syntax trees on
standard output as a Nix expression.
- `--find-file`\
Look up the given files in Nix’s search path (as specified by the
`NIX_PATH` environment variable). If found, print the corresponding
absolute paths on standard output. For instance, if `NIX_PATH` is
`nixpkgs=/home/alice/nixpkgs`, then `nix-instantiate --find-file
nixpkgs/default.nix` will print `/home/alice/nixpkgs/default.nix`.
- `--eval`
- `--strict`\
When used with `--eval`, recursively evaluate list elements and
attributes. Normally, such sub-expressions are left unevaluated
(since the Nix language is lazy).
Just parse and evaluate the input files, and print the resulting
values on standard output. No instantiation of store derivations
takes place.
> **Warning**
>
> This option can cause non-termination, because lazy data
> structures can be infinitely large.
> **Warning**
>
> This option produces output which can be parsed as a Nix expression which
> will produce a different result than the input expression when evaluated.
> For example, these two Nix expressions print the same result despite
> having different meaning:
>
> ```console
> $ nix-instantiate --eval --expr '{ a = {}; }'
> { a = <CODE>; }
> $ nix-instantiate --eval --expr '{ a = <CODE>; }'
> { a = <CODE>; }
> ```
>
> For human-readable output, `nix eval` (experimental) is more informative:
>
> ```console
> $ nix-instantiate --eval --expr 'a: a'
> <LAMBDA>
> $ nix eval --expr 'a: a'
> «lambda @ «string»:1:1»
> ```
>
> For machine-readable output, the `--xml` option produces unambiguous
> output:
>
> ```console
> $ nix-instantiate --eval --xml --expr '{ foo = <CODE>; }'
> <?xml version='1.0' encoding='utf-8'?>
> <expr>
> <attrs>
> <attr column="3" line="1" name="foo">
> <unevaluated />
> </attr>
> </attrs>
> </expr>
> ```
- `--json`\
When used with `--eval`, print the resulting value as an JSON
representation of the abstract syntax tree rather than as a Nix expression.
- `--find-file`
- `--xml`\
When used with `--eval`, print the resulting value as an XML
representation of the abstract syntax tree rather than as a Nix expression.
The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
Look up the given files in Nix’s search path (as specified by the
`NIX_PATH` environment variable). If found, print the corresponding
absolute paths on standard output. For instance, if `NIX_PATH` is
`nixpkgs=/home/alice/nixpkgs`, then `nix-instantiate --find-file
nixpkgs/default.nix` will print `/home/alice/nixpkgs/default.nix`.
- `--read-write-mode`\
When used with `--eval`, perform evaluation in read/write mode so
nix language features that require it will still work (at the cost
of needing to do instantiation of every evaluated derivation). If
this option is not enabled, there may be uninstantiated store paths
in the final output.
- `--strict`
When used with `--eval`, recursively evaluate list elements and
attributes. Normally, such sub-expressions are left unevaluated
(since the Nix language is lazy).
> **Warning**
>
> This option can cause non-termination, because lazy data
> structures can be infinitely large.
- `--json`
When used with `--eval`, print the resulting value as an JSON
representation of the abstract syntax tree rather than as a Nix expression.
- `--xml`
When used with `--eval`, print the resulting value as an XML
representation of the abstract syntax tree rather than as a Nix expression.
The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
- `--read-write-mode`
When used with `--eval`, perform evaluation in read/write mode so
nix language features that require it will still work (at the cost
of needing to do instantiation of every evaluated derivation). If
this option is not enabled, there may be uninstantiated store paths
in the final output.
{{#include ./opt-common.md}}
+22 -17
View File
@@ -39,27 +39,32 @@ the path of the downloaded file in the Nix store is also printed.
# Options
- `--type` *hashAlgo*\
Use the specified cryptographic hash algorithm,
which can be one of `md5`, `sha1`, `sha256`, and `sha512`.
The default is `sha256`.
- `--type` *hashAlgo*
- `--print-path`\
Print the store path of the downloaded file on standard output.
Use the specified cryptographic hash algorithm,
which can be one of `md5`, `sha1`, `sha256`, and `sha512`.
The default is `sha256`.
- `--unpack`\
Unpack the archive (which must be a tarball or zip file) and add the
result to the Nix store. The resulting hash can be used with
functions such as Nixpkgs’s `fetchzip` or `fetchFromGitHub`.
- `--print-path`
- `--executable`\
Set the executable bit on the downloaded file.
Print the store path of the downloaded file on standard output.
- `--name` *name*\
Override the name of the file in the Nix store. By default, this is
`hash-basename`, where *basename* is the last component of *url*.
Overriding the name is necessary when *basename* contains characters
that are not allowed in Nix store paths.
- `--unpack`
Unpack the archive (which must be a tarball or zip file) and add the
result to the Nix store. The resulting hash can be used with
functions such as Nixpkgs’s `fetchzip` or `fetchFromGitHub`.
- `--executable`
Set the executable bit on the downloaded file.
- `--name` *name*
Override the name of the file in the Nix store. By default, this is
`hash-basename`, where *basename* is the last component of *url*.
Overriding the name is necessary when *basename* contains characters
that are not allowed in Nix store paths.
# Examples
+51 -38
View File
@@ -60,55 +60,63 @@ All options not listed here are passed to `nix-store
--realise`, except for `--arg` and `--attr` / `-A` which are passed to
`nix-instantiate`.
- `--command` *cmd*\
In the environment of the derivation, run the shell command *cmd*.
This command is executed in an interactive shell. (Use `--run` to
use a non-interactive shell instead.) However, a call to `exit` is
implicitly added to the command, so the shell will exit after
running the command. To prevent this, add `return` at the end;
e.g. `--command "echo Hello; return"` will print `Hello` and then
drop you into the interactive shell. This can be useful for doing
any additional initialisation.
- `--command` *cmd*
- `--run` *cmd*\
Like `--command`, but executes the command in a non-interactive
shell. This means (among other things) that if you hit Ctrl-C while
the command is running, the shell exits.
In the environment of the derivation, run the shell command *cmd*.
This command is executed in an interactive shell. (Use `--run` to
use a non-interactive shell instead.) However, a call to `exit` is
implicitly added to the command, so the shell will exit after
running the command. To prevent this, add `return` at the end;
e.g. `--command "echo Hello; return"` will print `Hello` and then
drop you into the interactive shell. This can be useful for doing
any additional initialisation.
- `--exclude` *regexp*\
Do not build any dependencies whose store path matches the regular
expression *regexp*. This option may be specified multiple times.
- `--run` *cmd*
- `--pure`\
If this flag is specified, the environment is almost entirely
cleared before the interactive shell is started, so you get an
environment that more closely corresponds to the “real” Nix build. A
few variables, in particular `HOME`, `USER` and `DISPLAY`, are
retained.
Like `--command`, but executes the command in a non-interactive
shell. This means (among other things) that if you hit Ctrl-C while
the command is running, the shell exits.
- `--packages` / `-p` *packages*…\
Set up an environment in which the specified packages are present.
The command line arguments are interpreted as attribute names inside
the Nix Packages collection. Thus, `nix-shell --packages libjpeg openjdk`
will start a shell in which the packages denoted by the attribute
names `libjpeg` and `openjdk` are present.
- `--exclude` *regexp*
- `-i` *interpreter*\
The chained script interpreter to be invoked by `nix-shell`. Only
applicable in `#!`-scripts (described below).
Do not build any dependencies whose store path matches the regular
expression *regexp*. This option may be specified multiple times.
- `--keep` *name*\
When a `--pure` shell is started, keep the listed environment
variables.
- `--pure`
If this flag is specified, the environment is almost entirely
cleared before the interactive shell is started, so you get an
environment that more closely corresponds to the “real” Nix build. A
few variables, in particular `HOME`, `USER` and `DISPLAY`, are
retained.
- `--packages` / `-p` *packages*…
Set up an environment in which the specified packages are present.
The command line arguments are interpreted as attribute names inside
the Nix Packages collection. Thus, `nix-shell --packages libjpeg openjdk`
will start a shell in which the packages denoted by the attribute
names `libjpeg` and `openjdk` are present.
- `-i` *interpreter*
The chained script interpreter to be invoked by `nix-shell`. Only
applicable in `#!`-scripts (described below).
- `--keep` *name*
When a `--pure` shell is started, keep the listed environment
variables.
{{#include ./opt-common.md}}
# Environment variables
- `NIX_BUILD_SHELL`\
Shell used to start the interactive environment. Defaults to the
`bash` found in `<nixpkgs>`, falling back to the `bash` found in
`PATH` if not found.
- `NIX_BUILD_SHELL`
Shell used to start the interactive environment. Defaults to the
`bash` found in `<nixpkgs>`, falling back to the `bash` found in
`PATH` if not found.
{{#include ./env-common.md}}
@@ -289,3 +297,8 @@ with import <nixpkgs> {};
runCommand "dummy" { buildInputs = [ python pythonPackages.prettytable ]; } ""
```
The script's file name is passed as the first argument to the interpreter specified by the `-i` flag.
Aside from the very first line, which is a directive to the operating system, the additional `#! nix-shell` lines do not need to be at the beginning of the file.
This allows wrapping them in block comments for languages where `#` does not start a comment, such as ECMAScript, Erlang, PHP, or Ruby.
@@ -16,9 +16,13 @@ public url or broke since the download expression was written.
This operation has the following options:
- `--recursive`\
Use recursive instead of flat hashing mode, used when adding
directories to the store.
- `--recursive`
Use recursive instead of flat hashing mode, used when adding
directories to the store.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
@@ -11,6 +11,9 @@
The operation `--add` adds the specified paths to the Nix store. It
prints the resulting paths in the Nix store on standard output.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
+22 -18
View File
@@ -14,30 +14,34 @@ reachable via file system references from a set of “roots”, are deleted.
The following suboperations may be specified:
- `--print-roots`\
This operation prints on standard output the set of roots used by
the garbage collector.
- `--print-roots`
- `--print-live`\
This operation prints on standard output the set of “live” store
paths, which are all the store paths reachable from the roots. Live
paths should never be deleted, since that would break consistency —
it would become possible that applications are installed that
reference things that are no longer present in the store.
This operation prints on standard output the set of roots used by
the garbage collector.
- `--print-dead`\
This operation prints out on standard output the set of “dead” store
paths, which is just the opposite of the set of live paths: any path
in the store that is not live (with respect to the roots) is dead.
- `--print-live`
This operation prints on standard output the set of “live” store
paths, which are all the store paths reachable from the roots. Live
paths should never be deleted, since that would break consistency —
it would become possible that applications are installed that
reference things that are no longer present in the store.
- `--print-dead`
This operation prints out on standard output the set of “dead” store
paths, which is just the opposite of the set of live paths: any path
in the store that is not live (with respect to the roots) is dead.
By default, all unreachable paths are deleted. The following options
control what gets deleted and in what order:
- `--max-freed` *bytes*\
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
- `--max-freed` *bytes*
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
The behaviour of the collector is also influenced by the
`keep-outputs` and `keep-derivations` settings in the Nix
+106 -91
View File
@@ -24,122 +24,138 @@ symlink.
# Common query options
- `--use-output`; `-u`\
For each argument to the query that is a [store derivation], apply the
query to the output path of the derivation instead.
- `--use-output` / `-u`
- `--force-realise`; `-f`\
Realise each argument to the query first (see [`nix-store --realise`](./realise.md)).
For each argument to the query that is a [store derivation], apply the
query to the output path of the derivation instead.
- `--force-realise` / `-f`
Realise each argument to the query first (see [`nix-store --realise`](./realise.md)).
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
# Queries
- `--outputs`\
Prints out the [output paths] of the store
derivations *paths*. These are the paths that will be produced when
the derivation is built.
- `--outputs`
[output paths]: @docroot@/glossary.md#gloss-output-path
Prints out the [output paths] of the store
derivations *paths*. These are the paths that will be produced when
the derivation is built.
- `--requisites`; `-R`\
Prints out the [closure] of the store path *paths*.
[output paths]: @docroot@/glossary.md#gloss-output-path
[closure]: @docroot@/glossary.md#gloss-closure
- `--requisites` / `-R`
This query has one option:
Prints out the [closure] of the store path *paths*.
- `--include-outputs`
Also include the existing output paths of [store derivation]s,
and their closures.
[closure]: @docroot@/glossary.md#gloss-closure
This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
store derivation. A *binary deployment* is obtained by distributing
the closure of an output path. A *cache deployment* (combined
source/binary deployment, including binaries of build-time-only
dependencies) is obtained by distributing the closure of a store
derivation and specifying the option `--include-outputs`.
This query has one option:
- `--references`\
Prints the set of [references] of the store paths
*paths*, that is, their immediate dependencies. (For *all*
dependencies, use `--requisites`.)
- `--include-outputs`
Also include the existing output paths of [store derivation]s,
and their closures.
[references]: @docroot@/glossary.md#gloss-reference
This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
store derivation. A *binary deployment* is obtained by distributing
the closure of an output path. A *cache deployment* (combined
source/binary deployment, including binaries of build-time-only
dependencies) is obtained by distributing the closure of a store
derivation and specifying the option `--include-outputs`.
- `--referrers`\
Prints the set of *referrers* of the store paths *paths*, that is,
the store paths currently existing in the Nix store that refer to
one of *paths*. Note that contrary to the references, the set of
referrers is not constant; it can change as store paths are added or
removed.
- `--references`
- `--referrers-closure`\
Prints the closure of the set of store paths *paths* under the
referrers relation; that is, all store paths that directly or
indirectly refer to one of *paths*. These are all the path currently
in the Nix store that are dependent on *paths*.
Prints the set of [references] of the store paths
*paths*, that is, their immediate dependencies. (For *all*
dependencies, use `--requisites`.)
- `--deriver`; `-d`\
Prints the [deriver] that was used to build the store paths *paths*. If
the path has no deriver (e.g., if it is a source file), or if the
deriver is not known (e.g., in the case of a binary-only
deployment), the string `unknown-deriver` is printed.
The returned deriver is not guaranteed to exist in the local store, for
example when *paths* were substituted from a binary cache.
Use `--valid-derivers` instead to obtain valid paths only.
[references]: @docroot@/glossary.md#gloss-reference
[deriver]: @docroot@/glossary.md#gloss-deriver
- `--referrers`
- `--valid-derivers`\
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths subsituted from a binary cache.
Prints the set of *referrers* of the store paths *paths*, that is,
the store paths currently existing in the Nix store that refer to
one of *paths*. Note that contrary to the references, the set of
referrers is not constant; it can change as store paths are added or
removed.
- `--graph`\
Prints the references graph of the store paths *paths* in the format
of the `dot` tool of AT\&T's [Graphviz
package](http://www.graphviz.org/). This can be used to visualise
dependency graphs. To obtain a build-time dependency graph, apply
this to a store derivation. To obtain a runtime dependency graph,
apply it to an output path.
- `--referrers-closure`
- `--tree`\
Prints the references graph of the store paths *paths* as a nested
ASCII tree. References are ordered by descending closure size; this
tends to flatten the tree, making it more readable. The query only
recurses into a store path when it is first encountered; this
prevents a blowup of the tree representation of the graph.
Prints the closure of the set of store paths *paths* under the
referrers relation; that is, all store paths that directly or
indirectly refer to one of *paths*. These are all the path currently
in the Nix store that are dependent on *paths*.
- `--graphml`\
Prints the references graph of the store paths *paths* in the
[GraphML](http://graphml.graphdrawing.org/) file format. This can be
used to visualise dependency graphs. To obtain a build-time
dependency graph, apply this to a [store derivation]. To obtain a
runtime dependency graph, apply it to an output path.
- `--deriver` / `-d`
- `--binding` *name*; `-b` *name*\
Prints the value of the attribute *name* (i.e., environment
variable) of the [store derivation]s *paths*. It is an error for a
derivation to not have the specified attribute.
Prints the [deriver] that was used to build the store paths *paths*. If
the path has no deriver (e.g., if it is a source file), or if the
deriver is not known (e.g., in the case of a binary-only
deployment), the string `unknown-deriver` is printed.
The returned deriver is not guaranteed to exist in the local store, for
example when *paths* were substituted from a binary cache.
Use `--valid-derivers` instead to obtain valid paths only.
- `--hash`\
Prints the SHA-256 hash of the contents of the store paths *paths*
(that is, the hash of the output of `nix-store --dump` on the given
paths). Since the hash is stored in the Nix database, this is a fast
operation.
[deriver]: @docroot@/glossary.md#gloss-deriver
- `--size`\
Prints the size in bytes of the contents of the store paths *paths*
— to be precise, the size of the output of `nix-store --dump` on
the given paths. Note that the actual disk space required by the
store paths may be higher, especially on filesystems with large
cluster sizes.
- `--valid-derivers`
- `--roots`\
Prints the garbage collector roots that point, directly or
indirectly, at the store paths *paths*.
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths substituted from a binary cache.
- `--graph`
Prints the references graph of the store paths *paths* in the format
of the `dot` tool of AT\&T's [Graphviz
package](http://www.graphviz.org/). This can be used to visualise
dependency graphs. To obtain a build-time dependency graph, apply
this to a store derivation. To obtain a runtime dependency graph,
apply it to an output path.
- `--tree`
Prints the references graph of the store paths *paths* as a nested
ASCII tree. References are ordered by descending closure size; this
tends to flatten the tree, making it more readable. The query only
recurses into a store path when it is first encountered; this
prevents a blowup of the tree representation of the graph.
- `--graphml`
Prints the references graph of the store paths *paths* in the
[GraphML](http://graphml.graphdrawing.org/) file format. This can be
used to visualise dependency graphs. To obtain a build-time
dependency graph, apply this to a [store derivation]. To obtain a
runtime dependency graph, apply it to an output path.
- `--binding` *name* / `-b` *name*
Prints the value of the attribute *name* (i.e., environment
variable) of the [store derivation]s *paths*. It is an error for a
derivation to not have the specified attribute.
- `--hash`
Prints the SHA-256 hash of the contents of the store paths *paths*
(that is, the hash of the output of `nix-store --dump` on the given
paths). Since the hash is stored in the Nix database, this is a fast
operation.
- `--size`
Prints the size in bytes of the contents of the store paths *paths*
— to be precise, the size of the output of `nix-store --dump` on
the given paths. Note that the actual disk space required by the
store paths may be higher, especially on filesystems with large
cluster sizes.
- `--roots`
Prints the garbage collector roots that point, directly or
indirectly, at the store paths *paths*.
{{#include ./opt-common.md}}
@@ -225,4 +241,3 @@ $ nix-store --query --roots $(which svn)
/nix/var/nix/profiles/default-82-link
/home/eelco/.local/state/nix/profiles/profile-97-link
```
+19 -16
View File
@@ -32,7 +32,7 @@ If no substitutes are available and no store derivation is given, realisation fa
[store objects]: @docroot@/store/store-object.md
[closure]: @docroot@/glossary.md#gloss-closure
[substituters]: @docroot@/command-ref/conf-file.md#conf-substituters
[content-addressed derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
[content-addressed derivations]: @docroot@/development/experimental-features.md#xp-feature-ca-derivations
[Nix database]: @docroot@/glossary.md#gloss-nix-database
The resulting paths are printed on standard output.
@@ -42,23 +42,26 @@ For non-derivation arguments, the argument itself is printed.
# Options
- `--dry-run`\
Print on standard error a description of what packages would be
built or downloaded, without actually performing the operation.
- `--dry-run`
- `--ignore-unknown`\
If a non-derivation path does not have a substitute, then silently
ignore it.
Print on standard error a description of what packages would be
built or downloaded, without actually performing the operation.
- `--check`\
This option allows you to check whether a derivation is
deterministic. It rebuilds the specified derivation and checks
whether the result is bitwise-identical with the existing outputs,
printing an error if that’s not the case. The outputs of the
specified derivation must already exist. When used with `-K`, if an
output path is not identical to the corresponding output from the
previous build, the new output path is left in
`/nix/store/name.check.`
- `--ignore-unknown`
If a non-derivation path does not have a substitute, then silently
ignore it.
- `--check`
This option allows you to check whether a derivation is
deterministic. It rebuilds the specified derivation and checks
whether the result is bitwise-identical with the existing outputs,
printing an error if that’s not the case. The outputs of the
specified derivation must already exist. When used with `-K`, if an
output path is not identical to the corresponding output from the
previous build, the new output path is left in
`/nix/store/name.check.`
{{#include ./opt-common.md}}
@@ -14,10 +14,11 @@ access to a restricted ssh user.
The following flags are available:
- `--write`\
Allow the connected client to request the realization of
derivations. In effect, this can be used to make the host act as a
remote builder.
- `--write`
Allow the connected client to request the realization of
derivations. In effect, this can be used to make the host act as a
remote builder.
{{#include ./opt-common.md}}
+13 -11
View File
@@ -16,18 +16,20 @@ being modified by non-Nix tools, or of bugs in Nix itself.
This operation has the following options:
- `--check-contents`\
Checks that the contents of every valid store path has not been
altered by computing a SHA-256 hash of the contents and comparing it
with the hash stored in the Nix database at build time. Paths that
have been modified are printed out. For large stores,
`--check-contents` is obviously quite slow.
- `--check-contents`
- `--repair`\
If any valid path is missing from the store, or (if
`--check-contents` is given) the contents of a valid path has been
modified, then try to repair the path by redownloading it. See
`nix-store --repair-path` for details.
Checks that the contents of every valid store path has not been
altered by computing a SHA-256 hash of the contents and comparing it
with the hash stored in the Nix database at build time. Paths that
have been modified are printed out. For large stores,
`--check-contents` is obviously quite slow.
- `--repair`
If any valid path is missing from the store, or (if
`--check-contents` is given) the contents of a valid path has been
modified, then try to repair the path by redownloading it. See
`nix-store --repair-path` for details.
{{#include ./opt-common.md}}
+6 -5
View File
@@ -37,7 +37,7 @@ Most Nix commands accept the following command-line options:
Print even more informational messages.
- `4` “Debug”
Print debug information.
- `5` “Vomit”
@@ -143,7 +143,7 @@ Most Nix commands accept the following command-line options:
This option is accepted by `nix-env`, `nix-instantiate`, `nix-shell` and `nix-build`.
When evaluating Nix expressions, the expression evaluator will automatically try to call functions that it encounters.
It can automatically call functions for which every argument has a [default value](@docroot@/language/constructs.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
It can automatically call functions for which every argument has a [default value](@docroot@/language/syntax.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
With `--arg`, you can also call functions that have arguments without a default value (or override a default value).
That is, if the evaluator encounters a function with an argument named *name*, it will call it with value *value*.
@@ -187,11 +187,12 @@ Most Nix commands accept the following command-line options:
For `nix-shell`, this option is commonly used to give you a shell in which you can build the packages returned by the expression.
If you want to get a shell which contain the *built* packages ready for use, give your expression to the `nix-shell --packages ` convenience flag instead.
- <span id="opt-I">[`-I`](#opt-I)</span> *path*
- <span id="opt-I">[`-I` / `--include`](#opt-I)</span> *path*
Add an entry to the [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path).
Add an entry to the list of search paths used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
This option may be given multiple times.
Paths added through `-I` take precedence over [`NIX_PATH`](@docroot@/command-ref/env-common.md#env-NIX_PATH).
Paths added through `-I` take precedence over the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path) and the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH).
- <span id="opt-option">[`--option`](#opt-option)</span> *name* *value*
@@ -1,24 +1,67 @@
# Hacking
# Building Nix
This section provides some notes on how to hack on Nix. To get the
latest version of Nix from GitHub:
This section provides some notes on how to start hacking on Nix.
To get the latest version of Nix from GitHub:
```console
$ git clone https://github.com/NixOS/nix.git
$ cd nix
```
The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the [installation instructions].
> **Note**
>
> The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment.
> If you don't have it installed, follow the [installation instructions](../installation/index.md).
[installation instructions]: ../installation/index.md
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
```console
$ nix-shell
```
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix-shell --attr devShells.x86_64-linux.native-clangStdenvPackages
```
> **Note**
>
> You can use `native-ccacheStdenvPackages` to drastically improve rebuild time.
> By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ ./configure $configureFlags --prefix=$(pwd)/outputs/out
[nix-shell]$ make -j $NIX_BUILD_CORES
```
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ make install
[nix-shell]$ make installcheck -j $NIX_BUILD_CORES
[nix-shell]$ ./outputs/out/bin/nix --version
nix (Nix) 2.12
```
To build a release version of Nix for the current operating system and CPU architecture:
```console
$ nix-build
```
You can also build Nix for one of the [supported platforms](#platforms).
## Building Nix with flakes
This section assumes you are using Nix with the [`flakes`] and [`nix-command`] experimental features enabled.
See the [Building Nix](#building-nix) section for equivalent instructions using stable Nix interfaces.
[`flakes`]: @docroot@/contributing/experimental-features.md#xp-feature-flakes
[`nix-command`]: @docroot@/contributing/experimental-features.md#xp-nix-command
[`flakes`]: @docroot@/development/experimental-features.md#xp-feature-flakes
[`nix-command`]: @docroot@/development/experimental-features.md#xp-nix-command
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
@@ -67,50 +110,6 @@ $ nix build
You can also build Nix for one of the [supported platforms](#platforms).
## Building Nix
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
```console
$ nix-shell
```
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix-shell --attr devShells.x86_64-linux.native-clangStdenvPackages
```
> **Note**
>
> You can use `native-ccacheStdenvPackages` to drastically improve rebuild time.
> By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ ./configure $configureFlags --prefix=$(pwd)/outputs/out
[nix-shell]$ make -j $NIX_BUILD_CORES
```
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ make install
[nix-shell]$ make installcheck -j $NIX_BUILD_CORES
[nix-shell]$ ./outputs/out/bin/nix --version
nix (Nix) 2.12
```
To build a release version of Nix for the current operating system and CPU architecture:
```console
$ nix-build
```
You can also build Nix for one of the [supported platforms](#platforms).
## Makefile variables
You may need `profiledir=$out/etc/profile.d` and `sysconfdir=$out/etc` to run `make install`.
@@ -122,7 +121,6 @@ Run `make` with [`-e` / `--environment-overrides`](https://www.gnu.org/software/
The docs can take a while to build, so you may want to disable this for local development.
- `ENABLE_FUNCTIONAL_TESTS=yes` to enable building the functional tests.
- `ENABLE_UNIT_TESTS=yes` to enable building the unit tests.
- `OPTIMIZE=1` to enable optimizations.
- `libraries=libutil programs=` to only build a specific library.
@@ -295,81 +293,3 @@ If it fails, run `git add --patch` to approve the suggestions _and commit again_
To refresh pre-commit hook's config file, do the following:
1. Exit the development shell and start it again by running `nix develop`.
2. If you also use the pre-commit hook, also run `pre-commit-hooks-install` again.
## Add a release note
`doc/manual/rl-next` contains release notes entries for all unreleased changes.
User-visible changes should come with a release note.
### Add an entry
Here's what a complete entry looks like. The file name is not incorporated in the document.
```
---
synopsis: Basically a title
issues: 1234
prs: 1238
---
Here's one or more paragraphs that describe the change.
- It's markdown
- Add references to the manual using @docroot@
```
Significant changes should add the following header, which moves them to the top.
```
significance: significant
```
<!-- Keep an eye on https://codeberg.org/fgaz/changelog-d/issues/1 -->
See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog).
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
## Branches
- [`master`](https://github.com/NixOS/nix/commits/master)
The main development branch. All changes are approved and merged here.
When developing a change, create a branch based on the latest `master`.
Maintainers try to [keep it in a release-worthy state](#reverting).
- [`maintenance-*.*`](https://github.com/NixOS/nix/branches/all?query=maintenance)
These branches are the subject of backports only, and are
also [kept](#reverting) in a release-worthy state.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [`latest-release`](https://github.com/NixOS/nix/tree/latest-release)
The latest patch release of the latest minor version.
See [`maintainers/release-process.md`](https://github.com/NixOS/nix/blob/master/maintainers/release-process.md)
- [`backport-*-to-*`](https://github.com/NixOS/nix/branches/all?query=backport)
Generally branches created by the backport action.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [_other_](https://github.com/NixOS/nix/branches/all)
Branches that do not conform to the above patterns should be feature branches.
## Reverting
If a change turns out to be merged by mistake, or contain a regression, it may be reverted.
A revert is not a rejection of the contribution, but merely part of an effective development process.
It makes sure that development keeps running smoothly, with minimal uncertainty, and less overhead.
If maintainers have to worry too much about avoiding reverts, they would not be able to merge as much.
By embracing reverts as a good part of the development process, everyone wins.
However, taking a step back may be frustrating, so maintainers will be extra supportive on the next try.
@@ -0,0 +1,79 @@
# Contributing
## Add a release note
`doc/manual/rl-next` contains release notes entries for all unreleased changes.
User-visible changes should come with a release note.
### Add an entry
Here's what a complete entry looks like. The file name is not incorporated in the document.
```
---
synopsis: Basically a title
issues: 1234
prs: 1238
---
Here's one or more paragraphs that describe the change.
- It's markdown
- Add references to the manual using @docroot@
```
Significant changes should add the following header, which moves them to the top.
```
significance: significant
```
<!-- Keep an eye on https://codeberg.org/fgaz/changelog-d/issues/1 -->
See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog).
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
## Branches
- [`master`](https://github.com/NixOS/nix/commits/master)
The main development branch. All changes are approved and merged here.
When developing a change, create a branch based on the latest `master`.
Maintainers try to [keep it in a release-worthy state](#reverting).
- [`maintenance-*.*`](https://github.com/NixOS/nix/branches/all?query=maintenance)
These branches are the subject of backports only, and are
also [kept](#reverting) in a release-worthy state.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [`latest-release`](https://github.com/NixOS/nix/tree/latest-release)
The latest patch release of the latest minor version.
See [`maintainers/release-process.md`](https://github.com/NixOS/nix/blob/master/maintainers/release-process.md)
- [`backport-*-to-*`](https://github.com/NixOS/nix/branches/all?query=backport)
Generally branches created by the backport action.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [_other_](https://github.com/NixOS/nix/branches/all)
Branches that do not conform to the above patterns should be feature branches.
## Reverting
If a change turns out to be merged by mistake, or contain a regression, it may be reverted.
A revert is not a rejection of the contribution, but merely part of an effective development process.
It makes sure that development keeps running smoothly, with minimal uncertainty, and less overhead.
If maintainers have to worry too much about avoiding reverts, they would not be able to merge as much.
By embracing reverts as a good part of the development process, everyone wins.
However, taking a step back may be frustrating, so maintainers will be extra supportive on the next try.
File renamed without changes.
@@ -24,7 +24,7 @@ nix build .#^doc
and open `./result-doc/share/doc/nix/manual/index.html`.
To build the manual incrementally, [enter the development shell](./hacking.md) and run:
To build the manual incrementally, [enter the development shell](./building.md) and run:
```console
make manual-html-open -j $NIX_BUILD_CORES
@@ -196,15 +196,16 @@ You can also build and view it yourself:
[Doxygen API documentation]: https://hydra.nixos.org/job/nix/master/internal-api-docs/latest/download-by-type/doc/internal-api-docs
```console
# nix build .#hydraJobs.internal-api-docs
# xdg-open ./result/share/doc/nix/internal-api/html/index.html
$ nix build .#hydraJobs.internal-api-docs
$ xdg-open ./result/share/doc/nix/internal-api/html/index.html
```
or inside `nix-shell` or `nix develop`:
```
# make internal-api-html
# xdg-open ./outputs/doc/share/doc/nix/internal-api/html/index.html
```console
$ mesonConfigurePhase
$ ninja src/internal-api-docs/html
$ xdg-open src/internal-api-docs/html/index.html
```
## C API documentation
@@ -216,13 +217,14 @@ You can also build and view it yourself:
[C API documentation]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs
```console
# nix build .#hydraJobs.external-api-docs
# xdg-open ./result/share/doc/nix/external-api/html/index.html
$ nix build .#hydraJobs.external-api-docs
$ xdg-open ./result/share/doc/nix/external-api/html/index.html
```
or inside `nix-shell` or `nix develop`:
```
# make external-api-html
# xdg-open ./outputs/doc/share/doc/nix/external-api/html/index.html
$ mesonConfigurePhase
$ ninja src/external-api-docs/html
$ xdg-open src/external-api-docs/html/index.html
```
@@ -5,4 +5,4 @@ Check the [contributing guide](https://github.com/NixOS/nix/blob/master/CONTRIBU
This chapter is a collection of guides for making changes to the code and documentation.
If you're not sure where to start, try to [compile Nix from source](./hacking.md) and consider [making improvements to documentation](./documentation.md).
If you're not sure where to start, try to [compile Nix from source](./building.md) and consider [making improvements to documentation](./documentation.md).
@@ -90,7 +90,7 @@ This representation is extensible and preserves the ordering:
## Self-describing values
As described in the previous section, it's crucial that schemas can be extended with with new fields without breaking compatibility.
As described in the previous section, it's crucial that schemas can be extended with new fields without breaking compatibility.
However, that should *not* mean we use the presence/absence of fields to indicate optional information *within* a version of the schema.
Instead, always include the field, and use `null` to indicate the "nothing" case.
@@ -59,15 +59,15 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> …
> ```
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `tests/unit/${library_name_without-nix}`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `tests/unit/libexpr/tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `tests/unit/libexpr-support/tests/value/context.{hh,cc}`.
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `src/nix-expr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/nix-expr-test-support/tests/value/context.{hh,cc}`.
Data for unit tests is stored in a `data` subdir of the directory for each unit test executable.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `tests/unit/libstore/data`.
The path to the `tests/unit/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/nix-store-tests/data`.
The path to the `src/${library_name_without-nix}-test/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
Note that each executable only gets the data for its tests.
The unit test libraries are in `tests/unit/${library_name_without-nix}-lib`.
The unit test libraries are in `src/${library_name_without-nix}-test-support`.
All headers are in a `tests` subdirectory so they are included with `#include "tests/"`.
The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing.
@@ -76,8 +76,25 @@ there is no risk of any build-system wildcards for the library accidentally pick
### Running tests
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`.
Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option, or the `GTEST_FILTER` environment variable, e.g. `GTEST_FILTER='ErrorTraceTest.*' make check`.
You can run the whole testsuite with `meson test` from the Meson build directory, or the tests for a specific component with `meson test nix-store-tests`.
A environment variables that Google Test accepts are also worth knowing:
1. [`GTEST_FILTER`](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests)
This is used for finer-grained filtering of which tests to run.
2. [`GTEST_BRIEF`](https://google.github.io/googletest/advanced.html#suppressing-test-passes)
This is used to avoid logging passing tests.
Putting the two together, one might run
```bash
GTEST_BRIEF=1 GTEST_FILTER='ErrorTraceTest.*' meson test nix-expr-tests -v
```
for short but comprensive output.
### Characterisation testing { #characaterisation-testing-unit }
@@ -86,7 +103,7 @@ See [functional characterisation testing](#characterisation-testing-functional)
Like with the functional characterisation, `_NIX_TEST_ACCEPT=1` is also used.
For example:
```shell-session
$ _NIX_TEST_ACCEPT=1 make libstore-tests_RUN
$ _NIX_TEST_ACCEPT=1 meson test nix-store-tests -v
...
[ SKIPPED ] WorkerProtoTest.string_read
[ SKIPPED ] WorkerProtoTest.string_write
@@ -114,6 +131,8 @@ On other platforms they wouldn't be run at all.
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/local.mk`.
Each test is a bash script.
Functional tests are run during `installCheck` in the `nix` package build, as well as separately from the build, in VM tests.
### Running the whole test suite
The whole test suite can be run with:
@@ -252,13 +271,30 @@ Regressions are caught, and improvements always show up in code review.
To ensure that characterisation testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
### Running functional tests on NixOS
We run the functional tests not just in the build, but also in VM tests.
This helps us ensure that Nix works correctly on NixOS, and environments that have similar characteristics that are hard to reproduce in a build environment.
The recommended way to run these tests during development is:
```shell
nix build .#hydraJobs.tests.functional_user.quickBuild
```
The `quickBuild` attribute configures the test to use a `nix` package that's built without integration tests, so that you can iterate on the tests without performing recompilations due to the changed sources for `installCheck`.
Generally, this build is sufficient, but in nightly or CI we also test the attributes `functional_root` and `functional_trusted`, in which the test suite is run with different levels of authorization.
## Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`.
If you are testing a build of `nix` that you haven't compiled yet, you may iterate faster by appending the `quickBuild` attribute: `nix build .#hydraJobs.tests.{testName}.quickBuild`.
## Installer tests
@@ -272,7 +308,7 @@ Creating a Cachix cache for your installer tests and adding its authorisation to
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
- The `installer_test` job (which runs on `ubuntu-24.04` and `macos-14`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup
+8 -9
View File
@@ -71,10 +71,9 @@
[`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}
- [fixed-output derivation]{#gloss-fixed-output-derivation} (FOD)
A derivation which includes the
[`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute.
A [derivation] where a cryptographic hash of the [output] is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network.
- [store]{#gloss-store}
@@ -169,7 +168,7 @@
- [impure derivation]{#gloss-impure-derivation}
[An experimental feature](#@docroot@/contributing/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure,
[An experimental feature](#@docroot@/development/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure,
so that they are always rebuilt, and their outputs not reused by subsequent calls to realise them.
- [Nix database]{#gloss-nix-database}
@@ -313,7 +312,7 @@
- [package attribute set]{#package-attribute-set}
An [attribute set](@docroot@/language/values.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
An [attribute set](@docroot@/language/types.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
- attributes that refer to the files of a [package], typically in the form of [derivation outputs](#output),
- attributes that declare something about how the package is supposed to be installed or used,
- other metadata or arbitrary attributes.
@@ -326,9 +325,9 @@
See [String interpolation](./language/string-interpolation.md) for details.
[string]: ./language/values.md#type-string
[path]: ./language/values.md#type-path
[attribute name]: ./language/values.md#attribute-set
[string]: ./language/types.md#type-string
[path]: ./language/types.md#type-path
[attribute name]: ./language/types.md#attribute-set
- [base directory]{#gloss-base-directory}
@@ -354,7 +353,7 @@
Not yet stabilized functionality guarded by named experimental feature flags.
These flags are enabled or disabled with the [`experimental-features`](./command-ref/conf-file.html#conf-experimental-features) setting.
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/contributing/experimental-features.md).
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/development/experimental-features.md).
[Nix language]: ./language/index.md
+8
View File
@@ -14,6 +14,14 @@ This option requires either:
* Linux running systemd, with SELinux disabled
* MacOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
```console
$ bash <(curl -L https://nixos.org/nix/install) --daemon
```
@@ -1,5 +1,13 @@
# Installing a Binary Distribution
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
To install the latest version Nix, run the following command:
```console
@@ -77,7 +85,7 @@ $ su root
# Installing from a binary tarball
You can also download a binary tarball that contains Nix and all its dependencies:
- Choose a [version](https://releases.nixos.org/?prefix=nix/) and [system type](../contributing/hacking.md#platforms)
- Choose a [version](https://releases.nixos.org/?prefix=nix/) and [system type](../development/building.md#platforms)
- Download and unpack the tarball
- Run the installer
@@ -39,8 +39,6 @@
`pkgconfig` and the Boehm garbage collector, and pass the flag
`--enable-gc` to `configure`.
For `bdw-gc` <= 8.2.4 Nix needs a [small patch](https://github.com/NixOS/nix/blob/ac4d2e7b857acdfeac35ac8a592bdecee2d29838/boehmgc-traceable_allocator-public.diff) to be applied.
- The `boost` library of version 1.66.0 or higher. It can be obtained
from the official web site <https://www.boost.org/>.
+12 -2
View File
@@ -19,7 +19,7 @@ If you are on Linux with systemd:
Remove files created by Nix:
```console
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix
```
Remove build users and their group:
@@ -43,6 +43,14 @@ which you may remove.
### macOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
1. If system-wide shell initialisation files haven't been altered since installing Nix, use the backups made by the installer:
```console
@@ -133,7 +141,9 @@ which you may remove.
diskutil list
```
If you _do_ find a "Nix Store" volume, delete it by running `diskutil deleteVolume` with the store volume's `diskXsY` identifier.
If you _do_ find a "Nix Store" volume, delete it by running `diskutil apfs deleteVolume` with the store volume's `diskXsY` identifier.
If you get an error that the volume is in use by the kernel, reboot and immediately delete the volume before starting any other process.
> **Note**
>
+19 -12
View File
@@ -113,19 +113,18 @@ Derivations can declare some infrequently used optional attributes.
> `nix-build`.
If the [`configurable-impure-env` experimental
feature](@docroot@/contributing/experimental-features.md#xp-feature-configurable-impure-env)
feature](@docroot@/development/experimental-features.md#xp-feature-configurable-impure-env)
is enabled, these environment variables can also be controlled
through the
[`impure-env`](@docroot@/command-ref/conf-file.md#conf-impure-env)
configuration setting.
- [`outputHash`]{#adv-attr-outputHash}; [`outputHashAlgo`]{#adv-attr-outputHashAlgo}; [`outputHashMode`]{#adv-attr-outputHashMode}\
These attributes declare that the derivation is a so-called
*fixed-output derivation*, which means that a cryptographic hash of
the output is already known in advance. When the build of a
fixed-output derivation finishes, Nix computes the cryptographic
hash of the output and compares it to the hash declared with these
attributes. If there is a mismatch, the build fails.
These attributes declare that the derivation is a so-called *fixed-output derivation* (FOD), which means that a cryptographic hash of the output is already known in advance.
As opposed to regular derivations, the [`builder`] executable of a fixed-output derivation has access to the network.
Nix computes a cryptographic hash of its output and compares that to the hash declared with these attributes.
If there is a mismatch, the derivation fails.
The rationale for fixed-output derivations is derivations such as
those produced by the `fetchurl` function. This function downloads a
@@ -227,7 +226,7 @@ Derivations can declare some infrequently used optional attributes.
- [`__contentAddressed`]{#adv-attr-__contentAddressed}
> **Warning**
> This attribute is part of an [experimental feature](@docroot@/contributing/experimental-features.md).
> This attribute is part of an [experimental feature](@docroot@/development/experimental-features.md).
>
> To use this attribute, you must enable the
> [`ca-derivations`][xp-feature-ca-derivations] experimental feature.
@@ -279,7 +278,9 @@ Derivations can declare some infrequently used optional attributes.
> **Note**
>
> If set to `false`, the [`builder`](./derivations.md#attr-builder) should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
> If set to `false`, the [`builder`] should be able to run on the system type specified in the [`system` attribute](./derivations.md#attr-system), since the derivation cannot be substituted.
[`builder`]: ./derivations.md#attr-builder
- [`__structuredAttrs`]{#adv-attr-structuredAttrs}\
If the special attribute `__structuredAttrs` is set to `true`, the other derivation
@@ -301,6 +302,12 @@ Derivations can declare some infrequently used optional attributes.
(associative) arrays. For example, the attribute `hardening.format = true`
ends up as the Bash associative array element `${hardening[format]}`.
> **Warning**
>
> If set to `true`, other advanced attributes such as [`allowedReferences`](#adv-attr-allowedReferences), [`allowedReferences`](#adv-attr-allowedReferences), [`allowedRequisites`](#adv-attr-allowedRequisites),
[`disallowedReferences`](#adv-attr-disallowedReferences) and [`disallowedRequisites`](#adv-attr-disallowedRequisites), maxSize, and maxClosureSize.
will have no effect.
- [`outputChecks`]{#adv-attr-outputChecks}\
When using [structured attributes](#adv-attr-structuredAttrs), the `outputChecks`
attribute allows defining checks per-output.
@@ -363,6 +370,6 @@ Derivations can declare some infrequently used optional attributes.
ensures that the derivation can only be built on a machine with the `kvm` feature.
[xp-feature-ca-derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
[xp-feature-dynamic-derivations]: @docroot@/contributing/experimental-features.md#xp-feature-dynamic-derivations
[xp-feature-git-hashing]: @docroot@/contributing/experimental-features.md#xp-feature-git-hashing
[xp-feature-ca-derivations]: @docroot@/development/experimental-features.md#xp-feature-ca-derivations
[xp-feature-dynamic-derivations]: @docroot@/development/experimental-features.md#xp-feature-dynamic-derivations
[xp-feature-git-hashing]: @docroot@/development/experimental-features.md#xp-feature-git-hashing
@@ -1,5 +0,0 @@
# Built-in Constants
These constants are built into the Nix language evaluator:
<dl>
@@ -1 +0,0 @@
</dl>
+6 -4
View File
@@ -1,9 +1,11 @@
# Built-in Functions
# Built-ins
This section lists the functions built into the Nix language evaluator.
All built-in functions are available through the global [`builtins`](./builtin-constants.md#builtins-builtins) constant.
This section lists the values and functions built into the Nix language evaluator.
All built-ins are available through the global [`builtins`](#builtins-builtins) constant.
For convenience, some built-ins can be accessed directly:
Some built-ins are also exposed directly in the global scope:
<!-- TODO(@rhendric, #10970): this list is incomplete -->
- [`derivation`](#builtins-derivation)
- [`import`](#builtins-import)
-486
View File
@@ -1,486 +0,0 @@
# Language Constructs
## Recursive sets
Recursive sets are like normal [attribute sets](./values.md#attribute-set), but the attributes can refer to each other.
> *rec-attrset* = `rec {` [ *name* `=` *expr* `;` `]`... `}`
Example:
```nix
rec {
x = y;
y = 123;
}.x
```
This evaluates to `123`.
Note that without `rec` the binding `x = y;` would
refer to the variable `y` in the surrounding scope, if one exists, and
would be invalid if no such variable exists. That is, in a normal
(non-recursive) set, attributes are not added to the lexical scope; in a
recursive set, they are.
Recursive sets of course introduce the danger of infinite recursion. For
example, the expression
```nix
rec {
x = y;
y = x;
}.x
```
will crash with an `infinite recursion encountered` error message.
## Let-expressions
A let-expression allows you to define local variables for an expression.
> *let-in* = `let` [ *identifier* = *expr* ]... `in` *expr*
Example:
```nix
let
x = "foo";
y = "bar";
in x + y
```
This evaluates to `"foobar"`.
## Inheriting attributes
When defining an [attribute set](./values.md#attribute-set) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes).
This can be shortened using the `inherit` keyword.
Example:
```nix
let x = 123; in
{
inherit x;
y = 456;
}
```
is equivalent to
```nix
let x = 123; in
{
x = x;
y = 456;
}
```
and both evaluate to `{ x = 123; y = 456; }`.
> **Note**
>
> This works because `x` is added to the lexical scope by the `let` construct.
It is also possible to inherit attributes from another attribute set.
Example:
In this fragment from `all-packages.nix`,
```nix
graphviz = (import ../tools/graphics/graphviz) {
inherit fetchurl stdenv libpng libjpeg expat x11 yacc;
inherit (xorg) libXaw;
};
xorg = {
libX11 = ...;
libXaw = ...;
...
}
libpng = ...;
libjpg = ...;
...
```
the set used in the function call to the function defined in
`../tools/graphics/graphviz` inherits a number of variables from the
surrounding scope (`fetchurl` ... `yacc`), but also inherits `libXaw`
(the X Athena Widgets) from the `xorg` set.
Summarizing the fragment
```nix
...
inherit x y z;
inherit (src-set) a b c;
...
```
is equivalent to
```nix
...
x = x; y = y; z = z;
a = src-set.a; b = src-set.b; c = src-set.c;
...
```
when used while defining local variables in a let-expression or while
defining a set.
In a `let` expression, `inherit` can be used to selectively bring specific attributes of a set into scope. For example
```nix
let
x = { a = 1; b = 2; };
inherit (builtins) attrNames;
in
{
names = attrNames x;
}
```
is equivalent to
```nix
let
x = { a = 1; b = 2; };
in
{
names = builtins.attrNames x;
}
```
both evaluate to `{ names = [ "a" "b" ]; }`.
## Functions
Functions have the following form:
```nix
pattern: body
```
The pattern specifies what the argument of the function must look like,
and binds variables in the body to (parts of) the argument. There are
three kinds of patterns:
- If a pattern is a single identifier, then the function matches any
argument. Example:
```nix
let negate = x: !x;
concat = x: y: x + y;
in if negate true then concat "foo" "bar" else ""
```
Note that `concat` is a function that takes one argument and returns
a function that takes another argument. This allows partial
parameterisation (i.e., only filling some of the arguments of a
function); e.g.,
```nix
map (concat "foo") [ "bar" "bla" "abc" ]
```
evaluates to `[ "foobar" "foobla" "fooabc" ]`.
- A *set pattern* of the form `{ name1, name2, …, nameN }` matches a
set containing the listed attributes, and binds the values of those
attributes to variables in the function body. For example, the
function
```nix
{ x, y, z }: z + y + x
```
can only be called with a set containing exactly the attributes `x`,
`y` and `z`. No other attributes are allowed. If you want to allow
additional arguments, you can use an ellipsis (`...`):
```nix
{ x, y, z, ... }: z + y + x
```
This works on any set that contains at least the three named
attributes.
It is possible to provide *default values* for attributes, in
which case they are allowed to be missing. A default value is
specified by writing `name ? e`, where *e* is an arbitrary
expression. For example,
```nix
{ x, y ? "foo", z ? "bar" }: z + y + x
```
specifies a function that only requires an attribute named `x`, but
optionally accepts `y` and `z`.
- An `@`-pattern provides a means of referring to the whole value
being matched:
```nix
args@{ x, y, z, ... }: z + y + x + args.a
```
but can also be written as:
```nix
{ x, y, z, ... } @ args: z + y + x + args.a
```
Here `args` is bound to the argument *as passed*, which is further
matched against the pattern `{ x, y, z, ... }`.
The `@`-pattern makes mainly sense with an ellipsis(`...`) as
you can access attribute names as `a`, using `args.a`, which was
given as an additional attribute to the function.
> **Warning**
>
> `args@` binds the name `args` to the attribute set that is passed to the function.
> In particular, `args` does *not* include any default values specified with `?` in the function's set pattern.
>
> For instance
>
> ```nix
> let
> f = args@{ a ? 23, ... }: [ a args ];
> in
> f {}
> ```
>
> is equivalent to
>
> ```nix
> let
> f = args @ { ... }: [ (args.a or 23) args ];
> in
> f {}
> ```
>
> and both expressions will evaluate to:
>
> ```nix
> [ 23 {} ]
> ```
Note that functions do not have names. If you want to give them a name,
you can bind them to an attribute, e.g.,
```nix
let concat = { x, y }: x + y;
in concat { x = "foo"; y = "bar"; }
```
## Conditionals
Conditionals look like this:
```nix
if e1 then e2 else e3
```
where *e1* is an expression that should evaluate to a Boolean value
(`true` or `false`).
## Assertions
Assertions are generally used to check that certain requirements on or
between features and dependencies hold. They look like this:
```nix
assert e1; e2
```
where *e1* is an expression that should evaluate to a Boolean value. If
it evaluates to `true`, *e2* is returned; otherwise expression
evaluation is aborted and a backtrace is printed.
Here is a Nix expression for the Subversion package that shows how
assertions can be used:.
```nix
{ localServer ? false
, httpServer ? false
, sslSupport ? false
, pythonBindings ? false
, javaSwigBindings ? false
, javahlBindings ? false
, stdenv, fetchurl
, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null
}:
assert localServer -> db4 != null; ①
assert httpServer -> httpd != null && httpd.expat == expat; ②
assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); ③
assert pythonBindings -> swig != null && swig.pythonSupport;
assert javaSwigBindings -> swig != null && swig.javaSupport;
assert javahlBindings -> j2sdk != null;
stdenv.mkDerivation {
name = "subversion-1.1.1";
...
openssl = if sslSupport then openssl else null; ④
...
}
```
The points of interest are:
1. This assertion states that if Subversion is to have support for
local repositories, then Berkeley DB is needed. So if the Subversion
function is called with the `localServer` argument set to `true` but
the `db4` argument set to `null`, then the evaluation fails.
Note that `->` is the [logical
implication](https://en.wikipedia.org/wiki/Truth_table#Logical_implication)
Boolean operation.
2. This is a more subtle condition: if Subversion is built with Apache
(`httpServer`) support, then the Expat library (an XML library) used
by Subversion should be same as the one used by Apache. This is
because in this configuration Subversion code ends up being linked
with Apache code, and if the Expat libraries do not match, a build-
or runtime link error or incompatibility might occur.
3. This assertion says that in order for Subversion to have SSL support
(so that it can access `https` URLs), an OpenSSL library must be
passed. Additionally, it says that *if* Apache support is enabled,
then Apache's OpenSSL should match Subversion's. (Note that if
Apache support is not enabled, we don't care about Apache's
OpenSSL.)
4. The conditional here is not really related to assertions, but is
worth pointing out: it ensures that if SSL support is disabled, then
the Subversion derivation is not dependent on OpenSSL, even if a
non-`null` value was passed. This prevents an unnecessary rebuild of
Subversion if OpenSSL changes.
## With-expressions
A *with-expression*,
```nix
with e1; e2
```
introduces the set *e1* into the lexical scope of the expression *e2*.
For instance,
```nix
let as = { x = "foo"; y = "bar"; };
in with as; x + y
```
evaluates to `"foobar"` since the `with` adds the `x` and `y` attributes
of `as` to the lexical scope in the expression `x + y`. The most common
use of `with` is in conjunction with the `import` function. E.g.,
```nix
with (import ./definitions.nix); ...
```
makes all attributes defined in the file `definitions.nix` available as
if they were defined locally in a `let`-expression.
The bindings introduced by `with` do not shadow bindings introduced by
other means, e.g.
```nix
let a = 3; in with { a = 1; }; let a = 4; in with { a = 2; }; ...
```
establishes the same scope as
```nix
let a = 1; in let a = 2; in let a = 3; in let a = 4; in ...
```
Variables coming from outer `with` expressions *are* shadowed:
```nix
with { a = "outer"; };
with { a = "inner"; };
a
```
Does evaluate to `"inner"`.
## Comments
- Inline comments start with `#` and run until the end of the line.
> **Example**
>
> ```nix
> # A number
> 2 # Equals 1 + 1
> ```
>
> ```console
> 2
> ```
- Block comments start with `/*` and run until the next occurrence of `*/`.
> **Example**
>
> ```nix
> /*
> Block comments
> can span multiple lines.
> */ "hello"
> ```
>
> ```console
> "hello"
> ```
This means that block comments cannot be nested.
> **Example**
>
> ```nix
> /* /* nope */ */ 1
> ```
>
> ```console
> error: syntax error, unexpected '*'
>
> at «string»:1:15:
>
> 1| /* /* nope */ *
> | ^
> ```
Consider escaping nested comments and unescaping them in post-processing.
> **Example**
>
> ```nix
> /* /* nested *\/ */ 1
> ```
>
> ```console
> 1
> ```
## Scoping rules
Nix is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope), but with multiple scopes and shadowing rules.
* primary scope --- explicitly-bound variables
* [`let`](#let-expressions)
* [`inherit`](#inheriting-attributes)
* function arguments
* secondary scope --- implicitly-bound variables
* [`with`](#with-expressions)
Primary scope takes precedence over secondary scope.
See [`with`](#with-expressions) for a detailed example.
@@ -4,9 +4,9 @@
>
> *lookup-path* = `<` *identifier* [ `/` *identifier* ]... `>`
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/values.md#type-path) if the identifier matches a search path entry.
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/types.md#type-path) if the identifier matches a search path entry.
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath).
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtins.md#builtins-nixPath).
See [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile) for details on lookup path resolution.
+6 -6
View File
@@ -12,7 +12,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
### Required
- [`name`]{#attr-name} ([String](@docroot@/language/values.md#type-string))
- [`name`]{#attr-name} ([String](@docroot@/language/types.md#type-string))
A symbolic name for the derivation.
It is added to the [store path] of the corresponding [store derivation] as well as to its [output paths](@docroot@/glossary.md#gloss-output-path).
@@ -31,7 +31,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> The store derivation's path will be `/nix/store/<hash>-hello.drv`.
> The [output](#attr-outputs) paths will be of the form `/nix/store/<hash>-hello[-<output>]`
- [`system`]{#attr-system} ([String](@docroot@/language/values.md#type-string))
- [`system`]{#attr-system} ([String](@docroot@/language/types.md#type-string))
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
@@ -64,9 +64,9 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> }
> ```
>
> [`builtins.currentSystem`](@docroot@/language/builtin-constants.md#builtins-currentSystem) has the value of the [`system` configuration option], and defaults to the system type of the current Nix installation.
> [`builtins.currentSystem`](@docroot@/language/builtins.md#builtins-currentSystem) has the value of the [`system` configuration option], and defaults to the system type of the current Nix installation.
- [`builder`]{#attr-builder} ([Path](@docroot@/language/values.md#type-path) | [String](@docroot@/language/values.md#type-string))
- [`builder`]{#attr-builder} ([Path](@docroot@/language/types.md#type-path) | [String](@docroot@/language/types.md#type-string))
Path to an executable that will perform the build.
@@ -113,7 +113,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
### Optional
- [`args`]{#attr-args} ([List](@docroot@/language/values.md#list) of [String](@docroot@/language/values.md#type-string))
- [`args`]{#attr-args} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
Default: `[ ]`
@@ -132,7 +132,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> };
> ```
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/values.md#list) of [String](@docroot@/language/values.md#type-string))
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
Default: `[ "out" ]`
+50
View File
@@ -0,0 +1,50 @@
# Identifiers
An *identifier* is an [ASCII](https://en.wikipedia.org/wiki/ASCII) character sequence that:
- Starts with a letter (`a-z`, `A-Z`) or underscore (`_`)
- Can contain any number of:
- Letters (`a-z`, `A-Z`)
- Digits (`0-9`)
- Underscores (`_`)
- Apostrophes (`'`)
- Hyphens (`-`)
- Is not one of the [keywords](#keywords)
> **Syntax**
>
> *identifier* ~ `[A-Za-z_][A-Za-z0-9_'-]*`
# Names
A name can be an [identifier](#identifier) or a [string literal](./syntax.md#string-literal).
> **Syntax**
>
> *name* → *identifier* | *string*
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting attributes).
# Keywords
These keywords are reserved and cannot be used as [identifiers](#identifiers):
- [`assert`](./syntax.md#assertions)
- [`else`][if]
- [`if`][if]
- [`in`][let]
- [`inherit`](./syntax.md#inheriting-attributes)
- [`let`][let]
- [`or`](./operators.md#attribute-selection) (see note)
- [`rec`](./syntax.md#recursive-sets)
- [`then`][if]
- [`with`](./syntax.md#with-expressions)
[if]: ./syntax.md#conditionals
[let]: ./syntax.md#let-expressions
> **Note**
>
> The Nix language evaluator currently allows `or` to be used as a name in some contexts, for backwards compatibility reasons.
> Users are advised not to rely on this.
>
> There are long-standing issues with how `or` is parsed as a name, which can't be resolved without making a breaking change to the language.
+30 -30
View File
@@ -53,7 +53,7 @@ This is an incomplete overview of language features, by example.
<td>
*Basic values ([primitives](@docroot@/language/values.md#primitives))*
*Basic values ([primitives](@docroot@/language/types.md#primitives))*
</td>
@@ -71,7 +71,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [string](@docroot@/language/values.md#type-string)
A [string](@docroot@/language/types.md#type-string)
</td>
</tr>
@@ -102,7 +102,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [comment](@docroot@/language/constructs.md#comments).
A [comment](@docroot@/language/syntax.md#comments).
</td>
</tr>
@@ -130,7 +130,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Booleans](@docroot@/language/values.md#type-boolean)
[Booleans](@docroot@/language/types.md#type-boolean)
</td>
</tr>
@@ -142,7 +142,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Null](@docroot@/language/values.md#type-null) value
[Null](@docroot@/language/types.md#type-null) value
</td>
</tr>
@@ -154,7 +154,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
An [integer](@docroot@/language/values.md#type-number)
An [integer](@docroot@/language/types.md#type-int)
</td>
</tr>
@@ -166,7 +166,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [floating point number](@docroot@/language/values.md#type-number)
A [floating point number](@docroot@/language/types.md#type-float)
</td>
</tr>
@@ -178,7 +178,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
An absolute [path](@docroot@/language/values.md#type-path)
An absolute [path](@docroot@/language/types.md#type-path)
</td>
</tr>
@@ -190,7 +190,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [path](@docroot@/language/values.md#type-path) relative to the file containing this Nix expression
A [path](@docroot@/language/types.md#type-path) relative to the file containing this Nix expression
</td>
</tr>
@@ -202,7 +202,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A home [path](@docroot@/language/values.md#type-path). Evaluates to the `"<user's home directory>/.config"`.
A home [path](@docroot@/language/types.md#type-path). Evaluates to the `"<user's home directory>/.config"`.
</td>
</tr>
@@ -238,7 +238,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
An [attribute set](@docroot@/language/values.md#attribute-set) with attributes named `x` and `y`
An [attribute set](@docroot@/language/types.md#attribute-set) with attributes named `x` and `y`
</td>
</tr>
@@ -262,7 +262,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [recursive set](@docroot@/language/constructs.md#recursive-sets), equivalent to `{ x = "foo"; y = "foobar"; }`.
A [recursive set](@docroot@/language/syntax.md#recursive-sets), equivalent to `{ x = "foo"; y = "foobar"; }`.
</td>
</tr>
@@ -278,7 +278,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Lists](@docroot@/language/values.md#list) with three elements.
[Lists](@docroot@/language/types.md#list) with three elements.
</td>
</tr>
@@ -362,7 +362,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Attribute selection](@docroot@/language/values.md#attribute-set) (evaluates to `1`)
[Attribute selection](@docroot@/language/types.md#attribute-set) (evaluates to `1`)
</td>
</tr>
@@ -374,7 +374,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Attribute selection](@docroot@/language/values.md#attribute-set) with default (evaluates to `3`)
[Attribute selection](@docroot@/language/types.md#attribute-set) with default (evaluates to `3`)
</td>
</tr>
@@ -410,7 +410,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Conditional expression](@docroot@/language/constructs.md#conditionals).
[Conditional expression](@docroot@/language/syntax.md#conditionals).
</td>
</tr>
@@ -422,7 +422,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
[Assertion](@docroot@/language/constructs.md#assertions) check (evaluates to `"yes!"`).
[Assertion](@docroot@/language/syntax.md#assertions) check (evaluates to `"yes!"`).
</td>
</tr>
@@ -434,7 +434,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
Variable definition. See [`let`-expressions](@docroot@/language/constructs.md#let-expressions).
Variable definition. See [`let`-expressions](@docroot@/language/syntax.md#let-expressions).
</td>
</tr>
@@ -448,7 +448,7 @@ This is an incomplete overview of language features, by example.
Add all attributes from the given set to the scope (evaluates to `1`).
See [`with`-expressions](@docroot@/language/constructs.md#with-expressions) for details and shadowing caveats.
See [`with`-expressions](@docroot@/language/syntax.md#with-expressions) for details and shadowing caveats.
</td>
</tr>
@@ -462,7 +462,7 @@ This is an incomplete overview of language features, by example.
Adds the variables to the current scope (attribute set or `let` binding).
Desugars to `pkgs = pkgs; src = src;`.
See [Inheriting attributes](@docroot@/language/constructs.md#inheriting-attributes).
See [Inheriting attributes](@docroot@/language/syntax.md#inheriting-attributes).
</td>
</tr>
@@ -476,14 +476,14 @@ This is an incomplete overview of language features, by example.
Adds the attributes, from the attribute set in parentheses, to the current scope (attribute set or `let` binding).
Desugars to `lib = pkgs.lib; stdenv = pkgs.stdenv;`.
See [Inheriting attributes](@docroot@/language/constructs.md#inheriting-attributes).
See [Inheriting attributes](@docroot@/language/syntax.md#inheriting-attributes).
</td>
</tr>
<tr>
<td>
*[Functions](@docroot@/language/constructs.md#functions) (lambdas)*
*[Functions](@docroot@/language/syntax.md#functions) (lambdas)*
</td>
<td>
@@ -500,7 +500,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects an integer and returns it increased by 1.
A [function](@docroot@/language/syntax.md#functions) that expects an integer and returns it increased by 1.
</td>
</tr>
@@ -512,7 +512,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
Curried [function](@docroot@/language/constructs.md#functions), equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum.
Curried [function](@docroot@/language/syntax.md#functions), equivalent to `x: (y: x + y)`. Can be used like a function that takes two arguments and returns their sum.
</td>
</tr>
@@ -524,7 +524,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) call (evaluates to 101)
A [function](@docroot@/language/syntax.md#functions) call (evaluates to 101)
</td>
</tr>
@@ -536,7 +536,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) bound to a variable and subsequently called by name (evaluates to 103)
A [function](@docroot@/language/syntax.md#functions) bound to a variable and subsequently called by name (evaluates to 103)
</td>
</tr>
@@ -548,7 +548,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y` and concatenates them
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and concatenates them
</td>
</tr>
@@ -560,7 +560,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y`
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attribute `x` and optional `y`, using `"bar"` as default value for `y`
</td>
</tr>
@@ -572,7 +572,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y` and ignores any other attributes
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y` and ignores any other attributes
</td>
</tr>
@@ -586,7 +586,7 @@ This is an incomplete overview of language features, by example.
</td>
<td>
A [function](@docroot@/language/constructs.md#functions) that expects a set with required attributes `x` and `y`, and binds the whole set to `args`
A [function](@docroot@/language/syntax.md#functions) that expects a set with required attributes `x` and `y`, and binds the whole set to `args`
</td>
</tr>
+58 -11
View File
@@ -3,7 +3,7 @@
| Name | Syntax | Associativity | Precedence |
|----------------------------------------|--------------------------------------------|---------------|------------|
| [Attribute selection] | *attrset* `.` *attrpath* \[ `or` *expr* \] | none | 1 |
| Function application | *func* *expr* | left | 2 |
| [Function application] | *func* *expr* | left | 2 |
| [Arithmetic negation][arithmetic] | `-` *number* | none | 3 |
| [Has attribute] | *attrset* `?` *attrpath* | none | 4 |
| List concatenation | *list* `++` *list* | right | 5 |
@@ -26,12 +26,16 @@
| Logical conjunction (`AND`) | *bool* `&&` *bool* | left | 12 |
| Logical disjunction (`OR`) | *bool* <code>\|\|</code> *bool* | left | 13 |
| [Logical implication] | *bool* `->` *bool* | right | 14 |
| [Pipe operator] (experimental) | *expr* `\|>` *func* | left | 15 |
| [Pipe operator] (experimental) | *func* `<\|` *expr* | right | 15 |
[string]: ./values.md#type-string
[path]: ./values.md#type-path
[number]: ./values.md#type-number
[list]: ./values.md#list
[attribute set]: ./values.md#attribute-set
[string]: ./types.md#type-string
[path]: ./types.md#type-path
[number]: ./types.md#type-float
[list]: ./types.md#type-list
[attribute set]: ./types.md#type-attrs
<!-- TODO(@rhendric, #10970): ^ rationalize number -> int/float -->
## Attribute selection
@@ -42,13 +46,23 @@
Select the attribute denoted by attribute path *attrpath* from [attribute set] *attrset*.
If the attribute doesn’t exist, return the *expr* after `or` if provided, otherwise abort evaluation.
An attribute path is a dot-separated list of [attribute names](./values.md#attribute-set).
[Attribute selection]: #attribute-selection
## Function application
> **Syntax**
>
> *attrpath* = *name* [ `.` *name* ]...
> *func* *expr*
[Attribute selection]: #attribute-selection
Apply the callable value *func* to the argument *expr*. Note the absence of any visible operator symbol.
A callable value is either:
- a [user-defined function][function]
- a [built-in][builtins] function
- an attribute set with a [`__functor` attribute](./syntax.md#attr-__functor)
> **Warning**
>
> [List][list] items are also separated by whitespace, which means that function calls in list items must be enclosed by parentheses.
## Has attribute
@@ -61,7 +75,7 @@ The result is a [Boolean] value.
See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr)
[Boolean]: ./values.md#type-boolean
[Boolean]: ./types.md#type-bool
[Has attribute]: #has-attribute
@@ -172,7 +186,7 @@ All comparison operators are implemented in terms of `<`, and the following equi
- Numbers are type-compatible, see [arithmetic] operators.
- Floating point numbers only differ up to a limited precision.
[function]: ./constructs.md#functions
[function]: ./syntax.md#functions
[Equality]: #equality
@@ -182,3 +196,36 @@ Equivalent to `!`*b1* `||` *b2*.
[Logical implication]: #logical-implication
## Pipe operators
- *a* `|>` *b* is equivalent to *b* *a*
- *a* `<|` *b* is equivalent to *a* *b*
> **Example**
>
> ```
> nix-repl> 1 |> builtins.add 2 |> builtins.mul 3
> 9
>
> nix-repl> builtins.add 1 <| builtins.mul 2 <| 3
> 7
> ```
> **Warning**
>
> This syntax is part of an
> [experimental feature](@docroot@/development/experimental-features.md)
> and may change in future releases.
>
> To use this syntax, make sure the
> [`pipe-operators` experimental feature](@docroot@/development/experimental-features.md#xp-feature-pipe-operators)
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = pipe-operators
> ```
[Pipe operator]: #pipe-operators
[builtins]: ./builtins.md
[Function application]: #function-application
+14
View File
@@ -0,0 +1,14 @@
# Scoping rules
Nix is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope), but with multiple scopes and shadowing rules.
* primary scope: explicitly-bound variables
* [`let`](./syntax.md#let-expressions)
* [`inherit`](./syntax.md#inheriting-attributes)
* [function](./syntax.md#functions) arguments
* secondary scope: implicitly-bound variables
* [`with`](./syntax.md#with-expressions)
Primary scope takes precedence over secondary scope.
See [`with`](./syntax.md#with-expressions) for a detailed example.
+1 -1
View File
@@ -111,7 +111,7 @@ It creates an [attribute set] representing the string context, which can be insp
[`builtins.hasContext`]: ./builtins.md#builtins-hasContext
[`builtins.getContext`]: ./builtins.md#builtins-getContext
[attribute set]: ./values.md#attribute-set
[attribute set]: ./types.md#attribute-set
## Clearing string contexts
@@ -4,9 +4,9 @@ String interpolation is a language feature where a [string], [path], or [attribu
Such a construct is called *interpolated string*, and the expression inside is an [interpolated expression](#interpolated-expression).
[string]: ./values.md#type-string
[path]: ./values.md#type-path
[attribute set]: ./values.md#attribute-set
[string]: ./types.md#type-string
[path]: ./types.md#type-path
[attribute set]: ./types.md#attribute-set
## Examples
@@ -43,6 +43,47 @@ configureFlags = "
Note that Nix expressions and strings can be arbitrarily nested;
in this case the outer string contains various interpolated expressions that themselves contain strings (e.g., `"-thread"`), some of which in turn contain interpolated expressions (e.g., `${mesa}`).
To write a literal `${` in an regular string, escape it with a backslash (`\`).
> **Example**
>
> ```nix
> "echo \${PATH}"
> ```
>
> "echo ${PATH}"
To write a literal `${` in an indented string, escape it with two single quotes (`''`).
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
`$${` can be written literally in any string.
> **Example**
>
> In Make, `$` in file names or recipes is represented as `$$`, see [GNU `make`: Basics of Variable Reference](https://www.gnu.org/software/make/manual/html_node/Reference.html#Basics-of-Variable-References).
> This can be expressed directly in the Nix language strings:
>
> ```nix
> ''
> MAKEVAR = Hello
> all:
> @export BASHVAR=world; echo $(MAKEVAR) $${BASHVAR}
> ''
> ```
>
> "MAKEVAR = Hello\nall:\n\t@export BASHVAR=world; echo $(MAKEVAR) $\${BASHVAR}\n"
See the [documentation on strings][string] for details.
### Path
Rather than writing
+866
View File
@@ -0,0 +1,866 @@
# Language Constructs
This section covers syntax and semantics of the Nix language.
## Basic Literals
### String {#string-literal}
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes, e.g., `"foo bar"`.
Strings can span multiple lines.
The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation].
[string interpolation]: ./string-interpolation.md
The following must be escaped to represent them within a string, by prefixing with a backslash (`\`):
- Double quote (`"`)
> **Example**
>
> ```nix
> "\""
> ```
>
> "\""
- Backslash (`\`)
> **Example**
>
> ```nix
> "\\"
> ```
>
> "\\"
- Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly"
> **Example**
>
> ```nix
> "\${"
> ```
>
> "\${"
The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> "$${"
> ```
>
> "$\${"
String values are output on the terminal with Nix-specific escaping.
Strings written to files will contain the characters encoded by the escaping.
The second way to write string literals is as an *indented string*, which is enclosed between pairs of *double single-quotes* (`''`), like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
> **Note**
>
> Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line.
> **Warning**
>
> Prefixed tab characters are not stripped.
>
> > **Example**
> >
> > The following indented string is prefixed with tabs:
> >
> > ''
> > all:
> > @echo hello
> > ''
> >
> > "\tall:\n\t\t@echo hello\n"
Indented strings support [string interpolation].
The following must be escaped to represent them in an indented string:
- `$` is escaped by prefixing it with two single quotes (`''`)
> **Example**
>
> ```nix
> ''
> ''$
> ''
> ```
>
> "$\n"
- `''` is escaped by prefixing it with one single quote (`'`)
> **Example**
>
> ```nix
> ''
> '''
> ''
> ```
>
> "''\n"
These special characters are escaped as follows:
- Linefeed (`\n`): `''\n`
- Carriage return (`\r`): `''\r`
- Tab (`\t`): `''\t`
`''\` escapes any other character.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> ''
> $${
> ''
> ```
>
> "$\${\n"
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
### Number {#number-literal}
<!-- TODO(@rhendric, #10970): split this into int and float -->
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
[comparison]: ./operators.md#comparison
### Path {#path-literal}
*Paths* can be expressed by path literals such as `./builder.sh`.
A path literal must contain at least one slash to be recognised as such.
For instance, `builder.sh` is not a path:
it's parsed as an expression that selects the attribute `sh` from the variable `builder`.
Path literals are resolved relative to their [base directory](@docroot@/glossary.md#gloss-base-directory).
Path literals may also refer to absolute paths by starting with a slash.
> **Note**
>
> Absolute paths make expressions less portable.
> In the case where a function translates a path literal into an absolute path string for a configuration file, it is recommended to write a string literal instead.
> This avoids some confusion about whether files at that location will be used during evaluation.
> It also avoids unintentional situations where some function might try to copy everything at the location into the store.
If the first component of a path is a `~`, it is interpreted such that the rest of the path were relative to the user's home directory.
For example, `~/foo` would be equivalent to `/home/edolstra/foo` for a user whose home directory is `/home/edolstra`.
Path literals that start with `~` are not allowed in [pure](@docroot@/command-ref/conf-file.md#conf-pure-eval) evaluation.
Path literals can also include [string interpolation], besides being [interpolated into other expressions].
[interpolated into other expressions]: ./string-interpolation.md#interpolated-expressions
At least one slash (`/`) must appear *before* any interpolated expression for the result to be recognized as a path.
`a.${foo}/b.${bar}` is a syntactically valid number division operation.
`./a.${foo}/b.${bar}` is a path.
[Lookup path](./constructs/lookup-path.md) literals such as `<nixpkgs>` also resolve to path values.
## List {#list-literal}
Lists are formed by enclosing a whitespace-separated list of values
between square brackets. For example,
```nix
[ 123 ./foo.nix "abc" (f { x = y; }) ]
```
defines a list of four elements, the last being the result of a call to
the function `f`. Note that function calls have to be enclosed in
parentheses. If they had been omitted, e.g.,
```nix
[ 123 ./foo.nix "abc" f { x = y; } ]
```
the result would be a list of five elements, the fourth one being a
function and the fifth being a set.
Note that lists are only lazy in values, and they are strict in length.
Elements in a list can be accessed using [`builtins.elemAt`](./builtins.md#builtins-elemAt).
## Attribute Set {#attrs-literal}
An attribute set is a collection of name-value-pairs called *attributes*.
Attribute sets are written enclosed in curly brackets (`{ }`).
Attribute names and attribute values are separated by an equal sign (`=`).
Each value can be an arbitrary expression, terminated by a semicolon (`;`)
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](#string-literal)).
[name]: ./identifiers.md#names
> **Syntax**
>
> *attrset* → `{` { *name* `=` *expr* `;` } `}`
Attributes can appear in any order.
An attribute name may only occur once in each attribute set.
> **Example**
>
> This defines an attribute set with attributes named:
> - `x` with the value `123`, an integer
> - `text` with the value `"Hello"`, a string
> - `y` where the value is the result of applying the function `f` to the attribute set `{ bla = 456; }`
>
> ```nix
> {
> x = 123;
> text = "Hello";
> y = f { bla = 456; };
> }
> ```
Attributes in nested attribute sets can be written using *attribute paths*.
> **Syntax**
>
> *attrset* → `{` { *attrpath* `=` *expr* `;` } `}`
An attribute path is a dot-separated list of [names][name].
> **Syntax**
>
> *attrpath* = *name* { `.` *name* }
<!-- -->
> **Example**
>
> ```nix
> { a.b.c = 1; a.b.d = 2; }
> ```
>
> {
> a = {
> b = {
> c = 1;
> d = 2;
> };
> };
> }
Attribute names can also be set implicitly by using the [`inherit` keyword](#inheriting-attributes).
> **Example**
>
> ```nix
> { inherit (builtins) true; }
> ```
>
> { true = true; }
Attributes can be accessed with the [`.` operator](./operators.md#attribute-selection).
Example:
```nix
{ a = "Foo"; b = "Bar"; }.a
```
This evaluates to `"Foo"`.
It is possible to provide a default value in an attribute selection using the `or` keyword.
Example:
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
```
```nix
{ a = "Foo"; b = "Bar"; }.c.d.e.f.g or "Xyzzy"
```
will both evaluate to `"Xyzzy"` because there is no `c` attribute in the set.
You can use arbitrary double-quoted strings as attribute names:
```nix
{ "$!@#?" = 123; }."$!@#?"
```
```nix
let bar = "bar"; in
{ "foo ${bar}" = 123; }."foo ${bar}"
```
Both will evaluate to `123`.
Attribute names support [string interpolation]:
```nix
let bar = "foo"; in
{ foo = 123; }.${bar}
```
```nix
let bar = "foo"; in
{ ${bar} = 123; }.foo
```
Both will evaluate to `123`.
In the special case where an attribute name inside of a set declaration
evaluates to `null` (which is normally an error, as `null` cannot be coerced to
a string), that attribute is simply not added to the set:
```nix
{ ${if foo then "bar" else null} = true; }
```
This will evaluate to `{}` if `foo` evaluates to `false`.
A set that has a [`__functor`]{#attr-__functor} attribute whose value is callable (i.e. is
itself a function or a set with a `__functor` attribute whose value is
callable) can be applied as if it were a function, with the set itself
passed in first , e.g.,
```nix
let add = { __functor = self: x: x + self.x; };
inc = add // { x = 1; };
in inc 1
```
evaluates to `2`. This can be used to attach metadata to a function
without the caller needing to treat it specially, or to implement a form
of object-oriented programming, for example.
## Recursive sets
Recursive sets are like normal [attribute sets](./types.md#attribute-set), but the attributes can refer to each other.
> *rec-attrset* = `rec {` [ *name* `=` *expr* `;` `]`... `}`
Example:
```nix
rec {
x = y;
y = 123;
}.x
```
This evaluates to `123`.
Note that without `rec` the binding `x = y;` would
refer to the variable `y` in the surrounding scope, if one exists, and
would be invalid if no such variable exists. That is, in a normal
(non-recursive) set, attributes are not added to the lexical scope; in a
recursive set, they are.
Recursive sets of course introduce the danger of infinite recursion. For
example, the expression
```nix
rec {
x = y;
y = x;
}.x
```
will crash with an `infinite recursion encountered` error message.
## Let-expressions
A let-expression allows you to define local variables for an expression.
> *let-in* = `let` [ *identifier* = *expr* ]... `in` *expr*
Example:
```nix
let
x = "foo";
y = "bar";
in x + y
```
This evaluates to `"foobar"`.
## Inheriting attributes
When defining an [attribute set](./types.md#attribute-set) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes).
This can be shortened using the `inherit` keyword.
Example:
```nix
let x = 123; in
{
inherit x;
y = 456;
}
```
is equivalent to
```nix
let x = 123; in
{
x = x;
y = 456;
}
```
and both evaluate to `{ x = 123; y = 456; }`.
> **Note**
>
> This works because `x` is added to the lexical scope by the `let` construct.
It is also possible to inherit attributes from another attribute set.
Example:
In this fragment from `all-packages.nix`,
```nix
graphviz = (import ../tools/graphics/graphviz) {
inherit fetchurl stdenv libpng libjpeg expat x11 yacc;
inherit (xorg) libXaw;
};
xorg = {
libX11 = ...;
libXaw = ...;
...
}
libpng = ...;
libjpg = ...;
...
```
the set used in the function call to the function defined in
`../tools/graphics/graphviz` inherits a number of variables from the
surrounding scope (`fetchurl` ... `yacc`), but also inherits `libXaw`
(the X Athena Widgets) from the `xorg` set.
Summarizing the fragment
```nix
...
inherit x y z;
inherit (src-set) a b c;
...
```
is equivalent to
```nix
...
x = x; y = y; z = z;
a = src-set.a; b = src-set.b; c = src-set.c;
...
```
when used while defining local variables in a let-expression or while
defining a set.
In a `let` expression, `inherit` can be used to selectively bring specific attributes of a set into scope. For example
```nix
let
x = { a = 1; b = 2; };
inherit (builtins) attrNames;
in
{
names = attrNames x;
}
```
is equivalent to
```nix
let
x = { a = 1; b = 2; };
in
{
names = builtins.attrNames x;
}
```
both evaluate to `{ names = [ "a" "b" ]; }`.
## Functions
Functions have the following form:
```nix
pattern: body
```
The pattern specifies what the argument of the function must look like,
and binds variables in the body to (parts of) the argument. There are
three kinds of patterns:
- If a pattern is a single identifier, then the function matches any
argument. Example:
```nix
let negate = x: !x;
concat = x: y: x + y;
in if negate true then concat "foo" "bar" else ""
```
Note that `concat` is a function that takes one argument and returns
a function that takes another argument. This allows partial
parameterisation (i.e., only filling some of the arguments of a
function); e.g.,
```nix
map (concat "foo") [ "bar" "bla" "abc" ]
```
evaluates to `[ "foobar" "foobla" "fooabc" ]`.
- A *set pattern* of the form `{ name1, name2, …, nameN }` matches a
set containing the listed attributes, and binds the values of those
attributes to variables in the function body. For example, the
function
```nix
{ x, y, z }: z + y + x
```
can only be called with a set containing exactly the attributes `x`,
`y` and `z`. No other attributes are allowed. If you want to allow
additional arguments, you can use an ellipsis (`...`):
```nix
{ x, y, z, ... }: z + y + x
```
This works on any set that contains at least the three named
attributes.
It is possible to provide *default values* for attributes, in
which case they are allowed to be missing. A default value is
specified by writing `name ? e`, where *e* is an arbitrary
expression. For example,
```nix
{ x, y ? "foo", z ? "bar" }: z + y + x
```
specifies a function that only requires an attribute named `x`, but
optionally accepts `y` and `z`.
- An `@`-pattern provides a means of referring to the whole value
being matched:
```nix
args@{ x, y, z, ... }: z + y + x + args.a
```
but can also be written as:
```nix
{ x, y, z, ... } @ args: z + y + x + args.a
```
Here `args` is bound to the argument *as passed*, which is further
matched against the pattern `{ x, y, z, ... }`.
The `@`-pattern makes mainly sense with an ellipsis(`...`) as
you can access attribute names as `a`, using `args.a`, which was
given as an additional attribute to the function.
> **Warning**
>
> `args@` binds the name `args` to the attribute set that is passed to the function.
> In particular, `args` does *not* include any default values specified with `?` in the function's set pattern.
>
> For instance
>
> ```nix
> let
> f = args@{ a ? 23, ... }: [ a args ];
> in
> f {}
> ```
>
> is equivalent to
>
> ```nix
> let
> f = args @ { ... }: [ (args.a or 23) args ];
> in
> f {}
> ```
>
> and both expressions will evaluate to:
>
> ```nix
> [ 23 {} ]
> ```
Note that functions do not have names. If you want to give them a name,
you can bind them to an attribute, e.g.,
```nix
let concat = { x, y }: x + y;
in concat { x = "foo"; y = "bar"; }
```
## Conditionals
Conditionals look like this:
```nix
if e1 then e2 else e3
```
where *e1* is an expression that should evaluate to a Boolean value
(`true` or `false`).
## Assertions
Assertions are generally used to check that certain requirements on or
between features and dependencies hold. They look like this:
```nix
assert e1; e2
```
where *e1* is an expression that should evaluate to a Boolean value. If
it evaluates to `true`, *e2* is returned; otherwise expression
evaluation is aborted and a backtrace is printed.
Here is a Nix expression for the Subversion package that shows how
assertions can be used:.
```nix
{ localServer ? false
, httpServer ? false
, sslSupport ? false
, pythonBindings ? false
, javaSwigBindings ? false
, javahlBindings ? false
, stdenv, fetchurl
, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null, j2sdk ? null
}:
assert localServer -> db4 != null; ①
assert httpServer -> httpd != null && httpd.expat == expat; ②
assert sslSupport -> openssl != null && (httpServer -> httpd.openssl == openssl); ③
assert pythonBindings -> swig != null && swig.pythonSupport;
assert javaSwigBindings -> swig != null && swig.javaSupport;
assert javahlBindings -> j2sdk != null;
stdenv.mkDerivation {
name = "subversion-1.1.1";
...
openssl = if sslSupport then openssl else null; ④
...
}
```
The points of interest are:
1. This assertion states that if Subversion is to have support for
local repositories, then Berkeley DB is needed. So if the Subversion
function is called with the `localServer` argument set to `true` but
the `db4` argument set to `null`, then the evaluation fails.
Note that `->` is the [logical
implication](https://en.wikipedia.org/wiki/Truth_table#Logical_implication)
Boolean operation.
2. This is a more subtle condition: if Subversion is built with Apache
(`httpServer`) support, then the Expat library (an XML library) used
by Subversion should be same as the one used by Apache. This is
because in this configuration Subversion code ends up being linked
with Apache code, and if the Expat libraries do not match, a build-
or runtime link error or incompatibility might occur.
3. This assertion says that in order for Subversion to have SSL support
(so that it can access `https` URLs), an OpenSSL library must be
passed. Additionally, it says that *if* Apache support is enabled,
then Apache's OpenSSL should match Subversion's. (Note that if
Apache support is not enabled, we don't care about Apache's
OpenSSL.)
4. The conditional here is not really related to assertions, but is
worth pointing out: it ensures that if SSL support is disabled, then
the Subversion derivation is not dependent on OpenSSL, even if a
non-`null` value was passed. This prevents an unnecessary rebuild of
Subversion if OpenSSL changes.
## With-expressions
A *with-expression*,
```nix
with e1; e2
```
introduces the set *e1* into the lexical scope of the expression *e2*.
For instance,
```nix
let as = { x = "foo"; y = "bar"; };
in with as; x + y
```
evaluates to `"foobar"` since the `with` adds the `x` and `y` attributes
of `as` to the lexical scope in the expression `x + y`. The most common
use of `with` is in conjunction with the `import` function. E.g.,
```nix
with (import ./definitions.nix); ...
```
makes all attributes defined in the file `definitions.nix` available as
if they were defined locally in a `let`-expression.
The bindings introduced by `with` do not shadow bindings introduced by
other means, e.g.
```nix
let a = 3; in with { a = 1; }; let a = 4; in with { a = 2; }; ...
```
establishes the same scope as
```nix
let a = 1; in let a = 2; in let a = 3; in let a = 4; in ...
```
Variables coming from outer `with` expressions *are* shadowed:
```nix
with { a = "outer"; };
with { a = "inner"; };
a
```
Does evaluate to `"inner"`.
## Comments
- Inline comments start with `#` and run until the end of the line.
> **Example**
>
> ```nix
> # A number
> 2 # Equals 1 + 1
> ```
>
> ```console
> 2
> ```
- Block comments start with `/*` and run until the next occurrence of `*/`.
> **Example**
>
> ```nix
> /*
> Block comments
> can span multiple lines.
> */ "hello"
> ```
>
> ```console
> "hello"
> ```
This means that block comments cannot be nested.
> **Example**
>
> ```nix
> /* /* nope */ */ 1
> ```
>
> ```console
> error: syntax error, unexpected '*'
>
> at «string»:1:15:
>
> 1| /* /* nope */ *
> | ^
> ```
Consider escaping nested comments and unescaping them in post-processing.
> **Example**
>
> ```nix
> /* /* nested *\/ */ 1
> ```
>
> ```console
> 1
> ```
+120
View File
@@ -0,0 +1,120 @@
# Data Types
Every value in the Nix language has one of the following types:
* [Integer](#type-int)
* [Float](#type-float)
* [Boolean](#type-bool)
* [String](#type-string)
* [Path](#type-path)
* [Null](#type-null)
* [Attribute set](#type-attrs)
* [List](#type-list)
* [Function](#type-function)
* [External](#type-external)
## Primitives
### Integer {#type-int}
An _integer_ in the Nix language is a signed 64-bit integer.
Non-negative integers can be expressed as [integer literals](syntax.md#number-literal).
Negative integers are created with the [arithmetic negation operator](./operators.md#arithmetic).
The function [`builtins.isInt`](builtins.md#builtins-isInt) can be used to determine if a value is an integer.
### Float {#type-float}
A _float_ in the Nix language is a 64-bit [IEEE 754](https://en.wikipedia.org/wiki/IEEE_754) floating-point number.
Most non-negative floats can be expressed as [float literals](syntax.md#number-literal).
Negative floats are created with the [arithmetic negation operator](./operators.md#arithmetic).
The function [`builtins.isFloat`](builtins.md#builtins-isFloat) can be used to determine if a value is a float.
### Boolean {#type-bool}
A _boolean_ in the Nix language is one of _true_ or _false_.
<!-- TODO: mention the top-level environment -->
These values are available as attributes of [`builtins`](builtins.md#builtins-builtins) as [`builtins.true`](builtins.md#builtins-true) and [`builtins.false`](builtins.md#builtins-false).
The function [`builtins.isBool`](builtins.md#builtins-isBool) can be used to determine if a value is a boolean.
### String {#type-string}
A _string_ in the Nix language is an immutable, finite-length sequence of bytes, along with a [string context](string-context.md).
Nix does not assume or support working natively with character encodings.
String values without string context can be expressed as [string literals](syntax.md#string-literal).
The function [`builtins.isString`](builtins.md#builtins-isString) can be used to determine if a value is a string.
### Path {#type-path}
A _path_ in the Nix language is an immutable, finite-length sequence of bytes starting with `/`, representing a POSIX-style, canonical file system path.
Path values are distinct from string values, even if they contain the same sequence of bytes.
Operations that produce paths will simplify the result as the standard C function [`realpath`] would, except that there is no symbolic link resolution.
[`realpath`]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html
Paths are suitable for referring to local files, and are often preferable over strings.
- Path values do not contain trailing or duplicate slashes, `.`, or `..`.
- Relative path literals are automatically resolved relative to their [base directory].
- Tooling can recognize path literals and provide additional features, such as autocompletion, refactoring automation and jump-to-file.
[base directory]: @docroot@/glossary.md#gloss-base-directory
A file is not required to exist at a given path in order for that path value to be valid, but a path that is converted to a string with [string interpolation] or [string-and-path concatenation] must resolve to a readable file or directory which will be copied into the Nix store.
For instance, evaluating `"${./foo.txt}"` will cause `foo.txt` from the same directory to be copied into the Nix store and result in the string `"/nix/store/<hash>-foo.txt"`.
Operations such as [`import`] can also expect a path to resolve to a readable file or directory.
[string interpolation]: string-interpolation.md#interpolated-expression
[string-and-path concatenation]: operators.md#string-and-path-concatenation
[`import`]: builtins.md#builtins-import
> **Note**
>
> The Nix language assumes that all input files will remain _unchanged_ while evaluating a Nix expression.
> For example, assume you used a file path in an interpolated string during a `nix repl` session.
> Later in the same session, after having changed the file contents, evaluating the interpolated string with the file path again might not return a new [store path], since Nix might not re-read the file contents.
> Use `:r` to reset the repl as needed.
[store path]: @docroot@/store/store-path.md
Path values can be expressed as [path literals](syntax.md#path-literal).
The function [`builtins.isPath`](builtins.md#builtins-isPath) can be used to determine if a value is a path.
### Null {#type-null}
There is a single value of type _null_ in the Nix language.
<!-- TODO: mention the top-level environment -->
This value is available as an attribute on the [`builtins`](builtins.md#builtins-builtins) attribute set as [`builtins.null`](builtins.md#builtins-null).
## Compound values
### Attribute set {#type-attrs}
<!-- TODO(@rhendric, #10970): fill this out -->
An attribute set can be constructed with an [attribute set literal](syntax.md#attrs-literal).
The function [`builtins.isAttrs`](builtins.md#builtins-isAttrs) can be used to determine if a value is an attribute set.
### List {#type-list}
<!-- TODO(@rhendric, #10970): fill this out -->
A list can be constructed with a [list literal](syntax.md#list-literal).
The function [`builtins.isList`](builtins.md#builtins-isList) can be used to determine if a value is a list.
## Function {#type-function}
<!-- TODO(@rhendric, #10970): fill this out -->
A function can be constructed with a [function expression](syntax.md#functions).
The function [`builtins.isFunction`](builtins.md#builtins-isFunction) can be used to determine if a value is a function.
## External {#type-external}
An _external_ value is an opaque value created by a Nix [plugin](../command-ref/conf-file.md#conf-plugin-files).
Such a value can be substituted in Nix expressions but only created and used by plugin code.
Loaded 100 of 1151 files, more files were not shown because too many files have changed in this diff. Show more