Compare commits

...
3010 Commits
Author SHA1 Message Date
Jörg Thalheim c55a7e1423 Merge pull request #13744 from NixOS/mergify/bp/2.24-maintenance/pr-13741
libexpr: Canonicalize TOML timestamps for toml11 > 4.0 (backport #13741)
2025-08-29 07:36:10 +02:00
Sergei Zimmerman ae47fd6886 libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339

(cherry picked from commit dc769d72cb)
2025-08-13 23:13:08 +03:00
Sergei Zimmerman b78b79349e libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.

(cherry picked from commit df4e55ffc1)
2025-08-13 22:39:58 +03:00
Sergei Zimmerman 79b932277a libexpr: Format primops/fromTOML.cc 2025-08-13 22:37:03 +03:00
Sergei Zimmerman d52210fae3 tests/functional/lang: Add more tests for TOML timestamps
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.

(cherry picked from commit 7ed0229d1a)
2025-08-13 19:22:48 +00:00
Eelco Dolstra c2914c0d58 Bump version 2025-06-24 19:04:04 +02:00
Eelco Dolstra 148585470c Merge remote-tracking branch 'cve/fod-cves-2.24' into 2.24-maintenance 2025-06-24 16:08:50 +02:00
mergify[bot] ed56b2741d Merge pull request #13389 from NixOS/mergify/bp/2.24-maintenance/pr-13348
Make the repl test more robust (backport #13348)
2025-06-22 19:34:41 +00:00
Eelco Dolstra bcffa0ce59 Make the repl test more robust
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:

  nix-functional-tests> grep: repl_output: No such file or directory
  nix-functional-tests> +(repl.sh:174) cat repl_output

This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.

(cherry picked from commit 9eb46e9cc0)
2025-06-22 19:08:35 +00:00
Eelco Dolstra b0fab9f90b Fixes for GHSA-g948-229j-48j3
Squashed commit of the following:

commit 04fff3a637d455cbb1d75937a235950e43008db9
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:30:32 2025 +0200

    Chown structured attr files safely

commit 5417ad445e414c649d0cfc71a05661c7bf8f3ef5
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:14:04 2025 +0200

    Replace 'bool sync' with an enum for clarity

    And drop writeFileAndSync().

commit 7ae0141f328d8e8e1094be24665789c05f974ba6
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:35:28 2025 +0200

    Drop guessOrInventPathFromFD()

    No need to do hacky stuff like that when we already know the original path.

commit 45b05098bd019da7c57cd4227a89bfd0fa65bb08
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:15:58 2025 +0200

    Tweak comment

commit 0af15b31209d1b7ec8addfae9a1a6b60d8f35848
Author: Raito Bezarius <raito@lix.systems>
Date:   Thu Mar 27 12:22:26 2025 +0100

    libstore: ensure that temporary directory is always 0o000 before deletion

    In the case the deletion fails, we should ensure that the temporary
    directory cannot be used for nefarious purposes.

    Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 2c20fa37b15cfa03ac6a1a6a47cdb2ed66c0827e
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 12:42:55 2025 +0100

    libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds

    When calling `_deletePath` with a parent file descriptor, `openat` is
    made effective by using relative paths to the directory file descriptor.

    To avoid the problem, the signature is changed to resist misuse with an
    assert in the prologue of the function.

    Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit d3c370bbcae48bb825ce19fd0f73bb4eefd2c9ea
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:07:47 2025 +0100

    libstore: ensure that `passAsFile` is created in the original temp dir

    This ensures that `passAsFile` data is created inside the expected
    temporary build directory by `openat()` from the parent directory file
    descriptor.

    This avoids a TOCTOU which is part of the attack chain of CVE-????.

    Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 45d3598724f932d024ef6bc2ffb00c1bb90e6018
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:06:03 2025 +0100

    libutil: writeFile variant for file descriptors

    `writeFile` lose its `sync` boolean flag to make things simpler.

    A new `writeFileAndSync` function is created and all call sites are
    converted to it.

    Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 732bd9b98cabf4aaf95a01fd318923de303f9996
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:05:34 2025 +0100

    libstore: chown to builder variant for file descriptors

    We use it immediately for the build temporary directory.

    Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 962c65f8dcd5570dd92c72370a862c7b38942e0d
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:59 2025 +0100

    libstore: open build directory as a dirfd as well

    We now keep around a proper AutoCloseFD around the temporary directory
    which we plan to use for openat operations and avoiding the build
    directory being swapped out while we are doing something else.

    Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit c9b42462b75b5a37ee6564c2b53cff186c8323da
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:12 2025 +0100

    libutil: guess or invent a path from file descriptors

    This is useful for certain error recovery paths (no pun intended) that
    does not thread through the original path name.

    Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
    Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 16:55:43 +02:00
Jörg Thalheim 7d15dbf9d5 Merge pull request #12801 from pascalj/2.24-maintenance
Exclude 'dir' from the FlakeRef's URL
2025-05-18 23:35:53 +02:00
Pascal Jungblut fa2a3c6de8 Merge branch '2.24-maintenance' into 2.24-maintenance 2025-05-14 22:58:17 +02:00
mergify[bot] 3e2c0680b6 Merge pull request #13091 from NixOS/mergify/bp/2.24-maintenance/pr-13086
libutil: Fix invalid boost format string in infinite symlink recursion error (backport #13086)
2025-04-25 10:11:28 +00:00
Sergei Zimmerman 28c2c78716 libutil: Fix invalid boost format string in infinite symlink recursion error
Found while working on an automated migration to `std::format`.

(cherry picked from commit bfb357c40b)
2025-04-25 09:20:56 +00:00
Pascal Jungblut 72bf563191 Exclude 'dir' from the FlakeRef's URL
This fixes an issue where nix would try to check out invalid URLs,
because it would pass 'dir' to the HTTP endpoint.

For later versions this was fixed in
b2be6fed86. This is a backport of just the
relevant part.

See #12417
2025-04-15 13:34:48 +02:00
mergify[bot] 57190f9a39 Merge pull request #12988 from NixOS/mergify/bp/2.24-maintenance/pr-12986
Fix another machine config parsing bug (backport #12986)
2025-04-09 21:41:41 +00:00
mergify[bot] 89ea1a8758 Merge pull request #12987 from NixOS/mergify/bp/2.24-maintenance/pr-12984
Fix `;` and `#` bug in machine file parsing (backport #12984)
2025-04-09 20:47:38 +00:00
John Ericson 142f94b7e1 Fix another machine config parsing bug
We were ignorning the result of `trim`, and after my last change we were
also trimmming too early.

(cherry picked from commit b74b0f4e1c)
2025-04-09 16:20:07 -04:00
John Ericson 0c6a207916 Fix ; and # bug in machine file parsing
Comments go to the end of the line, not merely the next ; *or* \n. Fix
by splitting on `;` *within* lines, and test.

(cherry picked from commit f8b13cce19)
2025-04-09 20:06:25 +00:00
Eelco Dolstra a568447137 Merge pull request #12952 from NixOS/mergify/bp/2.24-maintenance/pr-12951
tests/functional/repl: fix race condition (backport #12951)
2025-04-07 19:51:27 +02:00
Jörg Thalheim bc88d09a08 tests/functional/repl: fix race condition
the sleep 1 is not enough in some circumstances. Switching to a fifo
helps.

(cherry picked from commit 1de951d31d)
2025-04-07 12:19:00 +00:00
Eelco Dolstra ccea5cce42 Bump version 2025-04-07 11:28:46 +02:00
mergify[bot] e949cdbc05 Merge pull request #12946 from NixOS/mergify/bp/2.24-maintenance/pr-12944
maintainers/release-notes: Let it fail (backport #12944)
2025-04-07 09:11:03 +00:00
Robert Hensing de18f5ac2d maintainers/release-notes: Let it fail
Fail when a command fails.

Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.

(cherry picked from commit da36c34db7)
2025-04-07 08:52:30 +00:00
Jörg Thalheim adbf23e818 Merge pull request #12922 from NixOS/mergify/bp/2.24-maintenance/pr-12911
create cache entry for paths already in the nix store (backport #12911)
2025-04-04 16:32:47 +02:00
Jörg Thalheim 2a4e2ecd89 create cache entry for paths already in the nix store
This allows path:/nix/store/* paths to not be copied twice to the nix
store.

(cherry picked from commit 61c6210dbf)
2025-04-04 16:01:17 +02:00
mergify[bot] b47bd02a6a Merge pull request #12854 from NixOS/mergify/bp/2.24-maintenance/pr-12765
Add various clang tidy fixes (backport #12765)
2025-04-01 14:35:45 +00:00
Jörg Thalheim 8abb67b842 libstore/daemon: make sure monitor is not considered "unused"
(cherry picked from commit 5c3682d7a1)
2025-04-01 13:37:38 +00:00
Jörg Thalheim 97f9f8e909 test/ca-fd-leak: fix clang-tidy lints
(cherry picked from commit b050db951b)
2025-04-01 13:37:37 +00:00
Jörg Thalheim 7ef04ba6dd git/getStringUntil: fix uninitialized stack variable
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.

(cherry picked from commit 7e540059a3)
2025-04-01 13:37:37 +00:00
Jörg Thalheim bc6e78402c libstore/local-store: fix linting warning about unused variable
(cherry picked from commit 05082ea1c5)
2025-04-01 13:37:37 +00:00
mergify[bot] 4687dc99f7 Merge pull request #12840 from NixOS/mergify/bp/2.24-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:47:08 +00:00
Thomas Miedema 8d99b6578a nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:47:25 +00:00
John Ericson e1dad7daa5 Merge pull request #12795 from NixOS/mergify/bp/2.24-maintenance/pr-12794
Fix windows build (backport #12794)
2025-03-28 21:54:39 -04:00
John Ericson d92b509bda Fix windows build
PR #12767 accidentally broke it.

(cherry picked from commit 99041b4d84)
2025-03-28 21:07:08 -04:00
mergify[bot] 8adc1f3011 Merge pull request #12774 from NixOS/mergify/bp/2.24-maintenance/pr-12767
use createDirs consistently everywhere (backport #12767)
2025-03-28 16:48:21 +00:00
Jörg Thalheim 076fa01a84 use createDirs consistently everywhere
(cherry picked from commit a8217f2642)
2025-03-28 15:54:56 +00:00
Eelco Dolstra 61f8fd3e9c Bump version 2025-03-24 19:35:50 +01:00
Eelco Dolstra 5ce8d922fd Merge pull request #12737 from NixOS/mergify/bp/2.24-maintenance/pr-12736
`MonitorFdHup::~MonitorFdHup`: use proper close method instead of lib… (backport #12736)
2025-03-24 14:00:41 +01:00
Jörg Thalheim 333479e6be MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)
2025-03-24 12:16:00 +00:00
John Ericson 71ab003a0c Merge pull request #12731 from NixOS/mergify/bp/2.24-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-23 20:39:08 -04:00
John Ericson 3fccc71492 MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-23 23:54:09 +00:00
Jade Lovelace 729d18fd4c daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-23 23:54:08 +00:00
Félix Baylac Jacqué 699f27c4ea MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-23 23:54:08 +00:00
John Ericson 2d792a1867 MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-23 23:54:08 +00:00
John Ericson da213e8ee5 MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-23 23:54:08 +00:00
Jörg Thalheim 787a14b173 MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-23 23:54:07 +00:00
John Ericson 01aace706a monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-23 23:54:07 +00:00
mergify[bot] 74a0ab3eea Merge pull request #12683 from NixOS/mergify/bp/2.24-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 21:07:12 +00:00
Pierre-Etienne Meunier 589da451e4 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:20:48 +00:00
mergify[bot] 56e113d67a Merge pull request #12653 from NixOS/mergify/bp/2.24-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-14 14:11:27 +00:00
Sergei Zimmerman aaf86cc0d5 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 6faf66d2f7 libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 3ab83f507c libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-14 13:22:43 +00:00
Sergei Zimmerman 0033cf4270 {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:43 +00:00
mergify[bot] 79828c12e0 Merge pull request #12535 from NixOS/mergify/bp/2.24-maintenance/pr-12391
Only try to chmod /nix/var/nix/profiles/per-user when necessary (backport #12391)
2025-02-20 16:55:38 +00:00
Sandro Jäckel 2a795bb793 Only try to chmod /nix/var/nix/profiles/per-user when necessary
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit dcbf4dcc09)
2025-02-20 16:11:26 +00:00
mergify[bot] 9238c892d3 Merge pull request #12528 from NixOS/mergify/bp/2.24-maintenance/pr-12439
Set FD_CLOEXEC on sockets created by curl (backport #12439)
2025-02-19 20:30:16 +00:00
MaxHearnden 5862686ce2 Set FD_CLOEXEC on sockets created by curl
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

(cherry picked from commit 12d2527276)
2025-02-19 19:47:40 +00:00
John Ericson feadb3178d Merge pull request #12508 from NixOS/mergify/bp/2.24-maintenance/pr-12499
startDaemon(): Detect if the daemon crashes before creating the socket (backport #12499)
2025-02-18 12:21:39 -05:00
Eelco Dolstra be35515ef7 startDaemon(): Detect if the daemon crashes before creating the socket
This avoids timeouts like those seen in
https://github.com/NixOS/nix/actions/runs/13376958708/job/37358120348?pr=6962.

(cherry picked from commit 11c42cb2e1)
2025-02-18 16:38:55 +00:00
Eelco Dolstra 71f93012fd Merge pull request #12487 from NixOS/mergify/bp/2.24-maintenance/pr-12481
resolveLookupPathPath(): Fix caching of negative lookups (backport #12481)
2025-02-18 17:27:52 +01:00
Eelco Dolstra b706642133 resolveLookupPathPath(): Fix caching of negative lookups
This avoids spamming in case the missing search path entry does not
exist (#12480).

(cherry picked from commit df08e1e204)
2025-02-18 16:58:07 +01:00
Eelco Dolstra 658392e029 getDefaultNixPath(): Don't add symlinks if the target doesn't exist
(cherry picked from commit 8ac49ea5de)
2025-02-17 17:08:45 +00:00
mergify[bot] d3a52c2ef4 Merge pull request #12451 from NixOS/mergify/bp/2.24-maintenance/pr-12448
copyPathToStore(): Preserve symlinks (backport #12448)
2025-02-11 22:21:12 +00:00
Eelco Dolstra 3d6a9c7410 copyPathToStore(): Preserve symlinks
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

(cherry picked from commit 26b87e78b5)

# Conflicts:
#	tests/functional/meson.build
2025-02-11 21:51:20 +00:00
mergify[bot] 0f5f24223d Merge pull request #12434 from NixOS/mergify/bp/2.24-maintenance/pr-12373
pre-commit/check-merge-conflicts-2: fix use outside dev shell (backport #12373)
2025-02-10 14:37:20 +00:00
Robert Hensing fcea626e49 test: Fix shellcheck by giving git-hashing scripts shebangs
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.

It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
2025-02-10 15:09:36 +01:00
Robert Hensing c6e09073ea pre-commit/check-merge-conflicts-2: fix use outside dev shell
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.

`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
2025-02-10 15:08:54 +01:00
mergify[bot] b9e6e00fe1 Merge pull request #12357 from NixOS/mergify/bp/2.24-maintenance/pr-12356
Improve "illegal path references in fixed output derivation" error (backport #12356)
2025-01-28 09:37:45 +00:00
mergify[bot] 5fb6551217 Merge pull request #12365 from NixOS/mergify/bp/2.24-maintenance/pr-12362
refactor: Extract EvalState::realiseString (backport #12362)
2025-01-28 08:51:24 +00:00
Robert Hensing 0f19bfe811 Fix conflict 2025-01-28 09:20:56 +01:00
Robert Hensing 865239c732 refactor: Extract EvalState::realiseString
(cherry picked from commit 7465fbe926)

# Conflicts:
#	src/libexpr/primops.cc
2025-01-27 15:50:52 +00:00
Ben Millwood 9c5482167f Improve "illegal path references in fixed output derivation" error
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.

(cherry picked from commit 4e5d1b281e)
2025-01-26 19:38:38 +00:00
Robert Hensing a28791a91e Merge pull request #12352 from NixOS/nixfmt-on-2.24
Apply nixfmt (backport 2.24)
2025-01-24 22:01:53 +01:00
Robert Hensing ae0bb7d71c test: Fix shifted source positions after formatting 2025-01-24 21:32:07 +01:00
Robert Hensing 2f1b70a529 Format .nix files
This does not include any automation for the release branch, but
is based on the configuration of https://github.com/NixOS/nix/pull/12349

    pre-commit run -a nixfmt-rfc-style
2025-01-24 20:30:07 +01:00
Eelco Dolstra 42b22fe3de Actually bump the version
Fixes 532092a812.
2025-01-24 16:47:16 +01:00
mergify[bot] 62e7144ba7 Merge pull request #12344 from NixOS/mergify/bp/2.24-maintenance/pr-12336
libstore: Fix progress bars (backport #12336)
2025-01-24 11:52:17 +00:00
Philipp Otterbein a106e33bab libstore: fix progress bars
(cherry picked from commit be97dc1efc)
2025-01-24 11:10:16 +00:00
Eelco Dolstra 7616bae342 Merge pull request #12332 from NixOS/mergify/bp/2.24-maintenance/pr-12331
GitRepo::fetch(): Ignore $GIT_DIR (backport #12331)
2025-01-22 22:49:18 +01:00
Eelco Dolstra 1a402db046 GitRepo::fetch(): Ignore $GIT_DIR
Fixes #12325.

(cherry picked from commit 41983dba8f)
2025-01-22 22:10:09 +01:00
Eelco Dolstra 4511c47ab7 GitRepo::fetch(): Cleanup
(cherry picked from commit bd10b859f7)

# Conflicts:
#	src/libfetchers/git-utils.cc
2025-01-22 22:10:09 +01:00
Eelco Dolstra 532092a812 Bump version 2025-01-22 11:20:50 +01:00
Robert Hensing f22359ba1a Merge pull request #12287 from bryango/2.24-maintenance
Backport git+file:./ fixes to 2.24 (#12107 + #12277)
2025-01-21 22:36:06 +01:00
mergify[bot] f366beaa78 Merge pull request #12318 from NixOS/mergify/bp/2.24-maintenance/pr-12315
tests: Wait for network *online* targets and/or multi-user targets (more) (backport #12315)
2025-01-20 18:29:53 +00:00
Robert Hensing 011084b127 tests: Wait for network *online* targets
More prevalent than I thought in 15073e86a

See also https://github.com/NixOS/nix/actions/runs/12872412321/job/35887830320?pr=12310
which is a failed github-flakes test without "Network is Online"

(cherry picked from commit 6f0bdd9ae7)

# Conflicts:
#	tests/nixos/nix-docker.nix
2025-01-20 18:16:42 +00:00
Eelco Dolstra d8872fce2c Merge pull request #12311 from NixOS/mergify/bp/2.24-maintenance/pr-12302
tests: Wait for network *online* and multi-user targets (backport #12302)
2025-01-20 18:27:50 +01:00
Robert Hensing 12c90a0ec0 tests: Wait for network *online* and multi-user targets
This should help prevent some test stalls.

By default, multi-user.target does not imply that the network is fully up.

(cherry picked from commit 15073e86a8)
2025-01-20 16:40:04 +00:00
Eelco Dolstra 296769742b Merge pull request #12307 from NixOS/mergify/bp/2.24-maintenance/pr-12294
processGraph(): Don't throw ThreadPoolShutDown if there is an exception (backport #12294)
2025-01-20 17:20:40 +01:00
Eelco Dolstra d31be32c4b Merge pull request #12299 from NixOS/mergify/bp/2.24-maintenance/pr-12296
addMultipleToStore(): Move pathsToCopy (backport #12296)
2025-01-20 16:25:17 +01:00
Eelco Dolstra a1b4bc7292 processGraph(): Don't throw ThreadPoolShutDown if there is an exception
Fixes

  $ nix copy --derivation --to /tmp/nix /nix/store/...
  error: cannot enqueue a work item while the thread pool is shutting down

The ThreadPoolShutDown exception was hiding the reason for the thread
pool shut down, e.g.

  error: cannot add path '/nix/store/03sl46khd8gmjpsad7223m32ma965vy9-fix-static.patch' because it lacks a signature by a trusted key

(cherry picked from commit a8c69cc907)
2025-01-20 15:24:44 +00:00
Eelco Dolstra ff91a777a6 Add comment
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 2669e4ac4f)
2025-01-20 14:17:51 +00:00
Eelco Dolstra b4418c5c1d addMultipleToStore(): Move pathsToCopy
This allows RemoteStore::addMultipleToStore() to free the Source
objects early (and in particular the associated sinkToSource()
buffers). This should fix #7359. For example, memory consumption of

  nix copy --derivation --to ssh-ng://localhost?remote-store=/tmp/nix --derivation --no-check-sigs \
    /nix/store/4p9xmfgnvclqpii8pxqcwcvl9bxqy2xf-nixos-system-...drv

went from 353 MB to 74 MB.

(cherry picked from commit cc838e8181)
2025-01-20 14:17:51 +00:00
Eelco Dolstra 876d724061 Add link to tracking issue
(cherry picked from commit 3197c19a31)
2025-01-19 15:41:06 +08:00
Eelco Dolstra f27e263f89 Warn against the use of relative 'git+file:' flake inputs
(cherry picked from commit 12e14956e2)
2025-01-19 15:38:05 +08:00
Eelco Dolstra d944cb7f58 Use isAbsolute()
(cherry picked from commit ff9d886f3c)
2025-01-19 14:58:36 +08:00
Robert Hensing b2acf4ab97 Clarify cd call in tests/functional/flakes/flakes.sh
(cherry picked from commit d9a50c0af2)
2025-01-19 14:56:13 +08:00
Bryan Lai d4f0e8f4e3 tests/flake-in-submodule: git+file:./* input
(cherry picked from commit 37ac18d1d9)
2025-01-19 14:56:11 +08:00
Bryan Lai 84e3f4ad79 tests/flakes: check git+file:./${submodule} protocol
Relative, local git repo used to work (for submodules), but it
fails after 3e0129ce3b.

This commit adds a test to prevent such failure in the future.

(cherry picked from commit 9d088fa502)
2025-01-19 14:55:51 +08:00
Bryan Lai 6e44bc2a89 fetchers/git: make path absolute for local repo
(cherry picked from commit 96bd9bad2f)
2025-01-19 14:55:49 +08:00
mergify[bot] 2009cc137a Merge pull request #12278 from NixOS/mergify/bp/2.24-maintenance/pr-12258
sequoia-nixbld-user-migration: nail down PATH (backport #12258)
2025-01-16 16:49:44 +00:00
Travis A. Everett 0cefe354c8 sequoia-nixbld-user-migration: nail down PATH
Fixes a user report of trouble with toybox grep and avoids
potential of same basic issue with other utils.

(cherry picked from commit 6a874c2865)

# Conflicts:
#	scripts/sequoia-nixbld-user-migration.sh
2025-01-16 16:43:56 +00:00
mergify[bot] c6f504d827 Merge pull request #12239 from NixOS/mergify/bp/2.24-maintenance/pr-12237
Reject merge conflicts (backport #12237)
2025-01-15 19:40:36 +00:00
mergify[bot] 6e718c7c5c Merge pull request #12259 from NixOS/mergify/bp/2.24-maintenance/pr-11922
gc: resume GC after a pathinuse error (backport #11922)
2025-01-15 16:42:29 +00:00
Picnoir f68cdf1801 gc: resume GC after a pathinuse error
First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.

This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.

Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.

(cherry picked from commit ced8d311a5)
2025-01-15 15:59:24 +00:00
Robert Hensing 1422832f7c Resolve conflict
Conflict introduced in 9ee4a85a99
2025-01-15 07:10:34 +01:00
Jörg Thalheim 7012e1ff8c Merge pull request #12208 from NixOS/mergify/bp/2.24-maintenance/pr-11305
[2.24] Document function application operator (backport #11305)
2025-01-13 09:56:16 +01:00
Jörg Thalheim e81be91738 Merge pull request #12242 from NixOS/mergify/bp/2.24-maintenance/pr-12235
libutil: thread-pool: ensure threads finished on error (backport #12235)
2025-01-13 09:49:15 +01:00
Robert Hensing 9cc5786201 doc: Edit language/operators
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 06b18cff20)
2025-01-13 09:07:22 +01:00
Robert Hensing acd43b2239 doc: Document function application operator
(cherry picked from commit e225b63062)
2025-01-13 09:07:22 +01:00
Dominique Martinet 3cd8c1e53d libutil: thread-pool: ensure threads finished on error
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.

Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.

It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.

By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.

[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]

Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
(cherry picked from commit afac093b34)
2025-01-13 08:03:02 +00:00
Robert Hensing 10438e6cfd Reject merge conflicts
They're usually found by other checks, but docs would remain
susceptible.

(cherry picked from commit 29a1a21ce4)
2025-01-12 21:22:43 +00:00
mergify[bot] c803fe0785 Merge pull request #12197 from NixOS/mergify/bp/2.24-maintenance/pr-11541
nix-profile.sh.in: fix envvar condition (backport #11541)
2025-01-12 13:09:04 +00:00
mergify[bot] 9a85784be2 Merge pull request #12209 from NixOS/mergify/bp/2.24-maintenance/pr-11294
parser-state: fix attribute merging (backport #11294)
2025-01-12 12:36:06 +00:00
Robert Hensing a69d32d194 Resolve conflict
Some of the code before https://github.com/NixOS/nix/pull/11351 did not
have the problem. What remains wasn't all that problematic, but is good
to have anyway.
2025-01-12 13:15:03 +01:00
Ryan Hendrickson e0e50fb01f parser-state: fix attribute merging
(cherry picked from commit 8034589d7e)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson 6ae5aa7fa0 parser.y: move attr doc setting into addAttr
(cherry picked from commit b0a8430e85)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson cd169f14f0 parser.y: use names where I'll be refactoring
(cherry picked from commit 429a197d24)
2025-01-12 13:07:28 +01:00
Jörg Thalheim e3715053b1 Merge pull request #12183 from NixOS/mergify/bp/2.24-maintenance/pr-12051
Skip tests on systems with restricted usernamespaces (backport #12051)
2025-01-11 21:42:36 +01:00
Jörg Thalheim ada649ad7f functional-tests: skip tests if the kernel restricts unprivileged user namespaces
Update tests/functional/common/functions.sh

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: bryango <bryango@users.noreply.github.com>
(cherry picked from commit da7f7ba810)
2025-01-11 20:59:50 +01:00
mergify[bot] e1e219a299 Merge pull request #12211 from NixOS/mergify/bp/2.24-maintenance/pr-11240
fix: bash mangles flake ref completion (backport #11240)
2025-01-10 23:43:24 +00:00
mergify[bot] a658bafce7 Merge pull request #12207 from NixOS/mergify/bp/2.24-maintenance/pr-11343
withFramedSink(): Don't use a thread to monitor the other side (backport #11343)
2025-01-10 23:43:21 +00:00
mergify[bot] e76757083a Merge pull request #12206 from NixOS/mergify/bp/2.24-maintenance/pr-11381
Fix few duplicate word occurrences (backport #11381)
2025-01-10 23:43:18 +00:00
mergify[bot] c9dd4a7c28 Merge pull request #12205 from NixOS/mergify/bp/2.24-maintenance/pr-11407
add removal of ~root/.cache/nix to uninstall instructions (backport #11407)
2025-01-10 23:43:16 +00:00
mergify[bot] 01cb05a265 Merge pull request #12203 from NixOS/mergify/bp/2.24-maintenance/pr-11424
Pull fut.get() out of the lock (backport #11424)
2025-01-10 22:12:00 +00:00
mergify[bot] 9328fc7529 Merge pull request #12204 from NixOS/mergify/bp/2.24-maintenance/pr-11371
Fix diskutil deleteVolume command in Uninstall guide (backport #11371)
2025-01-10 22:11:57 +00:00
mergify[bot] 92e5f64068 Merge pull request #12202 from NixOS/mergify/bp/2.24-maintenance/pr-11514
nix derivation add: Remove reference to "installable" (backport #11514)
2025-01-10 22:11:54 +00:00
mergify[bot] 283249a972 Merge pull request #12201 from NixOS/mergify/bp/2.24-maintenance/pr-11611
doc, src: Fix various spelling typos (backport #11611)
2025-01-10 22:11:52 +00:00
mergify[bot] 4d2cc07b48 Merge pull request #12200 from NixOS/mergify/bp/2.24-maintenance/pr-11613
doc/nix fmt: Do not document behavior we do not control (backport #11613)
2025-01-10 22:11:49 +00:00
mergify[bot] 5c0f2b744a Merge pull request #12199 from NixOS/mergify/bp/2.24-maintenance/pr-11526
docs: Provide an example of updating multiple inputs (backport #11526)
2025-01-10 20:51:42 +00:00
mergify[bot] 0e95d8b467 Merge pull request #12198 from NixOS/mergify/bp/2.24-maintenance/pr-11530
add description + example for nix flake archive (backport #11530)
2025-01-10 20:51:38 +00:00
mergify[bot] 53bc8b543c Merge pull request #12196 from NixOS/mergify/bp/2.24-maintenance/pr-11621
Improve documentation of `nix flake lock` (backport #11621)
2025-01-10 20:51:35 +00:00
mergify[bot] 86da8bbbf3 Merge pull request #12194 from NixOS/mergify/bp/2.24-maintenance/pr-11684
libstore: Make our sandbox pivot_root directory accessible to ourself (backport #11684)
2025-01-10 20:51:33 +00:00
mergify[bot] fcdca4f8d1 Merge pull request #12195 from NixOS/mergify/bp/2.24-maintenance/pr-11629
doc/manual: Fix some broken fragments (backport #11629)
2025-01-10 20:51:30 +00:00
mergify[bot] f393260a5d Merge pull request #12192 from NixOS/mergify/bp/2.24-maintenance/pr-11717
docs: clarify syntax for escaping dollar curlies (backport #11717)
2025-01-10 19:35:02 +00:00
mergify[bot] d3cb938e07 Merge pull request #12191 from NixOS/mergify/bp/2.24-maintenance/pr-11723
doc: Explain why tryEval does not return the message (backport #11723)
2025-01-10 19:34:59 +00:00
mergify[bot] 51e41edbef Merge pull request #12190 from NixOS/mergify/bp/2.24-maintenance/pr-11813
fix(libstore-tests): remove use-after-free bug for `StringSource` (backport #11813)
2025-01-10 19:34:56 +00:00
mergify[bot] daf526f19d Merge pull request #12188 from NixOS/mergify/bp/2.24-maintenance/pr-11853
fix(libmain/common-args): do not exceed maximum allowed verbosity (backport #11853)
2025-01-10 19:34:54 +00:00
mergify[bot] d6217bc691 Merge pull request #12187 from NixOS/mergify/bp/2.24-maintenance/pr-11981
docker: Fix command "nix profile install", Don't require --impure (fix for 2.25) (backport #11981)
2025-01-10 19:34:51 +00:00
mergify[bot] 22c24a87a1 Merge pull request #12176 from NixOS/mergify/bp/2.24-maintenance/pr-12114
fix documentation of substring (backport #12114)
2025-01-10 17:21:50 +00:00
Eelco Dolstra f2bb7e043b Merge pull request #12185 from NixOS/mergify/bp/2.24-maintenance/pr-12016
Disable suid and atime on the /nix mount point on Darwin (backport #12016)
2025-01-10 15:16:44 +01:00
Eelco Dolstra 9a7196ce62 Merge pull request #12180 from NixOS/mergify/bp/2.24-maintenance/pr-12091
libstore: fixup unformatted uri when S3 getObject fails (backport #12091)
2025-01-10 15:11:08 +01:00
Eelco Dolstra 7a2b1cbd71 Merge pull request #12173 from NixOS/mergify/bp/2.24-maintenance/pr-12127
toJSON: re-throw serialization exception (backport #12127)
2025-01-10 13:32:42 +01:00
mergify[bot] 534473e90b Merge pull request #12170 from NixOS/mergify/bp/2.24-maintenance/pr-12155
Document `--max-freed` for `nix-collect-garbage` (backport #12155)
2025-01-10 11:05:01 +00:00
0x5a4 ce5c0c64af fix: bash mangles flake ref completion
(cherry picked from commit 088fa815d3)
2025-01-10 10:42:07 +00:00
Eelco Dolstra 931eb85f50 Add FIXME
(cherry picked from commit fac756fed4)
2025-01-10 10:36:06 +00:00
Eelco Dolstra a9020404cf select() -> poll() for Windows compat
(cherry picked from commit 270c8469d7)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 77a71c518f Update src/libutil/serialise.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit ce7cf4a2d3)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 36f3fb72e9 withFramedSink(): Don't use a thread to monitor the other side
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.

  nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298

would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.

So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).

This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).

(cherry picked from commit 39daa4a0d3)
2025-01-10 10:36:05 +00:00
Jan Hrcek 5099768970 Fix few duplicate word occurrences (#11381)
(cherry picked from commit cb4b9be458)
2025-01-10 10:29:59 +00:00
Jeremy Kerfs 6f5684f6d1 add removal of ~root/.cache/nix to uninstall instructions (#11407)
(cherry picked from commit 4decd6f8b3)
2025-01-10 10:29:55 +00:00
Seggy Umboh d6488efeae Fix diskutil deleteVolume command in Uninstall guide (#11371)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 8e63dc4455)
2025-01-10 10:28:54 +00:00
Jeremy Kolb bc8611a29b Remote nullptr
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 8152c5c828)
2025-01-10 10:26:53 +00:00
Jeremy Kolb 1a21eb43af Pull fut.get() out of the lock
This is https://gerrit.lix.systems/c/lix/+/1462 by @jade_

see: https://git.lix.systems/lix-project/lix/issues/366
see: https://gerrit.lix.systems/c/lix/+/1462
(cherry picked from commit 46f6505865)
2025-01-10 10:26:53 +00:00
Eelco Dolstra bcf34e89c0 nix derivation add: Remove reference to "installable"
It doesn't operate on an installable, so don't mention it.

(cherry picked from commit 65f138f669)
2025-01-10 10:23:49 +00:00
Jordan Justen 0060a4ba5b doc, src: Fix various spelling typos
Signed-off-by: Jordan Justen <jljusten@gmail.com>
(cherry picked from commit 3b6cf350e7)
2025-01-10 10:22:51 +00:00
Robert Hensing 24e63b14e5 doc/nix fmt: Do not document behavior we do not control
This leads to confusion about what the command does.
E.g. https://github.com/NixOS/nix/issues/9359

- Move the description up

- Remove details about the individual formatters

(cherry picked from commit 8c4842de8c)
2025-01-10 10:22:50 +00:00
quatquatt 93e8e5b347 docs: Provide an example of updating multiple inputs with nix flake update
docs: Provide an example of updating multiple inputs with `nix flake update`
(cherry picked from commit 9e335ee3b9)
2025-01-10 10:22:45 +00:00
Jörg Thalheim 792099081c add description + example for nix flake archive
Update src/nix/flake-archive.md

Update src/nix/flake-archive.md

(cherry picked from commit f0a4f19087)
2025-01-10 10:21:45 +00:00
Noam Yorav-Raphael 971748ace9 nix-profile.sh.in: fix envvar condition
(cherry picked from commit 97fffd8765)

# Conflicts:
#	scripts/nix-profile.sh.in
2025-01-10 10:20:46 +00:00
Kevin Cox 0e421e7a35 Improve documentation of nix flake lock
The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.

(cherry picked from commit d2f627dda9)
2025-01-10 10:18:03 +00:00
Robert Hensing 9ee4a85a99 doc/manual: Fix some broken fragments
(cherry picked from commit 071ddbed57)

# Conflicts:
#	doc/manual/src/language/operators.md
2025-01-10 10:17:08 +00:00
Geoffrey Thomas 24542a7b96 libstore: Make our sandbox pivot_root directory accessible to ourself
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)

Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory.  The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.

(cherry picked from commit 5a794d9366)
2025-01-10 10:16:07 +00:00
Eman Resu 6304fa5cd1 docs: clarify syntax for escaping dollar curlies
(cherry picked from commit defff01a51)

# Conflicts:
#	doc/manual/source/language/string-literals.md
2025-01-10 10:14:55 +00:00
Robert Hensing 9c54fac5ff doc: Explain why tryEval does not return the message
(cherry picked from commit 90d257b771)
2025-01-10 10:12:37 +00:00
Sergei Zimmerman e863e6ab83 fix(libstore-tests): remove use-after-free bug for StringSource
Unfortunately `StringSource` class is very easy was very easy to misuse
because the ctor took a plain `std::string_view` which has a bad habit
of being implicitly convertible from an rvalue `std::string`. This lead
to unintentional use-after-free bugs.

This patch makes `StringSource` much harder to misuse by disabling the ctor
from a `std::string &&` (but `const std::string &` is ok).

Fix affected tests from libstore-tests.
Reformat those tests with clangd's range formatting since the diff is tiny
and it seems appropriate.

(cherry picked from commit 5bc8957c73)
2025-01-10 10:08:26 +00:00
Sergei Zimmerman dea80c4fec fix(libmain/common-args): do not exceed maximum allowed verbosity
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:

../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'

(cherry picked from commit b9f8c4af40)
2025-01-10 10:04:39 +00:00
h0nIg 26866eef0c docker: Fix command "nix profile install", Don't require --impure
(cherry picked from commit 61950c3375)
2025-01-10 09:58:36 +00:00
Graham Christensen 56332fda1c Disable suid and atime on the /nix mount point on Darwin
The Determinate Nix Installer has set nosuid and noatime in https://github.com/DeterminateSystems/nix-installer/pull/1338, and figured this perf and security improvement is worthy of upstreaming.

The /nix volume shouldn't have setuid binaries anyway, and filesystems seem to generally be noatime on macOS.
Further, the garbage collector doesn't use atime.

(cherry picked from commit 4137ead7a1)
2025-01-10 09:56:35 +00:00
Cole Helbling 4a7c7f12f5 libstore: fixup unformatted uri when S3 getObject fails
(cherry picked from commit b978fa8450)
2025-01-10 09:48:23 +00:00
Cole Helbling fed54f3550 tests/nixos/s3-binary-cache-store: disable default substituter so it runs faster
Since networking is disabled in these VMs, trying to talk to the default
cache.nixos.org slows the test down (since it can't resolve it).

(cherry picked from commit f0c1262d23)
2025-01-10 09:48:23 +00:00
Cole Helbling 6c1dfab347 tests/nixos/s3-binary-cache-store: test that "object does not exist" error message is properly formatted
(cherry picked from commit 535724fd79)
2025-01-10 09:48:22 +00:00
Philipp Otterbein 4b4b6377b2 fix documentation of substring
(cherry picked from commit 1e2cace5f1)
2025-01-10 09:44:14 +00:00
Philipp Otterbein 9936cb21ab toJSON: re-throw serialization exception
(cherry picked from commit 4a2310a3a0)
2025-01-10 09:42:14 +00:00
Domagoj Mišković 7c8276b799 Document --max-freed for nix-collect-garbage (#12155)
* Update nix-collect-garbage.md

Referencing issue at: https://github.com/NixOS/nix/issues/12132

Copied the description of `--max-freed` option from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44

(cherry picked from commit 5230d3ecc4)
2025-01-10 09:34:09 +00:00
mergify[bot] e0c8b0fc4f Merge pull request #12150 from NixOS/mergify/bp/2.24-maintenance/pr-12143
mergify: fix installer test name (backport #12143)
2025-01-07 19:42:11 +00:00
Jörg Thalheim 55dc2a9c9a mergify: fix installer test name
(cherry picked from commit 438a20427f)
2025-01-07 19:18:05 +00:00
Jörg Thalheim 112d0a7d85 Merge pull request #12144 from NixOS/mergify/bp/2.24-maintenance/pr-12046
CLI symlink fixes (backport #12046)
2025-01-07 08:06:58 +01:00
Jörg Thalheim b59b317e47 Merge pull request #12136 from NixOS/mergify/bp/2.24-maintenance/pr-11876
Filter OSC 8 correctly (backport #11876)
2025-01-07 07:22:37 +01:00
Jörg Thalheim 1eda38420f Merge pull request #11743 from NixOS/mergify/bp/2.24-maintenance/pr-11729
nix/tests: run test help.sh only if nix is built with documentation (backport #11729)
2025-01-07 07:21:51 +01:00
Robert Hensing caeec0404e test: Avoid regressing accidental use of weakly_canonical instead of makeParentCanonical
I'd messed up a rebase in my previous iteration, causing `weakly_canonical` to reappear,
but not trigger a test failure.

These two functions behave similarly when the argument is a path that points to a broken
symlink. `weakly_canonical` would not resolve it because the target doesn't exist, and
`makeParentCanonical` would not resolve it, because it never resolves the final path
element.
This new test case now also tests a valid symlink, "differentiating" the two.

(cherry picked from commit 4c74d679b6)
2025-01-07 07:20:55 +01:00
Jörg Thalheim f145dde10b makeParentCanonical: test case where parent is empty
(cherry picked from commit 5a5a86949a)

# Conflicts:
#	src/libutil-tests/file-system.cc
2025-01-07 07:20:55 +01:00
Robert Hensing 73dd3db039 test: Add hydraJobs.tests.functional_symlinked-home
(cherry picked from commit 628c11d237)
2025-01-07 07:20:55 +01:00
Robert Hensing e31d071793 fix: Resolve CLI parent symlinks before adding to store
Fixes https://github.com/NixOS/nix/issues/11941

(cherry picked from commit ddbbf53767)
2025-01-07 07:20:55 +01:00
Robert Hensing 42104d650a refactor: Don't re-construct SourcePath unnecessarily
(cherry picked from commit c0b64f3377)
2025-01-07 07:20:55 +01:00
Robert Hensing 129f8aaab2 doc: Document nix-store --add-fixed symlink behavior
Tested with

    nix run nix/2.3-maintenance#nix-store -- --add some_symlink
    nix run nix/2.3-maintenance#nix-store -- --add-fixed sha256 --recursive some_symlink

(cherry picked from commit 91e91f62fa)
2025-01-07 07:20:55 +01:00
Robert Hensing b9c30a9c87 fix: Handle symlinks and FIFOs in nix hash where possible
Fixes https://github.com/NixOS/nix/issues/11756
Fixes https://github.com/NixOS/nix/issues/11681

(cherry picked from commit 36563c69a4)
2025-01-07 07:20:55 +01:00
Robert Hensing fa3fd41063 Add makeParentCanonical()
(cherry picked from commit 69853c067c)
2025-01-07 07:20:55 +01:00
Marian Hammer e79d66ceef nix/tests: run test help.sh only if nix is built with documentation
tests/functional/help.sh calls nix-* commands with option --help
if nix is built without documentation the option --help throws an error
because the man page it wants to display is missing

(cherry picked from commit 85b0cd320a)
2025-01-07 05:55:13 +00:00
Jack Wilsdon e144f61ec0 Filter OSC 8 correctly
This allows Nix to use lowdown 1.2.0 which outputs OSC-8 links.

(cherry picked from commit 1301f8434d)
2025-01-07 05:33:19 +00:00
Jörg Thalheim 2c89c38fa1 Merge pull request #12074 from NixOS/mergify/bp/2.24-maintenance/pr-12059
installer: make sure we can always test the installer in ci and locally (backport #12059)
2025-01-07 00:27:16 +01:00
Robert Hensing 1a710e54dc .github/ci: Use fixed names
This lets us update "runs-on" without creating a mismatch with the
required checks that are configured for the repo in the github ui.
2025-01-07 00:04:32 +01:00
NAHO 365e0a9ab1 ci: lock macOS runner to macos-14 2025-01-07 00:04:32 +01:00
NAHO b3ea1e9b19 ci: update Ubuntu runner to ubuntu-24.04
Link: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
NAHO c78b818a0c ci: lock Ubuntu runner to ubuntu-22.04
Lock the Ubuntu runner to ubuntu-22.04 to avoid accidental updates [1]
and increase reproducibility.

[1]: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
Jörg Thalheim 3109250faf mergify: updated required tests
(cherry picked from commit df3eb4f3f6)
2025-01-07 00:04:32 +01:00
Jörg Thalheim 2fbd0a943c installer: make sure we can always test the installer in ci and locally
Just now there is a dependency on cachix, which means we cannot test
the installer in CI if forks do not have the necessary secrets set up.
We replace this with a simple http server that serves the installer and
can be both used in CI and locally.

(cherry picked from commit 6162e1220e)
2025-01-07 00:04:32 +01:00
mergify[bot] bda59dee55 Merge pull request #12077 from NixOS/mergify/bp/2.24-maintenance/pr-12045
EvalState::realiseContext(): Allow access to the entire closure (backport #12045)
2024-12-31 15:33:23 +00:00
Robert Hensing 000db53f8f tests/functional: Add $config_nix 2024-12-31 15:14:56 +01:00
mergify[bot] e40d0352b2 Merge pull request #12118 from NixOS/mergify/bp/2.24-maintenance/pr-12115
libcmd: update to support lowdown-1.4 API (backport #12115)
2024-12-30 20:26:40 +00:00
Sergei Trofimovich c530b13321 libcmd: update to support lowdown-1.4 API
Upstream change
https://github.com/kristapsdz/lowdown/commit/bab1d75079ac3866fc14e1fa4825ae64a94a7c0e
moved a few fields from `lowdown_opts` toa  new `lowdown_opts_term`
struct. As a result the build started failing as:

    nix-cmd> [2/17] Compiling C++ object libnixcmd.so.p/markdown.cc.o
    nix-cmd> FAILED: libnixcmd.so.p/markdown.cc.o
    nix-cmd> g++ -Ilibnixcmd.so.p -I. -I.. -I/nix/store/b0bnrk5lacxbpgxgnc28r8q3wcazrgxj-nix-util-2.26.0pre-dev/include/nix -I/nix/store/cxnynq9ykyj4xxv6wf6dw7r0aw5x6n9k-libarchive-3.7.7-dev/include -I/nix/store/bfgjwkcb8snkizx578rzdahi75m8zyh4-nlohmann_json-3.11.3/include -I/nix/store/3sx8bq3sip6j2nv1m5xx4gbdp33v7iy6-nix-store-2.26.0pre-dev/include/nix -I/nix/store/sih2dgqzvsbv7p510lkfmas7s7wbsl4j-nix-fetchers-2.26.0pre-dev/include/nix -I/nix/store/68p8s20fsiiakj7nys7grbaixfnhsdzs-nix-expr-2.26.0pre-dev/include/nix -I/nix/store/gw7wknhzhfzzj9zww2kyi5xrzgf1ndki-boehm-gc-8.2.8-dev/include -I/nix/store/3jwb9j4vnsk5saq3wfyyp9il3mhs41l9-nix-flake-2.26.0pre-dev/include/nix -I/nix/store/8nwjvmq7m48v8g646jrxkikv6x47bc3m-nix-main-2.26.0pre-dev/include/nix -I/nix/store/rb0hzsw5wc1a7daizhpj824mbxlvijrq-lowdown-1.4.0-dev/include -I/nix/store/m388ywpk53fsp8r98brfd7nf1f5sskv0-editline-1.17.1-dev/include -fdiagnostics-color=always -D_GLIBCXX_ASSERTIONS=1 -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++2a -include config-util.hh -include config-store.hh -include config-expr.hh -include config-main.hh -include config-cmd.hh -Wdeprecated-copy -Werror=suggest-override -Werror=switch -Werror=switch-enum -Werror=unused-result -Wignored-qualifiers -Wimplicit-fallthrough -Wno-deprecated-declarations -O3 -fPIC -pthread -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -MD -MQ libnixcmd.so.p/markdown.cc.o -MF libnixcmd.so.p/markdown.cc.o.d -o libnixcmd.so.p/markdown.cc.o -c ../markdown.cc
    nix-cmd> ../markdown.cc: In function 'std::string nix::doRenderMarkdownToTerminal(std::string_view)':
    nix-cmd> ../markdown.cc:28:5: error: 'lowdown_opts' has no non-static data member named 'cols'
    nix-cmd>    28 |     };
    nix-cmd>       |     ^

The change adds version-based conditional to support both pre-1.4 and
1.4 forms of the initialization.

Closes: https://github.com/NixOS/nix/issues/12113
(cherry picked from commit edbfe863ce)
2024-12-30 19:16:43 +00:00
Eelco Dolstra a924db7d0b Resolve merge conflict 2024-12-16 18:01:52 +01:00
Eelco Dolstra f986f7e89b EvalState::realiseContext(): Allow access to the entire closure
Fixes #11030.

(cherry picked from commit 08361f031d)

# Conflicts:
#	src/libexpr/eval.cc
#	tests/functional/import-from-derivation.nix
#	tests/functional/import-from-derivation.sh
2024-12-16 15:36:40 +00:00
Jörg Thalheim 6a791e946f Merge pull request #11904 from NixOS/mergify/bp/2.24-maintenance/pr-11902
Fix issue #11892 (backport #11902)
2024-12-16 13:22:30 +01:00
mergify[bot] aa22f98b25 Merge pull request #12054 from NixOS/mergify/bp/2.24-maintenance/pr-12050
speed up ci by splitting off eval and build + fix ci with restricted namespaces (backport #12050)
2024-12-14 15:46:25 +00:00
Jörg Thalheim b3dc855569 ci: disable apparmor restrictions
For our tests we need to map the root user for some tests.
However ubuntu no longer allows this by default:
https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces

(cherry picked from commit 9f2b9256b6)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 7bafbfd50e binary-tarball: create tarball without listing files
This is producing quite a bit of log output in CI.

(cherry picked from commit 31b3a34e5b)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 578f185daf speed up ci by splitting off eval and build
- This speeds up macOS builds from 30 minutes to 11 minutes (3x faster).
- Also improve error reporting e.g. printing out what actually failed to build.
- As a result we also no longer need swap.

(cherry picked from commit 691b1ea237)
2024-12-14 15:24:48 +00:00
Eelco Dolstra 628961d136 Bump version 2024-12-10 17:11:49 +01:00
Jörg Thalheim a4f978bd9b Merge pull request #12034 from NixOS/mergify/bp/2.24-maintenance/pr-11921
Make @nix json structured build log parsing warn instead of fail (backport #11921)
2024-12-10 05:37:28 +01:00
Robert Hensing 180031fb5a tests: Make unusual logging test conditional on fixed daemon version
(cherry picked from commit c783cd22ac)
2024-12-10 00:12:19 +01:00
Robert Hensing 29d91db99c Remove redundant warning: prefix from structured build log warning
(cherry picked from commit f3c722cab2)
2024-12-09 21:40:12 +00:00
Robert Hensing 6b2722fc84 test: Move unusual-logging to run only in logging test case
(cherry picked from commit 1421420e86)
2024-12-09 21:40:07 +00:00
Robert Hensing c951bb0689 Push log source description out of libutil and report build hook @nix warning correctly
(cherry picked from commit 03d4bfd852)
2024-12-09 21:40:04 +00:00
Robert Hensing 60eadfac7f tests/functional/dependencies.nix: Check that we tolerate syntax and type errors
(cherry picked from commit 1485937b89)
2024-12-09 21:40:00 +00:00
Linus Heckemann 9fed865d1d libutil: handle json builder log messages with unexpected format
Before this change, expressions like:

with import <nixpkgs> {};
runCommand "foo" {} ''
  echo '@nix {}' >&$NIX_LOG_FD
''

would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.

Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.

Fixes #544.

Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
(cherry picked from commit e55cd3beea710db727fd966f265a1b715b7285f3)
(cherry picked from commit ee03fd478e)
2024-12-09 21:39:58 +00:00
Robert Hensing 3cd4e9de1e tests/functional/dependencies.nix: Refactor, replace arcane let
(cherry picked from commit e82ff51726)
2024-12-09 21:39:55 +00:00
Eelco Dolstra 0f8fef49c3 Merge pull request #12017 from NixOS/mergify/bp/2.24-maintenance/pr-12013
nix hash convert: Support SRI hashes that lack trailing '=' characters (backport #12013)
2024-12-09 13:14:14 +01:00
Eelco Dolstra bd1b11ec66 Resolve conflict 2024-12-06 12:13:12 +01:00
Eelco Dolstra 6a8e9736e0 nix hash: Don't print 'nix hash' deprecation message
Fixes #11997.

(cherry picked from commit 408c2faf93)

# Conflicts:
#	src/nix/hash.cc
2024-12-05 22:30:24 +00:00
Eelco Dolstra dd79a490bf nix hash convert: Don't fail on uppercase base-16 hashes
(cherry picked from commit 33b645cedf)
2024-12-05 22:30:24 +00:00
Eelco Dolstra 00d0e63c61 nix hash convert: Support SRI hashes that lack trailing '=' characters
Fixes #11996.

(cherry picked from commit 52f1cd0595)
2024-12-05 22:30:24 +00:00
Jörg Thalheim fc11659148 Merge pull request #11932 from NixOS/mergify/bp/2.24-maintenance/pr-11915
Bump fetcher cache version (backport #11915)
2024-11-23 09:31:55 +01:00
Robert HensingandEelco Dolstra bf67741ff1 Bump fetcher cache version
We're getting more reports in https://github.com/NixOS/nix/issues/10985

It appears that something hasn't gone right process-wise.
I find this mistake not to be worth investigating, but rather something
to pay attention to going forward.

Let's nip this in the bud.

Closes https://github.com/NixOS/nix/issues/10985

(cherry picked from commit e948c8e033)

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-11-23 09:05:52 +01:00
Robert Hensing 403f388fdb Fix issue 11892
It seems that I copied the expression for baseDir thoughtlessly and
did not come back to it.

- `baseDir` was only used in the `fromArgs` branch.
- `fromArgs` is true when `packages` is true.

(cherry picked from commit c4b95dbdd1)
2024-11-18 21:21:13 +00:00
Eelco Dolstra 8e8a572d37 Merge pull request #11889 from NixOS/mergify/bp/2.24-maintenance/pr-11884
nix path-info: Don't write to std::cout directly (backport #11884)
2024-11-15 16:29:29 +01:00
Eelco Dolstra 3bfc35ba74 nix path-info: Don't write to std::cout directly
This interferes with the progress bar, resulting in output like

  evaluating derivation 'git+file:///home/eelco/Dev/nix-master#packages.x86_64-linux.default'/nix/store/zz8v96j5md952x0mxfix12xqnvq5qv5x-nix-2.26.0pre20241114_a95f6ea.drv

(cherry picked from commit 33a0fa882f)
2024-11-15 14:08:22 +00:00
Eelco Dolstra aaeaa4e133 Merge pull request #11879 from NixOS/mergify/bp/2.24-maintenance/pr-11878
Remove warning that interpretation of .drv has changed (backport #11878)
2024-11-13 18:23:23 +01:00
Robert Hensing 22e1613814 Remove warning that interpretation of .drv has changed
This was first tagged as 2.15.0, 1½ years ago; plenty of time for
everyone to catch up.

By now, the warning is causing more confusion than that it is helpful,
because passing a `.drv` or `drvPath` has legitimate use cases.

(cherry picked from commit f534a7a524)
2024-11-13 16:12:58 +00:00
Jörg Thalheim ef21dfa221 Merge pull request #11838 from NixOS/mergify/bp/2.24-maintenance/pr-11837
fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug (backport #11837)
2024-11-08 16:02:45 +01:00
Sergei Zimmerman c556c205a9 fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug
Naming class member variables the same as constructor arguments is a very
slippery slope because of how member variable names get resolved. Compiler
is not very helpful here and we need static analysis to forbid this kind of
stuff.

The following example illustrates the cause quite well:

```cpp

struct B {
    B(int) {}
};

struct A {
    A(int b): b([&](){
        return b;
        static_assert(std::is_same_v<decltype(b), int>);
    }()) {
       static_assert(std::is_same_v<decltype(b), int>);
    }
    void member() {
        static_assert(std::is_same_v<decltype(b), B>);
    }
    B b;
};

int main() {
    A(1).member();
}
```

From N4861 6.5.1 Unqualified name lookup:

> In all the cases listed in [basic.lookup.unqual], the scopes are searched
> for a declaration in the order listed in each of the respective categories;
> name lookup ends as soon as a declaration is found for the name.
> If no declaration is found, the program is ill-formed.

In the affected code there was a use-after-move for all accesses in the constructor
body, but this UB wasn't triggered.

These types of errors are trivial to catch via clang-tidy's [clang-analyzer-cplusplus.Move].

(cherry picked from commit 3e0129ce3b)
2024-11-08 13:58:17 +00:00
Jörg Thalheim 6e095ddef9 Merge pull request #11823 from NixOS/mergify/bp/2.24-maintenance/pr-11821
initRepoAtomically: Catch directory_not_empty (backport #11821)
2024-11-07 08:03:52 +01:00
Robert Hensing c3e2419e73 initRepoAtomically: Catch directory_not_empty
(cherry picked from commit 388271e8ec)
2024-11-07 06:23:15 +00:00
Eelco Dolstra 960c288fa0 Merge pull request #11777 from NixOS/mergify/bp/2.24-maintenance/pr-11759
fix: make sure directory exists before using `ln` (backport #11759)
2024-11-01 16:56:25 +01:00
Valentin Gagarin d14b46f9ba Merge pull request #11783 from NixOS/mergify/bp/2.24-maintenance/pr-11775
doc/rl-2.19: add entry for always-allow-substitutes option (backport #11775)
2024-11-01 13:12:58 +01:00
Bjørn Forsman c9325fe343 doc/rl-2.19: add entry for always-allow-substitutes option (#11775)
* doc/rl-2.19: add entry for always-allow-substitutes option

Fixes https://github.com/NixOS/nix/issues/9427.

(cherry picked from commit 020dbac0e0)
2024-11-01 10:57:56 +00:00
Pol Dellaiera 2a2d484986 fix: make sure directory exists before using ln
(cherry picked from commit 5f71ebb956)
2024-11-01 10:19:42 +00:00
Eelco Dolstra 9728a4ac94 maintainers/upload-release.pl: Delete temporary directories when we're done
(cherry picked from commit 02bb633a58)
2024-10-31 12:12:17 +01:00
Eelco Dolstra 75b00e52ac 'build' is now 'build.nix'
(cherry picked from commit 617e711820)
2024-10-31 12:12:12 +01:00
Eelco Dolstra 6044907c13 Bump version 2024-10-31 12:11:43 +01:00
Eelco Dolstra 597fcc98e1 Merge remote-tracking branch 'nix-ghsa-wf4c-57rh-9pjg/advisory-fix-1-2.24' into 2.24-maintenance 2024-10-30 21:44:39 +01:00
Eelco Dolstra 82abed901f Merge pull request #11769 from DeterminateSystems/fix-release-notes
Fix 2.24 release notes
2024-10-30 16:12:43 +01:00
Eelco Dolstra f9180f12c4 release notes: 2.24.9 2024-10-30 15:32:37 +01:00
John Ericson baa7565710 Merge pull request #11763 from NixOS/mergify/bp/2.24-maintenance/pr-11751
Add support for `utimensat` as an alternative to `lutimes` (backport #11751)
2024-10-28 15:27:39 +01:00
John Ericson 7e6e75fd48 Merge pull request #11762 from NixOS/mergify/bp/2.24-maintenance/pr-11750
Fix building on OpenBSD (backport #11750)
2024-10-28 15:27:23 +01:00
Artemis Tosini 803943fce4 Add support for utimensat as an alternative to lutimes
OpenBSD doesn't support `lutimes`, but does support `utimensat` which
subsumes it. In fact, all the BSDs, Linux, and newer macOS all support
it. So lets make this our first choice for the implementation.

In addition, let's get rid of the `lutimes` `ENOSYS` special case. The
Linux manpage says

> ENOSYS
>
> The kernel does not support this call; Linux 2.6.22 or later is
> required.

which I think is the origin of this check, but that's a very old version
of Linux at this point. The code can be simplified a lot of we drop
support for it here (as we've done elsewhere, anyways).

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
(cherry picked from commit d023202811)
2024-10-28 10:27:39 +00:00
Artemis Tosini 0ae90918db package.nix: Disable GC on OpenBSD
Nix fails to build on OpenBSD with a linking error due to a non-found
symbol in boehm-gc. Just disable the GC until we can find a proper
workaround.

(cherry picked from commit fecc1ca205)
2024-10-28 10:27:37 +00:00
Artemis Tosini d6ece7e94a Fix OpenBSD build with Makefiles
OpenBSD dynamic libraries never link to libc directly.
Instead, they have undefined symbols for all libc functions they use
that ld.so resolves to the libc referred to in the main executable.

Thus, disallowing undefined symbols will always fail

(cherry picked from commit c49bff2434)
2024-10-28 10:27:37 +00:00
Robert Hensing 50f83e4bbd local-derivation-goal: Move builder preparation to non-builtin code path 2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt 047ee50db2 local-derivation-goal: Refactor
This works because the `builder` and `args` variables are only used
in the non-builtin code path.

Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt ae7a2ea741 local-derivation-goal: Print sandbox error detail on darwin
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert Hensing f8a1a149c7 packaging: Add darwin -lsandbox in meson 2024-10-22 17:56:06 +02:00
Puck Meerburg 170242cf0c fix: Run all derivation builders inside the sandbox on macOS 2024-10-22 17:56:06 +02:00
Eelco Dolstra 7718688f52 Merge pull request #11730 from NixOS/mergify/bp/2.24-maintenance/pr-11713
fix env-vars beeing written to `/tmp` (backport #11713)
2024-10-22 15:01:56 +02:00
Jörg Thalheim 0e9b04a66e fix env-vars beeing written to /tmp
This overall seems like insecure tmp file handling to me. Because other
users could replace files in /tmp with a symlink and make the nix-shell
override other files.

fixes https://github.com/NixOS/nix/issues/11470

(cherry picked from commit 2105574702)
2024-10-22 12:13:47 +00:00
Eelco Dolstra 411ec33db3 Merge pull request #11692 from NixOS/mergify/bp/2.24-maintenance/pr-11690
Handle tarballs where directory entries are not contiguous (backport #11690)
2024-10-14 16:01:43 +02:00
Eelco Dolstra 31df105f45 Merge pull request #11691 from NixOS/mergify/bp/2.24-maintenance/pr-11677
builtins.fetchurl: Fix segfault on s3:// URLs (backport #11677)
2024-10-14 15:23:49 +02:00
Eelco Dolstra 57ace600af Add a test
(cherry picked from commit a7b9877da9)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 9da1300617 Handle tarballs where directory entries are not contiguous
I.e. when not all entries underneath a directory X follow eachother,
but there is some entry Y that isn't a child of X in between.

Fixes #11656.

(cherry picked from commit 4012954b59)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 1294442c6c Add assert
(cherry picked from commit d2f4d07619)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 339236d32e Make S3 downloads slightly more interruptable
(cherry picked from commit d38f62f64d)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 4912a9e7fd builtins.fetchurl: Fix segfault on s3:// URLs
Also, add an activity to show that we're downloading an s3:// file.

Fixes #11674.

(cherry picked from commit 0500fba56a)
2024-10-14 14:44:28 +02:00
Eelco Dolstra d80bf54e3b Add a VM test for S3BinaryCacheStore
Fixes #11238.

(cherry picked from commit 2950f9e18a)
2024-10-14 14:44:28 +02:00
Robert Hensing f1dc3b7d55 Merge pull request #11649 from NixOS/mergify/bp/2.24-maintenance/pr-11610
fix passing CA files into builtins:fetchurl sandbox (backport #11610)
2024-10-13 12:43:33 +02:00
Jörg Thalheim 5f1b132187 tests/nixos/fetchurl: drop unused variables
(cherry picked from commit 410853ddcf)
2024-10-07 12:45:04 +00:00
Puck Meerburg 742eb0f815 fix passing CA files into builtins:fetchurl sandbox
This patch has been manually adapted from
https://github.com/lix-project/lix/commit/14dc84ed03f1b7e5a41bb6fdce00916faab32b60

Tested with:

$ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 16:57:50 after 1s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1

Now returns:

nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 17:05:48 after 0s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

(cherry picked from commit c1ecf0bee9)
2024-10-07 12:45:03 +00:00
Robert Hensing 13e200df45 Merge pull request #11601 from Mic92/git-cache-backport
[2.24] create git caches atomically
2024-09-30 13:11:22 +02:00
Eelco Dolstra a1d841bf2c Bump version 2024-09-28 00:05:03 +02:00
Eelco Dolstra 048cfe51c9 Merge pull request #11604 from NixOS/mergify/bp/2.24-maintenance/pr-11600
HttpBinaryCacheStore::getFile(): Fix uncaught exception (backport #11600)
2024-09-27 13:26:21 +02:00
Eelco Dolstra 15a2b49115 HttpBinaryCacheStore::getFile(): Fix uncaught exception
This method is marked as `noexcept`, but `enqueueFileTransfer()` can
throw `Interrupted` if the user has hit Ctrl-C or if the `ThreadPool`
that the thread is a part of is shutting down.

(cherry picked from commit 4566854981)
2024-09-27 10:38:03 +00:00
Jörg Thalheim 34fd00accc create git caches atomically
When working on speeding up the CI,
I triggered a race condition in the creation of the tarball cache.
This code now instead will ensure that half-initialized repositories
are no longer visible to any other nix process.

This is the error message that I got before:

error: opening Git repository '"/Users/runner/.cache/nix/tarball-cache"': could not find repository at '/Users/runner/.cache/nix/tarball-cache'
(cherry picked from commit 12d5b2cfa1)
2024-09-27 10:06:58 +02:00
Eelco Dolstra b23812a59c Bump version 2024-09-26 03:25:40 +02:00
Eelco Dolstra 618a0cc987 Merge pull request #11592 from NixOS/mergify/bp/2.24-maintenance/pr-11585
builtin:fetchurl: Enable TLS verification (backport #11585)
2024-09-26 01:04:39 +02:00
Eelco Dolstra ba81598017 Resolve conflict 2024-09-26 00:17:03 +02:00
Eelco Dolstra e87be60055 Typo
(cherry picked from commit ef8987955b)
2024-09-26 00:16:17 +02:00
Eelco Dolstra 345a264a39 Add release note
(cherry picked from commit 7b39cd631e)
2024-09-25 21:55:36 +00:00
Eelco Dolstra ee6a5faf4b Add a test for builtin:fetchurl cert verification
(cherry picked from commit f2f47fa725)

# Conflicts:
#	tests/nixos/default.nix
2024-09-25 21:55:36 +00:00
Eelco Dolstra d4824c8ff7 builtin:fetchurl: Enable TLS verification
This is better for privacy and to avoid leaking netrc credentials in a
MITM attack, but also the assumption that we check the hash no longer
holds in some cases (in particular for impure derivations).

Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038.

(cherry picked from commit c04bc17a5a)
2024-09-25 21:55:36 +00:00
Eelco Dolstra b4fcd27590 Merge pull request #11578 from Mic92/key-backport
[2.24-maintainence] Ensure error messages don't leak private key
2024-09-24 13:45:43 +02:00
John Ericson 082f6bb35d Ensure error messages don't leak private key
Since #8766, invalid base64 is rendered in errors, but we don't actually
want to show this in the case of an invalid private keys.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 2b6b03d8df)
2024-09-24 06:39:03 +02:00
John Ericson 1e03ea386b Revert "base64Decode: clearer error message when an invalid character is detected"
We have a safer way of doing this.

This reverts commit dc3ccf02bf.

(cherry picked from commit d0c351bf43)
2024-09-24 06:31:50 +02:00
John Ericson b523e4de34 Merge pull request #11571 from NixOS/mergify/bp/2.24-maintenance/pr-11390
Don't refer to public keys as secret keys in error (backport #11390)
2024-09-23 18:50:03 -04:00
Alyssa Ross 563dedcf64 Don't refer to public keys as secret keys in error
This constructor is used for public keys as well.

(cherry picked from commit 9cc550d652)
2024-09-23 22:00:10 +00:00
Eelco Dolstra a7fdef6858 Bump version 2024-09-20 01:19:15 +02:00
Eelco Dolstra b5154deba3 Merge pull request #11553 from NixOS/mergify/bp/2.24-maintenance/pr-11548
Fix missing GC root in zipAttrsWith (backport #11548)
2024-09-19 22:09:56 +02:00
Eelco Dolstra ecd83dc155 Use HAVE_BOEHMGC
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 4449b0da74)
2024-09-19 19:04:17 +00:00
Eelco Dolstra 5b5e1920eb Fix missing GC root in zipAttrsWith
My SNAFU was that I assumed that all the `Value *`s we put in
`attrsSeen` are already reachable (which they are), but I forgot about
the `elems` pointer in `ListBuilder`.

Fixes #11547.

(cherry picked from commit 0c2fdd2f3c)
2024-09-19 19:04:17 +00:00
Valentin Gagarin fc1d6b2f03 Merge pull request #11521 from NixOS/mergify/bp/2.24-maintenance/pr-8766
base64Decode: clearer error message when an invalid character is detected (backport #8766)
2024-09-19 11:29:21 +02:00
Brian McGee 9941f620c4 base64Decode: clearer error message when an invalid character is detected
Output the offending string in its entirety to provide context.

Closes #8479

(cherry picked from commit dc3ccf02bf)
2024-09-17 14:03:19 +00:00
Valentin Gagarin 5b2a8c223e Merge pull request #11497 from Mic92/mergify/bp/2.24-maintenance/pr-11378 2024-09-17 07:09:32 +02:00
mergify[bot]andTravis A. Everett 1b076b4f84 doc: add admonitions for macOS 15 Sequoia update (#11487) (#11509)
The impending release of macOS 15 Sequoia will break many existing nix
installs on macOS, which may lead to an increased number of people who
are looking to try to reinstall Nix without noticing the open/pinned
issue (#10892) that explains the problem and outlines how to migrate
existing installs.

These admonitions are a short-term measure until we are over the hump
and support volumes dwindle.

(cherry picked from commit 48477d4a3e)

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2024-09-16 16:03:05 +02:00
Robert Hensing d9ef3dd012 Merge pull request #11484 from NixOS/mergify/bp/2.24-maintenance/pr-11473
Fix making the build directory kept by `keep-failed` readable (backport #11473)
2024-09-16 12:46:23 +02:00
Robert Hensing f9714bac34 Merge pull request #11456 from NixOS/mergify/bp/2.24-maintenance/pr-11321
replace backport github action with mergify (backport #11321)
2024-09-16 12:41:14 +02:00
Jörg Thalheim 684a690480 update filesystem-errors changelog to 2.24 release 2024-09-13 14:20:34 +02:00
John Ericson 4354d90384 tweak unpack channel built-in, std::filesystem::path for tarball
(cherry picked from commit 193dc49097)
2024-09-13 14:11:36 +02:00
Jörg Thalheim 60001b1936 add release notes for filesystem fixes
Update doc/manual/rl-next/filesystem-errors.md

Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit 04ce0e648a)
2024-09-13 14:11:36 +02:00
Jörg Thalheim c84fc0120f builtins.unpackChannel: wrap filesystem errors and sanitize channelName
Otherwise these errors are not caught correctly

(cherry picked from commit 70c52d72f4)
2024-09-13 14:11:36 +02:00
Jörg Thalheim cd97688bce builtins.readDir: fix nix error trace on filesystem errors
Before:

nix-env % ./src/nix/nix eval --impure --expr 'let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f' --show-trace
error: filesystem error: directory iterator cannot open directory: No such file or directory [/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo]

After:

error:
       … while calling the 'readDir' builtin
         at «string»:1:9:
            1| let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f
             |         ^

       error: reading directory '/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo': No such file or directory

(cherry picked from commit 22ba4dc78d)
2024-09-13 11:56:41 +00:00
Eelco Dolstra 07909de6ed Merge pull request #11493 from NixOS/mergify/bp/2.24-maintenance/pr-11423
Git fetcher: Ignore .gitmodules entries that are not submodules (backport #11423)
2024-09-12 20:30:45 +02:00
Eelco Dolstra 751907dc8a Git fetcher: Ignore .gitmodules entries that are not submodules
Fixes #10739.

(cherry picked from commit 9d24080090)
2024-09-12 16:16:36 +00:00
Eelco Dolstra d9dd6c62d6 Merge pull request #11486 from NixOS/mergify/bp/2.24-maintenance/pr-11466
Git fetcher: Don't update mtime of ref file if fetching by rev (backport #11466)
2024-09-11 19:59:10 +02:00
Eelco Dolstra 97c5ac5752 Git fetcher: Don't update mtime of ref file if fetching by rev
This fixes the warning

  $ nix eval --store /tmp/nix --expr 'builtins.fetchTree { type = "git"; url = "https://github.com/DeterminateSystems/attic"; ref = "fixups-for-magic-nix-cache"; rev = "635753a2069d4b8228e846dc5c09ad361c75cd1a"; }'
  warning: could not update mtime for file '/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache': error: changing modification time of '"/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache"': No such file or directory

When we're fetching by rev, that file doesn't necessarily exist, and we
don't care about it anyway.

(cherry picked from commit b80b091bac)
2024-09-11 15:37:36 +00:00
Artturin 40461a8e0e Fix making the build directory kept by keep-failed readable
Caused by 1d3696f0fb

Without this fix the kept build directory is readable only by root

```
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5
drwx------ root root 60 B Wed Sep 11 00:09:48 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/

$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build
drwxr-xr-x nixbld1 nixbld 80 B Wed Sep 11 00:09:58 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build/
```

(cherry picked from commit ebebe626ff)
2024-09-11 12:56:18 +00:00
Eelco Dolstra 0f825b38f4 Bump version 2024-09-10 13:45:04 +02:00
Eelco Dolstra eb11c14998 Merge commit from fork
Fix unsafe NAR unpacking (2.24)
2024-09-10 12:42:55 +02:00
Eelco Dolstra a6ad5565ef Improve use-case-hack description slightly
(cherry picked from commit 5ca2f58798)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 2e1cb495c1 Typo
(cherry picked from commit 4cfa59fdb3)
2024-09-10 09:51:50 +02:00
Eelco Dolstra e25410c788 Test that deserializing regular files / symlinks is exclusive
(cherry picked from commit 52ba3cc5ea)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 25510ba66f RestoreSink::createDirectory(): Use append()
On macOS, `mkdir("x/')` behaves differently than `mkdir("x")` if `x` is
a dangling symlink (the formed succeed while the latter fails). So make
sure we always strip the trailing slash.

(cherry picked from commit 9fcb588dd8)
2024-09-10 09:51:50 +02:00
Eelco Dolstra d9b60b3902 Fix test on macOS
(cherry picked from commit 21dcbd7e83)
2024-09-10 09:51:50 +02:00
Eelco Dolstra a041688133 Test that deserializing NARs with names with equal Unicode normal forms fails on macOS
The test is based on the one by @puckipedia but with the file names
swapped to make them sorted.

(cherry picked from commit 7a765a6aaf)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 1288970496 Detect NAR directory entries that collide with another path after case-hacking
The test was made by @puckipedia.

(cherry picked from commit 3557587381)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 0cfc9bf133 More tests
(cherry picked from commit 77c090cdbd)
2024-09-10 09:51:50 +02:00
Eelco Dolstra f160d3ac68 Test that nix-store --restore fails if the output already exists
This restores the behaviour from before the std::filesystem
refactorings.

(cherry picked from commit da1ad28912)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 6187ee468f Add test case for NARs with duplicate directory entries
This test was made by @puckipedia.

(cherry picked from commit 83d5b32803)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 12fa019ae5 NAR parser: Fix check for duplicate / incorrectly sorted entries
"prevName" was always empty because it was declared in the wrong scope.

(cherry picked from commit 495d32e1b8)
2024-09-10 09:51:50 +02:00
Jörg Thalheim f0cffa7300 replace backport github action with mergify
The current backport action cannot automerge because
the github action bot does not trigger github CI actions.
Mergify instead does not have this limitation and can also
use a merge queue.

On top we have now a declarative configuration to allow
contributers to add new tests to required without having access
to the github org.

An example pull request and backport can be seen here:

https://github.com/Mic92/nix-1/pull/4

and here:

https://github.com/Mic92/nix-1/pull/5

To complete the setup the mergify app must be enabled for this repository.
It's already installed in the nixos organization for nixos-hardware and
other repositories.

(cherry picked from commit 80f20fa4cb)
2024-09-09 16:12:24 +00:00
Robert Hensing ae486b2910 Merge pull request #11446 from NixOS/mergify/bp/2.24-maintenance/pr-11285
fetchers::downloadTarball(): Return a cacheable accessor (backport #11285)
2024-09-09 18:05:32 +02:00
Eelco Dolstra 437f7a0042 fetchers::downloadTarball(): Return a cacheable accessor
downloadTarball() is used by `-I foo=<url>` etc. fetchToStore() needs
the accessor to have a fingerprint to enable caching.

Fixes #11271.

(cherry picked from commit 9f6ee93f48)
2024-09-09 12:46:07 +00:00
Eelco Dolstra b53b07b30b Merge pull request #11421 from NixOS/mergify/bp/2.24-maintenance/pr-10919
install-darwin: fix _nixbld uids for macOS sequoia (backport #10919)
2024-09-04 18:55:18 +02:00
Emily 8d0414d682 install-darwin: increment base UID by 1 (#15)
(cherry picked from commit 11cf29b15c)
2024-09-03 23:59:16 +00:00
Travis A. Everett c5a0e624d9 install-darwin: move nixbld gid to match first UID
(cherry picked from commit 75567423fb)
2024-09-03 23:59:15 +00:00
Travis A. Everett 0679505d8c install-darwin: fix _nixbld uids for macOS sequoia
Starting in macOS 15 Sequoia, macOS daemon UIDs are encroaching on our
default UIDs of 301-332. This commit relocates our range up to avoid
clashing with the current UIDs of 301-304 and buy us a little time
while still leaving headroom for people installing more than 32 users.

(cherry picked from commit df36ff0d1e)
2024-09-03 23:59:15 +00:00
Eelco Dolstra ea7abb58b5 Bump version 2024-09-03 16:51:36 +02:00
Eelco Dolstra f1ab41b2bc Merge pull request #11408 from NixOS/mergify/bp/2.24-maintenance/pr-11402
Respect max-substitution-jobs again (backport #11402)
2024-09-03 14:23:49 +02:00
Eelco Dolstra c21f664e82 "unsigned" -> size_t
Slight cleanup.

(cherry picked from commit b7acd1c414)
2024-09-03 11:36:40 +00:00
Eelco Dolstra 0c25bea7cc Respect max-substitution-jobs again
This broke in #11005. Any number of PathSubstitutionGoals would
be woken up by a single build slot becoming available. If there
are a lot of substitution goals active, this could lead to us
running out of file descriptors (especially on macOS where the
default limit is 256).

(cherry picked from commit a33cb8af56)
2024-09-03 11:36:39 +00:00
Eelco Dolstra 9d8669b14a Bump version 2024-08-23 16:15:11 +02:00
Eelco Dolstra cb0439f0c2 Merge pull request #11360 from DeterminateSystems/backport-11152
[Backport 2.24-maintenance] addToStore(): Evaluate on the main stack
2024-08-23 15:08:56 +02:00
Eelco Dolstra 30a57328d2 Backport https://github.com/NixOS/nix/pull/11152 2024-08-23 13:15:30 +02:00
Robert Hensing 79a43160ca Merge pull request #11334 from NixOS/backport-11329-to-2.24-maintenance
[Backport 2.24-maintenance] fix: check to see if there are any lines before
2024-08-19 16:38:44 +02:00
Tom Bereknyei 7befd60c01 fix: check to see if there are any lines before
(cherry picked from commit 59db8fd62b)
2024-08-19 13:40:38 +00:00
Eelco Dolstra 90fb4e8890 Bump version 2024-08-19 12:46:17 +02:00
tomberek 3ac5d736e2 Merge pull request #11319 from NixOS/backport-11270-to-2.24-maintenance
[Backport 2.24-maintenance] libstore: fix port binding in __darwinAllowLocalNetworking sandbox
2024-08-17 03:17:29 -04:00
Andrew Marshall 4e707b8e57 libstore: fix port binding in __darwinAllowLocalNetworking sandbox
In d60c3f7f7c, this was changed to close a
hole in the sandbox. Unfortunately, this was too restrictive such that it
made local port binding fail, thus making derivations that needed
`__darwinAllowLocalNetworking` gain nearly nothing, and thus largely
fail (as the primary use for it is to enable port binding).

This unfortunately does mean that a sandboxed build process can, in
coordination with an actor outside the sandbox, escape the sandbox by
binding a port and connecting to it externally to send data. I do not
see a way around this with my experimentation and understanding of the
(quite undocumented) macOS sandbox profile API. Notably it seems not
possible to use the sandbox to do any of:

- Restrict the remote IP of inbound network requests
- Restrict the address being bound to

As such, the `(local ip "*:*")` here appears to be functionally no
different than `(local ip "localhost:*")` (however it *should* be
different than removing the filter entirely, as that would make it also
apply to non-IP networking). Doing `(allow network-inbound (require-all
(local ip "localhost:*") (remote ip "localhost:*")))` causes listening
to fail.

Note that `network-inbound` implies `network-bind`.

(cherry picked from commit 00f6db36fd)
2024-08-17 03:17:45 +00:00
Eelco Dolstra 20cae372f4 Merge pull request #11314 from cole-h/backport-2.24-update-nixpkgs-input-fix-darwin
[2.24] Update nixpkgs input to fix darwin ccache evaluation, have CI check that all outputs on all systems evaluate
2024-08-16 20:48:49 +02:00
Cole Helbling d550139191 ci: check that all outputs for all systems can evaluate
(cherry picked from commit aa3d35c1f4)
2024-08-16 11:46:29 -07:00
Cole Helbling 5b62a1dbd6 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/c3d4ac725177c030b1e289015989da2ad9d56af0?narHash=sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz%2BNG82pbdg%3D' (2024-08-15)

(cherry picked from commit 8866d2cd83)
2024-08-16 11:46:24 -07:00
Eelco Dolstra 450252c92c Bump version 2024-08-08 17:21:00 +02:00
Eelco Dolstra 4036c3aafb Bump version 2024-08-08 15:02:48 +02:00
Robert Hensing 935bf1157d Merge pull request #11267 from NixOS/backport-11244-to-2.24-maintenance
[Backport 2.24-maintenance] allow to c api with older c versions
2024-08-08 01:43:24 +02:00
Jörg Thalheim b1941c9f8a allow to c api with older c versions
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix

(cherry picked from commit 739418504c)
2024-08-07 23:12:17 +00:00
Eelco Dolstra 40832b0a95 Merge pull request #11262 from NixOS/backport-11257-to-2.24-maintenance
[Backport 2.24-maintenance] PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
2024-08-06 11:20:31 +02:00
Eelco Dolstra fa78d7f72f PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
It is not an error if queryPathInfo() indicates that a path does not
exist in the substituter.

Fixes #11198. This was broken in 846869da0e.

(cherry picked from commit 0a00bd07b2)
2024-08-06 08:33:46 +00:00
Eelco Dolstra 2382a52c84 Merge pull request #11239 from NixOS/backport-11237-to-2.24-maintenance
[Backport 2.24-maintenance] Fix the S3 store
2024-08-01 17:56:18 +02:00
Eelco Dolstra fe6a7c805c Fix the S3 store
It was failing with:

   error: AWS error fetching 'nix-cache-info': The specified bucket does not exist

because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.

(cherry picked from commit 9b5b7b7963)
2024-08-01 15:46:45 +00:00
Eelco Dolstra 0a167ffd1f Bump version 2024-08-01 10:41:11 +02:00
Eelco Dolstra 206e32e2d7 Mark release 2024-07-31 23:37:43 +02:00
Eelco Dolstra b24757f08a Merge pull request #11231 from DeterminateSystems/release-notes
2.24 release notes
2024-07-31 23:24:39 +02:00
Eelco Dolstra 794a50065b base32 -> nix32 2024-07-31 22:33:41 +02:00
Qyriad cb5a5dd4f3 docs: clarify how ^ works for -E/-f installables
We didn't even realize you *could* use this syntax with -E and -f, much
less that the attribute path could be *empty*.

Change-Id: Id1a6715609f3a76a5ce477bd43a7832effbbe07b
2024-07-31 22:27:58 +02:00
Qyriad 8ff169715d docs: clarify how the different kinds of installables are selected
Change-Id: I146736bb97ebe035e04be69ce9fb60a557e38c6c
2024-07-31 22:27:58 +02:00
Eelco Dolstra f136ec5290 Add contributors 2024-07-31 22:16:44 +02:00
Eelco Dolstra 22ad0e653f Edit release notes 2024-07-31 22:14:27 +02:00
John Ericson 733c816d34 Small windows cross fixes (#11230) 2024-07-31 20:04:18 +00:00
Eelco Dolstra c952d933e5 release notes: 2.24.0 2024-07-31 21:57:31 +02:00
Valentin Gagarin db5bacb637 reword documentation on nix-path config option (#7772)
* docs: unify documentation on search paths

- put all the information on search path semantics into `builtins.findFile`
- put all the information on determining the value of `builtins.nixPath` into the
  `nix-path` setting

  maybe `builtins.nixPath` is a better place for this, but those bits
  can still be moved around now that it's all next to each other.
- link to the syntax page for lookup paths from all places that are
  concerned with it
- add or clarify examples
- add a test verifying a claim from documentation
2024-07-31 21:41:26 +02:00
Eelco Dolstra e8b37e9e70 Merge pull request #11220 from NixOS/release-credits
Credit all contributors in release notes
2024-07-31 21:12:03 +02:00
Eelco Dolstra ed0934b884 Merge pull request #11140 from DeterminateSystems/protocol-features
WorkerProto: Support fine-grained protocol feature negotiation
2024-07-31 17:47:38 +02:00
Eelco Dolstra 69fb2848a5 Merge pull request #11170 from NixOS/release-notes-2.24
Release notes for 2.24
2024-07-31 17:06:49 +02:00
tomberek 4c007bf88f Merge pull request #11216 from tomberek/tomberek.add_flake_headers
fix: add flake headers
2024-07-30 21:19:14 -04:00
Tom Bereknyei ef80217448 fix: add flake headers 2024-07-30 13:19:55 -04:00
Robert Hensing f011cfd28d maintainers/release-*: Add mode line
This lets various tools figure out the language more easily.
2024-07-30 17:54:37 +02:00
Corbin SimpsonandValentin Gagarin ee86e7f361 doc/command-ref/nix-shell: Shebangs can occur anywhere (#11202)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-30 12:51:47 +00:00
Eelco Dolstra c77b671a66 Merge pull request #11221 from pinotree/hurd-fixes
Some fixes for GNU/Hurd
2024-07-30 13:59:19 +02:00
Pino Toscano a1ccf60613 tests: define fallback PATH_MAX
Few filesystem-related tests rely on PATH_MAX for buffers, and PATH_MAX
is optional in POSIX (and not available on the Hurd). To make them build
and pass, provide a fallback definition of PATH_MAX in case not
available.

Ideally speaking, the tests ought to not unconditionally rely on
PATH_MAX, do alternative strategies (e.g. dynamically allocate buffers,
expand them as needed, etc); OTOH this is test code, so it would be more
work that what it would be worth, so IMHO the define fallback is good
enough.
2024-07-30 05:34:34 +02:00
Pino Toscano 7442f4a161 libutil: use /proc/self/exe on Hurd as well
Rely on the Linux-compatible procfs available on the Hurd to get the
path of the current executable.
2024-07-30 05:31:42 +02:00
Pino Toscano d7f46cf28e makefiles: recognize GNU/Hurd
Set HOST_HURD & HOST_UNIX for GNU/Hurd in the makefile-based build
system; the latter variable is important as it will include all the
commit Unix bits.
2024-07-30 05:29:32 +02:00
John Ericson 2b78561335 Merge pull request #11219 from obsidiansystems/better-warning-solution
Make sure we use `-isystem` with Meson on some deps
2024-07-29 17:57:36 -05:00
Robert Hensing 850e4779d6 release-credits: Remove bots 2024-07-30 00:15:49 +02:00
Robert Hensing f380becffa Credit all contributors in release notes 2024-07-29 23:58:38 +02:00
John Ericson 12717325cc Make sure we use -isystem with Meson on some deps
Otherwise we get warnings on external code.
2024-07-29 13:06:26 -04:00
Eelco Dolstra 0b96c586e0 Merge pull request #11195 from DeterminateSystems/tarball-roots
Improve handling of tarballs that don't consist of a single top-level directory
2024-07-29 16:58:59 +02:00
Eelco Dolstra 84243027ec Merge pull request #11127 from NixOS/issue-10635-c-api-error-enum
C API: Make nix_err an enum
2024-07-29 16:00:58 +02:00
Eelco Dolstra f9d55b4d51 Merge pull request #11191 from DeterminateSystems/hash-symbol
Use std::unordered_map for ValueMap
2024-07-29 15:30:37 +02:00
Eelco DolstraandRobert Hensing a3171cec54 Update src/libfetchers/git-utils.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-29 15:12:01 +02:00
Eelco Dolstra e8bf2e74a5 Add release note 2024-07-29 15:09:06 +02:00
Eelco Dolstra 71865dee2d Fix fetchTarball docs 2024-07-29 15:04:55 +02:00
Robert Hensing 6e3bba5e26 Merge pull request #11171 from DeterminateSystems/speed-up-tarball-downloads
Increase download buffer size and improve tarball import logging
2024-07-29 15:02:35 +02:00
Eelco Dolstra 7c18b4d060 Don't dereference top-level regular files
Since this yielded an empty directory as far back as Nix 2.3, we don't
really need special handling for executables vs non-executables.
2024-07-29 14:34:02 +02:00
Eelco Dolstra e0012b97ab Split tarball-specific logic from GitFileSystemObjectSink 2024-07-29 14:26:25 +02:00
Eelco Dolstra 3c0963487e Merge pull request #11196 from NixOS/rename-lock-read
Rename SyncBase::read() -> readLock()
2024-07-29 14:10:46 +02:00
Eelco Dolstra 836d24d6e8 Merge pull request #11209 from pinotree/libutil-current-process-includes
libutil: fix/improve includes in current-process.cc
2024-07-29 14:01:02 +02:00
Eelco Dolstra 9e2bed7827 Merge pull request #11206 from tie/getxattr-enotsup
libstore: return ENOTSUP for getxattr functions
2024-07-29 14:00:36 +02:00
Eelco Dolstra 3faa77bb82 Merge pull request #11200 from NixOS/buildNoTests-no-unit-tests
buildNoTests: Disable unit tests
2024-07-29 13:57:23 +02:00
Eelco Dolstra 673f2dcadb Merge pull request #11203 from pinotree/libutil-ctor-remove-template-id
libutil: remove template id from constructors
2024-07-29 13:56:55 +02:00
Pino Toscano c34077578e libutil: fix/improve includes in current-process.cc
- move <sys/resource.h> from a __linux__ block to a !_WIN32 block: this
  matches what the actual code does, using getrlimit() & setrlimit() in
  !_WIN32 blocks
- drop <sys/mount.h>, which is not portable, and it is not used
2024-07-28 17:33:24 +02:00
Pino Toscano 96e06b2b06 libutil: remove template id from constructors
This is not allowed in C++20, and GCC 14 warns about it:

../src/libutil/ref.hh:26:20: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   26 |     explicit ref<T>(const std::shared_ptr<T> & p)
      |                    ^
../src/libutil/ref.hh:26:20: note: remove the '< >'
../src/libutil/ref.hh:33:21: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
   33 |     explicit ref<T>(T * p)
      |                     ^
../src/libutil/ref.hh:33:21: note: remove the '< >'
2024-07-28 16:35:09 +02:00
Robert Hensing 0e151bcbf0 Merge pull request #11204 from pinotree/libcmd-editline-helpers
libcmd: do not compile editline helpers when building w/ readline
2024-07-28 16:10:32 +02:00
Valentin Gagarin 933f2c086a docs: fix link to building instructions (#11207) 2024-07-28 13:34:48 +00:00
Ivan Trubach 1b47748e5a libstore: return ENOTSUP for getxattr functions
This change updates the seccomp profile to return ENOTSUP for getxattr
functions family. This reflects the behavior of filesystems that don’t
support extended attributes (or have an option to disable them), e.g.
ext2.

The current behavior is confusing for some programs because we can read
extended attributes, but only get to know that they are not supported
when setting them. In addition to that, ACLs on Linux are implemented
via extended attributes internally and if we don’t return ENOTSUP, acl
library converts file mode to ACL.
https://git.savannah.nongnu.org/cgit/acl.git/tree/libacl/acl_get_file.c?id=d9bb1759d4dad2f28a6dcc8c1742ff75d16dd10d#n69
2024-07-28 13:28:52 +03:00
Pino Toscano e0198c513a libcmd: do not compile editline helpers when building w/ readline
The internal "completionCallback" and "listPossibleCallback" helpers
are used only when building with editline; hence, do not build then
when using readline, matching their usage in
"ReadlineLikeInteracter::init()".
2024-07-28 11:40:16 +02:00
Robert Hensing aa2b1d10e2 Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md 2024-07-27 14:58:57 +02:00
Robert Hensing 9f1e73ed37 Merge pull request #11199 from NixOS/troubleshoot-remote-build-tests
Troubleshoot remote build tests
2024-07-27 14:25:10 +02:00
Robert Hensing cc5b8cdc85 buildNoTests: Disable unit tests
This seems to have been the intent all along.

The odd combination of unit tests, but no functional tests caused a
build error where some data for the unit test was source-filtered out.
Apparently. It's unclear to me why that happened, so I'm proposing this
alternate "fix" to get the buildNoTests to pass.

It would be nice to test more configurations, but this mode of building
is on the way out anyway, so let's just make it pass and see what
configurations make sense to test as part of the meson migration.
2024-07-27 13:42:03 +02:00
Robert Hensing 7c5a0b06a4 tests/nixos/remote-builds: Wait for multi-user
This should make the test more robust, considering the strange hang
in https://hydra.nixos.org/build/267517233/nixlog/8

`builder` seems to have reached `multi-user.target` before the
SSH connection was established, but this seems to be coincidental.
This does tell us that enforcing this has a minimal cost in terms
of runtime.

Waiting for `multi-user.target` on the client is honestly paranoid,
but flaky tests are very bad for productivity.
2024-07-27 13:08:30 +02:00
Robert Hensing f4464873f5 tests/nixos/remote-builds: Print hello world to stderr
Trying to learn more about enigmatic spurious hang at
https://hydra.nixos.org/build/267517233/nixlog/8
- builder1 seems to have started properly
- ssh connection and session are established
- ssh client doesn't exit or client.succeed does not return
  for some reason.

Seeing the stdout on the console might give a tiny bit more info.
2024-07-27 13:01:56 +02:00
Robert Hensing 04c20dc0c0 Merge pull request #11197 from NixOS/aws-sdk-cpp-like-nixpkgs
dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
2024-07-27 03:30:39 +02:00
Robert Hensing 17b5d40445 package.nix: Empty build inputs if not doBuild 2024-07-27 02:39:55 +02:00
Robert Hensing 22f943bb1f dependencies: Centralize aws-sdk-cpp and sync with Nixpkgs
By syncing with Nixpkgs, we reuse the same derivation, which is
generally a good idea, and has the benefit that it is transitively
a channel blocker.

Changes:

- https://github.com/NixOS/nixpkgs/pull/163313 (SuperSandro2000)

  > nix: disable big-parallel for aws-sdk-cpp

  > aws-sdk-cpp only takes ~1m52s on a 4 core machine under 50% load
  > which does not justify the requirement on big parallel.

  > Tested with `nix-build -A nixVersions.nix_2_6.aws-sdk-cpp`.

  > I can finally build nix without requiring a big-parallel machine.

- https://github.com/NixOS/nixpkgs/pull/227506 (Artturin)

  > nix: use [ ] instead null to empty requiredSystemFeatures

  > fixes 'error: value is null while a list was expected' with 'nixpkgs.hostPlatform.gcc.arch = "x86_64";'
2024-07-27 02:16:05 +02:00
Robert Hensing 6af40f488a Rename SyncBase::read() -> readLock()
Make it explicit so it's clear what it's about when I and other
contributors read its call sites.
2024-07-27 01:39:13 +02:00
Robert Hensing 95845d92f7 Merge pull request #11192 from DeterminateSystems/store-sharedsync
Store: Use SharedSync
2024-07-27 01:32:13 +02:00
Robert Hensing 861bd102a6 Merge pull request #11167 from NixOS/repl-test-rejiggle
Fix repl test for `buildReadlineNoMarkdown`
2024-07-27 00:55:57 +02:00
Robert Hensing 88e8c9017a Merge pull request #11187 from Mic92/diff-closure-fix
diff-closures: fix a use after free
2024-07-27 00:52:38 +02:00
Eelco Dolstra c1f7ba7a98 Merge pull request #11190 from DeterminateSystems/unnecessary-eval-string
nix repl: Remove unnecessary call to evalString
2024-07-26 22:27:51 +02:00
Eelco Dolstra 5e83c0427f Fix test 2024-07-26 20:46:07 +02:00
Eelco Dolstra b88950ec77 Update fetchTree docs 2024-07-26 20:34:04 +02:00
Eelco Dolstra 06b686b62d Handle tarballs that don't consist of a single top-level directory
Fixes #4785 (top-level directories are no longer merged into one).

Fixes #10983 (top-level non-directories are no longer discarded).
2024-07-26 20:24:58 +02:00
Eelco Dolstra d9ba2a1634 Fix error message 2024-07-26 19:06:49 +02:00
Eelco Dolstra ea46264bd3 Store: Use SharedSync for state 2024-07-26 16:14:03 +02:00
Eelco Dolstra ce663d75e3 LRUCache: Mark size() as const 2024-07-26 16:13:00 +02:00
Eelco Dolstra 6d843ce9fe Provide std::hash<Symbol> 2024-07-26 16:06:09 +02:00
Eelco Dolstra 2141a52ca3 nix repl: Remove unnecessary call to evalString
This crashes with the multithreaded evaluator, which checks against
attempts to finish an already finished value.
2024-07-26 15:40:32 +02:00
Robert Hensing c274e005b6 Merge pull request #11186 from cole-h/fixup-docs-build
Fix reference to experimental features docs
2024-07-25 21:47:18 +02:00
Jade Lovelace 07aeedd37e diff-closures: remove gratuitous copy
This was done originally because std::smatch does not accept `const char
*` as iterators. However, this was because we should have been using
std::cmatch instead.

(cherry picked from commit https://git.lix.systems/lix-project/lix/commit/12a5838d11f790d36e2ac626e8916a2c19bcb80b)
2024-07-25 21:41:31 +02:00
Jade Lovelace 492715c0bb diff-closures: fix a use after free
Found by looking for interesting asan reports from the test suite.

What happened here is that name got overwritten, but it was what
actually held the backing memory for the thing it got overwritten by,
which was a by-reference value coming out of std::regex.

Due to absurd reasons I cannot seem to use a string_view iterator here,
so I just copy the string with a longer lifetime instead. idk lol

==3796364==ERROR: AddressSanitizer: heap-use-after-free on address 0x503000014c61 at pc 0x74843523bf1d bp 0x7ffc68351330 sp 0x7ffc68350af0
READ of size 3 at 0x503000014c61 thread T0
    0 0x74843523bf1c in __asan_memcpy (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x159f1c)
    1 0x6403cf6cbff4 in std::char_traits<char>::copy(char*, char const*, unsigned long) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/char_traits.h:445:33
    <...>
    7 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::str() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:966:6
    8 0x6403cf6cbff4 in std::__cxx11::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>::operator std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>() const /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/regex.h:955:16
    9 0x6403cf6cbff4 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:37:26
    10 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    11 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

0x503000014c61 is located 17 bytes inside of 21-byte region [0x503000014c50,0x503000014c65)
freed by thread T0 here:
    0 0x748435250470 in operator delete(void*) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16e470)
    <...>
    6 0x6403cf6cbda2 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string() /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:792:9
    7 0x6403cf6cbda2 in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:36:13
    8 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    <...>

previously allocated by thread T0 here:
    0 0x74843524fa38 in operator new(unsigned long) (/nix/store/mzhqknx2mc94jdz4n320hn1lml86398y-clang-wrapper-17.0.6/resource-root/lib/linux/libclang_rt.asan-x86_64.so+0x16da38)
    <...>
    9 0x6403cf6cb68c in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::basic_string<std::basic_string_view<char, std::char_traits<char>>, void>(std::basic_string_view<char, std::char_traits<char>> const&, std::allocator<char> const&) /nix/store/14c6s4xzhy14i2b05s00rjns2j93gzz4-gcc-13.2.0/include/c++/13.2.0/bits/basic_string.h:784:4
    10 0x6403cf6cb68c in nix::getClosureInfo[abi:cxx11](nix::ref<nix::Store>, nix::StorePath const&) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:33:21
    11 0x6403cf6cd70c in nix::printClosureDiff(nix::ref<nix::Store>, nix::StorePath const&, nix::StorePath const&, std::basic_string_view<char, std::char_traits<char>>) /home/jade/lix/lix2/build/src/nix/diff-closures.cc:54:25
    12 0x6403cf873331 in CmdProfileDiffClosures::run(nix::ref<nix::Store>) /home/jade/lix/lix2/build/src/nix/profile.cc:479:17
    <...>

(cherry-picked from https://git.lix.systems/lix-project/lix/commit/b9b1bbd22fc3b34a4d8e2090f0d033f742c32ee0)
2024-07-25 21:41:31 +02:00
Cole Helbling 90459e60dc Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 21:41:31 +02:00
Cole Helbling 6e178cd899 Fix reference to experimental features docs
Arose because https://github.com/NixOS/nix/pull/9014 merged before
https://github.com/NixOS/nix/pull/11131, but the latter did not rebase /
merge against the latest master.
2024-07-25 11:38:45 -07:00
Robert Hensing 6ec123ad6b Merge pull request #11131 from rhendric/rhendric/pipe-operators
libexpr: experimental pipe operators
2024-07-25 16:58:43 +02:00
Robert Hensing 76f04b4146 Merge pull request #11111 from NixOS/grep-safety-AGAIN
Grep newline safety
2024-07-25 16:30:33 +02:00
Eelco Dolstra 499a9b5f89 Merge pull request #11182 from Mic92/less-alloc
build-remote: only allocate storeUri once
2024-07-25 16:23:06 +02:00
Robert Hensing 55a654abfd Make panic() and unreachable() robust
Plus one or two tweaks.
2024-07-25 15:50:01 +02:00
Robert HensingandJörg Thalheim f0fe1d880d Update doc/manual/rl-next/10734-nix3-build-show-all-fod-errors-with-keep-going.md
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
2024-07-25 15:39:15 +02:00
Robert HensingandJohn Ericson baa28159d3 Update tests/functional/test-infra.sh
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-25 15:38:02 +02:00
Jörg Thalheim 9b5ce9acc2 build-remote: only allocate storeUri once
also it's probably not much overhead compared to the networking stuff
it, but it's less code at least.
2024-07-25 14:04:03 +02:00
Eelco Dolstra 99c20d6624 Merge pull request #11179 from obsidiansystems/misc-fix
Misc fixes
2024-07-25 14:01:54 +02:00
John Ericson ca1e1e983f Merge pull request #11180 from Mic92/override-warnings
add werror=suggest-override
2024-07-25 03:00:50 -04:00
John Ericson 36252fed6b Merge pull request #11176 from obsidiansystems/flake-test-group
Put flake functional tests in their own group
2024-07-25 02:41:11 -04:00
Jörg Thalheim 63e50a4b56 add werror=suggest-override
Improves code readability by making overrides explicit.
Inspired by lix code-base
2024-07-25 07:41:12 +02:00
Jörg Thalheim 2c07ea8abb build-utils-meson: remove oudated meson build comment 2024-07-25 07:40:36 +02:00
Jörg Thalheim 6c38bc0952 {src/perl,build-utils-meson/diagnostics}: sort cflags
This makes them easier to copy between places.
2024-07-25 07:38:00 +02:00
Robert Hensing 6662c9012d Merge pull request #11177 from obsidiansystems/skip-exit-code
Change skipped test error code from 99 to 77
2024-07-25 06:59:18 +02:00
John Ericson 90f7f2139e Merge pull request #11115 from NixOS/doc-derivation
Document builtins.derivation
2024-07-25 00:21:53 -04:00
John Ericson 1ae5738317 Fix some warnings
I think they came from the last Nixpkgs bump.
2024-07-25 00:02:43 -04:00
Robert Hensing b711fcbef9 rl-next: Drop zzz-other. Number soup. 2024-07-25 06:00:59 +02:00
John Ericson e92dd06a7b build-remote: Cope with long store URLs by falling back on hashing
I hit this in the Meson port of the functional tests, because the use of
standalone build directories.
2024-07-25 00:00:52 -04:00
Robert Hensing 7275d68d3b rl-next: Add top 10 by +1 reactions on PRs
We should use a metric that weighs the related issues.
Counterbalancing time doesn't make much sense to me.
If it's around for longer, the fix will be relevant to more people.
2024-07-25 05:57:53 +02:00
Robert Hensing 8a7e31362a rl-next: Add credit 2024-07-25 05:57:06 +02:00
John Ericson 3b49f7a143 Deduplicate our many package.nix a bit (#11175)
- They should all be built in parallel

- They should all use strict deps by default
2024-07-25 03:12:39 +00:00
Valentin Gagarin a2fed6db9e manual: Contributing -> Development, Hacking -> Building (#9014)
* manual: Contributing -> Development, Hacking -> Building

what's currently called "hacking" are really instructions for setting up
a development environment and compiling from source. we have
a contribution guide in the repo (which rightly focuses on GitHub
workflows), and the material in the manual is more about working
on the code itself.

since we'd otherwise have three headings that amount to "Building Nix",
this change also moves the "classic Nix" instructions to the top.

we may want to reorganise this in the future, and bring
contributor-oriented information closer to the code, but for now let's
stick to more accurate names to ease navigation.
2024-07-25 02:53:06 +00:00
John Ericson dcbe2453f5 Change skipped test error code from 99 to 77
Meson uses a venerable GNU convention described in
https://www.gnu.org/software/automake/manual/html_node/Scripts_002dbased-Testsuites.html
in which:

> When no test protocol is in use, an exit status of 0 from a test
> script will denote a success, an exit status of 77 a skipped test, an
> exit status of 99 a hard error, and any other exit status will denote
> a failure.

77 is thus what we want, not 99.
2024-07-24 22:36:43 -04:00
John Ericson f4915af71c Put flake functional tests in their own group
This is a nice thing to have, and it made it easier to work on the
Meson-ifcation of these functional tests too.
2024-07-24 22:31:37 -04:00
Valentin GagarinandRyan Hendrickson dba1142c01 docs: add identifiers (#11174)
* docs: add identifiers

* clarify attribute set notation and add examples

* add definition of names

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-25 03:45:34 +02:00
e062021314 fix NIX_PATH for real (#11079)
* fix NIX_PATH overriding

- test restricted evaluation
- test precedence for setting the search path

Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-07-24 21:17:15 +00:00
Eelco Dolstra 01839b525c Show when we're unpacking an archive into the Git cache
This happens in parallel with the download (which starts later), so
you only see this message when the download has finished but the
import hasn't.
2024-07-24 20:22:26 +02:00
Eelco Dolstra f6a9a71b38 Warn if the download buffer is full 2024-07-24 20:14:31 +02:00
Eelco Dolstra 8ffea0a018 Add 'download-buffer-size' setting
We are piping curl downloads into `unpackTarfileToSink()`, but the
latter is typically slower than the former if you're on a fast
connection. So the download could appear unnecessarily slow. (There is
even a risk that if the Git import is *really* slow for whatever
reason, the TCP connection could time out.)

So let's make the download buffer bigger by default - 64 MiB is big
enough for the Nixpkgs tarball. Perhaps in the future, we could have
an unlimited buffer that spills data to disk beyond a certain
threshold, but that's probably overkill.
2024-07-24 20:10:45 +02:00
Eelco Dolstra caf4e98f0c Log download durations 2024-07-24 20:10:41 +02:00
Robert Hensing 4bfc96f376 Fix and update release notes 2024-07-24 19:36:49 +02:00
Robert Hensing 459ee00563 Render the release notes when building the manual from dev shell 2024-07-24 19:36:49 +02:00
Ryan Hendrickson e086d5d899 libexpr: experimental pipe operators 2024-07-24 13:17:28 -04:00
Robert Hensing 3172e88af5 Make abort() call sites log first 2024-07-24 16:52:04 +02:00
Eelco Dolstra 874c1bdbbf Merge pull request #11169 from NixOS/flake-update
Flake update
2024-07-24 16:52:03 +02:00
Eelco Dolstra 3be7c0037e WorkerProto: Support fine-grained protocol feature negotiation
Currently, the worker protocol has a version number that we increment
whenever we change something in the protocol. However, this can cause
a collision between Nix PRs / forks that make protocol changes
(e.g. PR #9857 increments the version, which could collide with
another PR). So instead, the client and daemon now exchange a set of
protocol features (such as `auth-forwarding`). They will use the
intersection of the sets of features, i.e. the features they both
support.

Note that protocol features are completely distinct from
`ExperimentalFeature`s.
2024-07-24 16:23:37 +02:00
Eelco Dolstra b13ba7490c Merge pull request #11162 from NixOS/rename-stack-limit
Rename osStackLow -> osStackLimit
2024-07-24 16:18:26 +02:00
Robert Hensing 171ef75218 Rename pre-commit-hooks -> git-hooks-nix
Following the upstream rename
2024-07-24 15:55:57 +02:00
Robert Hensing c316f1557d flake: Switch to nixos-24.05 channel
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
  → 'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
2024-07-24 15:31:30 +02:00
Robert Hensing f9a23c8d21 flake.nix: Remove outdated comment 2024-07-24 15:30:28 +02:00
Robert Hensing 650f189402 flake.nix: Pin libgit2 to a release 2024-07-24 15:24:10 +02:00
Robert Hensing a16df88252 flake.lock: Update
Flake lock file updates:

• Updated input 'flake-compat':
    'github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9?narHash=sha256-4gtG9iQuiKITOjNQQeQIpoIB6b16fm%2B504Ch3sNKLd8%3D' (2023-01-17)
  → 'github:edolstra/flake-compat/0f9255e01c2351cc7d116c072cb317785dd33b33?narHash=sha256-kvjfFW7WAETZlt09AgDn1MrtKzP7t90Vf7vypd3OL1U%3D' (2023-10-04)
• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/9126214d0a59633752a136528f5f3b9aa8565b7d?narHash=sha256-sB4SWl2lX95bExY2gMFG5HIzvva5AVMJd4Igm%2BGpZNw%3D' (2024-04-01)
  → 'github:hercules-ci/flake-parts/9227223f6d922fee3c7b190b2cc238a99527bbb7?narHash=sha256-pQMhCCHyQGRzdfAkdJ4cIWiw%2BJNuWsTX7f0ZYSyz0VY%3D' (2024-07-03)
• Updated input 'libgit2':
    'github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D' (2023-10-18)
  → 'github:libgit2/libgit2/503b66cf00ad7dca940148529f60b1a409ccc462?narHash=sha256-tDUQi%2Bs8sxJ30SmUH7Ln9WmDz5jGatlgKumjwi7KnCo%3D' (2024-07-17)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/be3ca229c85e978880babdeda9748b14e6aa008f?narHash=sha256-L61BXz7n/yNzOeZ3FqlnUmxj4145JOVeq9fvQTQzbNM%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/d0907b75146a0ccc1ec0d6c3db287ec287588ef6?narHash=sha256-PhmkdTJs2SfqKzSyDB74rDKp1MH4mGk0pG/%2BWqrnGEw%3D' (2024-07-24)
• Updated input 'pre-commit-hooks':
    'github:cachix/pre-commit-hooks.nix/40e6053ecb65fcbf12863338a6dcefb3f55f1bf8?narHash=sha256-nMirxrGteNAl9sWiOhoN5tIHyjBbVi5e2tgZUgZlK3Y%3D' (2024-04-12)
  → 'github:cachix/pre-commit-hooks.nix/f451c19376071a90d8c58ab1a953c6e9840527fd?narHash=sha256-6FPUl7HVtvRHCCBQne7Ylp4p%2BdpP3P/OYuzjztZ4s70%3D' (2024-07-15)
• Removed input 'pre-commit-hooks/flake-utils'
2024-07-24 15:18:50 +02:00
Robert Hensing 68693276f9 Update fixupBoehmStackPointer doc
lo might have made sense in the bdwgc code, maybe?, but not here.
2024-07-24 13:02:49 +02:00
Robert Hensing 907b0a371a Rename osStackLow -> osStackLimit
This is in accordance with ARM's naming convention.
"Low" is confusing, because it could refer to either the cold end
of the stack as an abstract data type, or a low address.
These are different places, because the stack grows down through
the address space.
2024-07-24 13:02:49 +02:00
Robert Hensing e48e0cbab0 markdown.hh: Improve includes 2024-07-24 12:54:40 +02:00
Robert Hensing 0bd2d36375 Document renderMarkdownToTerminal 2024-07-24 12:53:37 +02:00
Robert Hensing 7d4d34a27d eval-gc.cc: Fix warning 2024-07-24 12:48:38 +02:00
Robert Hensing 6e680a6644 tests/functional/repl: Improve failure reporting 2024-07-24 12:48:38 +02:00
Robert Hensing c4ae9bb45b tests/functional/repl: Normalize final prompt 2024-07-24 12:48:38 +02:00
Robert Hensing ca2cc26e12 tests/functional/repl: Improve precision and readability
... as well as match buildReadlineNoMarkdown.

Unfortunately it doesn't support long inputs or multiline inputs
for now.
This needs to make better use of the interacter interface.
2024-07-24 12:48:38 +02:00
Robert Hensing 712ce2feac ReadlineLikeInteracter::getLine: Add _NIX_TEST_REPL_ECHO env var
... for testing
2024-07-24 12:48:38 +02:00
Robert Hensing 97b0114ab8 renderMarkdownToTerminal: Add _NIX_TEST_RAW_MARKDOWN env var
For testing only.
2024-07-24 12:48:38 +02:00
Robert Hensing 2b4e3f04a4 markdown.cc: Format
Slightly custom because the automated formatting messes up the
braced initializer with named fields.
2024-07-24 01:02:48 +02:00
Eelco Dolstra fb450de20e Merge pull request #11163 from NixOS/rejiggle-getGCCyles-buildNoGc
Rejiggle getGCCycles() for buildNoGc
2024-07-23 17:17:47 +02:00
Robert Hensing 5d6bc484be Hide getGCCycles when we have no GC
Alternatively, we could make it return 0, but we don't need it in
the first place because the caller exists conditionally too.
2024-07-23 16:24:43 +02:00
Robert Hensing eb89e50cbb Rejiggle getGCCycles() for buildNoGc 2024-07-23 16:21:43 +02:00
John Ericson e20b0a5dda Merge pull request #11159 from DeterminateSystems/framedsource-terminate
~FramedSource(): Don't throw an exception if the remote has disconnected
2024-07-23 09:19:39 -04:00
Eelco Dolstra 498eed0a25 ~FramedSource(): Don't throw an exception if the remote has disconnected
This would cause the daemon to crash with a call to terminate().
2024-07-23 10:56:29 +02:00
Robert Hensingandtomberek 4457cebe05 Update comment in tests//vars-and-functions.sh
Co-authored-by: tomberek <tomberek@users.noreply.github.com>
2024-07-23 10:24:18 +02:00
Eelco Dolstra babfd0cd99 Merge pull request #11150 from hercules-ci/issue-11141-broken-sp-corrector
Fix issue #11141 broken stack pointer corrector
2024-07-22 19:59:46 +02:00
John Ericson 046cc8cbb3 Merge pull request #11156 from obsidiansystems/fix-bsd-pthread
Fix some BSD builds missing pthread functions
2024-07-22 13:58:41 -04:00
John Ericson 00fd180a79 Merge pull request #11153 from nix-windows/wine-ci
Run unit tests with wine
2024-07-22 13:35:32 -04:00
John Ericson d39bbcabb9 Fix some BSD builds missing pthread functions
In addition to adding the missing thread deps in the last commit, we
also appear to need to skip `-Wl,--as-needed` flags that Meson wants to
use, but doesn't work with our *BSD toolchains.

See https://github.com/mesonbuild/meson/issues/3593
2024-07-22 13:10:03 -04:00
John Ericson da625b8817 Merge pull request #11155 from obsidiansystems/build-system-fixes
Two Build system fixes
2024-07-22 12:16:56 -04:00
John Ericson d3cee8160c Add missing threads deps 2024-07-22 11:46:54 -04:00
Robert Hensing f2e0cecf34 tests/functional/lang-gc: Disable for now 2024-07-22 17:46:08 +02:00
Eelco Dolstra fe158e3e92 Merge pull request #10603 from edolstra/flake-regressions
Run the flake regressions test suite
2024-07-22 17:45:27 +02:00
Eelco Dolstra 9b9fc405d2 Merge pull request #11154 from DeterminateSystems/ping-store
nix ping-store: Redirect to 'nix store info'
2024-07-22 17:38:51 +02:00
Eelco Dolstra d08bb025e1 Merge pull request #11043 from hercules-ci/assert-eq
`assert`: Report why values aren't equal
2024-07-22 17:34:28 +02:00
John Ericson d7024ac9b7 Add S3 opt dep to Meson, and simplify build
Numeric version macros are now defined upstream, so we don't need roll
our own.
2024-07-22 11:11:38 -04:00
John Ericson 823baa25f3 Meson build: libstore check for statvfs 2024-07-22 11:09:53 -04:00
Eelco Dolstra 4878c31815 nix ping-store: Redirect to 'nix store info'
This avoids the double warning

   warning: 'ping-store' is a deprecated alias for 'store ping'
   warning: 'nix store ping' is a deprecated alias for 'nix store info'
2024-07-22 16:39:02 +02:00
John Ericson eea63d5f99 Run unit tests with wine
Fix #10547

The tests do not all run successfully, but that is a separate issue.
2024-07-22 10:31:26 -04:00
John Ericson dc6dbbc1a5 Test exes do not need dev outputs 2024-07-22 10:26:05 -04:00
John Ericson 112373c03c flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/88269ab3044128b7c2f4c7d68448b2fb50456870' (2024-06-03)
  → 'github:NixOS/nixpkgs/be3ca229c85e978880babdeda9748b14e6aa008f' (2024-07-21)
2024-07-22 10:26:05 -04:00
Eelco Dolstra f343364918 Merge remote-tracking branch 'origin/master' into flake-regressions 2024-07-22 16:03:45 +02:00
Eelco Dolstra 40f80e1b5c Merge pull request #11142 from detroyejr/fix-alias-flags
Allow flag aliases
2024-07-22 15:53:41 +02:00
Eelco Dolstra 56a2f7ab1d Merge pull request #11102 from fzakaria/issue-10532
doc: Add comment for fetchurl for name & url
2024-07-22 15:49:41 +02:00
Eelco Dolstra 3ef72040bf Merge pull request #11149 from hercules-ci/gc-stats
libexpr: Track and show GC time and cycle number
2024-07-22 15:39:50 +02:00
Robert Hensing 380becf0db Fix #11141 broken sp corrector 2024-07-22 14:52:08 +02:00
Robert Hensing b16861d82e libexpr: Track and show GC time and cycle number 2024-07-22 12:56:01 +02:00
Robert Hensing c00e396494 maintainers/README: Update Monday meeting time (#11147) 2024-07-22 12:05:50 +02:00
0ec5e3a1bc Progress on Wine CI support, MinGW dev shell with Meson (#10975)
* Only build perl subproject on Linux

* Fix various Windows regressions

* Don't put the emulator hook in test builds

  We run the tests in a separate derivation. Only need it for the dev shell.

* Fix native dev shells

* Fix cross dev shells we don't know how to emulate

Co-authored-by: PoweredByPie <poweredbypie@users.noreply.github.com>
Co-authored-by: Joachim Schiele <js@lastlog.de>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-21 22:03:04 +00:00
Robert Hensing 56757e15cf Merge pull request #11123 from tweag/fix-non-det-parser-printing
Fix non-deterministic inherit printing
2024-07-20 19:12:48 +02:00
Robert Hensing 584f8cb065 Merge pull request #11120 from tweag/early-string-cutoff
parser: Remove empty multiline string parts earlier
2024-07-20 19:10:50 +02:00
detroyejr d54dfbf879 addFlag: test that alias flags are allowed 2024-07-19 15:56:54 -04:00
detroyejr 74dccef004 addFlag: use aliases 2024-07-19 15:05:00 -04:00
Eelco Dolstra c4213f0e6c Merge pull request #11125 from DeterminateSystems/basic-connection
Factor out commonality between WorkerProto::Basic{Client,Server}Connection
2024-07-19 14:26:04 +02:00
Eelco Dolstra 58a79b6943 performOp(): Take a WorkerProto::BasicServerConnection 2024-07-19 13:35:46 +02:00
John Ericson d629b27e3c Merge pull request #11138 from emilazy/push-znxusvxozllv
libmain: add missing header include
2024-07-18 21:46:47 -04:00
Silvan MosbergerandRobert Hensing 0c91bb97e5 parser: Remove empty multiline string parts earlier
Makes parsing more consistent and is a super minor optimisation

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:43:44 +02:00
Silvan MosbergerandRobert Hensing a6dccae223 Fix non-deterministic parser printing
In _very_ rare cases (I had about 7 cases out of 32200 files!),
the order of how inherit-from bindings are printed when using
`nix-instantiate --parse` gets messed up.

The cause of this seems to be because the std::map the bindings are
placed in is keyed on a _pointer_, which then uses an
[implementation-defined strict total order](https://en.cppreference.com/w/cpp/language/operator_comparison#Pointer_total_order).

The fix here is to key the bindings on their displacement instead,
which maintains the same order as they appear in the file.

Unfortunately I wasn't able to make a reproducible test for this in the
source, there's something about the local environment that makes it
unreproducible for me.

However I was able to make a reproducible test in a Nix build on a Nix
version from a very recent master:

    nix build github:infinisil/non-det-nix-parsing-repro

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-19 00:42:00 +02:00
Emily 31e151386b libmain: add missing header include 2024-07-18 23:26:22 +01:00
Eelco Dolstra fa7aa0389a FdSource: Fix operator =
This wasn't moving the underlying buffer, so if the buffer was
non-empty, it could lose data.
2024-07-18 16:10:48 +02:00
Eelco Dolstra d231d802f5 Typo 2024-07-18 16:10:48 +02:00
Eelco Dolstra c1d5cf6f34 Factor out commonality between WorkerProto::Basic{Client,Server}Connection
This also renames clientVersion and daemonVersion to the more correct
protoVersion (since it's the version agreed to by both sides).
2024-07-18 16:10:48 +02:00
Robert Hensing 8ce4287409 Merge pull request #11086 from kognise/eval-cache-fixes
Eval cache: fix cache regressions
2024-07-18 14:57:07 +02:00
John Ericson 606ace41d2 Merge pull request #11112 from obsidiansystems/move-schemes-set
Move `uriSchemes` to `*StoreConfig`
2024-07-18 00:39:07 -04:00
John Ericson 2aa9cf34dd Move uriSchemes to *StoreConfig
It is a property of the configuration of a store --- how a store URL is
parsed into a store config, not a store itself.

Progress towards #10766
2024-07-17 23:48:19 -04:00
Farid ZakariaandJohn Ericson 57399bfc0e Refactor unix domain socket store config (#11109)
Following what is outlined in #10766 refactor the uds-remote-store such
that the member variables (state) don't live in the store itself but in
the config object.

Additionally, the config object includes a new necessary constructor
that takes a scheme & authority.

Tests are commented out because of linking errors with the current config system.
When there is a new config system we can reenable them.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-07-17 23:32:27 -04:00
Eelco Dolstra 17051ca80a Merge pull request #11061 from rhendric/rhendric/reference-manual
docs: fill out language/types.md#type-path
2024-07-17 21:59:05 +02:00
Eelco Dolstra 1e1a8e8ad1 Merge pull request #11058 from hercules-ci/more-nix-shell
Make `#!nix-shell` arguments and options relative to script
2024-07-17 21:52:34 +02:00
John Ericson a915862450 Merge pull request #11126 from DeterminateSystems/dont-send-version-twice
BasicClientConnection::handshake(): Don't send our version twice
2024-07-17 15:50:17 -04:00
Eelco Dolstra 6867cb1096 Merge pull request #11103 from fzakaria/issue-10795
lint: fix shellcheck for misc/systemv/nix-daemon
2024-07-17 21:46:17 +02:00
Eelco Dolstra b23da1ceca Merge pull request #11100 from NixOS/pretty-print-idempotent
Pretty print idempotently
2024-07-17 21:35:27 +02:00
Eelco Dolstra 05751de32b Merge pull request #10931 from hercules-ci/test-run-and-shell-envs
Test the `run` and `shell` envs for stray variables
2024-07-17 21:21:14 +02:00
RTUnrealandValentin Gagarin f0a1c130a1 doc: add example usage for Gitea in tarball fetcher (#11116)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-07-17 19:08:33 +00:00
Robert Hensing 83d585b423 C API: Make nix_err an enum
This generally gives a better experience with bindings generators,
possibly other tooling.

A possible risk is that some generators may not represent unknown
codes correctly.
Rust bindgen by default generates suitable code:
  * a type alias nix_err = c_int
  * individual constants for the known enum values
It does _not_ generate a closed type that can only hold the values
that were known at code generation time.

If this proves to be a problem, we could instead split the type:

`typedef int nix_err;` for return values
`enum nix_known_err` for code generation.

This would complicate the interface, so let's not do it unless it
is shown to be needed.
2024-07-17 16:54:30 +02:00
Eelco Dolstra 87f8ff23fe BasicClientConnection::handshake(): Don't send our version twice
This was accidentally introduced
in f71b4da0b3.  We didn't notice this
because the version got interpreted by the daemon as the obsolete "CPU
affinity will follow" field, and being non-zero, it would then read
another integer for the ignored CPU affinity.
2024-07-17 16:51:53 +02:00
Eelco Dolstra 621c23bbea Merge pull request #11113 from NixOS/doc-comment-unordered-map
Doc comments: use std::unordered_map
2024-07-17 16:50:48 +02:00
Las Safin 464e5925cb Avoid accessing uninitialized settings in own init (#11117)
The default value for the setting was evaluated by
calling a method on the object _being currently constructed_,
so we were using it before all fields were initialized.

This has been fixed by making the called method static,
and not using the previously used fields at all.

But functionality hasn't changed!
The fields were usually always zero (by chance?) anyway,
meaning the conditional path was always taken.

Thus the current logic has been kept, the code simplified,
and UB removed.

This was found with the helper of UBSan.
2024-07-17 14:10:01 +02:00
Robert Hensing da3eff60bc printList: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing a0635a80b2 printAttrs: Force item before determining whether to print multi-line 2024-07-17 13:42:05 +02:00
Robert Hensing ece334b532 tests/functional/repl: Characterize side effecting print behavior
Reported on matrix by aleksana:
https://matrix.to/#/!VRULIdgoKmKPzJZzjj:nixos.org/$7wZp5lUDTd-_u6MYo8kWWcysjtqTiQqP8dLI0RDNVVM?via=nixos.org&via=matrix.org&via=nixos.dev
2024-07-17 13:42:05 +02:00
Robert Hensing f5ebaea277 Simplify PosIdx::hash()
In C++ we don't need to salt the hash.
2024-07-17 13:31:31 +02:00
Robert Hensing b230c01f73 Merge pull request #11014 from obsidiansystems/plugins-libmain
Move plugins infra to `libnixmain`
2024-07-17 09:42:09 +02:00
Silvan Mosberger 9fae50ed4b Add parser test for indented strings
So that in the next commit we can see what changes about this test
2024-07-17 02:43:47 +02:00
John Ericson 463256b9e8 Merge pull request #11122 from L-as/fix-ub
Fix ub
2024-07-16 18:51:17 -04:00
Las Safin a1f3f103bc Check if drv is initialized in DerivationGoal::waiteeDone
It might not be set, in which case we shouldn't do anything.
Surprisingly, this somehow did not cause segfaults before?

Caught by UBSan.
2024-07-16 22:01:39 +00:00
Las Safin 5b6a21acc5 Avoid casting function pointer in libutil test support
Casting function pointers seems to be almost always UB.
See https://stackoverflow.com/questions/559581/casting-a-function-pointer-to-another-type

Fixed by doing the casting of `void*` to `std::string*` inside the function instead.

Caught by UBSan.
2024-07-16 22:01:34 +00:00
Robert Hensing 8f22245fdf Merge pull request #11119 from philiptaron/attrbute
builtins.genericClosure: fix documentation typo
2024-07-16 23:54:59 +02:00
Philip Taron 0a1a116f4b builtins.genericClosure: fix documentation typo 2024-07-16 13:51:52 -07:00
Robert Hensing 7dce074634 tests/functional/lang: Avoid /usr/bin/env for sandbox 2024-07-16 22:43:56 +02:00
Robert Hensing d0e9878389 Remove unused boost include and split out std-hash.hh
Splitting it out immediately answers questions like [this],
without increasing the number of compilation units.

I did consider using boost::hash_combine instead, but it doesn't seem
to be quite as capable, accepting only two arguments.

[this]: https://github.com/NixOS/nix/pull/11113#discussion_r1679991573
2024-07-16 22:31:25 +02:00
Robert Hensing f9a1d6b018 tests/functional/lang: Add post processing and remove certain line numbers 2024-07-16 17:36:30 +02:00
Eelco Dolstra 9300f855fc Merge pull request #11114 from DeterminateSystems/fix-warning
Pos::getSnippetUpTo(): Fix warning
2024-07-16 17:24:43 +02:00
Robert Hensing 74698d54c8 Document builtins.derivation 2024-07-16 17:20:09 +02:00
Robert Hensing 64b46000ad Add std::hash<PosIdx> 2024-07-16 16:46:41 +02:00
Eelco Dolstra 3d8fa9f668 Pos::getSnippetUpTo(): Fix warning 2024-07-16 16:34:13 +02:00
Robert HensingandEelco Dolstra 6c9d62dceb Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-16 15:19:39 +02:00
Eelco Dolstra 9c6678da0e Merge pull request #11092 from DeterminateSystems/hash-SourcePath
Use std::unordered_map for the EvalState caches
2024-07-16 11:06:43 +02:00
John Ericson 4bbadba83a Merge pull request #11108 from obsidiansystems/remote-store-constructors
Ensure we can construct remote store configs in isolation
2024-07-15 22:51:22 -04:00
Robert Hensing 41a03738d6 tests/functional: Also keep plain grep calls safe from newlines 2024-07-16 01:54:12 +02:00
Robert Hensing 644b97ce25 tests/functional: Make our grep* helpers reject newlines in the query
Newlines behave like *OR*; not "and then".
2024-07-16 01:41:22 +02:00
Robert Hensing f2df3f0c6c tests/vars-and-functions: Add callerPrefix helper 2024-07-16 01:40:33 +02:00
Robert Hensing 783a8341ee tests/functional: Support negative codes in expect, expectStderr 2024-07-16 01:32:54 +02:00
Robert Hensing 03326d606f Merge pull request #11110 from GoldsteinE/fix-repl-tests
tests/functional/repl.sh: fail test on wrong stdout
2024-07-16 00:23:42 +02:00
John Ericson 808082ea03 Ensure we can construct remote store configs in isolation
Progress towards #10766

I thought that #10768 achieved, but when I went to use this stuff (in
Hydra), turns out it did not. (Those `using FooConfig;` lines were not
working --- they are so finicky!) This PR gets the job done, and adds
some trivial unit tests to make sure I did what I intended.

I had to add add a header to expose `SSHStoreConfig`, after which the
preexisting `ssh-store-config.*` were very confusingly named files, so I
renamed them to `common-ssh-store-config.hh` to match the type defined
therein.
2024-07-15 17:32:49 -04:00
John Ericson 0feeab755a Move plugins infra to libnixmain
They are not actually part of the store layer, but instead part of the
Nix executable infra (libraries don't need plugins, executables do).

This is part of a larger project of moving all of our legacy settings
infra to libmain, and having the underlying libraries just have plain
configuration structs detached from any settings infra / UI layer.

Progress on #5638
2024-07-15 17:26:03 -04:00
John Ericson 1a273a623f Inline settings.pluginFiles.name
In theory the warning is more noisy now, but in practice this will not
happen unless the client is older than 2.14 (highly unlikely).
2024-07-15 16:50:57 -04:00
Las Safin 846869da0e Make goals use C++20 coroutines (#11005)
undefined
2024-07-15 16:49:15 -04:00
Goldstein 1bec90e3c4 tests/functional/repl.sh: fail test on wrong stdout
Previous test implementation assumed that grep supports newlines
in patterns. It doesn't, so tests spuriously passed, even though
some tests outputs were broken.

This patches output (and expected output) before grepping,
so there're no newlines in pattern.
2024-07-15 23:11:26 +03:00
Robert Hensing c6b5503190 Merge pull request #11072 from NixOS/doc-comments
Doc comments
2024-07-15 21:00:42 +02:00
Robert Hensing 61a4d3d45c getSnippetUpTo: Return optional
This makes it possible to certain discern failures from empty
snippets, which I think is an ok review comment.

Maybe it should do so for swapped column indexes too, but I'm not
sure.

I don't think it matters in the grand scheme. We don't even have
a real use case for `nullopt` now anyway.

Since we don't have a use case, I'm not applying this logic to
higher level functions yet.
2024-07-15 20:10:45 +02:00
Robert Hensing 03d33703ef Revert "Use HintFmt for doc comments"
Unfortunately these don't render correctly, because they go into the
markdown renderer, instead of the terminal.

```
nix-repl> :doc lib.version
Attribute '[35;1mversion[0m'

    … defined at [35;1m/home/user/h/nixpkgs/lib/default.nix:73:40[0m
```

We could switch that to go direct to the terminal, but then we should
do the same for the primops, to get a consistent look.

Reverting for now.

This reverts commit 3413e0338cbee1c7734d5cb614b5325e51815cde.
2024-07-15 19:56:40 +02:00
Robert Hensing ce31a0457f Use HintFmt for doc comments 2024-07-15 19:56:40 +02:00
Robert Hensing 6a125e65d0 Revert "Doc comments: use std::unordered_map"
hash<SourcePath> isn't implemented yet, and I can't cherry-pick
a bug-free commit yet.

This reverts commit 95529f31e3bbda99111c5ce98a33484dc6e7a462.
2024-07-15 19:56:40 +02:00
Robert Hensing ac89df815d libcmd/repl.cc: Explain evalString call and defend 2024-07-15 19:56:40 +02:00
Robert HensingandEelco Dolstra 21817473e8 Doc comments: use std::unordered_map
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 19:56:40 +02:00
Robert Hensing 131b6ccc71 nixexpr.hh: Avoid the warning and pragmas 2024-07-15 19:56:40 +02:00
Robert Hensing 6bbd493d49 libcmd/repl-interacter: INT_MAX -> numeric_limits 2024-07-15 19:56:40 +02:00
Robert Hensing 8a855296f5 tests/function/repl: Characterise the missing doc comment behavior 2024-07-15 19:56:40 +02:00
Robert Hensing 71cb8bf509 libexpr: Rename "column" fields to offset
... because that's what they are.
2024-07-15 19:56:40 +02:00
Robert Hensing 77e9f9ee82 libexpr: Get rid of unused line tracking fields 2024-07-15 19:56:40 +02:00
Robert Hensing f9243eca75 tests/functional/repl.sh: Work around GHA failure 2024-07-15 19:56:40 +02:00
Robert Hensing cef11b23e8 Add missing .sh in _NIX_TEST_ACCEPT=1 message 2024-07-15 19:56:40 +02:00
Robert Hensing d4f576b0b2 nix repl: Render docs for attributes 2024-07-15 19:56:40 +02:00
Robert Hensing 491b9cf415 Refactor: extract DocComment::getInnerText(PosTable) 2024-07-15 19:56:40 +02:00
Robert Hensing e68234c4f9 libexpr: Rearrange lexer files so that yylex_init_extra can be found 2024-07-15 19:56:40 +02:00
Robert Hensing 7fae378835 Track doc comments and render them in :doc 2024-07-15 19:56:40 +02:00
Robert Hensing e5af7cbeb9 libutil: Add Pos::getSnippetUpTo(Pos) 2024-07-15 18:41:11 +02:00
Robert Hensing 63f520fd00 doc/testing: Typo 2024-07-15 18:41:11 +02:00
Farid ZakariaandEelco Dolstra 945fff5674 Apply suggestions from code review
Add @edolstra suggestion fixes.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-15 09:12:56 -07:00
Eelco Dolstra 550b3479cf Include the accessor in the SourcePath hash 2024-07-15 15:46:30 +02:00
Farid Zakaria 104aba0fad Remove nix-daemon from exclusion 2024-07-14 19:57:55 -07:00
Farid Zakaria bc801e2c59 lint: fix shellcheck for misc/systemv/nix-daemon
Got shellcheck passing for misc/systemv/nix-daemon

Not sure how to test this since it's not running on my NixOS machine and
I see no references to it in the directory otherwise.

See #10795
2024-07-14 19:56:03 -07:00
Farid Zakaria a4ce96e5f1 doc: Add comment for fetchurl for name & url
fetchurl can be given a name and url aside from just the url.
Giving a name can be useful if the url has invalid characters such as
tilde for the store.
2024-07-14 19:07:18 -07:00
Robert Hensing 9d7397c4ba Merge pull request #11093 from NixOS/fix-concatStringsSep
Fix C++ `concatStringsSep`
2024-07-14 13:11:29 +02:00
Robert Hensing 7e604f716c concatStrings: Give compiler access to definition for inlining
... at call sites that are may be in the hot path.

I do not know how clever the compiler gets at these sites.
My primary concern is to not regress performance and I am confident
that this achieves it the easy way.
2024-07-14 12:20:45 +02:00
Robert Hensing 97e01107ec dropEmptyInitThenConcatStringsSep -> concatStringSep: empty separator
When the separator is empty, no difference is observable.

Note that concatStringsSep has centralized definitions. This adds the
required definitions. Alternatively, `strings-inline.hh` could be
included at call sites.
2024-07-14 12:10:39 +02:00
Robert Hensing d40fdb5711 dropEmptyInitThenConcatStringsSep: Update doc and deprecate 2024-07-14 11:50:20 +02:00
Robert Hensing 1c97718146 dropEmptyInitThenConcatStringsSep: Allow it to drop items again
It's usually harmless, if it occurs at all.
2024-07-13 03:06:24 +02:00
Robert Hensing 6b2c277c36 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are not empty
... but if they are, I'd like to see at least a hint of it so that
I'd know to fix it.
2024-07-13 03:06:24 +02:00
Robert Hensing 76b2d5ef3d dropEmptyInitThenConcatStringsSep -> concatStringSep: PATH handling
It's still wrong, but one step closer to correct.
Not that anyone should use "" or "." in their PATH, but that is not
for us to intervene.
2024-07-13 03:06:24 +02:00
Robert Hensing 9ca42d5da2 dropEmptyInitThenConcatStringsSep -> concatStringSep: setting value was already harmed
Considering that `value` was probably parsed with tokenizeString
prior, it's unlikely to contain empty strings, and we have no
reason to remove them either.
2024-07-13 03:06:24 +02:00
Robert Hensing 4029426ca8 dropEmptyInitThenConcatStringsSep -> concatStringSep: tokens from tokenizeString are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0fe3525223 illegal configuration line -> syntax error in configuration line
The law has nothing to do with this, although I do feel like a
badass when I mess with the config.
I'm a conf artist.
2024-07-13 03:06:24 +02:00
Robert Hensing cf3c5cd189 dropEmptyInitThenConcatStringsSep -> concatStringSep: showVersions version is not empty 2024-07-13 03:06:24 +02:00
Robert Hensing d9043021df dropEmptyInitThenConcatStringsSep -> concatStringSep: break nix help "" "" "" build
Garbage in, error out. Experimental CLI. Zero derivations given.
2024-07-13 03:06:24 +02:00
Robert Hensing 062672b022 dropEmptyInitThenConcatStringsSep -> concatStringSep: CLI commands are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 0480bfe50b dropEmptyInitThenConcatStringsSep -> concatStringSep: do not drop attributes with empty names
Empty attributes are probably not well supported, but the least we
could do is leave a hint.
Attribute path rendering and parsing should be done according to
Nix expression syntax in my opinion.
2024-07-13 03:06:24 +02:00
Robert Hensing 4b34feb4c2 dropEmptyInitThenConcatStringsSep -> concatStringSep: system string should not be empty 2024-07-13 03:06:24 +02:00
Robert Hensing 837c3612d4 dropEmptyInitThenConcatStringsSep -> concatStringSep: escaped shell args are never empty 2024-07-13 03:06:24 +02:00
Robert Hensing 3b77f13451 dropEmptyInitThenConcatStringsSep -> concatStringSep: experimental features do not render as empty strings 2024-07-13 03:06:24 +02:00
Robert Hensing e64643bf63 dropEmptyInitThenConcatStringsSep -> concatStringSep: feature should not be empty
(System) features are unlikely to be empty strings, but when they
come in through structuredAttrs, they probably can.
I don't think this means we should drop them, but most likely they
will be dropped after this because next time they'll be parsed with
tokenizeString.

TODO: We should forbid empty features.
2024-07-13 03:06:24 +02:00
Robert Hensing f1966e22d9 dropEmptyInitThenConcatStringsSep -> concatStringSep: store paths are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 49d100ba8b dropEmptyInitThenConcatStringsSep -> concatStringSep: output name empty not feasible
I don't think it's completely impossible, but I can't construct
one easily as derivationStrict seems to (re)tokenize the outputs
attribute, dropping the empty output.

It's not a scenario we have to account for here.
2024-07-13 03:06:24 +02:00
Robert Hensing d3e49ac881 dropEmptyInitThenConcatStringsSep -> concatStringSep: shortRefs are not empty 2024-07-13 03:06:24 +02:00
Robert Hensing 608a425550 dropEmptyInitThenConcatStringsSep -> concatStringSep: diag 2024-07-13 03:06:24 +02:00
Robert Hensing 75dde71ff9 dropEmptyInitThenConcatStringsSep -> concatStringSep: sigs are non-empty
The sigs field is produced by tokenizeStrings, which does not return
empty strings.
2024-07-13 03:06:24 +02:00
Robert Hensing 3f37785afd NIX_REMOTE_SYSTEMS: actually support multiple :-separated entries
Bug not reported in 6 years, but here you go.

Also it is safe to switch to normal concatStringsSep behavior
because tokenizeString does not produce empty items.
2024-07-13 03:06:24 +02:00
Robert Hensing 39878c8979 dropEmptyInitThenConcatStringsSep -> concatStringSep: preserve empty attr
The empty attribute name should not be dropped from attribute paths.
Rendering attribute paths with concatStringsSep is lossy and wrong,
but this is just a first improvement while dealing with the
dropEmptyInitThenConcatStringsSep problem.
2024-07-13 03:06:24 +02:00
Robert Hensing ea966a70fc dropEmptyInitThenConcatStringsSep -> concatStringSep: diagnostics and docs
These are non-critical, so their behavior is ok to change.
Dropping empty items is not needed and usually not expected.
2024-07-13 03:06:24 +02:00
Robert Hensing a681d354e7 Add fresh concatStringsSep without bug
The buggy version was previously renamed to
dropEmptyInitThenConcatStringsSep
2024-07-13 03:06:24 +02:00
Robert Hensing 79eb0adf9d dropEmptyInitThenConcatStringSep: Check that we don't drop...
... initial empty strings.

The tests pass, which is encouraging.
2024-07-13 03:05:50 +02:00
Robert Hensing 1a8defd06f Refactor: rename C++ concatStringsSep -> dropEmptyInitThenConcatStringsSep 2024-07-13 03:05:50 +02:00
tomberek b1effc9649 Merge pull request #11080 from NixOS/fix-build-x86_64-darwin-minSDK
packaging: Set darwinMinVersion to fix x86_64-darwin build
2024-07-12 20:50:50 -04:00
John Ericson db3e99d9d4 Merge pull request #10838 from obsidiansystems/spaceship-no-comparator
Remove `comparator.hh` and switch to `<=>` in a bunch of places
2024-07-12 15:43:17 -04:00
John Ericson bc83b9dc1f Remove comparator.hh and switch to <=> in a bunch of places
Known behavior changes:

- `MemorySourceAccessor`'s comparison operators no longer forget to
  compare the `SourceAccessor` base class.

Progress on #10832

What remains for that issue is hopefully much easier!
2024-07-12 14:54:18 -04:00
Lexi Mattick 6c4470ec2a Clean up cache for all commands 2024-07-12 11:54:12 -07:00
Lexi Mattick e764ed31f6 Eval cache: fix cache regressions
- Fix eval cache not being persisted in `nix develop` (since #10570)
- Don't attempt to commit cache transaction if there is no active transaction, which will spew errors in edge cases
- Drive-by: trivial typo fix
2024-07-12 09:45:35 -07:00
Eelco Dolstra 2a95a2d780 Merge pull request #11091 from NixOS/darwin-no-map-spaceship
Remove unused `operator<=>`'s that darwin can't generate
2024-07-12 18:40:20 +02:00
Eelco Dolstra cdc23b67a6 Provide std::hash<SourcePath> 2024-07-12 17:54:27 +02:00
Robert Hensing 6c5d2a1506 Merge pull request #11090 from amarshall/fix-stackoverflow-build
Fix stackoverflow during doc generation
2024-07-12 17:51:29 +02:00
Robert Hensing 11a6db5993 Remove unused operator<=>'s that darwin can't generate
It was complaining *a lot*, with dozens of MB of logs.
2024-07-12 17:37:27 +02:00
Andrew Marshall 51a12b38bd Fix stackoverflow during doc generation
On some systems, previous usage of `match` may cause a stackoverflow
(presumably due to the large size of the match result). Avoid this by
(ab)using `replaceStrings` to test for containment without using
regexes, thereby avoiding the issue. The causal configuration seems to
be the stack size hard limit, which e.g. Amazon Linux sets, whereas most
Linux distros leave unlimited.

Match the fn name to similar fn in nixpkgs.lib, but different
implementation that does not use `match`. This impl gives perhaps
unexpected results when the needle is `""`, but the scope of this is
narrow and that case is a bit odd anyway.

This makes for some duplication-of-work as we do a different
`replaceStrings` if this one is true, but this only runs during doc
generation at build time so has no runtime impact.

See https://github.com/NixOS/nix/issues/11085 for details.
2024-07-12 10:59:40 -04:00
John Ericson 337a5a23b7 Merge pull request #11089 from NixOS/warnings-includes
Fix warnings and optimize includes
2024-07-12 10:29:26 -04:00
John Ericson dfb169ca68 Merge pull request #11007 from obsidiansystems/push-down-fetcher-flake-settings
No global settings in `libnixfetchers` and `libnixflake`
2024-07-12 09:41:10 -04:00
Robert Hensing 27eaeebc41 nar-accessor.cc: Silence unused variable warning 2024-07-12 15:38:17 +02:00
Robert Hensing 8df041cbc6 Solve unused header warnings reported by clangd 2024-07-12 15:37:54 +02:00
John Ericson 3fc77f281e No global settings in libnixfetchers and libnixflake
Progress on #5638

There are still a global fetcher and eval settings, but they are pushed
down into `libnixcmd`, which is a lot less bad a place for this sort of
thing.

Continuing process pioneered in
52bfccf8d8.
2024-07-12 08:50:28 -04:00
Robert Hensing f070d68c32 Add BaseError assignment operators
The move assignment was implicitly generated and used in

    src/libstore/build/goal.cc:90:22:
       90 |             this->ex = std::move(*ex);

Clang warns about this generated method being deprecated, so making
them explicit fixes the warning.
2024-07-12 14:25:16 +02:00
Eelco Dolstra b57c361097 Merge pull request #11082 from DeterminateSystems/symbol-table-string-view
SymbolStr: Remove std::string conversion
2024-07-12 13:39:43 +02:00
Eelco Dolstra 61080554ab SymbolStr: Remove std::string conversion
This refactoring allows the symbol table to be stored as something
other than std::strings.
2024-07-11 17:43:10 +02:00
Eelco Dolstra 0363dbf2b9 Merge pull request #11081 from DeterminateSystems/remove-dead-code
Remove unused InstallableFlake::getFlakeOutputs()
2024-07-11 17:11:28 +02:00
Eelco Dolstra 87323a5689 Remove unused InstallableFlake::getFlakeOutputs() 2024-07-11 16:21:27 +02:00
Robert HensingandEmily 0395ff9bd3 packaging: Set darwinMinVersion to fix x86_64-darwin build
Ported from https://github.com/NixOS/nixpkgs/pull/326172

Co-authored-by: Emily <vcs@emily.moe>
2024-07-11 15:04:38 +02:00
Robert Hensing bb312a7174 Edit CompatibilitySettings 2024-07-11 13:59:49 +02:00
Robert Hensing 6f5f741157 doc/rl-next/shebang-relative: Update with example 2024-07-11 13:52:03 +02:00
John Ericson 426e2af6f7 Merge pull request #11077 from hercules-ci/support-hardlinks-in-tarballs
Support hardlinks in tarballs
2024-07-11 07:10:25 -04:00
Robert Hensing 56bf39e905 eqValues/assertEqValues: Clean up assertions
It's still paranoid, and probably a waste of words, but at least
now it's consistent and readily identifyable from a log.
2024-07-11 12:19:29 +02:00
Robert Hensing 4fd8f19ecf Fix build to use CanonPath in new FSO sinks 2024-07-11 12:14:48 +02:00
Robert Hensing 86420753ec Merge remote-tracking branch 'upstream/master' into support-hardlinks-in-tarballs 2024-07-11 11:43:02 +02:00
Robert Hensing 61577402ba Add EvalErrorBuilder::panic()
An nicer alternative to printError + abort, or assert(false /* foo */)
2024-07-11 11:35:58 +02:00
Robert Hensing c4e3e2dc27 Soft-deprecate the compatibility settings 2024-07-10 16:24:31 +02:00
Robert Hensing d942d505ca Merge remote-tracking branch 'upstream/master' into more-nix-shell 2024-07-10 16:19:18 +02:00
Robert Hensing 13522229a9 assertEqValues: clarify potential bug error message 2024-07-10 16:08:10 +02:00
Eelco Dolstra 142e566adb Merge pull request #11071 from DeterminateSystems/remove-stray-line
Remove reference to check-hydra-status
2024-07-09 17:33:39 +02:00
Eelco Dolstra 4c788504fa Remove reference to check-hydra-status 2024-07-09 16:44:01 +02:00
John Ericson 3755493642 Merge pull request #11060 from obsidiansystems/meson-nix
Package the Nix CLI with Meson
2024-07-09 09:49:03 -04:00
Robert Hensing d69fcbc9d7 Merge pull request #11065 from DeterminateSystems/remove-hydra-status-check
Remove the Hydra status check workflow
2024-07-09 10:28:48 +02:00
John EricsonandQyriad 7a6269ba7b Package the Nix CLI with Meson
Co-Authored-By: Qyriad <qyriad@qyriad.me>
2024-07-08 17:49:44 -04:00
John Ericson f1d88e0f27 Merge pull request #11064 from obsidiansystems/header-generater
Use a meson "generator" to deduplicate `.gen.hh` creation
2024-07-08 12:03:59 -04:00
John Ericson 1439de874f Merge pull request #11063 from obsidiansystems/config-expr
Forgot to include `config-expr.hh` in two places
2024-07-08 11:27:57 -04:00
John Ericson 6e5cec292b Use a meson "generator" to deduplicate .gen.hh creation 2024-07-08 11:13:11 -04:00
John Ericson c5284a84f3 Forgot to include config-expr.hh in some places 2024-07-08 10:37:34 -04:00
Eelco Dolstra ab0b54b905 Remove the Hydra status check workflow
I'm sick of receiving an email about this every 30 minutes.
2024-07-08 14:39:33 +02:00
Robert Hensing cfe3ee3de8 nix-shell: look up shell.nix when argument is a directory (#11057)
* Refactor: rename runEnv -> isNixShell

* Refactor: rename left -> remainingArgs

* nix-build.cc: Refactor: extract baseDir variable

* nix-build.cc: Refactor: extract sourcePath, resolvedPath variables

* nix-shell: Look for shell.nix when directory is specified

* Add legacy setting: nix-shell-always-looks-for-shell-nix

* rl-next: Add note about shell.nix lookups

* tests/functional/shell.nix: Implement runHook for dummy stdenv
2024-07-08 14:36:36 +02:00
Ryan Hendrickson 48804cffbf docs: fill out language/types.md#type-path 2024-07-08 00:41:19 -04:00
John Ericson d8850618b6 Merge pull request #11059 from rhendric/rhendric/reference-manual
docs: merge builtin-constants into builtins
2024-07-07 21:47:33 -04:00
Ryan Hendrickson 95890b3e1d docs: merge builtin-constants into builtins 2024-07-07 15:57:23 -04:00
Robert Hensing a9592077fb Merge pull request #11055 from NixOS/packaging-for-nixpkgs
Indirections for packaging meson-based granular build for Nixpkgs
2024-07-07 21:33:01 +02:00
Robert Hensing 193dd5d934 Fixup: add missing test file 2024-07-07 14:49:52 +02:00
Robert Hensing 3e424b1040 Merge branch 'nix-shell-lookup-shell-nix' into more-nix-shell 2024-07-07 14:16:15 +02:00
Robert Hensing e1106b45a3 tests/functional/nix-shell.sh: Fix Polo test for VM test
It is unclear to me why this worked when not in a VM test, but the
explanation would be in the part of nix-shell we're getting rid of
with the devShell attribute.
2024-07-07 13:03:19 +02:00
Robert Hensing 0f8a655023 tests/functional/shell.nix: Implement runHook for dummy stdenv 2024-07-07 13:02:21 +02:00
Robert Hensing fd4b17ab2c Merge pull request #11056 from romain-neil/s3-allow-use-of-proxy
Configure aws s3 lib to use system defined proxy, if existent
2024-07-07 02:47:24 +02:00
Robert Hensing c4a20a4101 rl-next: Add note about shell.nix lookups 2024-07-07 01:22:00 +02:00
Robert Hensing 2f1fada76b Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-07 01:22:00 +02:00
Robert Hensing 73602a7c6f nix-shell: Look for shell.nix when directory is specified 2024-07-07 01:22:00 +02:00
Robert Hensing 63262e78c7 Add opt-out: nix-shell-shebang-arguments-relative-to-script 2024-07-07 00:55:33 +02:00
Robert Hensing 4c59d6e9f5 Merge branch 'nix-shell-lookup-shell-nix' into more-nix-shell 2024-07-07 00:27:07 +02:00
Robert Hensing 6959ac157b rl-next: Add note about shell.nix lookups 2024-07-07 00:25:56 +02:00
Robert Hensing 6c6d5263e2 Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-07 00:25:56 +02:00
Robert Hensing f5b59fbc64 Fix and extend nix-shell baseDir test 2024-07-07 00:23:22 +02:00
Robert Hensing d5854f33e2 rl-next: Typo 2024-07-07 00:18:26 +02:00
Robert Hensing 8838f5c746 Merge remote-tracking branch 'matthewbauer/nix-shell-relative-shebang' into more-nix-shell 2024-07-07 00:18:03 +02:00
Robert Hensing afbe7c3d08 rl-next: Enter PR 2024-07-06 23:15:01 +02:00
John Ericson 41b6c735eb Merge pull request #11054 from NixOS/meson-fixes
Meson fixes
2024-07-06 17:11:11 -04:00
Robert Hensing b865625a8e nix-shell: Look for shell.nix when directory is specified 2024-07-06 23:05:34 +02:00
Robert Hensing a22f8b5276 rl-next: Add note about shell.nix lookups 2024-07-06 23:05:34 +02:00
Robert Hensing 32fb127b9c Add legacy setting: nix-shell-always-looks-for-shell-nix 2024-07-06 23:05:34 +02:00
Romain NEIL 514062c227 feat: configure aws s3 lib to use system defined proxy, if existent 2024-07-06 21:46:58 +02:00
Robert Hensing 76245ffbeb nix-build.cc: Refactor: extract sourcePath, resolvedPath variables 2024-07-06 20:55:27 +02:00
Robert Hensing e9479b272f nix-build.cc: Refactor: extract baseDir variable 2024-07-06 20:51:45 +02:00
Robert Hensing 5c367ece89 Refactor: rename left -> remainingArgs 2024-07-06 20:03:30 +02:00
Robert Hensing 13181356fc Refactor: rename runEnv -> isNixShell 2024-07-06 20:01:46 +02:00
Robert Hensing bea54d116e Add resolvePath, filesetToSource indirections for Nixpkgs 2024-07-06 19:49:55 +02:00
Robert Hensing da4c55995b ci.yml: Build non unit-tested components in meson_build 2024-07-06 19:15:53 +02:00
Robert Hensing 0729f0a113 packaging: Pass version directly 2024-07-06 17:52:57 +02:00
Robert Hensing efd5f50f5e nix-perl: Add deps, use mkMesonDerivation 2024-07-06 17:52:57 +02:00
Robert Hensing 4c014e238b nix-main: Add openssl 2024-07-06 17:52:57 +02:00
Robert Hensing 4d0c55ae55 api docs: Use mkMesonDerivation 2024-07-06 17:52:57 +02:00
Robert Hensing b7e5446b81 flake.nix: Remove unused binding 2024-07-06 17:52:57 +02:00
Robert Hensing 896eb7a44b Merge pull request #11034 from obsidiansystems/meson-nix
Package libnixmain and libnixcmd with Meson
2024-07-06 01:26:59 +02:00
John EricsonandRobert Hensing 3acf3fc746 Package libnixmain and libnixcmd with Meson
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-05 16:40:55 -04:00
Robert Hensing 0b901e10ee Merge pull request #11050 from hercules-ci/issue-10677
Explain when `man` is missing
2024-07-05 22:25:38 +02:00
John Ericson ff9b6d0e1f Merge pull request #11037 from fricklerhandwerk/document-config-parsing
use self-descriptive name for config file parser, document
2024-07-05 15:21:26 -04:00
Eelco Dolstra d5461b9009 Merge pull request #11051 from Mic92/fix-prefetch
src/nix/prefetch: fix prefetch containing current directory instead o…
2024-07-05 20:33:05 +02:00
Robert Hensing ddff76f667 Merge pull request #10973 from NixOS/meson-libexpr
Meson build for libexpr libflake, external C API, unit tests
2024-07-05 20:27:12 +02:00
Jörg ThalheimandEelco Dolstra 05381c0b30 Update src/nix/prefetch.cc
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-07-05 19:45:03 +02:00
Jörg Thalheim 8cea1fbd97 src/nix/prefetch: fix prefetch containing current directory instead of tarball
When --unpack was used the nix would add the current directory to the
nix store instead of the content of unpacked.
The reason for this is that std::distance already consumes the iterator.
To fix this we re-instantiate the directory iterator in case the
directory only contains a single entry.
2024-07-05 19:18:49 +02:00
Robert Hensing 6ef00a503a Explain when man is missing
Have you seen this man?

Fixes #10677
2024-07-05 19:18:23 +02:00
Eelco Dolstra 8f280d72ff Merge pull request #11019 from DeterminateSystems/fix-failed-to-open-archive
Tarball fetcher: Fix handling of cached tarballs
2024-07-05 17:10:02 +02:00
Robert Hensing a476383f46 Merge pull request #11031 from emilazy/push-xsrvoyspsvqx
libstore: fix sandboxed builds on macOS
2024-07-05 17:08:39 +02:00
Robert Hensing d63bd8295e assert: Report why values aren't equal 2024-07-05 16:43:48 +02:00
Eelco Dolstra 98bef7c38e Merge pull request #11035 from siddhantk232/refactor
Factor duplicate code into util function `append`
2024-07-05 16:34:23 +02:00
Eelco Dolstra 61e1880847 Merge pull request #11041 from hercules-ci/trace-nix-env-attribute-names
getDerivations: add attributes to trace
2024-07-05 16:32:27 +02:00
Eelco Dolstra e1b6b3ce27 Merge pull request #11020 from DeterminateSystems/fix-tarball-caching
Tarball fetcher: Fix fetchToStore() and eval caching
2024-07-05 16:30:12 +02:00
Eelco Dolstra e7e070d36b Document 2024-07-05 16:29:16 +02:00
Robert Hensing 09763c7cad getDerivations: add attributes to trace
This improves the error message of nix-env -qa, among others, which
is crucial for understanding some ofborg eval error reports, such as
https://gist.github.com/GrahamcOfBorg/89101ca9c2c855d288178f1d3c78efef

After this change, it will report the same trace, but also start with

```
error:
       … while evaluating the attribute 'devShellTools'

       … while evaluating the attribute 'nixos'

       … while evaluating the attribute 'docker-tools-nix-shell'

       … while evaluating the attribute 'aarch64-darwin'

       … from call site
         at /home/user/h/nixpkgs/outpaths.nix:48:6:
           47|   tweak = lib.mapAttrs
           48|     (name: val:
             |      ^
           49|       if name == "recurseForDerivations" then true

<same>
```
2024-07-05 15:30:07 +02:00
John EricsonandRobert Hensing e4056b9afd Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 17:48:27 -04:00
Emily af2e1142b1 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
2024-07-04 16:28:37 +01:00
Emily 76e4adfaac libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
2024-07-04 16:22:02 +01:00
Valentin GagarinandRobert Hensing c66079f1e8 use self-descriptive name for config file parser, document
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-07-04 10:36:48 +02:00
siddhantCodes 976c05879f factor duplicate code into util function append 2024-07-04 11:09:23 +05:30
Eelco Dolstra 5b4102c3b2 Tarball fetcher: Include revCount/lastModified in the fingerprint
This can influence the evaluation result so they should be included in
the fingerprint.
2024-07-03 22:05:45 +02:00
John Ericson 509be0e77a Merge pull request #11022 from obsidiansystems/fix-openbsd-socket-peercred
Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
2024-07-03 11:56:39 -04:00
kn 10ccdb7a41 Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).
2024-07-03 11:16:39 -04:00
John Ericson a09360400b Ident some CPP in nix daemon
Makes it easier for me to read.
2024-07-03 11:15:56 -04:00
Eelco Dolstra 1ff186fc6e nix flake metadata: Show flake fingerprint
This is useful for testing/debugging and maybe for sharing eval caches
(since it tells you what file in ~/.cache/nix/eval-cache-v5 to copy).
2024-07-03 17:00:30 +02:00
Eelco Dolstra 9d95c228ee Tarball fetcher: Fix fetchToStore() and eval caching 2024-07-03 16:50:45 +02:00
John Ericson 30de61f16d Merge pull request #11018 from siddhantk232/canonpath-fs-sink
Use `CanonPath` in `fs-sink.hh`
2024-07-03 10:36:18 -04:00
Eelco Dolstra 8bdd0ecd80 Add a test 2024-07-03 15:57:05 +02:00
Eelco Dolstra 79ed3df8f8 Tarball fetcher: Fix handling of cached tarballs
Fixes a regression introduced in
5a9e1c0d20 where downloading a cached
file causes the error "Failed to open archive (Unrecognized archive
format)".
2024-07-03 14:14:20 +02:00
siddhantCodes 2cf24a2df0 fix tests and minor changes
- use the iterator in `CanonPath` to count `level`
- use the `CanonPath::basename` method
- use `CanonPath::root` instead of `CanonPath{""}`
- remove `Path` and `PathView`, use `std::filesystem::path` directly
2024-07-03 17:43:55 +05:30
Valentin GagarinandRyan Hendrickson a713476790 docs: split types from syntax (#11013)
move together all syntactic and semantic information into one
page, and add a page on data types, which in turn links to the syntax and
semantics.

also split out the note on scoping rules into its own page.

Co-authored-by: Ryan Hendrickson <ryan.hendrickson@alum.mit.edu>
2024-07-03 09:03:41 +02:00
John Ericson b0bc2a97bf Put unit tests back in old build system for now 2024-07-02 09:26:22 -04:00
John Ericson 451f8a8c19 Put back files for now
We'll revert this sometime later
2024-07-02 09:26:22 -04:00
John Ericson 11946817f0 fileset for store unit test data 2024-07-02 09:26:22 -04:00
John Ericson 4d6bc61b8d Fix things 2024-07-02 09:26:22 -04:00
John Ericson 4727d5c3c5 Fix format blacklist 2024-07-02 09:26:22 -04:00
John Ericson 11dab30be9 Update docs on the unit tests 2024-07-02 09:26:22 -04:00
John Ericson 224c6c3256 Fix test symlinks 2024-07-02 09:26:22 -04:00
John Ericson 496b4a9cd2 Move around unit test dirs to match new names 2024-07-02 09:26:22 -04:00
John Ericson 3ad39d2afb Fix library name 2024-07-02 09:26:22 -04:00
John Ericson 513f6b9718 meson: Prelink links to avoid missing C++ initializers
This is the same as what the old build system did in
7eca8a16ea, done for the same reasons.
2024-07-02 09:26:22 -04:00
John Ericson 912c517bc0 Fix build of unit tests 2024-07-02 09:26:22 -04:00
John Ericson f7ce10dbc1 Fix static build 2024-07-02 09:26:22 -04:00
John Ericson 874ff000d4 Fix format 2024-07-02 09:26:22 -04:00
John Ericson 7312d13acc Keep another test dir 2024-07-02 09:26:22 -04:00
John Ericson 17c843c5c5 Fix more issues 2024-07-02 09:26:22 -04:00
John Ericson 479befa76d More fixes 2024-07-02 09:26:22 -04:00
John Ericson 5ba9f6cec6 Fix typo 2024-07-02 09:26:22 -04:00
John Ericson 6a0582d9fd Rename file to avoid reserved name 2024-07-02 09:26:22 -04:00
John Ericson 79e0ef88bf Include missing components 2024-07-02 09:26:22 -04:00
John Ericson 2c184f694b Ensure we have data dir for libexpr unit tests 2024-07-02 09:26:22 -04:00
John Ericson 46ec69a483 Everything builds in the dev shell now 2024-07-02 09:26:22 -04:00
John Ericson 429d6ae2b5 Add missing package.nix 2024-07-02 09:26:22 -04:00
John Ericson 92d3a06b25 Remove overrides of removed flags since unit tests broken out 2024-07-02 09:26:22 -04:00
John Ericson 0b539dea4a Improve boost hacks 2024-07-02 09:26:22 -04:00
John Ericson 8399bd6b8f Dedup 2024-07-02 09:26:21 -04:00
John Ericson 8198888bc4 More dedup 2024-07-02 09:23:25 -04:00
John Ericson d6f57f3260 More dedup 2024-07-02 09:23:25 -04:00
John Ericson c88f83b471 More dedup 2024-07-02 09:23:25 -04:00
John Ericson 4609ab318c Fix internal API docs 2024-07-02 09:23:25 -04:00
John Ericson d902481a36 Better org 2024-07-02 09:23:25 -04:00
John Ericson a81e319528 Deduplicating 2024-07-02 09:23:24 -04:00
John Ericson 17a8c2bfce Unit tests and external libraries 2024-07-02 09:23:24 -04:00
John Ericson 4fa8068b78 Mesonify other external API 2024-07-02 09:23:24 -04:00
John Ericson 31257009e1 Meson build for libexpr and libflake 2024-07-02 09:23:24 -04:00
John Ericson fbdc554908 Fix Nix shell for building Perl too 2024-07-02 09:23:24 -04:00
John Ericson f809edba4f Merge pull request #11011 from Mic92/compiler-checks
enable -Werror=unused-result
2024-07-02 09:23:03 -04:00
Jörg Thalheim 101915c9b7 enable -Werror=unused-result
Inspired by
https://git.lix.systems/lix-project/lix/commit/010ff57ebb40f1a9aaff99867d2886f0e59f774a

From the original PR:

> We do not have any of these warnings appearing at the moment, but
> it seems like a good idea to enable [[nodiscard]] checking anyway.
> Once we start introducing more functions with must-use conditions we will
> need such checking, and the rust stdlib has proven them very useful.
2024-07-02 08:46:06 +02:00
Robert Hensing 80d67d603b Merge pull request #11009 from NixOS/installer-tests
Installer tests
2024-07-02 00:45:12 +02:00
Robert Hensing df3e92ff96 installerScriptForGHA: aarch64-darwin
GitHub Actions seems to have magically switched architectures
without changing their identifiers.
See https://github.com/actions/runner-images/blob/2813ee66cbe85b31a8322ff8967148548b1a5db9/README.md#available-images
Maybe they have more complete documentation elsewhere, but it
seems to be incapable of selecting a runner based on architecture.
2024-07-01 20:40:53 +02:00
Eelco Dolstra b01b60f08c Merge pull request #11006 from DeterminateSystems/fix-eval-cache-test
tests/functional/flakes/eval-cache.sh: Don't write a result symlink in the wrong location
2024-07-01 19:47:54 +02:00
Eelco Dolstra 6600b1c7e0 tests/functional/flakes/eval-cache.sh: Don't write a result symlink in the wrong location 2024-07-01 19:10:41 +02:00
Robert Hensing f0329568b5 GitFileSystemObjectSink: catch an overflow 2024-07-01 17:29:07 +02:00
Robert Hensing a409c1a882 Start unit testing GitFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Robert Hensing 1fac22b16e GitFileSystemObjectSink: Add path context to some messages 2024-07-01 17:29:07 +02:00
Robert Hensing 992912f3b4 test-support: Add TracingFileSystemObjectSink 2024-07-01 17:29:07 +02:00
Eelco Dolstra 10c9764c27 Merge pull request #10992 from hercules-ci/issue-10947-dont-cache-disallowed-ifd
Fix #10947; don't cache disallowed IFD
2024-07-01 11:20:27 +02:00
John Ericson 10ceb7d173 Merge pull request #10995 from NixOS/meson-remove-symlink-hack
Remove symlink hack from meson packaging
2024-06-30 17:35:46 -04:00
Robert Hensing 93b50857ed packaging: Restore .version value altering behavior 2024-06-30 19:44:36 +02:00
Robert Hensing 7dd938b228 libutil/package.nix: Remove .version symlink replacement solution 2024-06-30 19:44:04 +02:00
Robert Hensing e084316130 Add mkMesonPackage for local meson packages
This helper makes it easy to use filesets that include files from
parent directories, which we'll need more of in
https://github.com/NixOS/nix/pull/10973
2024-06-30 19:42:19 +02:00
Sergei TrofimovichandJohn Ericson 5a16bf86c5 doc: fix directory definition in nix-archive.md (#10997)
* doc: fix `directory` definition in nix-archive.md

Before the change the document implied that directory of a single entry
contained entry:

    "type" "directory" "type" directory" "entry" ...

After the change document should expand into:

   "type" "directory" "entry" ...

Co-authored-by: John Ericson <git@JohnEricson.me>
2024-06-30 16:22:04 +00:00
Robert Hensing c74b5aa3e0 Merge pull request #10999 from NixOS/ci-cachix-verbose
ci.yml: Put installer cachix in verbose mode
2024-06-30 17:45:22 +02:00
Robert Hensing 39154ed9be ci.yml: Put installer cachix in verbose mode
Maybe it prints something, as a workaround for https://github.com/cachix/cachix-action/issues/153,
which might explain our flaky installer_test.
2024-06-30 17:10:34 +02:00
John Ericson dc538ad2f7 Merge pull request #10994 from hercules-ci/fix-value-print-elided
Fix underflow in `printAttrs`, `printList`
2024-06-30 10:11:47 -04:00
siddhantCodes 72bb530141 use CanonPath in fs-sink and its derivatives 2024-06-30 19:03:15 +05:30
Robert Hensing b2c7f09b0a Fix underflow in Printer::printList
Analogous to 9b88bf8adf / three commits back
2024-06-29 14:10:58 +02:00
Robert Hensing bfc5416240 Refactor: rename ValuePrinter::totalListItemsPrinted 2024-06-29 14:02:28 +02:00
Robert Hensing ce1dc87711 Refactor: rename ValuePrinter::totalAttrsPrinted
Make it more distinct from the attrs printed of any specific
attrset.
2024-06-29 14:01:15 +02:00
Robert Hensing 9b88bf8adf Fix underflow in Printer::printAttrs
The code that counts the number of elided attrs incorrectly used the
per-printer "global" attribute counter instead of a counter that
was relevant only to the current attribute set.

This bug flew under the radar because often the attribute sets aren't
nested, not big enough, or we wouldn't pay attention to the numbers.
I've noticed the issue because the difference underflowed.

Although this behavior is tested by the functional test
lang/eval-fail-bad-string-interpolation-4.nix, the underflow slipped
through review. A simpler reproducer would be as follows, but I
haven't added it to the test suite to keep it simple and marginally
faster.

```
$ nix run nix/2.23.1 -- eval --expr '"" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)'
error:
       … while evaluating a path segment
         at «string»:1:6:
            1| "" + (let v = { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; b = 1; c = 1; }; }; in builtins.deepSeq v v)
             |      ^

       error: cannot coerce a set to a string: { a = { a = 1; b = 2; c = 1; d = 1; e = 1; f = 1; g = 1; h = 1; }; b = { a = 1; «4294967289 attributes elided» }; }
```
2024-06-29 13:53:05 +02:00
Robert Hensing fd94b74ee5 Fix #10947; don't cache disallowed IFD 2024-06-29 13:19:04 +02:00
Robert Hensing 32e6cc64b5 Merge pull request #10988 from cole-h/restore-summary-rename
Restore commit-lock-file-summary rename for consistency
2024-06-29 11:51:50 +02:00
Cole Helbling d906d42330 Restore commit-lock-file-summary rename for consistency
It was originally renamed in https://github.com/NixOS/nix/pull/10691,
but https://github.com/NixOS/nix/pull/9063 accidentally removed the new
name and alias.
2024-06-28 14:53:40 -07:00
Cole Helbling 9e9730ef0f Test that commit-lock-file-summary and its alias work 2024-06-28 14:53:40 -07:00
John Ericson 02963f712e Merge pull request #10987 from tweag/remove-thufschmitt-from-maintainers
maintainers: Drop thufschmitt
2024-06-28 16:19:14 -04:00
Théophane Hufschmitt 26089183e6 maintainers: Drop thufschmitt
https://github.com/NixOS/nixos-homepage/pull/1490
2024-06-28 15:56:53 +02:00
Valentin Gagarin b44909ac22 add many more examples on escaping in strings (#10974) 2024-06-27 10:58:59 +02:00
3b388f6629 string interpolation escape example (#10966)
* string interpolation escape example

Make it easier to find the documentation, and the example might be enough for most cases.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-27 08:30:21 +00:00
John Ericson aff0fbc764 Merge pull request #10972 from NixOS/no-vendor-toml11
Stop vendoring toml11
2024-06-27 00:10:09 -04:00
John Ericson b87e048b3b Override toml11 so it evaluates on Windows too 2024-06-26 23:30:38 -04:00
Winter 149d8eb8aa Stop vendoring toml11
We don't apply any patches to it, and vendoring it locks users into
bugs (it hasn't been updated since its introduction in late 2021).

Closes https://git.lix.systems/lix-project/lix/issues/164

Change-Id: Ied071c841fc30b0dfb575151afd1e7f66970fdb9
(cherry picked from commit 80405d06264f0de1c16ee2646388ab501df20628)
2024-06-26 22:27:13 -04:00
John Ericson f002f85861 Avoid libmain header in libexpr
We just don't need it!
2024-06-26 22:26:45 -04:00
John Ericson 7e66f0d91c Merge pull request #9063 from obsidiansystems/libfetchers-prep
Introduce `libnixflake`
2024-06-26 20:32:46 -04:00
John Ericson ed129267dc Merge pull request #10971 from NixOS/user-sandbox-escape-post
Slight fixes to get CI passing again
2024-06-26 20:28:56 -04:00
John Ericson 7181d1f4a1 Reformat
Factored out code is now elegible for formatting.
2024-06-26 19:56:21 -04:00
John EricsonandRobert Hensing 0084a486cc Split out a new libnixflake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John EricsonandRobert Hensing 52730d38e2 Factor out flake:... lookup path from evaluator
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-26 19:56:21 -04:00
John Ericson 88f9d8ccb1 Don't format the just-added test .c file
On one hand, new things should be formatted. On the other, we just
bacported this file to many prior branches, and if we need to make
changes to it and backport them also, formatting the file on master but
not the release branches would cause issues.
2024-06-26 19:55:06 -04:00
John Ericson 32e67eba8b Remove invalid release notes YAML field
There is no PR for this, since it was an embargoed fix before
disclosure.
2024-06-26 19:35:57 -04:00
tomberek 2dd7f8f42d Merge pull request from GHSA-q82p-44mg-mgh5
Fix user sandbox escape
2024-06-26 18:49:21 -04:00
John Ericson 8a420162ab Merge branch 'master' into fix-sandbox-escape 2024-06-26 18:11:39 -04:00
John Ericson 5e4e3345d4 Merge pull request #10963 from hercules-ci/build-with-nixpkgs-scope
Build with nixpkgs scope
2024-06-26 11:08:45 -04:00
Robert Hensing 8eb804b0e6 Merge pull request #10967 from NixOS/ci-meson-darwin
ci.yml: Build meson on darwin
2024-06-26 11:41:31 +02:00
Robert Hensing 9f8e387c3f ci.yml: Build meson on darwin
We're building a bit of Darwin meson indirectly through `checks`,
but it'd be annoying to encounter broken un-`check`-ed stuff
during the porting process, so let's just do the right thing now.
2024-06-26 11:03:47 +02:00
Robert Hensing e644289161 Remove unused boehmgc patch 2024-06-26 10:54:17 +02:00
Robert Hensing 985c211061 flake.nix: Move {in,ex}ternal-api-docs into nixComponents scope 2024-06-26 10:48:45 +02:00
Robert Hensing 409eded541 flake.nix: Move dependencies scope to packaging/dependencies.nix 2024-06-26 10:41:56 +02:00
Robert Hensing 65802da98d Move maintainers/hydra.nix -> packaging/hydra.nix 2024-06-26 04:24:50 +02:00
Robert Hensing c24dbf1457 components.nix: Simplify 2024-06-26 04:17:30 +02:00
Robert Hensing 25dc12aab1 components.nix: Extract dependency scope
This avoids polluting nixComponents with things that aren't our
components.
Fixes the extraction of passthru tests, which failed for boehmgc
which had many irrelevant ones anyway.
2024-06-26 04:11:20 +02:00
Robert Hensing ebf77c79ae flake.nix: Use Nixpkgs convention for package variants 2024-06-26 04:11:20 +02:00
Robert Hensing 74b9b77c9f components.nix: Simplify with scope 2024-06-26 04:11:20 +02:00
Robert Hensing 85de5a60c7 Use lib instead of explicit fileset passing 2024-06-26 04:11:20 +02:00
Robert Hensing d40c59ed19 flake.nix: Use the nixComponents scope instead of bare pkgs packages
... which aren't around anymore.
2024-06-26 04:11:18 +02:00
Robert Hensing a14faa869d flake.nix: Use a Nixpkgs scope for components 2024-06-26 04:07:49 +02:00
John Ericson a92f3071bb Merge pull request #10962 from hercules-ci/tidy-packages-matrix
flake.nix: Tidy `packages` build matrix code
2024-06-25 22:06:33 -04:00
John Ericson 3f191cc0e7 Merge pull request #10961 from hercules-ci/tidy
Tidy
2024-06-25 22:04:32 -04:00
Robert Hensing e28e6b96bb flake.nix: Tidy packages build matrix code
flatMapAttrs is easier to read because it introduces the values
before using them, kind of like a `let` bindings with multiple
values.

The repeated comments remind the reader of the purpose of the
innermost attrsets, which is actually very simple.
Knowing that they go right into the result should help a lot
with building a mental model for this pattern.
2024-06-26 02:03:21 +02:00
Robert Hensing 6fe8fb967a libstore/worker.hh: Document Worker 2024-06-26 01:43:31 +02:00
Robert Hensing fd0b376c79 libstore/worker.cc: Remove outdated comment
It was added above this conditional

    Worker::Worker(LocalStore & store)
        : store(store)
    {
        /* Debugging: prevent recursive workers. */
        if (working) abort();
        working = true;

However, `working` has since been removed.

Source: https://github.com/NixOS/nix/blame/7f8e805c8ef2d7728648553de6b762964730a09a/src/libstore/build.cc#L2617
2024-06-26 01:43:31 +02:00
Robert Hensing 052f1320dd Merge pull request #10960 from rhendric/rhendric/reference-manual
docs: internal documentation touchup
2024-06-26 01:16:45 +02:00
John Ericson 65d7c80365 Merge pull request #10955 from NixOS/meson-nix-util-c
Build nix-util-c with meson + unit tests
2024-06-25 19:06:06 -04:00
Ryan Hendrickson fba81cf74b docs: internal documentation touchup
Make two comments more accurate for the next reader.
2024-06-25 18:44:56 -04:00
Robert Hensing ae3304bde9 Test static build of nix-util on non-darwin 2024-06-25 21:35:23 +02:00
Robert Hensing 1eaddb209d TMP: disable static meson build on darwin 2024-06-25 21:35:23 +02:00
Robert Hensing 6a28566db6 refact: concatMapAttrs -> flatMapAttrs
This should be slightly easier to read.
We could apply this to all concatMapAttrs calls.
2024-06-25 21:35:23 +02:00
Robert Hensing ac89828b5a Build nix-util-c with meson and unit test 2024-06-25 21:35:23 +02:00
John Ericson ccb679ee01 Merge pull request #10941 from hercules-ci/check-derivation-name
Improve error messages for invalid derivation names
2024-06-25 14:11:34 -04:00
Robert Hensing 7df9d6da65 Improve error messages for invalid derivation names 2024-06-25 19:41:29 +02:00
John Ericson 5f4f789144 Merge pull request #10954 from NixOS/ci-meson
ci.yml: Add meson_build
2024-06-25 09:02:33 -04:00
John Ericson d02d38f16d Merge pull request #7048 from puffnfresh/lutimes
Guard uses of lutimes, for portability
2024-06-25 06:06:45 -04:00
Robert Hensing 0674be8d49 nix-util: Fix build 2024-06-25 10:26:57 +02:00
Robert Hensing 1801119e29 ci.yml: Add meson_build
Restore meson CI after https://github.com/NixOS/nix/pull/10929
2024-06-25 10:15:28 +02:00
Brian McKenna 5be44d235a Guard uses of lutimes, for portability 2024-06-24 17:35:34 -04:00
John Ericson 05580a373f Fix error in the no-GC build 2024-06-24 17:18:16 -04:00
John Ericson 5c497a992b Merge pull request #10922 from hercules-ci/functional-tests-on-nixos
Run the functional tests in a NixOS environment
2024-06-24 13:36:13 -04:00
Robert Hensing 445a4a0298 ci.yml: Add swap and monitor it 2024-06-24 19:05:30 +02:00
Robert Hensing c66f1e7660 Merge pull request #10913 from NixOS/no-global-eval-settings-in-libexpr
No global eval settings in `libnixexpr`
2024-06-24 18:52:19 +02:00
John Ericson fda4c78921 Merge pull request #10951 from obsidiansystems/load-just-one-config
Small global config refactors
2024-06-24 12:38:04 -04:00
John Ericson 52bfccf8d8 No global eval settings in libnixexpr
Progress on #5638

There is still a global eval settings, but it pushed down into
`libnixcmd`, which is a lot less bad a place for this sort of thing.
2024-06-24 12:15:16 -04:00
Robert Hensing 5a7ccd6580 tests/functional: Print all args of fail() 2024-06-24 18:11:58 +02:00
Robert Hensing d4ca634508 tests/functional: Differentiate die and fail 2024-06-24 18:11:10 +02:00
John Ericson cb0c868da4 Allow loading config files into other config objects
This gives us some hope of moving away from global variables.
2024-06-24 12:07:56 -04:00
John Ericson b46e13840b Format config-global.{cc,hh}
Since the code is factored out, it is no longer avoding the formatter.
2024-06-24 12:07:56 -04:00
Robert Hensing 602c444411 Merge remote-tracking branch 'upstream/master' into functional-tests-on-nixos 2024-06-24 18:07:21 +02:00
John Ericson 1620ad4587 Split out GlobalConfig into its own header
This makes it easier to understand the reach of global variables /
global state in the config system.
2024-06-24 11:36:21 -04:00
Robert Hensing ce4bcef830 Merge pull request #10745 from obsidiansystems/ca-cleanup
Cleanup `ContentAddressMethod` to match docs
2024-06-24 17:25:45 +02:00
John EricsonandRobert Hensing b51e161af5 Cleanup ContentAddressMethod to match docs
The old `std::variant` is bad because we aren't adding a new case to
`FileIngestionMethod` so much as we are defining a separate concept ---
store object content addressing rather than file system object content
addressing. As such, it is more correct to just create a fresh
enumeration.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-24 10:24:06 -04:00
John Ericson 64e599ebe1 Rename Recursive -> NixArchive
For enums:

- `FileIngestionMethod`

- `FileSerialisationMethod`
2024-06-24 10:24:06 -04:00
Eelco Dolstra 903acc7c0f Merge pull request #10873 from siddhantk232/rm-createdirs
use `std::filesystem::create_directories` for createDirs
2024-06-24 14:54:37 +02:00
Eelco Dolstra bc21c54565 Merge pull request #10943 from pineapplehunter/master
Accept response from gitlab api with more than one entry in json
2024-06-24 14:23:47 +02:00
Robert Hensing 6f64154eea Merge pull request #10884 from tomberek/tomberek.warn_structuredAttrs_advanced
fix: warn and document when advanced attributes will have no impact d…
2024-06-24 07:56:26 +02:00
John Ericson 927b719bce Merge pull request #10874 from haenoe/derivation-tests
Tests for derivation "advanced attrs"
2024-06-23 22:09:10 -04:00
HaeNoeandJohn Ericson 7fb14201af Unit test for derivation "advanced attrs"
This tests the parser and JSON format using the DRV files from the tests
added in the previous commit.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
HaeNoeandJohn Ericson 9f9984e4d0 Functional test for derivation "advanced attrs"
This tests the Nix language side of things.

We are purposely skipping most of `common.sh` because it is overkill for
this test: we don't want to have an "overfit" test environment.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2024-06-23 21:42:56 -04:00
John Ericson 490ca93cf8 Factor out a bit more language testings infra
Will be used in a second test after `lang.sh`.
2024-06-23 15:33:45 -04:00
John Ericson df068734ac Merge pull request #10769 from poweredbypie/mingw-spawn
Implement runProgram for Windows
2024-06-23 14:12:36 -04:00
Shogo Takata 0468061dd2 accept response from gitlab with more than one entry 2024-06-23 00:52:19 +09:00
Eelco Dolstra d54590fdf3 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.
2024-06-21 17:06:19 +02:00
Eelco Dolstra 58b7b3fd15 Formatting 2024-06-21 17:06:19 +02:00
Eelco Dolstra ede95b1fc1 Put the chroot inside a directory that isn't group/world-accessible
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).

To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt d99c868b04 Add a release note for the build-dir hardening 2024-06-21 17:06:19 +02:00
Théophane Hufschmitt 1d3696f0fb Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.
2024-06-21 17:06:19 +02:00
Théophane Hufschmitt 717f3eea39 Add a test for the user sandboxing 2024-06-21 17:06:18 +02:00
Eelco Dolstra d4a70b67a0 Move flake-regressions repos to the NixOS org 2024-06-21 15:38:03 +02:00
Eelco Dolstra 6f3d2daee6 Fix spellcheck 2024-06-21 15:37:08 +02:00
Eelco Dolstra 0eec60968a flake-regressions.sh: Make the sort order deterministic 2024-06-21 15:37:08 +02:00
Eelco Dolstra 36cc8d5f4b Run the flake-regressions test suite 2024-06-21 15:37:08 +02:00
Robert Hensing d9684664c8 Revert "tests/functional/common/init.sh: Use parentheses around negation"
ShellCheck doesn't want us to add extra parentheses for show.

This reverts commit 7c9f3eeef8.
2024-06-20 22:31:32 +02:00
siddhantCodes 85b7989764 fix: handle errors in nix::createDirs
the `std::filesystem::create_directories` can fail due to insufficient
permissions. We convert this error into a `SysError` and catch it
wherever required.
2024-06-20 19:53:25 +05:30
siddhantCodes 857e380c7d Merge branch 'rm-createdirs' of github.com:siddhantk232/nix into rm-createdirs 2024-06-20 18:47:51 +05:30
Robert HensingandValentin Gagarin dcee46a0ef Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-20 14:54:11 +02:00
Robert HensingandEelco Dolstra 7c9f3eeef8 tests/functional/common/init.sh: Use parentheses around negation
roberth: Not strictly necessary, but probably a good habit

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-20 14:54:11 +02:00
Robert Hensing 648302b833 tests/functional: Enable more tests in NixOS VM 2024-06-20 14:54:11 +02:00
Robert Hensing f0abe4d8f0 ci: Build tests.functional_user for PRs 2024-06-20 14:54:11 +02:00
Robert Hensing fca160fbcd doc/contributing/testing: Describe functional VM tests and quickBuild 2024-06-20 14:54:11 +02:00
Robert Hensing d208e9dd9f tests: Add quickBuild to all VM tests 2024-06-20 14:54:11 +02:00
Robert Hensing 8557d79650 tests/functional: Skip tests that don't work in NixOS environment yet 2024-06-20 14:54:11 +02:00
Robert Hensing 211aec473e tests/functional/timeout.sh: Find missing test case
This reproduces an instance of
https://github.com/NixOS/nix/issues/4813
2024-06-20 14:54:11 +02:00
Robert Hensing 439022c5ac tests: Add hydraJobs.tests.functional_* 2024-06-20 14:54:11 +02:00
Robert Hensing dc720f89f2 flake.nix: Factor pkgs.nix_noTests out of buildNoTests
This is useful when iterating on the functional tests when trying
to run them in a VM test, for example.
2024-06-20 14:54:10 +02:00
Valentin Gagarin 1c131ec2b7 Port C API docs to Meson (#10936)
* Port C API docs to Meson

* don't cross-compile the docs
2024-06-19 22:43:54 +02:00
John Ericson 0c6029669d Merge pull request #10935 from fricklerhandwerk/cli-docs-formatting
use separate paragraphs inside list items
2024-06-18 15:24:44 -04:00
John Ericson 613d598daa Merge pull request #10934 from Artoria2e5/patch-1
optimize-store.cc: Update macos exclusion comments
2024-06-18 15:15:45 -04:00
Valentin Gagarin b975151c09 dedent lists
this indentation is unnecessary and probably an artefact from the
migration off XML.
2024-06-18 11:26:09 +02:00
PoweredByPie 8b81d083a7 Remove lookupPathForProgram and implement initial runProgram test
Apparently, CreateProcessW already searches path, so manual path search
isn't really necessary.
2024-06-18 01:01:52 -07:00
PoweredByPie fcb92b4fa4 Fix DWORD vs. int comparison warning 2024-06-17 22:14:38 -07:00
Mingye Wang ff1fc780d2 optimize-store.cc: Update macos exclusion comments
#2230 broadened the scope of macOS hardlink exclusion but did not change the comments. This was a little confusing for me, so I figured the comments should be updated.
2024-06-18 12:05:59 +08:00
PoweredByPie 4f6e3b9402 Implement tests for lookupPathForProgram and fix bugs caught by tests 2024-06-17 18:46:08 -07:00
PoweredByPie d7537f6955 Implement initial spawn tests (just testing windowsEscape for now) 2024-06-17 14:58:17 -07:00
PoweredByPie 4662e7d856 Implement windowsEscape 2024-06-17 14:57:57 -07:00
John Ericson daf1b6b23a Merge pull request #10933 from NixOS/meson-libfetchers
Meson for libfetchers
2024-06-17 17:56:11 -04:00
Tom Bereknyei 706edf26eb build: meson for libfetchers 2024-06-17 17:25:56 -04:00
John Ericson 93218dc62a Merge pull request #10930 from fricklerhandwerk/meson-docs
migrate internal API docs to Meson
2024-06-17 16:25:03 -04:00
PoweredByPie b11cf8166f Format runProgram declaration 2024-06-17 13:12:28 -07:00
Valentin Gagarin 6e34c68327 Convert the internal API doc build to Meson 2024-06-17 15:51:58 -04:00
John Ericson 69d404edad Merge pull request #10914 from NixOS/combo-shell-perl
Integrate perl with the other meson builds
2024-06-17 15:15:05 -04:00
John Ericson a83d95e26e Integrate perl with the other meson builds
One big dev shell!
2024-06-17 14:48:20 -04:00
PoweredByPie a58ca342ca Initial runProgram implementation for Windows
This is incomplete; proper shell escaping needs to be done
2024-06-17 11:13:22 -07:00
John Ericson a1bb668ccb Merge remote-tracking branch 'upstream/master' into rm-createdirs 2024-06-17 12:57:54 -04:00
Valentin Gagarin 4f340213bb use separate paragraphs inside list items 2024-06-17 17:55:29 +02:00
Robert Hensing 316b58dd5f tests/shell.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
Robert Hensing 68b8a28bc4 tests/run.sh: Check that env is mostly unmodified 2024-06-17 17:03:58 +02:00
John Ericson e0b4691754 Merge pull request #10929 from NixOS/avoid-building-too-many-jobs-at-once
Avoid building too many jobs at once
2024-06-17 09:53:43 -04:00
John Ericson c9cdc2423a Temporarily remove the Meson builds from packages in the flake
This will avoid some out-of-memory issues in GitHub actions that result
from num jobs > 1 and num cores = 4. Once we only have the Meson build
system, this problem should go away, and we can reenable these jobs.
2024-06-17 09:16:18 -04:00
John Ericson 5e806673c3 Make hydraJobs.build include the constituent packages
We were only doing that for the more exotic builds, just forgot.
2024-06-17 08:33:09 -04:00
Robert Hensing e48abec567 Merge pull request #10916 from jmbaur/read-only-no-chown
Don't chown when local-store is read-only
2024-06-17 13:49:19 +02:00
Eelco Dolstra 48d38b32d2 Merge pull request #10918 from andir/restrict-tarfile-formats
Restrict supported tarball formats to actual Tarballs
2024-06-17 13:20:23 +02:00
Robert Hensing 83d1bc95b3 Merge pull request #10925 from hercules-ci/junit-report
`testresults` output
2024-06-17 10:54:29 +02:00
Robert Hensing 5e0e0ec2d3 Merge pull request #10927 from poweredbypie/windows-fix
Fix a compile error on windows
2024-06-17 10:52:09 +02:00
PoweredByPie b0cfac8f93 Fix compile error on windows 2024-06-17 00:03:50 -07:00
Jared Baur de639ceafe Don't chown when local-store is read-only
If the local-store is using the read-only flag, the underlying
filesystem might be read-only, thus an attempt to `chown` would always
fail.
2024-06-16 23:03:33 -07:00
Robert Hensing 2894c1b38e WIP add testresults output 2024-06-16 16:34:54 +02:00
Andreas Rammhold 5a9e1c0d20 Restrict supported tarball formats to actual Tarballs
The documentation is clear about the supported formats (with at least
`builtins.fetchTarball`). The way the code was written previously it
supported all the formats that libarchive supported. That is a
surprisingly large amount of formats that are likely not on the radar
of the Nix developers and users. Before people end up relying on
this (or if they do) it is better to break it now before it becomes a
widespread "feature".

Zip file support has been retained as (at least to my knowledge)
historically that has been used to fetch nixpkgs in some shell
expressions *many* years back.

Fixes https://github.com/NixOS/nix/issues/10917
2024-06-15 14:28:20 +02:00
Robert Hensing 573e385a68 Merge pull request #10907 from hercules-ci/issue-10561
C API: Use opaque struct instead of void for `nix_value`
2024-06-15 10:12:13 +02:00
John Ericson 4728840ca8 Merge pull request #10908 from NixOS/meson-libstore
Build `nix-store` with Meson, adjust Perl bindings deps to integrate with other meson builds
2024-06-14 12:16:14 -04:00
Eelco Dolstra b3bd31c82d Merge pull request #10878 from kirillrdy/kr/sysv
darwin: allow ipc-sysv* in sandbox
2024-06-14 17:11:20 +02:00
+6 81004a05c6 Build nix-store with Meson
Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-by: Eli Schwartz <eschwartz93@gmail.com>
2024-06-14 10:25:14 -04:00
John Ericson ea8e49bea5 Force the cpuid option for libutil rather than relying on detection
This is more robust, and match's Nixpkgs policy to force enable flags
statically by default (a common distro thing).
2024-06-14 10:25:14 -04:00
John Ericson 2f5fdab06c Merge pull request #10905 from obsidiansystems/platform-namespace
Put some file descriptor functions in unix and windows namespaces
2024-06-14 08:45:31 -04:00
Robert Hensing 80f9b04f5e Merge pull request #10903 from DeterminateSystems/fix-stack-overflow
eval-fail-infinite-recursion-lambda: Reduce recursion depth
2024-06-14 14:20:42 +02:00
Hamir Mahal 03883f0d1d fix: copy in install-multi-user.sh (#10902) 2024-06-13 21:13:21 +00:00
Robert Hensing b94e1d6218 C API: Value -> nix_value
See issue https://github.com/NixOS/nix/issues/10434
2024-06-13 18:51:58 +02:00
Robert Hensing 5d8118d9cb C API: Docs 2024-06-13 18:23:21 +02:00
Robert Hensing c50db4e58c C API: Add nix_value_{inc,dec}ref
- Can be implemented more easily by more eval architectures.
- Better types in generated bindings remove some uncertainty and doubt.
2024-06-13 18:21:04 +02:00
Robert Hensing 0b56c98b1c C API: Value -> nix_value 2024-06-13 18:18:36 +02:00
John Ericson c9766c6489 Merge pull request #10904 from obsidiansystems/meson-document
Document the nix-util Meson build system more extensively
2024-06-13 12:15:39 -04:00
John EricsonandValentin Gagarin 7a5ee5d597 Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-13 11:42:17 -04:00
John EricsonandRobert Hensing 56f0b5304f Document the nix-util Meson build system more extensively
I hope this will make it easier to maintain, and also make it easier for
others to assist with porting the rest of the build system to Meson.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-13 11:39:59 -04:00
John Ericson ff87c1a318 Put some file descriptor functions in unix and windows namespaces
It is misleading when platform-specific functions are in the overall
`nix` namespace. More namespaces also makes for nicer doxygen.
2024-06-13 11:18:59 -04:00
Eelco Dolstra 1dc7c8e599 eval-fail-infinite-recursion-lambda: Reduce recursion depth
This prevents the test from failing in environments with a smaller
configured stack size.
2024-06-13 13:55:42 +02:00
Robert Hensing afdd12be5e Merge pull request #10900 from obsidiansystems/hash-order
hash: Compare hash algo second for back compat
2024-06-13 01:35:31 +02:00
John Ericson 96cf6b0f5f Merge pull request #10855 from NixOS/meson-libutil
Build `nix-util` with Meson
2024-06-12 19:19:22 -04:00
John Ericson 25a9894943 hash: Compare hash algo second for back compat
Previously (in cfc18a7739), we forgot to
compare the algo at all. This means we keep the same ordering as before
by making the stuff we always have compared take priority.
2024-06-12 18:35:47 -04:00
John Ericson 0e225596d6 Merge pull request #10857 from SkamDart/shellcheck-auto-fixup
Shellcheck: test scripts
2024-06-12 18:33:10 -04:00
+7 28d2af4ea6 Build nix-util with Meson
The idea is two-fold:

- Replace autotools with Meson

- Build each library in its own derivation

The interaction of these two features is that Meson's "subprojects"
feature (https://mesonbuild.com/Subprojects) allows us to have single
dev shell for building all libraries still, while also building things
separately. This allows us to break up the build without a huge
productivity lost.

I tested the Linux native build, and NetBSD and Windows cross builds.

Also do some clean ups of the Flake in the process of supporting new
jobs.

Special thanks to everyone that has worked on a Meson port so far,
@p01arst0rm and @Qyriad in particular.

Co-Authored-By: p01arst0rm <polar@ever3st.com>
Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Artemis Tosini <me@artem.ist>
Co-Authored-By: Felix Uhl <felix.uhl@outlook.com>
Co-Authored-By: Jade Lovelace <lix@jade.fyi>
Co-Authored-By: Lunaphied <lunaphied@lunaphied.me>
Co-Authored-By: Maximilian Bosch <maximilian@mbosch.me>
Co-Authored-By: Pierre Bourdon <delroth@gmail.com>
Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: Rebecca Turner <rbt@sent.as>
Co-Authored-By: Winter <winter@winter.cafe>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
Co-Authored-By: jade <lix@jade.fyi>
Co-Authored-By: julia <midnight@trainwit.ch>
Co-Authored-By: rebecca “wiggles” turner <rbt@sent.as>
Co-Authored-By: wiggles dog <rbt@sent.as>
Co-Authored-By: fricklerhandwerk <valentin@fricklerhandwerk.de>
Co-authored-By: Eli Schwartz <eschwartz93@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 18:31:02 -04:00
John Ericson 33241887d1 More quote coalescing 2024-06-12 17:47:54 -04:00
John Ericson d8ae28617d Try to fix quotes that don't go to end with sed 2024-06-12 17:41:16 -04:00
Cameron Dart 1afac8fbbc remove tests from pre-commit excludes 2024-06-12 17:41:16 -04:00
Cameron Dart 2d467b4731 housekeeping: shellcheck for tests/functional/import-derivation.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 1c93360989 housekeeping: shellcheck for tests/functional/hash-path.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d1c476865a housekeeping: shellcheck for tests/functional/gc-runtime.sh 2024-06-12 17:41:16 -04:00
Cameron Dart cd46ec17f9 housekeeping: shellcheck for tests/functional/function-trace.sh 2024-06-12 17:41:16 -04:00
Cameron Dart b764dd9aa4 housekeeping: shellcheck for tests/functional/flakes/unlocked-override.sh 2024-06-12 17:41:16 -04:00
Cameron Dart e1ce349d05 housekeeping: shellcheck for tests/functional/flakes/search-root.sh 2024-06-12 17:41:16 -04:00
Cameron Dart b9768b4872 housekeeping: shellcheck for tests/functional/flakes/mercurial.sh 2024-06-12 17:41:16 -04:00
Cameron Dart c7b3468968 housekeeping: shellcheck for tests/functional/flakes/inputs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d95adb531e housekeeping: shellcheck for tests/functional/flakes/init.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 71d5baca47 housekeeping: shellcheck for tests/functional/flakes/flake-in-submodule.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 3b853e795b housekeeping: shellcheck for tests/functional/flakes/circular.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ece86b7191 housekeeping: shellcheck for tests/functional/flakes/bundle.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ee57c610ae housekeeping: shellcheck for tests/functional/flakes/build-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d81fd4a1c3 housekeeping: shellcheck for tests/functional/flakes/absolute-attr-paths.sh 2024-06-12 17:41:16 -04:00
Cameron Dart ae6a842c55 housekeeping: shellcheck for tests/functional/filter-source.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 224f5515b9 housekeeping: shellcheck for tests/functional/fetchTree-file.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart f0492a6197 housekeeping: shellcheck for tests/functional/fetchPath.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 823d53c643 housekeeping: shellcheck for tests/functional/experimental-features.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 1c1abefdd2 housekeeping: shellcheck for tests/functional/dyn-drv/text-hashed-output.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 847842c4bb housekeeping: shellcheck for tests/functional/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 58a653dcc6 housekeeping: shellcheck for tests/functional/compression-levels.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 48520cb71e housekeeping: shellcheck for tests/functional/chroot-store.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 63272235e2 housekeeping: shellcheck for tests/functional/case-hacks.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart 4f04006bc1 housekeeping: shellcheck for tests/functional/check-reqs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart d7bb5bde48 housekeeping: shellcheck for tests/functional/check-refs.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 259b502773 housekeeping: shellcheck for tests/functional/ca/substitute.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 04876c39e4 housekeeping: shellcheck for tests/functional/ca/signatures.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 8f85537629 housekeeping: shellcheck for tests/functional/ca/nix-run.sh 2024-06-12 17:41:16 -04:00
Cameron Dart deacc421eb housekeeping: shellcheck for tests/functional/ca/nix-copy.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 195c0da849 housekeeping: shellcheck for tests/functional/ca/duplicate-realisation-in-closure.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 2dfbba3e5e housekeeping: shellcheck for tests/functional/ca/derivation-json.sh 2024-06-12 17:41:16 -04:00
Cameron Dart 627176fd54 housekeeping: shellcheck for tests/functional/ca/build.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 80c44138cb housekeeping: shellcheck for tests/functional/ca/build-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:16 -04:00
Cameron Dart c127625095 housekeeping: shellcheck for tests/functional/build-dry.sh 2024-06-12 17:41:16 -04:00
Cameron Dart f615489e0e housekeeping: shellcheck for tests/functional/build-delete.sh 2024-06-12 17:41:16 -04:00
Cameron DartandSandro 7186c68f75 housekeeping: shellcheck for tests/functional/brotli.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:15 -04:00
Cameron DartandSandro aeed835a2e housekeeping: shellcheck for tests/functional/binary-cache.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2024-06-12 17:41:08 -04:00
Cameron Dart 4a28ba7877 housekeeping: shellcheck for tests/functional/binary-cache-build-remote.sh 2024-06-12 16:58:25 -04:00
Cameron Dart 7738b295e5 housekeeping: shellcheck for tests/functional/bash-profile.sh 2024-06-12 16:58:25 -04:00
John Ericson 394286cf5e Merge pull request #10896 from obsidiansystems/fix-freebsd-build
Fix FreeBSD build
2024-06-12 16:49:25 -04:00
John Ericson 5b53d8fec3 Factor out GC initialization code
This is not really part of the evaluator: it is just an integration
between Boehm GC and Boost coroutines usable for any purpose. The
evaluator (merely) optionally uses it.
2024-06-12 16:00:03 -04:00
John EricsonandEelco Dolstra 7c2981fc55 Fix FreeBSD build
This restores some CPP'd code that was added in
c18911602e and accidentally lost in
2477e4e3b8.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-12 15:59:54 -04:00
John Ericson e74ce01b7f Fix precompiled headers building with clang
Since 24.05 (I think) we need to pass `-c` or Clang thinks we want to
compile *both* a final executable and precompiled header file, and
complains that we cannot use `-o` with multiple outputs. `-c` seems fine
with GCC too, so I just put it in there conditionally.
2024-06-12 10:51:43 -04:00
Eelco Dolstra 01644744ed Merge pull request #10895 from NixOS/bump-version
Bump version
2024-06-12 15:29:29 +02:00
Eelco Dolstra 258e2a32ec Bump version 2024-06-12 14:57:40 +02:00
Eelco DolstraandRobert Hensing efd4bf6533 Update src/libfetchers/git-utils.cc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-12 14:41:35 +02:00
Eelco DolstraandRobert Hensing bd37a70d8f Update tests/functional/tarball.sh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-11 19:39:42 +02:00
Valentin Gagarin b130d2f2e3 add more context on the README (#9871)
the thesis is still the defining document with all the motivation and
explanations.
adding it here for greater visibility.

also more emphasis and clarity around the community aspect.
the hydra build job seems a bit arbitrary right there. may be better for
the contributing guide.
2024-06-11 15:52:33 +00:00
Eelco Dolstra 35bdb9cee7 Support hard links in tarballs
Fixes #10395.
2024-06-11 16:05:57 +02:00
siddhantCodes 552a2cee21 Merge branch 'master' of github.com:NixOS/nix into rm-createdirs 2024-06-11 19:16:56 +05:30
d1dd7abbf0 mention the actual meaning of FODs in the glossary (#10888)
* mention the actual meaning of FODs in the glossary

Co-authored-by: Alex Groleau <source@proof.construction>
Co-authored-by: Daniel Baker <daniel.n.baker@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-10 20:50:35 +02:00
Robert Hensing de3fd52a95 Add tests/f/lang/eval-okay-derivation-legacy 2024-06-10 16:31:46 +02:00
Tom Bereknyei 4809e59b7e fix: warn and document when advanced attributes will have no impact due to __structuredAttrs 2024-06-10 09:31:21 -04:00
Robert Hensing e943ee32bd Merge pull request #10883 from tomberek/tomberek.remove_xdg_tmp
fix: remove usage of XDG_RUNTIME_DIR for TMP
2024-06-10 15:08:50 +02:00
Tom Bereknyei 1363f51bcb fix: remove usage of XDG_RUNTIME_DIR for TMP 2024-06-10 08:38:57 -04:00
Eelco Dolstra 12355e7ac5 Merge pull request #10880 from hercules-ci/bdwgc-8.2.6-has-sp-corrector
bdw-gc 8.2.6 has sp corrector
2024-06-10 13:28:05 +02:00
Robert Hensing f91f34aa65 bdwgc 8.2.4 has sp_corrector
> Support client-defined stack pointer adjustment before thread stack push

-- https://github.com/ivmai/bdwgc/releases/tag/v8.2.4

This fixes an inaccuracy in cc6f315252,
in the update to Nixpkgs 24.05 https://github.com/NixOS/nix/pull/10835

After this fixup, the build log won't ask for an upgrade, and we'll be
able to collect when a coroutine exists, e.g. during filterSource.
2024-06-10 12:57:13 +02:00
Robert Hensing 0a09597790 Typo 2024-06-10 12:46:04 +02:00
Robert Hensing 4755e133c4 Fix warning 2024-06-10 12:46:04 +02:00
siddhantCodes 7a21432e77 fix: catch filesystem_error thrown by createDirs 2024-06-10 11:30:39 +05:30
Kirill Radzikhovskyy 372d5a441e darwin: allow ipc-sysv* in sandbox 2024-06-10 11:21:17 +10:00
siddhantCodes 25b0242ca6 std::filesystem::create_directories for createDirs
The implementation of `nix::createDirs` allows it to be a simple wrapper
around `std::filesystem::create_directories` as its return value is not
used anywhere.
2024-06-09 19:49:39 +05:30
Robert Hensing 0ab9369572 Merge pull request #10854 from DeterminateSystems/thread-safe-RegexCache
Make RegexCache thread-safe
2024-06-08 18:07:35 +02:00
Robert Hensing e1b3716d50 Merge pull request #10853 from DeterminateSystems/thread-safe-srcToStore
Make EvalState::srcToStore thread-safe
2024-06-06 19:31:46 +02:00
John Ericson 5ff79efcad Merge pull request #10869 from Mic92/improve-github-actions-doc
Improve GitHub actions doc
2024-06-06 11:10:51 -04:00
John Ericson dd46ed85eb Merge pull request #10868 from DeterminateSystems/shared-sync
Add SharedSync class
2024-06-06 10:11:36 -04:00
Jörg Thalheim e505434332 document how to test github ci fully in your own fork 2024-06-06 12:55:45 +02:00
Jörg Thalheim 7e6a7c9258 make it possible to push to different docker registries in forks 2024-06-06 11:34:30 +02:00
Eelco Dolstra fd9e49480a PosixSourceAccessor: Use SharedSync 2024-06-06 10:44:53 +02:00
Eelco Dolstra 2f39caf180 Sync: Add support for shared locks 2024-06-06 10:44:09 +02:00
John Ericson b0283240a1 Merge pull request #10864 from Mic92/tarball-flakes-test
tests/nixos: make the tarball-flakes test better reflect real use cases
2024-06-05 18:40:16 -04:00
Robert Hensing dadb6f86cf Merge pull request #10865 from Mic92/better-cleanup
libutil: guard Finally against invalid exception throws
2024-06-05 23:59:51 +02:00
John Ericson 1a32490aca Merge pull request #10835 from hercules-ci/update-nixpkgs-24.05
Update to Nixpkgs 24.05
2024-06-05 17:56:16 -04:00
eldritch horrors e291087747 libutil: guard Finally against invalid exception throws
throwing exceptions is fine, but throwing exceptions during exception
handling is hard enough to do correctly that we should just forbid it
entirely out of an overabundance of caution. in cases where terminate
is the correct answer the users of Finally must call it manually now.

Source: https://git.lix.systems/lix-project/lix/commit/6c777476c9e97abfc5232f0707985caf6df2baea
2024-06-05 22:45:37 +02:00
Pierre Bourdon 162d94d975 tests/nixos: make the tarball-flakes test better reflect real use cases
In most real world cases, the Link header is set on the redirect, not on
the final file. This regressed in Lix earlier and while new unit tests
were added to cover it, this integration test should probably have also
caught it.

Source: https://git.lix.systems/lix-project/lix/commit/a3256a93759206dc64e597e2ad790ce582e84cf3
2024-06-05 20:14:24 +02:00
Eelco Dolstra edd445f745 Merge pull request #10861 from DeterminateSystems/fix-optional-deref
PackageInfo::queryDrvPath(): Don't dereference an empty optional
2024-06-05 16:48:09 +02:00
Eelco Dolstra 3e72ed9743 Typo 2024-06-05 16:19:01 +02:00
Eelco Dolstra d2eeabf3e6 PackageInfo::queryDrvPath(): Don't dereference an empty optional
Fixes a regression introduced in f923ed6b6a.

https://hydra.nixos.org/build/262267313
2024-06-05 16:17:24 +02:00
Robert Hensing 9d34815cb6 Merge pull request #10858 from elohmeier/master
flake check: Recognize well known homeModule/homeModules attribute
2024-06-05 13:11:15 +02:00
Enno Richter 80ba7778e7 flake check: Recognize well known homeModule/homeModules attributes 2024-06-05 07:38:20 +02:00
Jörg Thalheim 49c6f34911 docs: fixup description of builtins.importNative (#10810)
There was an argument missing and the fact that the imported function is
called.
2024-06-04 19:55:05 +00:00
Eelco Dolstra ef140c25d1 Merge pull request #10852 from siddhantk232/add-interrupts
add call to `checkInterrupt` in a bunch of places
2024-06-04 17:21:03 +02:00
Eelco Dolstra fbbca59453 Make RegexCache thread-safe 2024-06-04 16:59:30 +02:00
Eelco Dolstra 7d295c594e Make EvalState::srcToStore thread-safe 2024-06-04 16:56:06 +02:00
siddhantCodes 8f1a26667e add call to checkInterrupt in a bunch of places
This brings back the old behaviour. We check for interrupts in places
that may iterate over wide directories.
2024-06-04 19:35:40 +05:30
Eelco Dolstra b36aa04b53 Merge pull request #10836 from edolstra/release-notes
2.23 release notes
2024-06-04 13:00:06 +02:00
Eelco Dolstra 21be03b233 Merge pull request #10840 from obsidiansystems/libutil-pkg-config
Create and install a `nix-util.pc`
2024-06-04 12:33:37 +02:00
John Ericson 4e0d058fc3 eval.cc: Fix for Windows 2024-06-04 10:18:22 +02:00
John Ericson 6588420103 Bring back FreeBSD 2024-06-04 10:17:58 +02:00
PhilippandValentin Gagarin 214051ba79 clarify not on nix_value_force (#10842)
* clarify not on `nix_value_force`

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-04 07:41:04 +00:00
Eli FlanaganandJohn Ericson bf72b78ef2 docs: fix python nix-shell example (#10841)
* docs: fix python nix-shell example

This Python code snippet depended on Python 2 which has been marked as insecure in 24.05.

I modernized the example so new users will not be surprised upon copying and pasting the snippet for exploration.

Co-authored-by: John Ericson <git@JohnEricson.me>
2024-06-03 23:22:50 +02:00
John Ericson 06be6812a6 Create and install a nix-util.pc
Before, `-lnixutil` was just stuck in `nix-store.pc`, but that doesn't
seem so nice.

This prepares us to distribute `libnixutil` in a separate package if we
want, but it should be a good change either way. I suspect it wasn't
done before because libutil was an extra unstable interface, but I don't
think we need worry about that. *All* the C++ is less stable than the C
(or that's the goal at least).

For what it's worth, Lix also created this pkg-config file *en passant*
during their rename:
https://git.lix.systems/lix-project/lix/commit/c97e17144e0d0b666d7b79d8b4b0d581bfdf373b#diff-3c4f60cc44a0e35444c7f45331cfa50f76637118
2024-06-03 14:14:40 -04:00
Eelco Dolstra e0885fc216 Fix link 2024-06-03 20:02:03 +02:00
Robert Hensing 9019b7a37a doc/rl-2.23.md: Fix broken link 2024-06-03 18:56:04 +02:00
Robert Hensing 5d460d563e TMP: Disable tests.setuid.i686-linux
Temporarily(?) blocked on https://github.com/NixOS/nixpkgs/pull/297475#issuecomment-2145589501
2024-06-03 18:47:53 +02:00
Robert Hensing d494ac15e2 Use Nixpkgs changelog-d 2024-06-03 18:47:53 +02:00
Robert Hensing 5ddc11d0eb Update nixpkgs 2024-06-03 18:47:53 +02:00
Robert Hensing 27f880c098 Format after clang-format update 2024-06-03 18:38:56 +02:00
Robert Hensing e6ba450ce2 .clang-format: Remove duplicated key 2024-06-03 18:38:56 +02:00
Eelco Dolstra 879089e80d Edit release notes 2024-06-03 18:13:37 +02:00
Eelco Dolstra 754ea9058d release notes: 2.23.0 2024-06-03 18:06:42 +02:00
Robert Hensing 8a510f4ede Add tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing efc2508e8b Refactor hydraJobs.tests.remoteBuilds_*_2_18 2024-06-03 17:34:55 +02:00
Robert Hensing 6558025e77 Fix eval remoteBuilds_*_2_13 2024-06-03 17:34:55 +02:00
Robert Hensing 449e4b9232 Change checkOverrideNixVersion for NixOS 24.05 2024-06-03 17:34:55 +02:00
Eelco Dolstra da92ad7dd2 Merge pull request #10592 from hercules-ci/builtins-warn
Add `builtins.warn`
2024-06-03 17:16:32 +02:00
Eelco Dolstra a0e35d92d2 Merge pull request #10661 from edolstra/large-path-warning
Add setting to warn about copying/hashing large paths
2024-06-03 17:04:43 +02:00
Robert Hensing 6067525162 hydraJobs.installTests.<s>.againstCurrent{Unstable -> Latest}
Nixpkgs has reshuffled its Nix versions.
2024-06-03 16:37:39 +02:00
Robert Hensing f01f65b615 Fix nixpkgsLibTests 2024-06-03 16:37:39 +02:00
Robert Hensing cc6f315252 nix: Disable GC during coroutine when bdwgc < 8.4
This re-enables support for older bwdgc versions without complicating
the code too much.
Coroutines generally only interfere with GC during source filtering,
so it's not too bad of a regression on older bdwgc.
This seems preferable over conditional compilation to enable the patch
etc; we've already spent a lot of complexity budget on this GC-coroutine
interaction...
2024-06-03 16:37:39 +02:00
Robert Hensing b311f51f84 boehmgc-nix: Remove released traceable_allocator patch 2024-06-03 16:37:39 +02:00
Robert Hensing 2477e4e3b8 libexpr: Use GC_set_sp_corrector instead of patch
Manually tested by printing to stderr in both branches (sp in os
stack, or not), and triggering a GC in a filterSource function,
e.g.:

    let
      generateTree = n: if n == 0 then "ha" else { left = generateTree (n - 1); right = generateTree (n - 1); };
    in
      builtins.deepSeq (generateTree 18) ...

Note that the darwin still uses the strategy of disabling GC, despite
having an implementation that compiles. The proper solution will be
enabled and tested later.
2024-06-03 16:37:39 +02:00
Robert Hensing 2edcdf8508 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e' (2024-02-28)
  → 'github:NixOS/nixpkgs/88dca77be222aedd1f47d2cf0942dffefee76216' (2024-06-03)
2024-06-03 16:37:39 +02:00
Robert Hensing 8df206be54 Update nixpkgs ref to 24.05 2024-06-03 16:37:39 +02:00
Robert HensingandEelco Dolstra 3a0b0af2ac Fix typo in doc/manual/rl-next/builtins-warn.md
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-06-03 16:24:21 +02:00
Robert Hensing 70b1036224 builtins.warn: Use new EvalBaseError + "evaluation warning" 2024-06-03 16:24:21 +02:00
Robert Hensing 831d96d8d7 builtins.warn: Do not throw EvalError 2024-06-03 16:24:21 +02:00
Robert Hensing c07500e14d refactor: Extract EvalState::{runDebugRepl,canDebug} 2024-06-03 16:24:21 +02:00
Robert Hensing da82d67022 builtins.warn: Require string argument
... so that we may perhaps later extend the interface.
Note that Nixpkgs' lib.warn already requires a string coercible
argument, so this is reasonable. Also note that string coercible
values aren't all strings, but in practice, for warn, they are.
2024-06-03 16:24:21 +02:00
Robert Hensing 923cbea2af builtins.warn: Use logWarning
Constructing ErrorInfo is a little awkward for now, but this does
produce a richer log entry.
2024-06-03 16:24:21 +02:00
Robert Hensing 2d4c9d8f4a Add builtins.warn 2024-06-03 16:24:21 +02:00
Eelco Dolstra 54a9fbe5d6 Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 16:17:52 +02:00
Eelco Dolstra d07cdbd9c2 Merge pull request #10834 from obsidiansystems/fix-shellcheck
Fix shellcheck issue
2024-06-03 16:14:41 +02:00
Eelco Dolstra eb0d46fab6 Merge pull request #9897 from bryango/fix-submodule-subdir
libutil/url: fix git+file:./ parse error
2024-06-03 16:04:41 +02:00
Eelco Dolstra ac3e5d22e3 Merge pull request #10028 from DavHau/fetchTree-shallow-default
fetchTree: shallow git fetching by default
2024-06-03 16:02:34 +02:00
Eelco Dolstra d2bfc7e55a Add release note 2024-06-03 15:55:19 +02:00
John Ericson 4e62629a2d Merge pull request #10833 from obsidiansystems/hash-ordering
Modernize `Hash` ordering with C++20 `<=>`
2024-06-03 09:50:04 -04:00
Eelco Dolstra deac00c6d0 Rename large-path-warning-threshold -> warn-large-path-threshold 2024-06-03 15:49:15 +02:00
Eelco Dolstra 1450b553fa Merge pull request #10806 from jdek/riscv64_install
scripts/install.in: add riscv64 support to installer
2024-06-03 15:42:13 +02:00
John Ericson 1e99f324d9 Fix shellcheck issue
8b86f415c1 was merged from a CI run that
predated the new linting.
2024-06-03 09:36:48 -04:00
Eelco Dolstra 7f5b57d18f Merge remote-tracking branch 'origin/master' into large-path-warning 2024-06-03 15:32:27 +02:00
Eelco Dolstra ecfad6a828 Merge pull request #10564 from edolstra/remove-forceErrors
AttrCursor: Remove forceErrors
2024-06-03 15:30:01 +02:00
John Ericson cfc18a7739 Modernize Hash ordering with C++20 <=>
Progress on #10832

This doesn't switch to auto-deriving the fields, but by defining `<=>`
we allow deriving `<=>` in downstream types where `Hash` is used.
2024-06-03 09:24:33 -04:00
Eelco Dolstra d16fcaee21 Merge pull request #10782 from obsidiansystems/both-connections
Factor our connection code for worker proto like serve proto
2024-06-03 15:10:38 +02:00
John Ericson c6add8873e Merge pull request #9995 from NixOS/json-empty-sigs
`ValidPathInfo` JSON format should use `null` not omit field
2024-06-03 08:58:49 -04:00
Robert Hensing de5050f73b Merge pull request #9590 from wh0/patch-1
nix-profile: fix both profile links detection
2024-06-03 14:37:08 +02:00
John EricsonandRobert Hensing 84c65135a5 ValidPathInfo JSON format should use null not omit field
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
John EricsonandRobert Hensing 213a7a87b4 Decouple within-build (structured attrs) and unstable CLI path info JSON
See code comment for details.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-06-03 08:21:22 -04:00
c50e14276e manual: Extend JSON guidlines with optional field info
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-06-03 08:21:18 -04:00
John Ericson 57aa901071 manual: Put the JSON guideline on its own page 2024-06-03 08:11:02 -04:00
Robert Hensing d32ee396b0 Merge pull request #10820 from fricklerhandwerk/meson-1-factor-out-hydra
move Hydra jobs into a separate file
2024-06-03 12:35:43 +02:00
Robert Hensing f8bd4ba561 Merge pull request #10827 from SkamDart/skamdart/functional-add-sc
housekeeping: shellcheck test/functional/add.sh
2024-06-03 12:31:39 +02:00
Robert Hensing b74a0df645 Merge pull request #10825 from tie/output-spec-assert
Fix empty outputsToInstall for InstallableAttrPath
2024-06-03 12:27:50 +02:00
Philipp Zander 25e2b1f7f7 improve note in nix_value_force documentation 2024-06-03 09:55:44 +02:00
Cameron Dart 6a507f5d3b housekeeping: shellcheck test/functional/add.sh 2024-06-02 13:41:51 -07:00
John Ericson 8e9fc2853c Merge pull request #10824 from nix-windows/misc-windows-fixes
Misc Windows fixes
2024-06-02 10:10:34 -04:00
Ivan Trubach 68090d7ff1 Fix empty outputsToInstall for InstallableAttrPath
Fixes assertion failure if outputsToInstall is empty by defaulting to the "out"
output. That is, behavior between the following commands should be consistent:

	$ nix build --no-link --json .#nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

	$ nix build --no-link --file default.nix --json nothing-to-install-no-out
	error: derivation '/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-nothing-to-install-no-out.drv' does not have wanted outputs 'out'

Real-world example of this issue:

	$ nix build --json .#.legacyPackages.aarch64-linux.texlive.pkgs.iwona
	error: derivation '/nix/store/dj0h6b0pnlnan5nidnhqa0bmzq4rv6sx-iwona-0.995b.drv' does not have wanted outputs 'out'

	$ git rev-parse HEAD
	eee33247cf6941daea8398c976bd2dda7962b125
	$ nix build --json --file . texlive.pkgs.iwona
	nix: src/libstore/outputs-spec.hh:46: nix::OutputsSpec::Names::Names(std::set<std::__cxx11::basic_string<char> >&&): Assertion `!empty()' failed.
	Aborted (core dumped)
2024-06-02 14:26:18 +03:00
John Ericson e0b159549b Misc Windows fixes
1. Fix build by making the legacy SSH Storey's secret `logFD` setting
   not a setting on Windows. (It doesn't make sense to specify `void *`
   handles by integer cross-proccess, I don't think.)

2. Move some files that don't need to be Unix-only anymore back to their
   original locations.
2024-06-01 19:19:35 -04:00
John Ericson 300b129fc7 hydra.nix Can just return the obj for that name 2024-05-31 18:27:20 -04:00
fricklerhandwerk 1c46b9b2c5 fix path 2024-05-31 23:27:16 +02:00
fricklerhandwerk 0067f49e87 move more declarations 2024-05-31 20:37:58 +02:00
fricklerhandwerkandTom Bereknyei 5fde77b166 move Hydra jobs into a separate file
Co-Authored-By: Tom Bereknyei <tomberek@gmail.com>
2024-05-31 19:12:35 +02:00
Robert Hensing 802b4e403b Merge pull request #10814 from Mic92/fix-nix-edit
Fix nix edit
2024-05-31 13:30:24 +02:00
Robert Hensing 84e116379c Merge pull request #10812 from Mic92/build-perf
Remove 100s of CPU time (10%) from build times (1465s -> 1302s)
2024-05-31 13:28:24 +02:00
Jade Lovelace 473d2d56fc Remove 100s of CPU time (10%) from build times (1465s -> 1302s)
Result's from Mic92's framework 13th Gen Intel Core i7-1360P:

Before: 3595.92s user 183.01s system 1360% cpu 4:37.74 total
After: 3486.07s user 168.93s system 1354% cpu 4:29.79 total

I saw that boost/lexical_cast was costing about 100s in CPU time on our
compiles. We can fix this trivially by doing explicit template
instantiation in exactly one place and eliminating all other includes of
it, which is a code improvement anyway by hiding the boost.

Before:
```
lix/lix2 » ClangBuildAnalyzer --analyze buildtimeold.bin
Analyzing build trace from 'buildtimeold.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1465.3 s
  Codegen & opts (backend):   1110.9 s

<snip>

**** Expensive headers:
178153 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 3426 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

176217 ms: ../src/libutil/error.hh (included 246 times, avg 716 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: serialise.hh strings.hh
  6x: <direct include>
  6x: archive.hh serialise.hh strings.hh
  ...

173243 ms: ../src/libstore/store-api.hh (included 152 times, avg 1139 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

170482 ms: ../src/libutil/serialise.hh (included 201 times, avg 848 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh
  14x: store-api.hh nar-info.hh hash.hh
  11x: <direct include>
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh
  6x: fetchers.hh hash.hh
  ...

169397 ms: ../src/libcmd/installables.hh (included 53 times, avg 3196 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-derived-path.hh
  1x: installable-value.hh
  ...

159740 ms: ../src/libutil/strings.hh (included 221 times, avg 722 ms), included via:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh
  19x: <direct include>
  14x: store-api.hh nar-info.hh hash.hh serialise.hh
  11x: serialise.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh
  ...

156796 ms: ../src/libcmd/command.hh (included 51 times, avg 3074 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

150392 ms: ../src/libutil/types.hh (included 251 times, avg 599 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

133101 ms: /nix/store/644b90j1vms44nr18yw3520pzkrg4dd1-boost-1.81.0-dev/include/boost/lexical_cast.hpp (included 226 times, avg 588 ms), included via
:
  37x: command.hh installable-value.hh installables.hh built-path.hh realisation.hh hash.hh serialise.hh strings.hh
  19x: file-system.hh
  11x: store-api.hh nar-info.hh hash.hh serialise.hh strings.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  7x: libexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh source-path.hh archive.hh serialise.hh strings.hh
  ...

132887 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 507 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

  done in 0.6s.
```

After:
```
lix/lix2 » maintainers/buildtime_report.sh build
Processing all files and saving to '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
  done in 0.6s. Run 'ClangBuildAnalyzer --analyze /home/jade/lix/lix2/maintainers/../buildtime.bin' to analyze it.
Analyzing build trace from '/home/jade/lix/lix2/maintainers/../buildtime.bin'...
**** Time summary:
Compilation (551 times):
  Parsing (frontend):         1302.1 s
  Codegen & opts (backend):    956.3 s

<snip>

**** Expensive headers:
178145 ms: ../src/libutil/error.hh (included 246 times, avg 724 ms), included via:
  36x: command.hh installable-value.hh installables.hh derived-path.hh config.hh experimental-features.hh
  12x: globals.hh config.hh experimental-features.hh
  11x: file-system.hh file-descriptor.hh
  6x: <direct include>
  6x: serialise.hh strings.hh
  6x: fetchers.hh hash.hh serialise.hh strings.hh
  ...

154043 ms: ../src/libcmd/installable-value.hh (included 52 times, avg 2962 ms), included via:
  40x: command.hh
  5x: command-installable-value.hh
  3x: installable-flake.hh
  2x: <direct include>
  2x: installable-attr-path.hh

153593 ms: ../src/libstore/store-api.hh (included 152 times, avg 1010 ms), included via:
  55x: <direct include>
  39x: command.hh installable-value.hh installables.hh
  7x: libexpr.hh
  4x: local-store.hh
  4x: command-installable-value.hh installable-value.hh installables.hh
  3x: binary-cache-store.hh
  ...

149948 ms: ../src/libutil/types.hh (included 251 times, avg 597 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh
  11x: file-system.hh
  10x: globals.hh
  6x: fetchers.hh
  6x: serialise.hh strings.hh error.hh
  5x: archive.hh
  ...

144560 ms: ../src/libcmd/installables.hh (included 53 times, avg 2727 ms), included via:
  40x: command.hh installable-value.hh
  5x: command-installable-value.hh installable-value.hh
  3x: installable-flake.hh installable-value.hh
  2x: <direct include>
  1x: installable-value.hh
  1x: installable-derived-path.hh
  ...

136585 ms: ../src/libcmd/command.hh (included 51 times, avg 2678 ms), included via:
  42x: <direct include>
  7x: command-installable-value.hh
  2x: installable-attr-path.hh

133394 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/memory (included 262 times, avg 509 ms), included via:
  36x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh
  16x: gtest.h
  11x: file-system.hh types.hh ref.hh
  10x: globals.hh types.hh ref.hh
  10x: json.hpp
  6x: serialise.hh
  ...

89315 ms: ../src/libstore/derived-path.hh (included 178 times, avg 501 ms), included via:
  37x: command.hh installable-value.hh installables.hh
  25x: store-api.hh realisation.hh
  7x: primops.hh eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh
  6x: libexpr.hh value.hh context.hh
  6x: shared.hh
  ...

87347 ms: /nix/store/h2abv2l8irqj942i5rq9wbrj42kbsh5y-gcc-12.3.0/include/c++/12.3.0/ostream (included 273 times, avg 319 ms), included via:
  35x: command.hh installable-value.hh installables.hh path.hh types.hh ref.hh memory unique_ptr.h
  12x: regex sstream istream
  10x: file-system.hh types.hh ref.hh memory unique_ptr.h
  10x: gtest.h memory unique_ptr.h
  10x: globals.hh types.hh ref.hh memory unique_ptr.h
  6x: fetchers.hh types.hh ref.hh memory unique_ptr.h
  ...

85249 ms: ../src/libutil/config.hh (included 213 times, avg 400 ms), included via:
  37x: command.hh installable-value.hh installables.hh derived-path.hh
  20x: globals.hh
  20x: logging.hh
  16x: store-api.hh logging.hh
  6x: <direct include>
  6x: eval.hh attr-set.hh nixexpr.hh value.hh context.hh derived-path.hh
  ...

  done in 0.5s.
```

Adapated from https://git.lix.systems/lix-project/lix/commit/18aa3e1d570b4ecbb9962376e5fba5757dad8da9
2024-05-31 13:00:09 +02:00
Jörg Thalheim 69c159811e add regression test for nix edit 2024-05-31 12:58:47 +02:00
Robert Hensing 138aa2b0a7 Merge pull request #10807 from hercules-ci/issue-10504-nix-env-shell
Add `nix env shell`
2024-05-31 12:34:03 +02:00
Robert Hensing 962475d97f Merge pull request #10811 from Mic92/fetcher-fix
libfetchers: handle nonexistent refs in GitLab repos more gracefully
2024-05-31 12:24:28 +02:00
Jörg Thalheim e1a817fb1b fix nix edit in pure mode
FilteringSourceAccessor was not delegating getPhysicalPath to its inner accessor.
2024-05-31 10:39:30 +02:00
Linus Heckemann a9031978da libfetchers: handle nonexistent refs in GitLab repos more gracefully
Before:

$ nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent
fetching git input 'git+file:///home/linus/projects/lix'
fetching gitlab input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'
error: [json.exception.type_error.302] type must be string, but is null

After:

/tmp/inst/bin/nix flake lock --override-input nixpkgs gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent

warning: unknown experimental feature 'repl-flake'
error:
       … while updating the lock file of flake 'git+file:///home/joerg/git/nix?ref=refs/heads/master&rev=62693c2c37c8edd92f95114eb1387b461fc671df'

       … while updating the flake input 'nixpkgs'

       … while fetching the input 'gitlab:simple-nixos-mailserver/nixos-mailserver/nonexistent'

       error: No commits returned by GitLab API -- does the git ref really exist?

Adapted from: https://git.lix.systems/lix-project/lix/commit/3df013597d7a2b5e400839e6625c05bd47de4dca
2024-05-31 08:24:53 +02:00
J. Dekker 73f9afd716 upload-release.pl: add riscv64 to nix-fallback-paths.nix
This uses the x86_64-linux's cross-compiled output as we don't have a
native riscv64 builder.

Signed-off-by: J. Dekker <jdek@itanimul.li>
2024-05-30 20:11:37 +02:00
Robert Hensing d93cc11491 Format 2024-05-30 19:41:58 +02:00
Robert Hensing c692f6af13 nix env shell: Move from nix shell, add shorthand alias 2024-05-30 19:41:58 +02:00
Robert Hensing 98b85b2166 nix/main: Add AliasStatus::{Deprecated,AcceptedShorthand} 2024-05-30 19:41:58 +02:00
J. Dekker 0ed356f3c0 scripts/install.in: add riscv64 support to installer
The artifacts are already built and hosted, the install script just needs to be taught about riscv64.

Signed-off-by: J. Dekker <jdek@itanimul.li>
2024-05-30 15:03:20 +02:00
Robert Hensing ef5c846e25 Merge pull request #10768 from obsidiansystems/legacy-ssh-expose-ssh-master-for-hydra
Create `CommonSSHStoreConfig::createSSHMaster`
2024-05-29 22:53:29 +02:00
Robert Hensing 1054ff0873 Merge pull request #10789 from nix-windows/windows-substitution-goal
More work on the scheduler for windows
2024-05-29 22:45:55 +02:00
Robert Hensing 154769544d Merge pull request #10805 from hercules-ci/issue-10774
libcmd: Fix #10774
2024-05-29 22:39:00 +02:00
Robert Hensing 5df42223e2 Merge pull request #10797 from obsidiansystems/shellcheck-tests
Shellcheck some test scripts
2024-05-29 22:27:00 +02:00
Robert Hensing 1c70eb8eee libcmd: Fix #10774 2024-05-29 21:42:53 +02:00
Qyriad 18ac6545fc print type and value in "flake attr is not a derivation" errors
This turns errors like:

error: flake output attribute 'hydraJobs' is not a derivation or path

into errors like:

error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }

This change affects all InstallableFlake commands.

Source: https://git.lix.systems/lix-project/lix/commit/20981461d4a2a62c68f4bc7c4258473f7cd7d8e1
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-29 20:51:32 +02:00
Jörg Thalheim e2182d07d9 fixup extension of changelog entry 2024-05-29 20:51:32 +02:00
5786e1ae7c docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation (#10803)
* docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation

Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.

Co-authored-by: Qyriad <qyriad@qyriad.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-29 07:50:51 +00:00
PhilippandCole Helbling 2bd66922ee add empty line to documentation comments after @brief field (#10800)
* add empty line to documentation comments after `@brief` field

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2024-05-28 23:05:40 +00:00
John Ericson 10f864c5ae Ensure all functional scripts are (a) executable (b) have shebang
This is good for shebang, and also good for future build system
simplifications
2024-05-28 12:46:24 -04:00
John Ericson 2e12b58126 Shellcheck some test scripts
Progress on #10795
2024-05-28 12:32:22 -04:00
John Ericson bcdee80a0d More work on the scheduler for windows
- Get a rump derivation goal: hook instance will come later, local
  derivation goal will come after that.

- Start cleaning up the channel / waiting code with an abstraction.
2024-05-28 11:39:49 -04:00
John Ericson 1e2b26734b Merge pull request #10799 from hercules-ci/safer-tab-completion
Add repl completion test
2024-05-28 11:30:56 -04:00
John Ericson aa5f013d64 Merge pull request #10794 from obsidiansystems/per-hook-excludes
dev shell: excludes are per hook
2024-05-28 09:56:04 -04:00
Eelco Dolstra 7b471547e6 Merge pull request #10793 from NixOS/dependabot/github_actions/zeebe-io/backport-action-3.0.2
Bump zeebe-io/backport-action from 2.5.0 to 3.0.2
2024-05-28 15:54:15 +02:00
John Ericson ebc29017fc dev shell: excludes are per hook
As suggested by Robert in
https://github.com/NixOS/nix/pull/10787#discussion_r1617145374
2024-05-28 09:28:36 -04:00
dependabot[bot] d7b04d61a9 Bump zeebe-io/backport-action from 2.5.0 to 3.0.2
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.5.0 to 3.0.2.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.5.0...v3.0.2)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-28 13:10:48 +00:00
John Ericson cc98fce039 Merge pull request #10787 from NixOS/shellcheck-tests
Start getting all shell scripts passing shellcheck
2024-05-28 09:09:37 -04:00
John Ericson ef96a58ed7 Merge pull request #10791 from obsidiansystems/fix-format
Fix format
2024-05-27 22:58:04 -04:00
John Ericson 567265ae67 Start getting all shell scripts passing shellcheck
Like with the formatter, we are blacklisting most files by default.

Do a few files to get us started, and get a sense of what this looks
like.
2024-05-27 22:39:56 -04:00
John Ericson 1d5d748fe4 Fix format
39b2a399ad passed CI but was landed after
the formatting change in 1d6c2316a9.
2024-05-27 22:32:52 -04:00
John Ericson d0c7da131f Merge pull request #10678 from nix-windows/windows-substitution-goal
Start building the scheduler for Windows
2024-05-27 17:47:29 -04:00
John Ericson 3e9c3738d3 Create CommonSSHStoreConfig::createSSHMaster
By moving `host` to the config, we can do a lot further cleanups and
dedups. This anticipates a world where we always go `StoreReference` ->
`*StoreConfig` -> `Store*` rather than skipping the middle step too.

Progress on #10766

Progress on https://github.com/NixOS/hydra/issues/1164
2024-05-27 16:12:53 -04:00
John Ericson 17964441d9 Merge pull request #10781 from obsidiansystems/build-mode-parse
Worker proto use proper serialiser for `BuildMode`
2024-05-27 16:07:54 -04:00
John Ericson 263905da4b Merge pull request #10785 from mjoerg/fix-typos
fix typos
2024-05-27 10:24:53 -04:00
John Ericson 8527f4e7fa Merge pull request #10754 from qwqawawow/master
nix repl: make runNix() isInteractive is true by default
2024-05-27 10:07:15 -04:00
Martin Joerg e7ea5591a2 fix typos 2024-05-27 15:56:52 +02:00
John Ericson aa4a2927a7 Merge pull request #10778 from Mic92/safer-tab-completion
repl: do not crash when tab-completing import errors
2024-05-27 09:54:29 -04:00
Robert Hensing ebfada36a1 Add repl completion test 2024-05-27 09:58:49 +02:00
John Ericson f71b4da0b3 Factor our connection code for worker proto like serve proto
This increases test coverage, and gets the worker protocol ready to be
used by Hydra.

Why don't we just try to use the store interface in Hydra? Well, the
problem is that the store interface works on connection pools, with each
opreation getting potentially a different connection, but the way temp
roots work requires that we keep one logical "transaction" (temp root
session) using the same connection.

The longer-term solution probably is making connections themselves
implement the store interface, but that is something that builds on
this, so I feel OK that this is not churn in the wrong direction.

Fixes #9584
2024-05-27 00:43:46 -04:00
John Ericson 8ebd99c74e Back in enum values for BuildMode serializer
We don't want to rely on how C assigns numbers for enums in the wire
format. Sure, this is totally determined by the ABI, but it obscures the
code and makes it harder to safely change the enum definition (should we
need to) without accidentally breaking the wire format.
2024-05-27 00:22:55 -04:00
John EricsonandJade Lovelace eeb89c28b0 Worker proto use proper serialiser for BuildMode
Do this instead of an unchecked cast

I redid this to use the serialisation framework (including a unit test),
but I am keeping the reference to credit Jade for spotting the issue.

Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
(adapted from commit 2a7a824d83dc5fb33326b8b89625685f283a743b)

Co-Authored-By: Jade Lovelace <lix@jade.fyi>
2024-05-27 00:22:55 -04:00
John Ericson 7de033d63f Merge pull request #10777 from Mic92/copy-path
libstore: remove unused copyPath function
2024-05-26 10:58:38 -04:00
Pierre Bourdon ffe6ba69d6 repl: do not crash when tab-completing import errors
File not found while importing is not currently caught by the tab-completion handler.

Original bug report: https://git.lix.systems/lix-project/lix/issues/340
Fix has been adapted from https://gerrit.lix.systems/c/lix/+/1189

Example crash:

$ cat /tmp/foo.nix
{
  someImport = import ./this_file_does_not_exist;
}
$ ./src/nix/nix repl --file /tmp/foo.nix
warning: unknown experimental feature 'repl-flake'
Nix 2.23.0pre20240517_dirty
Type :? for help.
Loading installable ''...
Added 1 variables.
nix-repl> someImport.<TAB>
2024-05-25 23:15:28 +02:00
Jörg Thalheim f97da4b11c libutil/source-accessor: custom error if source does not exist
This allows better error handling by catching this error in particular.
2024-05-25 23:06:57 +02:00
eldritch horrors 2de4589e46 libstore: remove unused copyPath function
Source: https://git.lix.systems/lix-project/lix/commit/47523944c5f13250dd0eb9d56d5c6621f4722bea

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-25 22:28:41 +02:00
John Ericson 5cfa75ea16 Merge pull request #10737 from poweredbypie/mingw-stackSize
Implement `setStackSize` on Windows
2024-05-25 09:56:02 -04:00
John Ericson e0c94b91ee Merge pull request #10757 from obsidiansystems/fix-4977
Require `drvPath` attribute to end with `.drv`
2024-05-24 12:14:59 -04:00
Eelco Dolstra eeb4c40867 Typo 2024-05-24 16:35:06 +02:00
Eelco Dolstra 2c88930ef2 AttrCursor: Remove forceErrors
Instead, force evaluation of the original value only if we need to
show the exception to the user.
2024-05-24 16:34:57 +02:00
Eelco Dolstra 8b86f415c1 Add test for the evaluation cache 2024-05-24 16:34:49 +02:00
Robert Hensing c90a763273 Merge pull request #10767 from hercules-ci/fix-c-api-primop-for-strict-initializers
C API: Fix custom primops
2024-05-24 08:32:02 +02:00
PoweredByPie 0b7da099d1 Commit more stack size in some windows binaries
This way we can commit the same amount of stack size (64 MB) without a conditional.
Includes nix, libnixexpr-tests, libnixfetchers-tests, libnixstore-tests, libnixutil-tests.
2024-05-23 17:42:55 -07:00
Robert Hensing 97c3463291 C API: Refactor: use NIX_VALUE_CALL 2024-05-23 21:25:23 +02:00
Robert Hensing 2497d10351 C API: Add nix_value_call_multi, NIX_VALUE_CALL
_multi can be implemented more efficiently.
NIX_VALUE_CALL is a convenient way to invoke it.
2024-05-23 21:24:06 +02:00
Robert Hensing ab106c5ca3 C API: Test arity 2 primop 2024-05-23 21:23:15 +02:00
Robert Hensing 4bc4fb40ea C API: builtin -> custom function
Not all primops will be in `builtins`.
2024-05-23 21:23:14 +02:00
Robert Hensing 8ef6efc184 C API: Require non-thunk value from primop definition 2024-05-23 21:22:59 +02:00
Robert Hensing 8884227045 C API: Require initialized value from primop definition 2024-05-23 21:22:21 +02:00
Robert Hensing a942a34469 C API: Fix nix_c_primop_wrapper for strict initializers
https://github.com/NixOS/nix/pull/10555 added a check requiring
that output parameters always have an uninitialized Value as argument.

Unfortunately the output parameter of the primop callback received
a thunk instead.

See the comment for implementation considerations.
2024-05-23 18:32:49 +02:00
John Ericson 0f9099b517 Merge pull request #10764 from obsidiansystems/machine-ctor-field-comments
Document field being initialized in `Machine` constructor
2024-05-23 11:53:21 -04:00
John Ericson 5384ceacc3 Document field being initialized in Machine constructor 2024-05-23 11:28:25 -04:00
John Ericson 2c42e7b8d9 Merge pull request #10763 from obsidiansystems/expose-parsing-machines
Restore exposing machine file parsing
2024-05-23 09:58:55 -04:00
PoweredByPie 5f68e6d69f Get max stack size in setStackSize to match Linux 2024-05-23 03:54:35 -07:00
John Ericson f2bcebc450 Restore exposing machine file parsing
This was accidentally removed in
e989c83b44. I restored it and also did a
few other cleanups:

- Make a static method for namespacing purposes

- Put the test files in the data dir with the other test data

- Avoid mutating globals in the machine config tests

This will be used by Hydra.
2024-05-23 00:03:52 -04:00
Valentin Gagarin 5845fd59c3 CODEOWNERS: add fricklerhandwerk for documentation (#10759) 2024-05-23 01:24:15 +02:00
John Ericson 4a19f4a866 Merge pull request #9839 from obsidiansystems/more-machine-cleanup
Create `StoreReference` and use it in `Machine`
2024-05-22 17:01:57 -04:00
John Ericson 859e55d1e8 Merge pull request #10758 from obsidiansystems/fix-10747
Fix #10747
2024-05-22 16:59:39 -04:00
John Ericson dc7615dbbb tryUnshareFilesystem: Ignore ENOSYS too
Fixes #10747
2024-05-22 16:07:38 -04:00
John Ericson d5fdfdc592 unshareFilesystem: Do not assume caller 2024-05-22 16:07:34 -04:00
John Ericson f923ed6b6a Require drvPath attribute to end with .drv
Fixes #4977
2024-05-22 12:50:24 -04:00
John Ericson b3ebcc5aad Use the new StoreReference in Machine
This makes the remote builder abstract syntax more robust.
2024-05-22 09:20:15 -04:00
John Ericson b59a7a14c4 Add StoreReference::render
This will be needed for the next step.

Also allows us to write round trip tests.
2024-05-22 09:20:15 -04:00
c036d75f9e Factor out abstract syntax for Store URIs
Need to decouple parsing from actually opening a store for Machine
configs.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-22 09:20:15 -04:00
John Ericson 1d6c2316a9 Slightly change formatting style
For long expressions, one argument or parameter per line is just easier.
2024-05-22 09:20:15 -04:00
Robert Hensing bd7a074636 Merge pull request #10724 from prednaz/libexpr-c_example
remove redundant and outdated example from `libexpr-c` documentation
2024-05-22 12:59:14 +02:00
John Ericson 5f7673c7ee Merge pull request #9850 from obsidiansystems/missing-store-urls
Ensure all store types support "real" URIs
2024-05-21 13:06:13 -04:00
Valentin GagarinandRobert Hensing 77cb02b739 reword documentation on nix-copy-closure (#10709)
* reword documentation on `nix-copy-closure`

- one sentence per line
- be more precise with respect to which Nix stores are being accessed
- make a clear distinction between store paths and store objects
- add links to definitions of terms
- clarify which machine is which
- --to and --from don't take arguments

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-21 18:06:16 +02:00
John EricsonandThéophane Hufschmitt 470c0501eb Ensure all store types support "real" URIs
In particular `local://<path>` and `unix://` (without any path) now
work, and mean the same things as `local` and `daemon`, respectively. We
thus now have the opportunity to desguar `local` and `daemon` early.

This will allow me to make a change to
https://github.com/NixOS/nix/pull/9839 requested during review to
desugar those earlier.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-05-21 11:56:40 -04:00
Robert Hensing b462a92b15 Merge pull request #10755 from fricklerhandwerk/comments-examples
add examples of comments
2024-05-21 17:34:52 +02:00
John Ericson 3a7d62528d Merge pull request #10752 from darix/libseccomp-configure-cflags
Use CFLAGS for libseccomp from pkg-config also for the CFLAGS
2024-05-21 11:00:15 -04:00
Eelco Dolstra ec2a1e4ba7 Merge pull request #10751 from NixOS/dependabot/github_actions/cachix/install-nix-action-27
Bump cachix/install-nix-action from 26 to 27
2024-05-21 16:57:56 +02:00
Eelco Dolstra ff3f1e62f0 Merge pull request #10750 from NixOS/dependabot/github_actions/cachix/cachix-action-15
Bump cachix/cachix-action from 14 to 15
2024-05-21 16:57:44 +02:00
Valentin Gagarin 117dbc2c46 add examples of comments
make a suggestion for what to do if one wants to write nested comments
2024-05-21 16:49:46 +02:00
eihqnh bb1a4ea21a nix repl: make runNix() isInteractive is true by default 2024-05-21 20:05:43 +08:00
Philipp Zander 142222030c remove redundant and outdated example from libexpr-c documentation 2024-05-21 14:00:34 +02:00
Marcus Rückert 56afe228df Use CFLAGS for libseccomp from pkg-config also for the CFLAGS
Otherwise the configure check for fchmodat2 afterwards fails because it
can not find the header files.
2024-05-21 00:52:11 +02:00
dependabot[bot] 041fec0b5c ---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-20 22:51:54 +00:00
dependabot[bot] 7577597cc4 ---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-20 22:51:53 +00:00
John Ericson a57abbd143 Merge pull request #10749 from NixOS/serve-protocol-stuff-for-hydra
Query path infos (plural) and handshake version minimum for hydra
2024-05-20 18:10:47 -04:00
John Ericson 8b369f90fd Query path infos (plural) and handshake version minimum for hydra
1. Hydra currently queries for multiple path infos at once, so let us
   make a connection item for that.

2. The minimum of the two versions should always be used, see #9584.
   (The issue remains open because the daemon protocol needs to be
   likewise updated.)
2024-05-20 17:43:18 -04:00
Valentin GagarinandRobert Hensing 40b7fb4f11 expand example on nix-copy-closure (#10708)
* expand example on nix-copy-closure


Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-20 20:45:45 +00:00
Valentin Gagarin fc14378ae3 add cross-references for discoverability (#10714) 2024-05-20 20:39:34 +00:00
Robert Hensing 8953bdbf32 Merge pull request #10715 from fricklerhandwerk/remove-copy-closure-guide
redirect "Copying Closures via SSH" guide to `nix-copy-closure`
2024-05-20 18:10:27 +02:00
John Ericson c2d4c38d24 Merge pull request #10736 from lorenzleutgeb/citation
chore: PhD thesis as reference in `CITATION.cff`
2024-05-20 11:45:21 -04:00
Robert Hensing 9a2f21711d Merge pull request #10494 from obsidiansystems/primop-better-method
Slight refactors in preparation for #10480
2024-05-20 16:03:15 +02:00
Robert Hensing 20ed0c02b8 Merge pull request #10688 from hraban/fix/develop-aliases
fix: don’t expand aliases in develop stdenv setup
2024-05-20 15:59:21 +02:00
Robert Hensing 67db9e0c64 Merge pull request #10722 from obsidiansystems/ca-obj-docs
Content addressing store objects
2024-05-20 15:58:29 +02:00
John Ericson 4c91bc543c Remove store object content address reproducibility section 2024-05-20 09:34:38 -04:00
John EricsonandRobert Hensing 1c75af969a Document store object content addressing & improve JSON format
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese,
but separate fields.

Progress on #9866

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-20 09:34:35 -04:00
Robert Hensing e4be8abe42 Merge pull request #10701 from cole-h/nix-command-warn-unknown-settings
Warn on unknown settings when the first positional is an argument
2024-05-20 15:15:13 +02:00
John Ericson 43dc575fd7 Merge pull request #10738 from poweredbypie/mingw-windowSize
Implement `updateWindowSize` for Windows
2024-05-20 08:57:22 -04:00
John Ericson d48bbda2e7 Update the updateWindowSize documentation 2024-05-20 08:34:49 -04:00
Eelco Dolstra b7709d14a5 Merge pull request #10741 from edolstra/shutup-warning
value.hh: Shut up warning about useless const qualifier
2024-05-20 10:48:28 +02:00
Eelco Dolstra 927034e7ac value.hh: Shut up warning about useless const qualifier 2024-05-20 10:25:04 +02:00
Valentin Gagarin 209d75529c reword documentation on nix-store --export (#10713)
- add links to definitions of terms
- one sentence per line
- be more specific about which store is used for the import
- clearly distinguish store paths and store objects
- make a recommendation to use `nix-copy-closure` for efficient SSH transfers
2024-05-20 08:28:35 +02:00
Philipp e0bfa6c55f small additions to the documentation of nix_store_open and nix_state_create (#10728) 2024-05-20 08:27:33 +02:00
tomberek 7cb3c80bb5 Merge pull request #10711 from fricklerhandwerk/nix-store-export-example
make a more relevant example for `nix-store --export`
2024-05-19 13:29:54 -04:00
tomberek 6193737ca1 Merge pull request #10710 from fricklerhandwerk/nix-store-import-example
add example to `nix-store --import`
2024-05-19 13:27:36 -04:00
PoweredByPie a41f4223de Use setStackSize on Windows 2024-05-18 16:19:01 -07:00
PoweredByPie 6a3f906382 Implement setStackSize for Windows 2024-05-18 16:19:01 -07:00
PoweredByPie e42d00c961 Change rlim_t to size_t in setStackSize in preparation of Windows impl 2024-05-18 16:18:50 -07:00
PoweredByPie 53f0c44d6c Implement updateWindowSize for Windows 2024-05-18 16:14:20 -07:00
Lorenz Leutgeb a5de384cff chore: PhD thesis as reference in CITATION.cff 2024-05-18 20:05:22 +02:00
John Ericson beb3c2bc7a Merge pull request #10732 from drupol/add-citation-cff
chore: add `CITATION.cff` file
2024-05-17 18:49:08 -04:00
John Ericson 1ab107b0bf Merge pull request #10734 from Mic92/better-keepgoing
nix3-build: show all FOD errors with `--keep-going`
2024-05-17 18:46:57 -04:00
John Ericson d25e54d56a Merge pull request #10733 from alyssais/nix-store-delete
Improve nix-store --delete failure message
2024-05-17 18:41:23 -04:00
John Ericson bcfc722f81 Merge pull request #10735 from Mic92/more-warnings
Add more compiler warnings
2024-05-17 18:40:36 -04:00
Qyriad 05bc889b89 add and fix -Wignored-qualifiers
Change-Id: I4bffa766ae04dd80355f9b8c10e59700e4b406da
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 19:29:50 +02:00
Qyriad 52a16b7e59 add and fix -Wdeprecated-copy
*so* many warnings, from only two definitions

Change-Id: If2561cd500c05a1e33cce984faf9f3e42a8a95ac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 19:25:35 +02:00
Maximilian Bosch 19720d733f nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes https://git.lix.systems/lix-project/lix/issues/302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 18:48:52 +02:00
Alyssa Ross 979a019014 Improve nix-store --delete failure message
On several occasions I've found myself confused when trying to delete
a store path, because I am told it's still alive, but
nix-store --query --roots doesn't show anything.  Let's save future
users this confusion by mentioning that a path might be alive due to
having referrers, not just roots.
2024-05-17 18:20:22 +02:00
Pol Dellaiera a2cb90caba chore: add CITATION.cff file 2024-05-17 17:26:09 +02:00
Eelco Dolstra c9e7239e22 Merge pull request #10729 from fidgetingbits/fix-testing-typo
fix typo in testing documentation
2024-05-17 12:51:05 +02:00
fidgetingbits c66796d460 fix typo in testing documentation 2024-05-17 08:38:57 +08:00
Eelco Dolstra ba2911b03b Merge pull request #10723 from prednaz/embedding_the_nix_evaluator_example
fix "Embedding the Nix Evaluator" c api example
2024-05-16 10:14:21 +02:00
Eelco Dolstra 02d393d619 Merge pull request #10725 from prednaz/c_api_docs_improvements
small c api documentation fixes
2024-05-16 10:11:19 +02:00
Philipp Zander f63292462c document nix_external_print's printer parameter to be an out parameter 2024-05-16 02:55:38 +02:00
Philipp Zander 359043ed0d add missing c api parameter names to documentation 2024-05-16 02:55:38 +02:00
Philipp Zander 449404531d fix "Embedding the Nix Evaluator" c api example 2024-05-16 00:39:39 +02:00
Robert Hensing 303268bb71 Merge pull request #10479 from obsidiansystems/ca-fso-docs
Document file system object content addressing
2024-05-15 22:52:53 +02:00
Valentin Gagarin d50ce2df14 make a more relevant example for nix-store --export
given `nix-copy-closure` exists, it doesn't make much sense to do

    nix-store --export $paths | nix-store --import --store ssh://foo@bar

since that dumps everything rather than granularly transferring store
objects as needed.

therefore, pick an example where dumping the entire closure into a file
actually makes a difference, such as when deploying to airgapped systems.
2024-05-15 22:37:14 +02:00
John EricsonandRobert Hensing 043135a848 Document file system object content addressing
In addition:

- Take the opportunity to add a bunch more missing hyperlinks, too.

- Remove some glossary entries that are now subsumed by dedicated pages.
  We used to not be able to do this without breaking link fragments, but
  now we can, so pick up where we left off.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-15 16:28:48 -04:00
Eelco Dolstra 3026613893 Merge pull request #10675 from edolstra/zip-symlinks
Handle zip files containing symlinks
2024-05-15 22:07:22 +02:00
Daniel RamírezandValentin Gagarin 50bbe22a51 reword nix-env documentation (#10718)
* reword `nix-env` documentation

- added links
- added an overview of package sources
- clarified parsing and matching of package names

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-15 19:42:14 +00:00
Cole Helbling 39a269657e libutil/args: warn on unknown settings after parsing all flags 2024-05-15 12:25:03 -07:00
Cole Helbling 06e13465c5 tests/functional: test both clis warn on unknown settings 2024-05-15 12:25:03 -07:00
John Ericson bbe780b137 Merge pull request #10712 from fricklerhandwerk/reword-nix-store-import-docs
reword documentation on `nix-store --import`
2024-05-15 13:17:36 -04:00
Eelco Dolstra 45c83cd430 Merge pull request #10716 from fricklerhandwerk/documentation-labels
labeler: capture all docs files
2024-05-15 15:13:40 +02:00
Valentin Gagarin 6907eaad4f reword documentation on nix-store --import
- add links to definitions of terms
- one sentence per line
- be more specific about which store is used for the import
- clearly distinguish store paths and store objects
- make a recommendation to use `nix-copy-closure` for efficient SSH transfers
2024-05-15 01:59:07 +02:00
Valentin Gagarin 0c2c260180 labeler: capture all docs files 2024-05-15 01:40:25 +02:00
Valentin Gagarin 7c7aa79ebe redirect "Copying Closures via SSH" guide to nix-copy-closure
the individual commands' documentation should provide enough examples to
make sense of the options and judge what to use and when. proper guides,
which would require a more elaborate setup to show off Nix's
capabilities are out of scope for the reference manual.
2024-05-15 01:29:10 +02:00
Valentin Gagarin dcc2a51bac add example to nix-store --import
this also features specifying `--store` to give more pointers for
discoverability
2024-05-15 01:11:14 +02:00
Valentin Gagarin 49bd408c10 remove link to relocated manual page (#10703)
fix old anchor redirects to point to the correct location
2024-05-14 21:18:40 +00:00
Eli FlanaganandValentin Gagarin 05ad4e8806 doc: convention improvements for copying closure (#10702)
* doc: convention improvements for copying closure

   use -P, which only considers executables but not shell builtins

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-14 20:38:54 +00:00
Valentin Gagarin 2f0031aedc Revert "manual: fold sidebar sections" (#10698)
The original change arguably reduced ergonomics of navigation, since menu items weren't ctrl+f searchable any more.
2024-05-14 19:23:29 +00:00
Eelco Dolstra 52200474e1 Merge pull request #10697 from edolstra/fix-test-root-removal
tests/functional/common/init.sh: Make $TEST_ROOT writable before removing it
2024-05-14 16:47:21 +02:00
Eelco Dolstra 1da18e85ba tests/functional/common/init.sh: Make $TEST_ROOT writable before removing it
$TEST_ROOT typically contains read-only files/directories (e.g. the
Nix store). So we have to make it writable first.
2024-05-14 16:23:08 +02:00
Eelco Dolstra d352c52111 Merge pull request #10696 from edolstra/remove-verbose
tests/nixos/containers/containers.nix: Remove superfluous -v
2024-05-14 14:50:38 +02:00
Eelco Dolstra 9a58d90c73 tests/nixos/containers/containers.nix: Remove superfluous -v 2024-05-14 14:27:09 +02:00
Eelco Dolstra 39f7cbdc7c Merge pull request #10691 from DeterminateSystems/commit-lock-file-summary
Rename commit-lockfile-summary to commit-lock-file-summary for consistency
2024-05-14 10:06:48 +02:00
Eelco Dolstra 1623249745 Merge pull request #10684 from siddhantk232/rm-readDirectory
Inline the usage of `nix::readDirectory` and remove it
2024-05-13 15:41:59 +02:00
Graham Christensen e1e041ed8f Rename commit-lockfile-summary to commit-lock-file-summary for consistency 2024-05-13 09:23:59 -04:00
Valentin Gagarin 7822ecbadf tests: always clean the test directory
previously the test directory could have been left untouched before executing
a test when `init.sh` was not run - and sometimes it isn't
supposed to be run - which made the test suite highly stateful and thus
behaving surprisingly on multiple runs.
2024-05-13 15:19:49 +02:00
Valentin Gagarin 33ca905cdb tests: simplify initialisation and wiring
pararameterisation is not actually needed the way things are currently
set up, and it confused me when trying to understand what the code does.

all but one test sources vars-and-functions.sh, which nominally only
defines variables, but in practice is always coupled with the actual
initialisation. while the cleaner way of making this more legible would
be to source variables and initialisation separately, this would produce
a huge diff.

the change requires a few small fixes to keep the tests working:

- only create test home directory during initialisation

  that vars-and-functions.sh wrote to the file system seems not write

- fix creation of the test directory

  due to statefulness, the test home directory was implicitly creating
  the test root, too. decoupling that made it apparent that this was
  probably not intentional, and certainly confusing.

- only source vars-and-functions.sh if init.sh is not needed

  there is one test case that only needs a helper function but no
  initialisation side effects

- remove some unnecessary cleanups and split parts of re-used test code

  there were confusing bits in how initialisation code was repurposed,
  which break if trying to refactor the outer layers naively...
2024-05-13 15:19:49 +02:00
siddhantCodes 39e8aad446 Merge branch 'master' of github.com:NixOS/nix 2024-05-13 18:43:12 +05:30
Eelco Dolstra 56abd341bb Merge pull request #10685 from siddhantk232/fs-cleanup
inline the usage of `nix::renameFile`, `nix::getFileType` and `nix::copyFile`
2024-05-13 14:12:10 +02:00
siddhantCodes 62e1ea2f4b use path for from arg in nix::copyFile 2024-05-13 16:10:21 +05:30
Eelco DolstraandRobert Hensing 5534682166 Update src/libutil/util.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-13 12:14:43 +02:00
Eelco Dolstra f0b5628eb2 renderSize(): Add some unit tests 2024-05-13 12:08:51 +02:00
siddhantCodes 4d0777ca69 fix: copy fileName before calling std::distance 2024-05-13 15:36:00 +05:30
Eelco Dolstra cf3b044b7e Make large path warnings human-readable 2024-05-13 11:52:38 +02:00
Eelco Dolstra 5314430437 Move printSize() into libutil
Also always include the unit (i.e. "MiB" instead of "M").
2024-05-13 11:52:38 +02:00
Eelco Dolstra dbe1b51580 Add setting to warn about copying/hashing large paths
This is useful for diagnosing whether an evaluation is copying large
paths to the store. Example:

   $ nix build .#packages.x86_64-linux.default --large-path-warning-threshold 1000000
   warning: copied large path '/home/eelco/Dev/nix-master/' to the store (6271792 bytes)
   warning: copied large path '«github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e?narHash=sha256-7kkJQd4rZ%2BvFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo%3D»/' to the store (155263768 bytes)
   warning: copied large path '«github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D»/' to the store (22175416 bytes)
   warning: copied large path '/nix/store/z985088mcd6w23qwdlirsinnyzayagki-source' to the store (5885872 bytes)
2024-05-13 11:52:27 +02:00
Eelco Dolstra d8559cad8d Merge pull request #10686 from DeterminateSystems/long-commit-message
git putFile: support flake maximalists
2024-05-13 10:22:57 +02:00
Hraban Luyat 6bf7edb18b fix: don’t expand aliases in develop stdenv setup
This fixes https://github.com/NixOS/nixpkgs/pull/290775 by not expanding aliases
when sourcing the stdenv setup script. The way bash handles aliases is to expand
them when a function is defined, not when it is used. I.e.:

    $ alias echo="echo bar "
    $ echo foo
    bar foo
    $ xyzzy() { echo foo; }
    $ shopt -u expand_aliases
    $ xyzzy
    bar foo
    $ xyzzy2() { echo foo; }
    $ xyzzy2
    foo

The problem is that ~/.bashrc is sourced before the stdenv setup, and bashrc
commonly sets aliases for ‘cp’, ‘mv’ and ‘rm’ which you don’t want to take
effect in the stdenv derivation builders. The original commit introducing this
feature (5fd8cf7667) even mentioned this very
alias.

The only way to avoid this is to disable aliases entirely while sourcing the
stdenv setup, and reenable them afterwards.
2024-05-12 21:57:55 -04:00
Graham Christensen 8b5e8f4fba git putFile: support flake maximalists
Passing the commit message as an argument causes update failures on repositories with lots of flake inputs. In some cases, the commit message is over 250,000 bytes.
2024-05-12 16:42:43 -04:00
Robert Hensing c940d11fb0 Merge pull request #10666 from tie/derivation-outputs-drv-path
Forbid drvPath in strictDerivation outputs attribute
2024-05-12 21:14:32 +02:00
siddhantCodes ccf94545db rename copy -> copyFile and remove old copyFile
the old `copyFile` was just a wrapper that was calling the `copy`
function. This wrapper function is removed and the `copy` function is
renamed to `copyFile`.
2024-05-12 19:20:17 +05:30
siddhantCodes d3b7367c80 inline usage of nix::getFileType and remove it 2024-05-12 18:58:05 +05:30
siddhantCodes 4537663740 inline the usage of nix::renameFile
use `std::filesystem::rename` everywhere and remove `nix::renameFile`
2024-05-12 18:40:16 +05:30
siddhantCodes 1db7d1b840 inline the usage of nix::readDirectory
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
2024-05-12 17:42:18 +05:30
John Ericsonandvolth 39b2a399ad Start building the scheduler for Windows
Building derivations is a lot harder, but the downloading goals is
portable enough.

The "common channel" code is due to Volth. I wonder if there is a way we
can factor it out into separate functions / files to avoid some
within-function CPP.

Co-authored-by: volth <volth@volth.com>
2024-05-10 20:23:59 -04:00
John Ericson 87ab3c0ea4 Merge pull request #10674 from nix-windows/local-store-on-windows
Build the local store on Windows
2024-05-10 13:26:17 -04:00
e0ff8da9d5 Build the local store on Windows
Fixes #10558

Co-Authored-By: Eugene Butler <eugene@eugene4.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-05-10 13:05:23 -04:00
John Ericson 0998a3ac01 Remove LocalStore::OptimiseStats::blocksFreed as it is dead code 2024-05-10 12:55:21 -04:00
Eelco Dolstra cb7224a8c2 Merge pull request #10676 from edolstra/require-docker-images
Die rather than warn if a Docker image is missing
2024-05-10 11:55:46 +02:00
Eelco Dolstra 6df07f3e81 Die rather than warn if a Docker image is missing
The warning was done to handle older Nix releases that didn't have
Docker images (091f232896), but this was
a bad idea because it causes us to silently skip uploading Docker
images if e.g. Hydra hasn't finished building them yet.

Issue #10648.
2024-05-10 11:31:36 +02:00
Eelco Dolstra 9951e14ae0 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.
2024-05-09 19:33:09 +02:00
Eelco Dolstra de8c3c034c Merge pull request #10668 from edolstra/unit-prefixes
Support unit prefixes in configuration settings
2024-05-09 19:29:36 +02:00
b5605217ae Document string context (#8595)
* Document string context

Now what we have enough primops, we can document how string contexts
work.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Felix Uhl <iFreilicht@users.noreply.github.com>
2024-05-08 23:14:00 +02:00
Eelco Dolstra 77a406a5a6 Fix warning 2024-05-08 21:16:53 +02:00
Eelco Dolstra 79c7d6205c Support unit prefixes in configuration settings
E.g. you can now say `--min-free 1G`.
2024-05-08 21:11:09 +02:00
John Ericson d8d20307a8 Merge pull request #10620 from nh2/patch-2
manual: language: Explain that `with` does not shadow
2024-05-08 11:41:21 -04:00
Ivan Trubach 081faeda8c Forbid drvPath in strictDerivation outputs attribute
builtins.strictDerivation returns an attribute set with drvPath and
output paths. For some reason, current implementation forbids drv
instead of drvPath.
2024-05-08 18:20:37 +03:00
John Ericson a5252c9979 doc: Reword scoping section
"dynamic scope" is not accurate, so reword. The underlying idea is good however.
2024-05-08 11:18:17 -04:00
John Ericson 0930058189 Merge pull request #10665 from siddhantk232/stdfs
Remove `isLink` in favor of `std::filesystem::is_link`

This is one step closer to eventually getting rid of most of our file system utils (in `file-system.cc`) in favor of the `std::filesystem`.
2024-05-08 10:55:19 -04:00
siddhantCodes ddea4c6deb rm isLink
isLink util is removed in favour of std::filesystem::is_symlink
2024-05-08 19:59:37 +05:30
Valentin Gagarin 52ccaf7971 maintainers: update information on team meetings (#10663)
- specify meeting times in terms of a time zone rather than standard
  time (the first encompasses standard time changes)
- add information on who can participate and how
- unrelated but still important: add GitHub handle to contact the team
2024-05-08 11:29:27 +02:00
Eelco Dolstra 89f500b554 Merge pull request #10662 from Prince213/typo-fix
fix(doc/manual/src/command-ref/nix-env/install): fix typo
2024-05-08 11:28:34 +02:00
Sizhe Zhao f83617f052 fix(doc/manual/src/command-ref/nix-env/install): fix typo 2024-05-08 14:24:23 +08:00
Siddhant KumarandJohn Ericson fcbc36cf78 Use std::filesystem::path in more places (#10657)
Progress on #9205

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>

* Get rid of `PathNG`, just use `std::filesystem::path`
2024-05-07 22:28:50 +00:00
John Ericson 9ae6455b0e Merge pull request #10658 from nix-windows/more-std-filesystem
Use `std::filesystem` functions in more places
2024-05-07 16:52:37 -04:00
John Ericson 72a0d4b022 Try to fix macOS Nixpkgs lib test failure
Sometimes we read a directory with children we cannot stat. It's a pitty
we even try to stat at all (wasteful) in the `DT_UNKNOWN` case, but at
least this should get rid of the failure.
2024-05-07 16:21:02 -04:00
John Ericson a3c573950b Replace our DirEntry with std::filesystem's 2024-05-07 16:21:02 -04:00
Théophane Hufschmitt 9763eb2fcb Merge pull request #10659 from edolstra/fix-builtin-fetchurl-drvPath
builtin:fetchurl: Revert impureEnvVars attribute
2024-05-07 11:52:51 +02:00
Eelco Dolstra d641e8f717 builtin:fetchurl: Revert impureEnvVars attribute
This was changed in #10611, which caused the derivation paths of
anything using builtin:fetchurl to change (i.e. all of
Nixpkgs). However, impureEnvVars doesn't actually do anything for
builtin:fetchurl, so we can just set it to its historical value.
2024-05-07 11:25:07 +02:00
John Ericson c371070580 Use std::filesystem functions in more places
This makes for shorter and more portable code.

The only tricky part is catching exceptions: I just searched for near by
`catch (Error &)` or `catch (SysError &)` and adjusted them to `catch
(std::filesystem::filesystem_error &)` according to my human judgement.

Good for windows portability; will help @siddhantk232 with his GSOC
project.
2024-05-07 00:16:54 -04:00
b4950404ba Honor the same set of proxy environment variables (#10611)
Different parts of the project honor different sets of proxy environment
variables. With this commit all parts of the project will honor the same
set of proxy environment variables.

---------

Co-authored-by: Your Name <you@example.com>
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-05-06 15:39:22 -04:00
Eelco Dolstra da3381d51f Merge pull request #10465 from edolstra/remove-locked
Fetcher cache cleanups
2024-05-06 21:32:26 +02:00
Eelco DolstraandRobert Hensing c7216a416f Update src/libfetchers/cache.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-06 21:11:41 +02:00
John Ericson 1ad7b5451d Merge pull request #10655 from edolstra/use-source-path
Use `SourcePath` in more places
2024-05-06 14:57:01 -04:00
John Ericson 36150e6fce Merge pull request #10656 from obsidiansystems/fix-build-failure
Fix build failure with clang
2024-05-06 14:02:33 -04:00
Eelco Dolstra b7eb26e362 Fix perl build 2024-05-06 20:00:44 +02:00
John Ericson 5e189025ca Fix build failure with clang
A slight issue with feb1d10f60.
2024-05-06 13:39:21 -04:00
Eelco Dolstra ef28c7329c Rename makeFSSourceAccessor -> getFSSourceAccessor()
This makes it clearer that it returns a shared accessor object.
2024-05-06 19:16:52 +02:00
Eelco Dolstra eab2919119 Use SourcePath in more places
Now that SourcePath uses a SourceAccessor instead of an InputAccessor,
we can use it in function signatures instead of passing a
SourceAccessor and CanonPath separately.
2024-05-06 19:05:42 +02:00
Robert Hensing cbafa1ba2d Merge pull request #10560 from hercules-ci/doc-path-value
Improve path value documentation
2024-05-06 18:48:50 +02:00
Robert HensingandValentin Gagarin 0eababb5f7 doc: Edit
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-06 18:26:11 +02:00
Robert Hensing 038573279c doc/values: Refer to base directory definition 2024-05-06 18:23:15 +02:00
Robert Hensing 020edac1ca doc/values: Improve Path
See https://github.com/NixOS/nix/issues/8738 for a more pointed
criticism of absolute paths.
2024-05-06 18:22:13 +02:00
Eelco Dolstra 2926ef0e90 Merge pull request #10652 from tweag/check-additionalSandboxProfile
libstore: check additionalSandboxProfile
2024-05-06 17:59:25 +02:00
John Ericson 79f03b794c Merge pull request #10654 from edolstra/rename-input-accessor
Rename remaining instances of "InputAccessor" to "SourceAccessor"
2024-05-06 11:56:00 -04:00
9bd1191fcc libstore: check additionalSandboxProfile
Make sure that `extraSandboxProfile` is set before we check whether it's
empty or not (in the `sandbox=true` case).

Also adds a test case for this.

Co-Authored-By: Artemis Tosini <lix@artem.ist>
Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
2024-05-06 17:37:08 +02:00
Eelco Dolstra 709cd44d3e Rename remaining instances of "InputAccessor" to "SourceAccessor" 2024-05-06 17:29:03 +02:00
John Ericson ea70878a76 Merge pull request #10650 from fricklerhandwerk/simplify-shell-tests
tests: remove unneeded indirection
2024-05-06 10:24:45 -04:00
Valentin GagarinandThéophane Hufschmitt 27a02bc7d1 tests: remove unneeded indirection
the additional function calls obscured the actual logic

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-05-06 15:57:22 +02:00
HaeNoe feb1d10f60 _not_ round-trip tests for fetchers::PublicKey default type (#10637)
Another continuation of #10602
2024-05-06 09:50:26 -04:00
Eelco Dolstra ee2fa87a7e Merge pull request #10642 from edolstra/remove-input-accessor
Merge InputAccessor into SourceAccessor
2024-05-06 15:48:25 +02:00
Eelco Dolstra 20445dfeaf Merge pull request #10646 from cmoog/cmoog/doc-fix
document store url `trusted=true` option behavior
2024-05-06 10:32:51 +02:00
Charlie Moog 71c66de227 document store url trusted=true option behavior 2024-05-05 17:37:04 +00:00
Niklas Hambüchen 460d8fbaea language: Link examples to detail explanations.
Also, warn of the scoping caveats of `with`.
2024-05-05 16:56:35 +02:00
Eelco Dolstra ffc280f27a Formatting 2024-05-03 15:41:03 +02:00
Eelco Dolstra 20558e0462 Remove FSInputAccessor 2024-05-03 12:30:28 +02:00
Eelco Dolstra ba5929c7be Merge InputAccessor into SourceAccessor
After the removal of the InputAccessor::fetchToStore() method, the
only remaining functionality in InputAccessor was `fingerprint` and
`getLastModified()`, and there is no reason to keep those in a
separate class.
2024-05-03 12:14:01 +02:00
John Ericson 00ca2b05b8 Merge pull request #10639 from obsidiansystems/fix-format
Fix format errors
2024-05-02 22:05:44 -04:00
John Ericson 037c8d771d Fix format errors
Fix formatting violations, update blacklist to reflect moved files.

PR #10556 passed CI before the new formating rules were added, and our
CI has the race condition of allowing old results, resulting in master
getting broken.
2024-05-02 21:42:28 -04:00
John Ericson 840267491e Merge pull request #10633 from hercules-ci/fix-eval-state-baseEnv-gc-root
libexpr: Add missing GC root for `baseEnv`
2024-05-02 10:10:54 -04:00
John Ericson 1948ec390c Merge pull request #10556 from nix-windows/uds-remote-on-windows
Enable the `unix://` store on Windows
2024-05-02 09:53:00 -04:00
Robert Hensing f34b52b521 libexpr: Add missing GC root for baseEnv
This missing GC root wasn't much of a problem before, because the
heap would end up with a reference to the `baseEnv` pretty soon,
but when unit testing, the construction of `EvalState` doesn't
necessarily happen well before GC runs for the first time.

Found while unit testing the Rust bindings that currently reside
at https://github.com/nixops4/nixops4/tree/main/rust
2024-05-01 22:36:39 +02:00
Robert Hensing e17aad23d6 Merge pull request #10555 from jlesquembre/jl/c-api_check-init
Add isValid to nix::Value
2024-05-01 16:33:01 +02:00
Eelco Dolstra 5279e1f190 Merge pull request #10625 from edolstra/test-flake-root
Test that the root of a tree produces /nix/store/<hash1>-<hash2>-source
2024-05-01 10:05:22 +02:00
Eelco Dolstra de634a54a1 Merge pull request #10626 from mannahusum/patch-1
Update distibuted-builds.md not to use nix-store info
2024-04-30 20:23:32 +02:00
Eelco DolstraandJohn Ericson 1f41682217 Update tests/functional/flakes/flakes.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-04-30 18:10:16 +02:00
Eelco Dolstra f29a220b70 Test that the root of a tree produces /nix/store/<hash1>-<hash2>-source 2024-04-30 17:25:35 +02:00
Christian Albertsen 724132468a Update distibuted-builds.md not to use nix-store info
When trying the „nix-store info“ commands on this page I received the error "error: 'info' is not a recognised command". According to https://github.com/NixOS/nix/issues/9349 info seems to have been an alias for ping. So why not just replace info with ping?
2024-04-30 17:08:04 +02:00
Eelco Dolstra e18c3d4670 Merge pull request #10624 from edolstra/flake-root-tests
Add tests for dirOf/baseNameOf on the root of a flake
2024-04-30 16:12:07 +02:00
Eelco Dolstra 503be57bbd Test baseNameOf behaviour on the root of a flake 2024-04-30 15:43:33 +02:00
Eelco Dolstra 458441c637 Test dirOf behaviour on the root of a flake 2024-04-30 15:34:38 +02:00
Eelco Dolstra 4d99d07bc9 Whitespace 2024-04-30 15:34:35 +02:00
Eelco Dolstra 4161f3cfea Merge pull request #10618 from srhb/editor-pause-repl
nix repl: hide progress bar during :edit
2024-04-29 12:37:17 +02:00
Sarah Brofeldt e5f509ef0b nix repl: hide progress bar during :edit 2024-04-29 10:03:34 +02:00
Eelco Dolstra 2f678331d5 Merge pull request #10536 from hercules-ci/doc-doxygen-make-rebuild
Rebuild doxygen docs when headers change
2024-04-26 23:49:27 +02:00
Eelco Dolstra de51e5c335 Merge pull request #10570 from layus/shared_caches
Share evaluation caches across installables
2024-04-26 15:48:46 +02:00
John Ericson 84e0c464f1 Merge pull request #10610 from obsidiansystems/move-seccomp-custom-src
Move `libseccomp` source override outside `package.nix`
2024-04-25 17:00:34 -04:00
John Ericson 4722b0c9e9 Merge pull request #10602 from haenoe/json-infra-tests-misc
Json infra tests misc
2024-04-25 16:53:51 -04:00
Eelco Dolstra 0353d6f79e Merge pull request #10588 from hercules-ci/fix-nested-submodules
Fix fetchGit/fetchTree for nested submodules
2024-04-25 22:53:22 +02:00
John Ericson 28043fef69 Merge pull request #10378 from p01arst0rm/nix-perl-port
Port Nix-Perl to Meson
2024-04-25 16:48:22 -04:00
John Ericson 1a2f88491f Move libseccomp source override outside package.nix
This makes it match the current pattern:

- `package.nix` assumes deps are right version

- Overlay in `flake.nix` creates `*-nix` package variations

- Overlay manually passes in those packages to `package.nix`
2024-04-25 16:38:35 -04:00
John Ericson 1ac635d600 perl: Allow running yath test in the build directory
For most purposes, the stock `ninja test` should be fine, but this
allows for doing other things with the `yath` during development.
2024-04-25 16:25:50 -04:00
polar fc1d9023a2 perl: Rewrite build system using Meson 2024-04-25 16:20:03 -04:00
polar b406cf9b81 perl: Correct nebulous #include within Store.xs 2024-04-25 16:17:13 -04:00
Valentin Gagarin 1c2336ff5f add a recommendation for first-time contributors (#10605)
this is an idea that came up in a discussion among maintainers
2024-04-25 13:34:15 +02:00
Guillaume Maudoux 19cc50dcbf fixup: Compute fingerprint only if needed 2024-04-25 00:47:46 +02:00
Guillaume MaudouxandEelco Dolstra 1c4e392c64 Compute fingerprint only if needed
As per Eelco's review comments

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-04-25 00:44:47 +02:00
John Ericson 7e104840a9 Merge pull request #10604 from hercules-ci/doc-manual-c-api
doc/manual: Add C API to menu
2024-04-24 14:34:56 -04:00
Robert Hensing a92eb5fc33 doc/manual: Add C API to menu 2024-04-24 20:04:18 +02:00
HaeNoe 943a877a6a use default value in fetchers::PublicKey json deserialization 2024-04-24 18:23:04 +02:00
HaeNoe c73172e986 add unit tests for getNullable 2024-04-24 18:22:53 +02:00
HaeNoe 4ff7f5aa9c refactor fetchers::PublicKey tests 2024-04-24 18:22:41 +02:00
5747d244ed streamline macOS uninstall instructions (#10589)
* move single-user uninstall to the end

this is not the default method of installation, and therefore irrelevant
for most users.

* move the backup restore instructions to the first step

for most users we can expect that the system-wide shell init files were
not ever touched, so we can as well tell them to do the most likely
thing.

from experience, while it's not necessarily safe to just mess with these
files, most people are simply confused by the complexity of
instructions.

* provide more detailed instructions for using `sudo vifs`

we can expect most beginners not to ever have used `vi`, and they will
probably need some hand-holding.

* express instructions as a script

Co-authored-by: wamirez <wamirez@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-24 15:04:49 +00:00
Eelco Dolstra e3a4e40a35 Merge pull request #10594 from NixOS/bump-2.23.0
Bump version
2024-04-23 14:37:30 +02:00
Eelco Dolstra 26384a3187 Bump version 2024-04-23 14:14:13 +02:00
Eelco Dolstra 1cfc9da472 Merge pull request #10593 from NixOS/release-notes
2.22 release notes
2024-04-23 10:50:43 +02:00
Eelco Dolstra b219017b88 Typo 2024-04-23 10:21:45 +02:00
Eelco Dolstra 34c5346e98 release notes: 2.22.0 2024-04-23 10:19:32 +02:00
Eelco Dolstra df424667a3 Merge pull request #10591 from Ma27/require-libseccomp-2.5.5
Require at least libseccomp 2.5.5
2024-04-22 23:27:39 +02:00
Maximilian Bosch 73918b0ae4 Require at least libseccomp 2.5.5
Closes #10585

As it turns out, libseccomp maintains an internal syscall table and
validates each rule against it. This means that when using libseccomp
2.5.4 or older, one may pass `452` as syscall number against it, but
since it doesn't exist in the internal structure, `libseccomp` will refuse
to create a filter for that. This happens with nixpkgs-23.11, i.e. on
stable NixOS and when building Nix against the project's flake.

To work around that

* a backport of libseccomp 2.5.5 on upstream nixpkgs has been
  scheduled[1].

* the package now uses libseccomp 2.5.5 on its own already. This is to
  provide a quick fix since the correct fix for 23.11 is still a staging cycle
  away.

It must not be possible to build a Nix with an incompatible libseccomp
version (nothing can be built in a sandbox on Linux!), so configure.ac
rejects libseccomp if `__SNR_fchmodat2` is not defined.

We still need the compat header though since `SCMP_SYS(fchmodat2)`
internally transforms this into `__SNR_fchmodat2` which points to
`__NR_fchmodat2` from glibc 2.39, so it wouldn't build on glibc 2.38.
The updated syscall table from libseccomp 2.5.5 is NOT used for that
step, but used later, so we need both, our compat header and their
syscall table 🤷

[1] https://github.com/NixOS/nixpkgs/pull/306070
2024-04-22 22:37:34 +02:00
Guillaume Maudoux a60a1f09b2 Reuse eval caches and related values when possible 2024-04-22 20:32:41 +02:00
Dylan Greenandvolth aa165301d1 Pathlocks Implementation for Windows (#10586)
Based on Volth's original port.

Co-authored-by: volth <volth@volth.com>
2024-04-22 15:08:10 +00:00
Robert Hensing 750bcaa330 Fix fetchGit nested submodules 2024-04-22 16:41:40 +02:00
Eelco Dolstra e1fd0e0a8c Merge pull request #10581 from bam80/patch-3
Update uninstall.md: mention .profile
2024-04-22 11:33:58 +02:00
Eelco Dolstra 790202232c Merge pull request #10580 from bam80/patch-1
Update installing-binary.md: give TTY to the installer
2024-04-22 11:17:39 +02:00
Eelco Dolstra 16669ae445 Update doc/manual/src/installation/uninstall.md 2024-04-22 11:00:09 +02:00
Eelco Dolstra cf1adbdd64 Merge pull request #10572 from NorfairKing/no-more-prev
Remove the 'prev' check entirely.
2024-04-22 10:53:31 +02:00
José Luis Lafuente 6acf02b32a C API: source argument to nix_copy_value should be const 2024-04-21 22:46:18 +02:00
José Luis Lafuente 8d70db3251 C API: add check_value_[in,out] helper functions 2024-04-21 22:44:14 +02:00
José Luis Lafuente ff76dd2211 C API: fix test, nix float is a double internally 2024-04-21 22:44:14 +02:00
José Luis Lafuente ccad6e94e2 C API: add (un)initialized value checks 2024-04-21 22:44:14 +02:00
José Luis Lafuente 9d7dee4a8f nix::Value: Use more descriptive names 2024-04-21 22:44:13 +02:00
José Luis Lafuente 5cc4af5231 Add isInitialized to nix::Value
Add a method to check if a value has been initialized. This helps avoid
segfaults when calling `type()`.
Useful in the context of the new C API.

Closes #10524
2024-04-21 22:44:13 +02:00
Robert Hensing 6fd2f42c2d Merge pull request #10573 from RCoeurjoly/Rename_SearchPath
Rename SearchPath to LookupPath and searchPath to lookupPath
2024-04-21 17:33:46 +02:00
Andrey Butirsky a6d08e3502 Update uninstall.md: remove ~/.nix-* files also 2024-04-21 17:08:37 +03:00
Andrey Butirsky d4b44a41fb Update uninstall.md: mention .profile
~/.profile is auto-edited by the single-mode installer so we should mention it in the uninstall instructions
2024-04-21 16:46:03 +03:00
Andrey Butirsky 722dfe9908 Update installing-binary.md: give TTY to the installer
Run the installer with TTY so the process can go interactively
2024-04-21 16:24:09 +03:00
Robert Hensing c6526fa33b Merge pull request #10563 from hercules-ci/doc-glossary-base-directory
doc/glossary: Add base directory
2024-04-21 15:17:16 +02:00
Robert Hensing 6a5d222818 Merge pull request #7745 from hercules-ci/pre-commit
Add pre-commit hook and CI check, excluding currently unformatted files
2024-04-21 14:43:25 +02:00
Robert Hensing d871e7cc5b Merge pull request #10562 from hercules-ci/unit-test-clean-config
Run unit tests with clean config
2024-04-21 14:28:03 +02:00
Robert Hensing e8d267ad5b doc/values: Refer to base directory 2024-04-21 14:24:33 +02:00
Robert Hensing 73125e46fc doc/glossary: Add base directory 2024-04-21 14:24:33 +02:00
Robert Hensing b5f1d4cce9 Edit docs 2024-04-21 14:16:46 +02:00
Robert Hensing a3ff75fd7e devShells: null out src to avoid nix develop rebuild
Whenever src changed, nix develop would internally create a fresh
derivation, which it has to try and substitute and then build.
Let's not do that.
2024-04-21 13:54:34 +02:00
Robert Hensing 96c8a9a417 devShells: Prefix shell-for-
Without this, it's not clear from an error trace that it's the
shell that's evaluated. It would look like evaluating the nix
package.
2024-04-21 13:54:34 +02:00
Robert Hensing 8f042a5e6d pre-commit: Remove nixpkgs-fmt 2024-04-21 13:54:34 +02:00
Robert Hensing 5b36ee4c95 Add pre-commit hook and make format target
I've added the new local.mk to the package sources. While this
should not be needed for the build, it is the simplest solution,
and won't cause many extra rebuilds, because the file won't change
very often.
2024-04-21 13:54:34 +02:00
Tom Sydney Kerckhove ad65a50a94 Remove the 'prev' check 2024-04-20 14:46:23 +02:00
Robert Hensing 1b6cd1d2af Revert "tests/test-libstoreconsumer: Ignore config"
This reverts commit 62feb5ca09263c78ddb692836228223e5b58d3ae.

It runs as part of the functional tests, which control the environment,
solving some of the problems a default config has when run in the
sandbox.
2024-04-20 01:45:04 +02:00
Robert Hensing bcaa2e4a85 tests/libstore-support: Ignore config 2024-04-20 01:45:04 +02:00
Robert Hensing 0ecf7dac3b tests/test-libstoreconsumer: Ignore config 2024-04-20 01:45:04 +02:00
Robert Hensing e05b58b060 init: Add flag to avoid loading configuration 2024-04-20 01:45:04 +02:00
Robert Hensing 8c4c2156bd doc/glossary: Define output closure (#8311) 2024-04-19 15:48:56 +02:00
John Ericson b973cd494f Enable the unix:// store on Windows
Windows now has some basic Unix Domain Socket support, see
https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/

Building `nix daemon` on Windows I've left for later, because the daemon
currently forks per connection but this is not an option on Windows. But
we can get the client part working right away.
2024-04-18 16:58:32 -04:00
John Ericson 3a3c205fa7 Use rand not random for creating GC root indirect links
I don't think fewer bits matters for this, and `rand` but not `random`
is available on Windows.
2024-04-18 16:57:11 -04:00
Robert Hensing 538eb2617a Merge pull request #10537 from hercules-ci/c-api-nix-value-init-apply
C API: Add nix_init_apply
2024-04-18 20:09:24 +02:00
Robert Hensing ad643cde58 C API: Add nix_init_apply
Thunks are relevant when initializing attrsets and lists, passing
arguments. This is an important way to produce them.
2024-04-18 19:13:38 +02:00
Eelco Dolstra f00b3b37b5 Merge pull request #10554 from 0x4A6F/master-scripts/upload-release
scripts/upload-release: fix riscv64 call
2024-04-18 19:12:22 +02:00
Eelco Dolstra 40499bca99 Merge pull request #10549 from alyssais/exportReferencesGraph
Fix exportReferencesGraph when given store subpath
2024-04-18 17:23:12 +02:00
0x4A6F f8a67d7e26 scripts/upload-release: fix riscv64 call 2024-04-18 17:18:48 +02:00
John Ericson b2b776da4f Merge pull request #10501 from Ma27/seccomp-fchmodat2
libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
2024-04-18 10:07:05 -04:00
0x4A6F e3fa7c38d7 system: build for riscv64-unknown-linux-gnu (#10228)
* system: add support for riscv64-unknown-linux-gnu

* maintainers: upload riscv64-linux-gnu

* doc: add riscv64-linux to supported platforms
2024-04-18 11:10:52 +00:00
John Ericson fb9f4208ed Don't include linux/ in #include
The linux dirs are conditionally added to the `-I` path.
2024-04-18 12:20:20 +02:00
Maximilian Bosch ba68045187 libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through https://github.com/NixOS/nix/blob/9b88e5284608116b7db0dbd3d5dd7a33b90d52d7/src/libstore/build/local-derivation-goal.cc#L1650-L1663.
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
and seccomp 2.5.5 are needed to have the correct syscall number available
via `__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I tested the change by adding the diff below as patch to
`pkgs/tools/package-management/nix/common.nix` & then built a VM from
the following config using my dirty nixpkgs master:

    {
      vm = { pkgs, ... }: {
        virtualisation.writableStore = true;
        virtualisation.memorySize = 8192;
        virtualisation.diskSize = 12 * 1024;
        nix.package = pkgs.nixVersions.nix_2_21;
      };
    }

The original issue can be triggered via

    nix build -L github:nixos/nixpkgs/d6dc19adbda4fd92fe9a332327a8113eaa843894#lxc \
      --extra-experimental-features 'nix-command flakes'

however the problem disappears with this patch applied.

Closes #10424

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
2024-04-18 12:20:20 +02:00
Eelco Dolstra 731c389d32 Merge pull request #10552 from obsidiansystems/tiny-cpp-style-cleanup
`file-descriptor.hh`: Avoid some C-isms for better C++-isms
2024-04-18 10:25:33 +02:00
John Ericson 9c815db366 file-descriptor.hh: Avoid some Cism for better C++isms
- `reinterpret_cast` not C-style cast
- `using` not `typedef`
2024-04-17 19:55:40 -04:00
John Ericson c16fc2c1bd Merge pull request #10550 from obsidiansystems/less-cli-unix-only
Make a few commands that were Unix-only no longer
2024-04-17 17:22:06 -04:00
John Ericson 6fa3656a32 Make a few commands that were Unix-only no longer
Also clean up some more linux-specific (`setPersonality`) code in
alignment with recent best practices.
2024-04-17 16:20:56 -04:00
John Ericson cde0fae7d9 Merge pull request #10502 from obsidiansystems/local-overlay-doc-improvements
Improve `local-overlay` docs in a few ways
2024-04-17 15:52:14 -04:00
Alyssa Ross 0774e8ba33 Fix exportReferencesGraph when given store subpath
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:

	with import <nixpkgs> {};

	let
	  hello = writeShellScriptBin "hello" ''
	    echo ${toString builtins.currentTime}
	  '';
	in

	writeClosure [ "${hello}/bin/hello" ]

This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:

	error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store
2024-04-17 21:51:59 +02:00
John Ericson 21d9412ddc Improve local-overlay docs in a few ways
In response to https://discourse.nixos.org/t/super-colliding-nix-stores/28462/24
2024-04-17 15:28:19 -04:00
Eelco Dolstra 582c7f32e8 Merge pull request #10548 from a-kenji/wiki/link
docs: fix wiki link
2024-04-17 21:04:54 +02:00
a-kenji 5fd28eeba4 docs: fix wiki link
Link to the official wiki.
2024-04-17 19:34:22 +02:00
John Ericson ac253fb99e Merge pull request #8901 from nix-windows/mingw
Build a minimized Nix with MinGW
2024-04-17 12:52:47 -04:00
John Ericson 1063aa502a Relax store path canonicalization 2024-04-17 12:26:10 -04:00
John Ericson b529d91902 Prefix -DNIX_ paths to be windows-complient for windows
This is a hacky solution, but it will do for now.
2024-04-17 12:26:10 -04:00
John Ericson 98691b46e3 Get rid of shellCrossSystems
We don't need it now that our (minimized) Windows build of Nix succeeds!
2024-04-17 12:26:10 -04:00
John Ericson 8433027e35 Build a minimized Nix with MinGW
At this point many features are stripped out, but this works:

- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
2024-04-17 12:26:10 -04:00
John Ericson 2248a3f545 Create no-op Window pathlocks implementation
This keeps the call sites simple, eventually this should be filled in.
2024-04-17 11:48:14 -04:00
John Ericson 05b9dac754 Fix friend struct/class mismatch warning 2024-04-17 15:47:07 +00:00
Robert Hensing 0fade05e96 doc/internal-api/local.mk: Rebuild when headers change 2024-04-17 17:28:30 +02:00
Robert Hensing dd19cce9c4 doc/external-api/local.mk: Rebuild when headers change 2024-04-17 17:17:59 +02:00
Eelco Dolstra c40e2afd33 Merge pull request #10530 from hercules-ci/solve-make-warnings
local.mk: Solve warnings
2024-04-17 16:51:56 +02:00
Robert Hensing 062d3b339b Merge pull request #10533 from jlesquembre/jl/c-api_labeler
Auto label C API PRs
2024-04-17 16:50:21 +02:00
Eelco Dolstra 4638ddd855 Merge pull request #10322 from edolstra/finish-value
Ensure immutability of non-thunk values
2024-04-17 16:42:17 +02:00
José Luis Lafuente d42a2643b0 Auto label C API PRs 2024-04-17 16:20:37 +02:00
Eelco Dolstra 6a3ecdaa39 Merge remote-tracking branch 'origin/master' into finish-value 2024-04-17 16:02:44 +02:00
Robert Hensing 7a1135901d local.mk: Solve warnings
local.mk:5: warning: overriding recipe for target 'outputs/dev/include/nix/nix_api_expr.h'
local.mk:5: warning: ignoring old recipe for target 'outputs/dev/include/nix/nix_api_expr.h'
local.mk:5: warning: overriding recipe for target 'outputs/dev/include/nix/nix_api_external.h'
local.mk:5: warning: ignoring old recipe for target 'outputs/dev/include/nix/nix_api_external.h'
local.mk:5: warning: overriding recipe for target 'outputs/dev/include/nix/nix_api_value.h'
local.mk:5: warning: ignoring old recipe for target 'outputs/dev/include/nix/nix_api_value.h'
local.mk:5: warning: overriding recipe for target 'outputs/dev/include/nix/nix_api_store.h'
local.mk:5: warning: ignoring old recipe for target 'outputs/dev/include/nix/nix_api_store.h'
local.mk:5: warning: overriding recipe for target 'outputs/dev/include/nix/nix_api_util.h'
local.mk:5: warning: ignoring old recipe for target 'outputs/dev/include/nix/nix_api_util.h'
2024-04-17 15:37:14 +02:00
Eelco Dolstra d53ad516e8 Merge pull request #10517 from edolstra/path-display
Add missing setPathDisplay() calls
2024-04-17 15:36:08 +02:00
Eelco Dolstra e0a8d38dad Merge pull request #10527 from hercules-ci/devShell-api-docs
devShell: enable API docs
2024-04-17 15:35:35 +02:00
Eelco Dolstra d4860c8f17 Merge pull request #10528 from NixOS/thufschmitt-patch-1
Remove the git-submodule test from Github actions
2024-04-17 15:34:13 +02:00
Robert Hensing db948397ff Merge pull request #10523 from tweag/jl/c-api_nix_get_string
C API: nix_get_string now accepts a callback to return the value
2024-04-17 15:22:06 +02:00
José Luis Lafuente 1f1cd97c71 C API: Add section in Nix manual (#10519) 2024-04-17 15:21:17 +02:00
Théophane Hufschmitt deb71e5044 Merge pull request #10526 from hercules-ci/labeler-contributing
Auto-label contributor-experience in PRs
2024-04-17 15:20:49 +02:00
Théophane Hufschmitt a863a75f0b Remove the giy-submodule test from Github actions
It causes the CI to time out, so let's just run it on Hydra
2024-04-17 14:34:53 +02:00
Robert Hensing 45a1142a8e devShell: enable API docs
Affects both the deps and the configure flags in configurePhase.
2024-04-17 13:12:46 +02:00
Robert Hensing 94c861bebf labeler.yml: Add contributor-experience
Not exhaustive perhaps, but a good start.
2024-04-17 12:50:55 +02:00
José Luis Lafuente c75b143b6c C API: nix_get_string now accepts a callback to return the value 2024-04-16 22:36:39 +02:00
Eelco Dolstra 74e4bc9b1d Merge pull request #10486 from tweag/jl/c-api_function-pointer
C API: Safer function pointer casting
2024-04-16 18:56:08 +02:00
Eelco Dolstra 75687844e1 Merge pull request #10518 from crayor1/patch-1
Double word is superfluous
2024-04-16 18:07:57 +02:00
Eelco Dolstra 84572e7198 Merge pull request #10520 from crayor1/patch-2
Fix typo in hacking.md
2024-04-16 17:47:18 +02:00
Eelco Dolstra 28e0f0a04c Fix another typo 2024-04-16 17:43:17 +02:00
Eelco Dolstra 6892c9803c GitInputScheme: Fix path display for workdirs and submodules 2024-04-16 16:42:47 +02:00
Eelco Dolstra 79363b2273 MountedInputAccessor, FilteringInputAccessor: Respect the path display prefix/suffix
This was causing Git paths not to be rendered correctly.
2024-04-16 16:42:41 +02:00
crayor1 548a12c1fe Fix typo in hacking.md 2024-04-16 15:54:45 +02:00
crayor1 25265a9365 Double word is superfluous 2024-04-16 14:36:41 +02:00
Eelco Dolstra 6df58a0891 MercurialInputScheme: Improve path display 2024-04-16 14:14:50 +02:00
Eelco Dolstra fa01db9626 StorePathAccessor: Fix path display
Set the prefix to the store path to fix messages like "copying '/' to
the store" to "copying '/nix/store/bla' to the store".
2024-04-16 14:14:26 +02:00
Théophane Hufschmitt d2a07a96ba Merge pull request #10467 from edolstra/nix-shell-symlink
nix shell: Handle output paths that are symlinks
2024-04-16 12:31:14 +02:00
John Ericson 20f0472149 Merge pull request #10493 from obsidiansystems/cpp-indent-clang-format
Have `clang-format` indent conditional CPP
2024-04-15 10:29:04 -04:00
John Ericson 65cc237b3a Merge pull request #10482 from tweag/fix-symlink-in-sandbox
Fix the access of symlinks to host files in the sandbox
2024-04-15 09:29:00 -04:00
John Ericson bcda38c272 Have clang-format indent conditional CPP
This reflects the style I've been introducing in tandem with Windows
support.

See https://clang.llvm.org/docs/ClangFormatStyleOptions.html#indentppdirectives for this configuration option.
2024-04-15 08:33:45 -04:00
Eelco Dolstra cceae30aaf Combine the domain and key arguments into a single value for convenience 2024-04-15 13:03:27 +02:00
Eelco Dolstra aad11f4496 Simplify the fetcher cache 2024-04-15 13:03:27 +02:00
Eelco Dolstra d084c1cb41 Remove the "locked" flag from the fetcher cache
This also reworks the Mercurial fetcher (which was still using the
old cache interface) to have two distinct cache mappings:

* A ref-to-rev mapping, which is store-independent.
* A rev-to-store-path mapping.
2024-04-15 13:03:27 +02:00
José Luis Lafuente 774e7213e8 C API: Use nix_get_string_callback typedef 2024-04-15 12:05:57 +02:00
John Ericson 03eb4f7baa Merge pull request #10503 from haenoe/public-key-json-impl
`fetchers::PublicKey` json impl
2024-04-14 18:02:27 -04:00
HaeNoe e3fed2ebcf update fetchers::PublicKey json (de)serialization 2024-04-14 22:42:48 +02:00
HaeNoe ff4c286e80 add tests for optionalValueAt 2024-04-14 22:36:03 +02:00
HaeNoe bb939d3772 change implementation of optionalValueAt 2024-04-14 22:35:51 +02:00
José Luis Lafuente 76444a3958 C API: proper ifdef endif indentation 2024-04-14 16:18:32 +02:00
Roland Coeurjoly 62ce139e3f No need to undef now that there is no collision 2024-04-13 23:34:01 +02:00
John Ericson eff90af498 Slight refactors in preparation for #10480
Code operating on store objects (including creating them) should, in
general, use `ContentAddressMethod` rather than `FileIngestionMethod`.

See also dfc876531f which included some
similar refactors.
2024-04-13 12:13:47 -04:00
Roland Coeurjoly 40a6a9fdb8 Rename SearchPath to LookupPath and searchPath to lookupPath 2024-04-13 17:35:15 +02:00
John Ericson aa438b8fba Merge pull request #10414 from edolstra/remove-downloadFile-locked
downloadFile(): Remove the "locked" (aka "immutable") flag
2024-04-12 17:23:53 -04:00
José Luis Lafuente 01bad63c72 C API: Safer function pointer casting
See https://github.com/NixOS/nix/pull/8699#discussion_r1554312181

Casting a function pointer to `void*` is undefined behavior in the C
spec, since there are platforms with different sizes for these two kinds
of pointers. A safe alternative might be `void (*callback)()`
2024-04-12 21:41:15 +02:00
John Ericson 03e74e125e Merge pull request #10483 from obsidiansystems/docroot-glossary
docs: Refer to the glossary with `@docroot@` instead of `..`
2024-04-12 12:17:08 -04:00
John Ericson 95ae12b607 docs: Refer to the glossary with @docroot@ instead of ..
These unweildy relative paths probably predate the `@docroot@`
mechanism.
2024-04-12 11:46:47 -04:00
13c2005e7d add intermediate variables and clarifying comments (#9274)
* add intermediate variables and clarifying comments

Co-authored-by: Alexander Groleau <alex@proof.construction>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-12 17:43:35 +02:00
Théophane Hufschmitt cef677ddbc Test the inclusion of transitive symlinks in the sandbox 2024-04-12 16:10:22 +02:00
Théophane Hufschmitt acbb1523c1 Fix the access of symlinks to host files in the sandbox
https://github.com/NixOS/nix/pull/10456 fixed the addition of symlink
store paths to the sandbox, but also made it so that the hardcoded
sandbox paths (like `/etc/hosts`) were now bind-mounted without
following the possible symlinks. This made these files unreadable if
there were symlinks (because the sandbox would now contain a symlink to
an unreachable file rather than the underlying file).
In particular, this broke FOD derivations on NixOS as `/etc/hosts` is a
symlink there.

Fix that by canonicalizing all these hardcoded sandbox paths before
adding them to the sandbox.
2024-04-12 15:57:53 +02:00
John Ericson 5b9cb8b372 Merge pull request #10412 from roberth/c-string-context
C API: Add `nix_string_realise`
2024-04-11 12:07:31 -04:00
Robert Hensing f2522d4ecd libexpr-c: Add nix_store_path_name 2024-04-11 17:39:21 +02:00
Robert Hensing a512f4eebc test/libutil: Add OBSERVE_STRING macro
Makes string callback easier to pass, without mistakes.
2024-04-11 17:39:21 +02:00
Robert Hensing 876e70bc9a tests/unit/libexpr/local.mk
A proper build system would catch errors like this.
2024-04-11 17:39:21 +02:00
Robert Hensing 1233bcde37 libstore-c: Add nix_store_path_clone 2024-04-11 17:39:21 +02:00
Robert Hensing 48808a5320 tests/unit/libexpr: Enable nix_store_realise test, and add docs 2024-04-11 17:39:21 +02:00
Robert Hensing 94d9819bdc tests/unit/libexpr/main: Fix realisation 2024-04-11 17:39:19 +02:00
Robert Hensing ed13cf05a2 build-hook: Allow empty
Like always declining; local builds only, as can be inferred from the
docs. (Not worth spending too many words on this pretty obvious
behavior, I think. Also, plans to remove it? https://github.com/NixOS/nix/issues/1221)
2024-04-11 17:39:16 +02:00
Théophane Hufschmitt db6335d099 Merge pull request #10244 from bouk/submodule-fetch-default-method
git fetcher: fix resolveSubmoduleUrl to work with all repo URLs
2024-04-11 16:58:28 +02:00
Bouke van der Bijl 1e4f902b28 Add gitSubmodules test to github actions 2024-04-11 15:55:09 +02:00
Bouke van der Bijl cd06193d13 Add nixos test 2024-04-11 15:28:38 +02:00
Bouke van der Bijl 1a76ca4161 Set the origin instead of hacking in the URL resolving 2024-04-11 14:22:12 +02:00
Bouke van der Bijl 1f73de2629 git fetcher: relax absolute URL check of resolveSubmoduleUrl
This matches up the behavior with the internals of libgit2

Fixes #9979
2024-04-11 14:22:12 +02:00
Théophane Hufschmitt da1e977bf4 Merge pull request #10456 from NixOS/fixpermdeniedbind
Fix adding symlink to the sandbox paths
2024-04-11 13:41:49 +02:00
Eelco Dolstra 26a4688a86 nix shell: Test that store paths cannot link outside of the store 2024-04-11 11:06:35 +02:00
Eelco Dolstra 9d50f57fa3 Doh 2024-04-11 09:00:47 +02:00
Théophane Hufschmitt def00d7b52 Merge pull request #10455 from tie/structured-attrs-get-env
Do not rely on $stdenv/setup to set output variables
2024-04-11 06:48:36 +02:00
Eelco Dolstra 85b9f4ef4f nix shell: Handle output paths that are symlinks
This requires moving resolveSymlinks() into SourceAccessor. Also, it
requires LocalStoreAccessor::maybeLstat() to work on parents of the
store (to avoid an error like "/nix is not in the store").

Fixes #10375.
2024-04-10 23:49:19 +02:00
19c8867d2a Fix store-path.md (#10457)
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2024-04-10 21:11:57 +00:00
John Ericson 021488497d Merge pull request #10464 from obsidiansystems/nar-format-docs
Document the Nix Archive format
2024-04-10 16:26:27 -04:00
Eelco Dolstra 74aab19c4b Merge pull request #10458 from Ma27/rl-2.20-followup
doc/rl-2.20: clarify builders-use-substitutes vs. substitute-on-destion
2024-04-10 22:22:49 +02:00
John Ericson 3e5797e97f Document the Nix Archive format
This is adopted from Eelco's PhD thesis.
2024-04-10 15:21:22 -04:00
Ivan Trubach 664532c533 Do not rely on $stdenv/setup to set output variables
Instead of relying on setup script to set output variables when
structured attributes are enabled, iterate over the values of an
outputs associative array.

See also
https://github.com/NixOS/nixpkgs/blob/374fa3532ee7d7496165d3b5a6652a3dcad1ebc2/pkgs/stdenv/generic/setup.sh#L23-L26
2024-04-10 19:50:19 +03:00
Maximilian Bosch 50557adb3b doc/rl-2.20: clarify builders-use-substitutes vs. substitute-on-destination
...as this lead to confusion before.
2024-04-10 17:26:58 +02:00
Théophane Hufschmitt a268c0de71 Merge pull request #10413 from Ma27/path-info-all-binary-cache
path-info: print correct path when using `nix path-info --store file://... --all --json`
2024-04-10 15:25:50 +02:00
Théophane Hufschmitt ae4737294e doBind: Use our own lstat wrapper
Doesn't change much, but brings a bit more consistency to the code
2024-04-10 15:20:10 +02:00
Théophane HufschmittandArtturin 913db9f738 Fix permission denied when building symlink derivation which points to a symlink out of the store
Bind-mounting symlinks is apparently not possible, which is why the
thing was failing.

Fortunately, symlinks are small, so we can fallback to copy them at no cost.

Fix https://github.com/NixOS/nix/issues/9579

Co-authored-by: Artturin <Artturin@artturin.com>
2024-04-10 15:19:18 +02:00
Théophane Hufschmitt 872d93eb13 Add a test for depending on a symlink store path
Regression test for https://github.com/NixOS/nix/issues/9579
2024-04-10 15:17:39 +02:00
Théophane Hufschmitt 65d711351e Merge pull request #10449 from Ma27/rl-2.20-nix-copy-ssh-ng
doc/rl-2.20: add missing entry about `nix copy --to ssh-ng://...`
2024-04-10 11:44:31 +02:00
Eelco Dolstra 3fd8dfec4d Merge pull request #10021 from obsidiansystems/ca-type-names
Make `outputHashAlgo` accept `"nar"`, stay in sync
2024-04-09 23:06:43 +02:00
John EricsonandEelco Dolstra 93d68e18e5 Make outputHashAlgo accept "nar", stay in sync
Now that we have a few things identifying content address methods by
name, we should be consistent about it.

Move up the `parseHashAlgoOpt` for tidiness too.

Discussed this change for consistency's sake as part of #8876

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-04-09 16:17:23 -04:00
Maximilian Bosch f34b8de5b2 doc/rl-2.20: add missing entry about nix copy --to ssh-ng://...
This requires `--substitute-on-destination` if you want the remote side
to substitute instead of copying if possible.

For completeness sake, document it here.

Also, the stable Nix from nixpkgs is still 2.18, so more folks may
stumble upon this when this is bumped, so I'd expect this to be actually
useful.

Closes #10182
2024-04-09 21:27:00 +02:00
Eelco Dolstra c313394ae9 Merge pull request #10432 from obsidiansystems/delete-resolve-system-dependencies
Remove `resolve-system-dependencies`
2024-04-08 18:41:14 +02:00
Eelco Dolstra d29786f258 downloadFile(): Remove the "locked" (aka "immutable") flag
This was used in only one place, namely builtins.fetchurl with an
expected hash. Since this can cause similar issues as described
in #9814 and #9905 with the "locked" flag for fetchTarball and fetchTree,
let's just remove it.

Note that if an expected hash is given and the hash algorithm is
SHA-256, then we will never do a download anyway if the resulting
store path already exists. So removing the "locked" flag will only
cause potentially unnecessary HTTP requests (subject to the tarball
TTL) for non-SHA-256 hashes.
2024-04-08 15:56:16 +02:00
John Ericson e68f24f1e0 Remove resolve-system-dependencies
Fix #9769

As Abathur reports, it seems to be unused since #3429 in 2020.
2024-04-08 09:55:42 -04:00
Eelco Dolstra 334874d101 Merge pull request #10433 from edolstra/run-mercurial-tests
Actually run the Mercurial tests
2024-04-08 15:52:13 +02:00
Eelco Dolstra 737ce5e81f Actually run the Mercurial tests 2024-04-08 15:21:57 +02:00
Eelco Dolstra 9d2b9ef92e Merge pull request #10423 from AndrewDDavis/patch-1
Prevent `nix-daemon.sh` from leaking variable into user environment
2024-04-08 14:36:50 +02:00
Théophane Hufschmitt c749c115ab Merge pull request #10427 from lf-/jade/fix-nix-doctor
"but doctor, I AM the untrusted store": nix doctor had wrong trustedness
2024-04-08 11:29:43 +02:00
Théophane Hufschmitt bd8c276ddb Improve the config check output for stores that don't know about trust
Make it proper english
2024-04-08 11:02:39 +02:00
Jade Lovelace dea23c3c9b "but doctor, I AM the untrusted store": nix doctor had wrong trustedness
This probably snuck in in a refactor using truthiness or so. The
trustedness flag was having the optional fullness checked, rather than
the actual contained trust level.

Also adds some tests.

```
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix store ping
warning: 'nix store ping' is a deprecated alias for 'nix store info'
Store URL: daemon
Version: 2.20.4
Trusted: 0
m1@6876551b-255d-4cb0-af02-8a4f17b27e2e ~ % nix doctor
warning: 'doctor' is a deprecated alias for 'config check'
[PASS] PATH contains only one nix version.
[PASS] All profiles are gcroots.
[PASS] Client protocol matches store protocol.
[INFO] You are trusted by store uri: daemon
```
2024-04-07 22:43:02 -07:00
John Ericson fef952e258 Merge pull request #8397 from NixLayeredStore/overlayfs-store
Local Overlay Store
2024-04-07 22:57:01 -04:00
John Ericson bd7c26bc7b Add comment explaining LIBMOUNT_FORCE_MOUNT2=always 2024-04-07 21:55:19 -04:00
Robert Hensing 5f9aaa86eb Merge pull request #10416 from obsidiansystems/cgroup-linux-only
Make `cgroup.{cc,hh}` linux-only files
2024-04-07 16:52:31 +02:00
Andrew Davis a2a633d332 Prevent nix-daemon.sh from leaking variable into user environment
The script at `/nix/store/...-nix-2.21.0/etc/profile.d/nix-daemon.sh`  was leaving behind a variable, which was visible in the user's shell environment, but not used outside the script.
2024-04-06 10:26:29 -04:00
Théophane Hufschmitt 2dc06b3969 Merge pull request #10420 from stuebinm/markdown-doc-formatting
fix codeblocks in the repl's `:doc`
2024-04-06 14:12:15 +02:00
cidkidnix e73dc0e938 Use LIBMOUNT_FORCE_MOUNT2=always to workaround new mount API issues 2024-04-05 16:43:14 -05:00
stuebinm 910211f9ff avoid markdown which the repl's :doc cannot handle
code blocks, if not surrounded by empty lines, have the language
tags (in these cases, always `nix`) show up in the output of :doc.

for example:

  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example: nix builtins.parseFlakeRef
    "github:NixOS/nixpkgs/23.05?dir=lib"  evaluates to: nix { dir =
    "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type =
    "github"; }

is now instead:
  nix-repl> :doc builtins.parseFlakeRef
  Synopsis: builtins.parseFlakeRef flake-ref

    Parse a flake reference, and return its exploded form.

    For example:

      | builtins.parseFlakeRef "github:NixOS/nixpkgs/23.05?dir=lib"

    evaluates to:

      | { dir = "lib"; owner = "NixOS"; ref = "23.05"; repo = "nixpkgs"; type = "github"; }
2024-04-05 23:19:32 +02:00
John Ericson c99c80f075 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-04-05 16:32:02 -04:00
Maximilian Bosch c80cd6bb06 path-info: print correct path when using nix path-info --store file://... --all --json
When querying all paths in a binary cache store, the path's representation
is `<hash>-x` (where `x` is the value of `MissingName`) because the .narinfo
filenames only contain the hash.

Before cc46ea1630 this worked correctly,
because the entire path info was read and the path from this
representation was printed, i.e. in the form `<hash>-<name>`. Since then
however, the direct result from `queryAllValidPaths()` was used as `path`.

Added a regression test to make sure the behavior remains correct.
2024-04-05 21:29:55 +02:00
John Ericson 5a365b0c89 Delete dead openFile in binary-cache-store.cc (#10418)
d64cb33e90 / #5111 previously deleted the dead code where this was used, but missed this.
2024-04-05 18:31:43 +00:00
John Ericson a3d5a71c5f Slight cleanup of builtins.derivation outputHashAlgo logic (#10417)
This was part of approved PR #10021. Unfortunately that one is stalled
on a peculiar Linux test timeout, so trying to get bits of it merged
first to bisect failure.
2024-04-05 18:10:28 +00:00
John Ericson 513634ab5b Make cgroup.{cc,hh} linux-only files
Forcing a conditional include, vs making the headers content
conditional, I think is more maintainable.

It is also how the other platform-specific headers (like
`namespaces.hh`) have been adapted.
2024-04-05 12:29:14 -04:00
Eelco Dolstra 74130fd1f1 Merge pull request #10411 from edolstra/fix-warning
setInterruptCheck(): Remove declared but undefined function
2024-04-05 16:31:29 +02:00
Robert Hensing c145ce0e1a realiseContext: Remove no-op replacements
A possible use of them might have been to figure out the paths
(which can now be retrieved with maybePathsOut), but I have
not found evidence that it was used this way, and it would have
been broken, because non-CA outputs weren't recorded in the map.
2024-04-05 16:15:43 +02:00
Robert Hensing 02c41aba5b libexpr-c: Add nix_string_realise 2024-04-05 16:08:18 +02:00
Eelco Dolstra 75be3f23c6 setInterruptCheck(): Remove declared but undefined function 2024-04-05 16:03:25 +02:00
Eelco Dolstra 62f8d8c9a9 Merge pull request #10288 from tweag/refine-release-cycle
Update the release cycle
2024-04-05 13:07:31 +02:00
Eelco Dolstra 75fd09b3a1 Merge pull request #10401 from nix-windows/better-signals-interface
Better signals interface
2024-04-05 13:06:16 +02:00
Eelco Dolstra 3d0d9085d5 Merge pull request #10408 from NixOS/fix-make
fix: Remove duplicate imports from Makefile
2024-04-05 13:02:51 +02:00
Robert Hensing 25584e215e fix: Remove duplicate imports from Makefile 2024-04-05 12:03:53 +02:00
John Ericson 50f621b241 Better signals interface
This avoids some CPP and accidentally using Unix stuff in client code.
2024-04-05 01:38:28 -04:00
Théophane Hufschmitt 9d03c2b08b Merge pull request #10406 from obsidiansystems/fix-c-bindings-portability
Fix some portability issues with the new C bindings
2024-04-05 07:17:14 +02:00
John Ericson c1e0769355 Fix some portability issues with the new C bindings
Build without GC is unbroken

Fix #10403

Also building tests with Windows (assuming rest of Windows fixes) is
unbroken.
2024-04-05 00:54:47 -04:00
John Ericson 19016c7b52 Merge pull request #10400 from nix-windows/better-env-var
Clean up env var logic in preparation for Windows
2024-04-04 18:49:07 -04:00
John EricsonandThéophane Hufschmitt ef2d10f7e7 Clean up env var logic in preparation for Windows
It's a little weird we don't check the return status for these, but
changing that would introduce risk so I did not.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-04-04 18:17:12 -04:00
John Ericson 6b889e0588 Merge pull request #10399 from nix-windows/portable-sleep
Make SQLite busy back-off logic portable
2024-04-04 13:08:42 -04:00
John Ericson 1577b5fa67 Make SQLite busy back-off logic portable
Use C++ standard library not Unix functions for sleeping and randomness.

Suggested by @edolstra in https://github.com/NixOS/nix/pull/8901#discussion_r1550416615
2024-04-04 12:43:33 -04:00
Robert Hensing 12ec3154b8 Merge pull request #8699 from tweag/nix-c-bindings
(Towards) stable C bindings for libutil, libexpr
2024-04-04 17:50:52 +02:00
HaeNoeandJohn Ericson 50cb14fcf9 Improve checked json casting (#10087)
This introduces new utility functions to get elements from JSON — in an ergonomic way and with nice error messages if the expected type does not match.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-04-03 18:04:00 +00:00
Eelco Dolstra bf86b939f8 Merge pull request #10391 from edolstra/home-symlink
Handle the case where a parent of ~/.nix-defexpr is a symlink
2024-04-03 17:20:35 +02:00
John Ericson 36d92dc80a Merge pull request #10316 from dottharun/derivation-output-test
test: for derivation output selection with `.`
2024-04-03 10:48:04 -04:00
Eelco Dolstra 09551fabd0 Handle the case where a parent of ~/.nix-defexpr is a symlink
Fixes https://github.com/DeterminateSystems/nix-installer/issues/912 and probably #10247.
2024-04-03 16:46:23 +02:00
Valentin GagarinandRobert Hensing 59597628cb show Nix logo in the manual (#9870)
* show Nix logo in the manual

the location of files is hard-coded by mdBook.
there is also seems to be no way to define custom templates, therefore
all styling has to be done in the CSS override.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-03 14:22:47 +00:00
Théophane Hufschmitt 5536788404 Merge pull request #10386 from SkamDart/SkamDart/include-directive-config-test
Add functional tests for include directives in nix config file
2024-04-03 08:55:04 +02:00
Tharun T 8d84de455e outputSpecified doesnt exit in top attr-set 2024-04-03 08:26:42 +05:30
Cameron Dart ba0bd8fae6 Add functional tests for include directive in nix config file 2024-04-02 18:10:36 -07:00
John Ericson 478c05308c Merge pull request #10364 from obsidiansystems/split-out-unix
Start factoring out Unix-assuming code
2024-04-02 15:07:36 -04:00
John EricsonandRobert Hensing 02fa20622f Start factoring out Unix assumptions
This splits files and adds new identifiers in preperation for supporting
windows, but no Windows-specific code is actually added yet.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-02 14:43:38 -04:00
John Ericson 852391765d Add unix (and linux) dirs
In the Nix commit, platform-specific sources will go here.
2024-04-02 12:55:23 -04:00
Théophane Hufschmitt 290be6c906 Merge pull request #10345 from edolstra/fix-chroot-flakes
Fix flake evaluation in chroot stores
2024-04-02 17:49:45 +02:00
Théophane Hufschmitt 29c3e4f580 Merge pull request #10340 from edolstra/trust-github
Add trust-tarballs-from-git-forges setting
2024-04-02 16:08:57 +02:00
Eelco Dolstra bb5764e578 Merge pull request #10301 from a-n-n-a-l-e-e/wal-persist
enable persistent WAL mode for sqlite db
2024-04-02 12:02:12 +02:00
Eelco Dolstra 911febe56c Merge pull request #10358 from yshui/fix-include
Fix "include" directive in config file
2024-04-02 11:50:12 +02:00
Eelco Dolstra e473610e7a Merge pull request #10380 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.5.0
build(deps): bump zeebe-io/backport-action from 2.4.1 to 2.5.0
2024-04-02 11:41:39 +02:00
Eelco Dolstra c4f4726dbc Merge pull request #10381 from NixOS/dependabot/github_actions/actions/checkout-4
build(deps): bump actions/checkout from 3 to 4
2024-04-02 11:41:25 +02:00
dependabot[bot] 29239a2829 build(deps): bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 22:32:31 +00:00
dependabot[bot] 39c554aad4 build(deps): bump zeebe-io/backport-action from 2.4.1 to 2.5.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.4.1 to 2.5.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.4.1...v2.5.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 22:32:29 +00:00
John Ericson 9b88e52846 Merge pull request #10362 from obsidiansystems/maybeLstat
Factor out `nix::maybeLstat`
2024-03-30 11:23:29 -04:00
John EricsonandThéophane Hufschmitt 3752bbef28 Document maybeLstat
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-30 10:39:25 -04:00
Théophane Hufschmitt 845b2a9256 Merge pull request #10363 from obsidiansystems/is-root-user
Factor out `isRootUser` function
2024-03-30 07:15:56 +01:00
Théophane Hufschmitt c864e3b871 Merge pull request #10341 from xokdvium/xokdvium/fix-double-decompression
fix(libutil): apply only the specified filter to decompress archive
2024-03-30 07:13:09 +01:00
Sergei Zimmerman a66b5a1526 test(nixos): add integration test for doubly-compressed content
Add an integration test with a compressing proxy (nginx is used).
This test verifies that fetched archives do not get decompressed
excessively.
2024-03-30 01:29:34 +03:00
Sergei Zimmerman 500683a949 fix(libutil): apply only the specified filter to decompress archive
This patch makes `makeDecompressionSink` strip only a single layer
of compression specified via method. This fixes erroneous decompression
of doubly-compressed NARs fetched with curl.
2024-03-30 01:29:29 +03:00
Sergei Zimmerman 6d9bafb3b8 nfc(libutil): reformat files
Run clang-format on compression.{cc,hh} and tarfile{cc,hh}.
This way follow-up patches will be formatted properly and have
easier to read diffs.
2024-03-30 01:29:22 +03:00
John Ericson e4d9b207c2 Factor out isRootUser function 2024-03-29 16:33:01 -04:00
John Ericson 8be347afca Factor out nix::maybeLstat
This function is nice for more than `PosixSourceAccessor`. We can make a
few things simpler with it.

Note that the error logic slightly changes in some of the call sites, in
that we also count `ENOTDIR` and not just `ENOENT` as not having the
file, but that should be fine.
2024-03-29 16:32:53 -04:00
John Ericson eeecbb9c36 Merge pull request #10361 from obsidiansystems/fix-exception-type
`throwExceptionSelfCheck` throw `Error`
2024-03-29 16:31:39 -04:00
John Ericson 555181c3fd throwExceptionSelfCheck throw Error
`SysError` is not appropriate because there is no (Unix) syscall involved.

The catch block in `initLibUtil` is already for `Error` and still works.
2024-03-29 16:05:05 -04:00
Yuxuan Shui 0be87c023f Fix "include" directive in config files 2024-03-29 16:33:13 +00:00
Théophane Hufschmitt b72e1c79da Merge pull request #10299 from detroyejr/remove-repl-flake
flakes: remove experimental repl-flake
2024-03-29 15:06:23 +01:00
José Luis Lafuente 926fbadcc3 C API: add more tests 2024-03-29 14:00:19 +01:00
Jonathan De TroyeandThéophane Hufschmitt a09d0e19c1 flakes: Reword doc/manual/rl-next/remove-repl-flake.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-29 08:57:41 -04:00
Eelco Dolstra 00ce36fafe Add test 2024-03-29 13:38:55 +01:00
Eelco Dolstra 46d9e70c20 Add trust-tarballs-from-git-forges setting
If enabled, GitHub flakerefs don't require a content hash, a Git
revision is enough.

Fixes #10297.
2024-03-29 13:38:47 +01:00
Théophane Hufschmitt 90f5189451 Merge pull request #10131 from intelfx/work/notty
`TERM=dumb` fixes
2024-03-29 13:38:37 +01:00
Eelco Dolstra 8930772864 Add regression test for #10331, #10267 2024-03-29 13:20:50 +01:00
Eelco Dolstra dffc22f30f Rename local-store.sh -> chroot-store.sh 2024-03-29 12:51:14 +01:00
José Luis Lafuente 2d84433a3b C API: update documentation 2024-03-29 10:05:21 +01:00
José Luis Lafuente 2bb609bce2 C API: rename nix_observe_string -> nix_get_string_callback 2024-03-29 10:01:16 +01:00
Théophane Hufschmitt de101417eb Merge pull request #10305 from hercules-ci/addErrorContext-is-great
Always print addErrorContext
2024-03-29 09:10:09 +01:00
Robert Hensing ef26133df3 Merge pull request #10337 from Cyclic4179/patch-1
`nix shell` shebang: support O'Caml comments
2024-03-28 19:54:03 +01:00
José Luis Lafuente 061140fc8f C API: remove unused argument 2024-03-28 19:38:12 +01:00
José Luis Lafuente 925a8fda6e C API: Use new ListBuilder helper
See https://github.com/NixOS/nix/pull/10251
2024-03-28 19:02:01 +01:00
José Luis Lafuente c57de60522 C API: Keep the structure flat
See https://github.com/NixOS/nix/pull/10329
2024-03-28 19:00:04 +01:00
Eelco Dolstra 7829caab49 Merge pull request #10347 from Atry/patch-2
docs: clarify that `builtins.stringLength` is counting bytes
2024-03-28 15:56:22 +01:00
José Luis Lafuente d96b52bd8b C api: nix_export_std_string -> nix_observe_string 2024-03-28 10:52:02 +01:00
José Luis Lafuente 940ff6535c C API: update libstore tests 2024-03-28 10:52:02 +01:00
José Luis Lafuente 31fbb24329 C API: refactor nix_store_realise 2024-03-28 10:52:02 +01:00
José Luis Lafuente 1a574c6c60 C API: refactor ListBuilder 2024-03-28 10:52:01 +01:00
José Luis Lafuente 34d15e8f2f C API: rename nix_store_build -> nix_store_realise 2024-03-28 10:52:01 +01:00
José Luis Lafuente 1093ab64a2 C API: add more tests 2024-03-28 10:52:01 +01:00
José Luis Lafuente 2e1dbbe307 C API: refactor test support 2024-03-28 10:52:01 +01:00
José Luis Lafuente 693e8ec8fe C API: unify makefile after rebase 2024-03-28 10:52:01 +01:00
José Luis Lafuente c49b88b066 C API: update docs based on PR feedback 2024-03-28 10:52:01 +01:00
José Luis Lafuente 7c602d9f01 C API: add tests for external values 2024-03-28 10:52:01 +01:00
José Luis Lafuente 2349185c96 C API: fix after rebase 2024-03-28 10:52:00 +01:00
José Luis Lafuente 6c231dcf68 C API: disable test 2024-03-28 10:52:00 +01:00
José Luis Lafuente b9cd24a4a8 C API: fix api_expr tests 2024-03-28 10:52:00 +01:00
José Luis Lafuente 24c8f6864d C API: if store doesn't have a version, return an empty string 2024-03-28 10:52:00 +01:00
José Luis Lafuente dfdb90dc8e C API: Consolidate initializers 2024-03-28 10:52:00 +01:00
José Luis Lafuente 51ff547d9a C API: add more tests to nix_api_expr 2024-03-28 10:52:00 +01:00
José Luis Lafuente 415583a500 C API: use bool argument consistently 2024-03-28 10:52:00 +01:00
José Luis Lafuente d5ec1d0617 C API: nix_store_open, check for empty strings 2024-03-28 10:51:59 +01:00
José Luis Lafuente 535694122e C API: rename State to EvalState 2024-03-28 10:51:59 +01:00
José Luis Lafuente 24604d024a C API: fix docs build after rebase 2024-03-28 10:51:59 +01:00
José Luis Lafuenteandasymmetric 92dacec0e4 C API: Apply documentation suggestions
Co-authored-by: asymmetric <lorenzo@mailbox.org>
2024-03-28 10:51:59 +01:00
José Luis Lafuente ac3a9c6605 C API: add nix_api_expr tests 2024-03-28 10:51:48 +01:00
José Luis Lafuente 55601963b3 C API: fix documentation build 2024-03-28 10:50:08 +01:00
José Luis Lafuente 41f1669dea C API: add tests for libutil and libstore 2024-03-28 10:50:02 +01:00
José Luis Lafuente 46f5d0ee7b Apply suggestions from code review 2024-03-28 10:47:56 +01:00
José Luis Lafuente 550af113c6 String value refactor
Related to https://github.com/NixOS/nix/pull/9047
2024-03-28 10:47:56 +01:00
Yorick van Pelt c6e28d8da2 C API: fix: macos doesn't have std::bind_front 2024-03-28 10:47:56 +01:00
Yorick van Pelt ab9250286a C API: add a way to throw errors from primops 2024-03-28 10:47:56 +01:00
Yorick van Pelt 3d79f38709 C API: add user_data argument to nix_alloc_primop
Also add a helper function for primops, that converts to C argument
types
(and eventually handles errors)
2024-03-28 10:47:55 +01:00
Yorick van Pelt 48aa57549d primops: change to std::function, allowing the passing of user data 2024-03-28 10:47:55 +01:00
Yorick van Pelt 9e423dee11 C API: update after rebase 2024-03-28 10:47:55 +01:00
Yorick van Pelt e1bb799da9 C API: reformat according to proposed clang-format file 2024-03-28 10:47:55 +01:00
YorickandValentin Gagarin 91e53de7d3 C API: update README example
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-28 10:47:55 +01:00
Yorick van Pelt 9d380c0f76 C API: clarify some documentation 2024-03-28 10:47:55 +01:00
Yorick van Pelt 5d82d6e733 nix_api: fix missing includes in headers
Forward declaration doesn't work here, since we define classes that
contain the objects
2024-03-28 10:47:55 +01:00
YorickandValentin Gagarin 40f5d48d3c Apply documentation suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-28 10:47:55 +01:00
Yorick van Pelt 3b41830a96 docs/external-api: write main page 2024-03-28 10:47:54 +01:00
Yorick van Pelt e642bbc2a7 C API: move to src/lib*/c/ 2024-03-28 10:46:39 +01:00
Yorick van Pelt df9401eb4e nix_api_store: add nix_init_plugins 2024-03-28 10:39:07 +01:00
Yorick van Pelt dc0f7d8f96 initPlugins: run nix_plugin_entry() on dlopen'd plugins
Only when it exists.
2024-03-28 10:39:07 +01:00
Yorick van Pelt 713f10aeaa nix_api_value: Add nix_register_primop to add builtins 2024-03-28 10:39:07 +01:00
Yorick van Pelt e891aac2e4 nix_api_value: add nix_get_attr_name_byidx get attr names without forcing 2024-03-28 10:39:07 +01:00
Yorick van Pelt 9cccb8bae0 nix_api_expr: always force values before giving them to the user 2024-03-28 10:39:07 +01:00
Yorick van Pelt c48b9b8a83 nix_api_util: tests 2024-03-28 10:39:07 +01:00
Yorick van Pelt f0afe7f9b9 nix_api_util: throw nix::error instead of new nix::Error for null ctx's 2024-03-28 10:39:06 +01:00
Yorick van Pelt e74d6c1b3d nix_api_expr: document nix_value_force 2024-03-28 10:39:06 +01:00
Yorick van Pelt e58a9384c6 nix_api_expr, nix_api_util: slightly improve documentation 2024-03-28 10:39:06 +01:00
Yorick van Pelt f41a7e326b nix_err_code: do not fail 2024-03-28 10:39:06 +01:00
Yorick van Pelt b0741f7128 external-api-doc: introduce and improve documentation 2024-03-28 10:39:06 +01:00
Yorick van Pelt 866558af34 nix_api_expr: add error handling to incref, decref 2024-03-28 10:39:06 +01:00
Yorick van Pelt ada2af4f88 nix_api_expr: add nix_gc_now() 2024-03-28 10:39:06 +01:00
Yorick van Pelt ded0ef6f6c nix_api_expr: switch to refcounting
Remove GCRef, keep references in a map. Change to nix_gc_incref and
nix_gc_decref, where users will mostly use nix_gc_decref.
2024-03-28 10:39:05 +01:00
Yorick van Pelt bebee700ea nix_api_external: own return strings on the nix side
Change from nix_returned_string that passes ownership, into a
nix_string_return parameter that can be set using nix_set_string_return.
2024-03-28 10:39:05 +01:00
Yorick van Pelt 022b918db1 nix_api_expr: remove bindingsbuilder refcounting 2024-03-28 10:39:05 +01:00
Yorick van Pelt aa85f7d917 nix_api_expr: merge nix_parse_expr and nix_expr_eval, remove Expr 2024-03-28 10:39:05 +01:00
Yorick van Pelt 1777e4a5bb nix_api_store: add userdata param to nix_store_build 2024-03-28 10:39:05 +01:00
Yorick van Pelt 1e583c4ebd nix_api_value: nix_{get,set}_double -> nix_{get,set}_float 2024-03-28 10:39:05 +01:00
Yorick van Pelt efcddcdd2f nix_api_external: fix missing void* self param 2024-03-28 10:39:05 +01:00
Yorick van Pelt c3b5b8eb62 nix_api_expr, store: fix minor documentation issues 2024-03-28 10:39:05 +01:00
Yorick van Pelt 4a4936136b nix_api_value: fix documentation for get_attr_byname 2024-03-28 10:39:04 +01:00
Puck Meerburg 748b322ddd nix_api_value: fix primop arity 2024-03-28 10:39:04 +01:00
Yorick van Pelt e76652a5d3 libexpr: add C bindings 2024-03-28 10:39:04 +01:00
Yorick van Pelt 1d41600498 libstore: add C bindings 2024-03-28 10:39:04 +01:00
Yorick van Pelt 4702317506 libutil: add C bindings 2024-03-28 10:39:04 +01:00
Yang, Bo c39afb28db Clarify stringLength is counting bytes 2024-03-27 16:46:50 -07:00
Robert Hensing c0b6907ccd doc/local.mk: Add manual-html-open phony target for auto-opening the browser (#10308) 2024-03-27 22:28:05 +01:00
vac (Brendan) 37f8edce99 docs: add cacert in macos multi-user upgrade (#10237) 2024-03-27 22:26:50 +01:00
Eelco Dolstra 1b71a4270a Merge pull request #10329 from obsidiansystems/mingw-devops
Build system changes for MinGW support
2024-03-27 22:17:11 +01:00
Tharun T 52359ca00a move test to correct file 2024-03-28 02:20:33 +05:30
Eelco Dolstra c0dd111af1 Fix flake evaluation in chroot stores
This is a temporary fix until we can pass `SourcePath`s rather than
`StorePath`s to `call-flake.nix`.

Fixes #10331.
2024-03-27 20:49:56 +01:00
John Ericson 77205b2042 Allow for ergnomically putting Unix-only files in subdirs by creating INLCUDE_$(pkg) vars
Separate platform-specific files will allow avoiding a lot of CPP.
2024-03-27 12:01:59 -04:00
John Ericson 4a61827d2d Hack to make sure the DLL linking job works with the check output 2024-03-27 12:01:59 -04:00
John Ericson ae2b2849c9 Disable GC on windows
We can build the dep and the our GC code is totally portable, but for
some reason we get link errors saying `GC_throw_bad_alloc` is missing.
2024-03-27 12:01:59 -04:00
annalee 7205a6bbc9 enable persistent WAL mode for sqlite db
allow processes without write access to the directory containing the db
to read the db when all connections are closed. Without this setting and
with WAL enabled and no open db connections unprivileged processes will
fail to open the db due the WAL files not existing and not able to
create them. When the WAL files are persistent unprivileged processeses
can read the db when there are no open connections.

Additionally, journal_size_limit is set to 2^40, which results in the
WAL files being truncated to 0 on exit, as well as limiting the WAL
files to 2^40 bytes following a checkpoint.

https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal
https://www.sqlite.org/pragma.html#pragma_journal_size_limit
https://github.com/sqlite/sqlite/blob/ed517a708284b6e00b6ae5f1e3f702bbfcbd32ed/src/wal.c#L2518

Fixes https://github.com/NixOS/nix/issues/10300
2024-03-27 15:36:09 +00:00
Robert HensingandRebecca Turner d4fa0a84a5 refact: TraceKind -> TracePrint
Co-authored-by: Rebecca Turner <rbt@sent.as>
2024-03-27 16:32:49 +01:00
Robert Hensing bed541b04e error.cc: Make printTrace static 2024-03-27 16:28:04 +01:00
Robert Hensing 981c309057 Remove trace item: while calling the 'addErrorContext' builtin 2024-03-27 16:28:04 +01:00
Robert Hensing bebacc475c Always print addErrorContext traces 2024-03-27 16:28:04 +01:00
Robert Hensing 189e40214e Merge pull request #10339 from edolstra/gha-vm-tests
Run some VM tests on GitHub
2024-03-27 16:13:41 +01:00
Cyclic4179 9f7b2b93ce fixup! add flakes.sh test for shebang supported types of comments 2024-03-27 15:03:54 +01:00
Cyclic4179 6227cd06bd add flakes.sh test for shebang supported types of comments 2024-03-27 14:55:02 +01:00
Eelco Dolstra 6f3972498b Run some VM tests on GitHub 2024-03-27 14:11:06 +01:00
Cyclic4179 a2c3333b97 fix #10336 2024-03-27 07:47:36 +01:00
Théophane Hufschmitt 6c10cc0eda Merge pull request #10312 from hercules-ci/add-build-dir
Add build-dir setting
2024-03-26 13:51:50 +01:00
Théophane Hufschmitt 0bf01917ef Merge pull request #10325 from K900/ooformatting
build-remote: fix format string shenanigans
2024-03-26 11:45:32 +01:00
Théophane Hufschmitt 1394d4e9c5 Merge pull request #10306 from hercules-ci/baseNameOf
Test and document `builtins.baseNameOf`, improve internal `baseNameOf()`
2024-03-26 09:59:27 +01:00
K900 2d4edb945b build-remote: fix format string shenanigans
HintFmt(string) invokes the HintFmt("%s", literal) constructor,
which is not what we want here. Add a constructor with a proper name
and call that.

Next step: rename all the other ones to HintFmt::literal(string).

Fixes https://github.com/NixOS/nix/issues/10238
2024-03-26 11:01:03 +03:00
Robert Hensing 8b16cced18 Add build-dir setting 2024-03-25 19:48:59 +01:00
Robert Hensing b1fe388d33 Remove uncalled for message 2024-03-25 19:48:57 +01:00
Robert Hensing 6221770c9d tests/functional: Add count() 2024-03-25 19:46:34 +01:00
Eelco Dolstra c82623a6cc Remove value clearing since it no longer has an effect 2024-03-25 19:21:25 +01:00
Eelco Dolstra 8c0590fa32 Never update values after setting the type
Thunks are now overwritten by a helper function
`Value::finishValue(newType, payload)` (where `payload` is the
original anonymous union inside `Value`). This helps to ensure we
never update a value elsewhere, since that would be incompatible with
parallel evaluation (i.e. after a value has transitioned from being a
thunk to being a non-thunk, it should be immutable).

There were two places where this happened: `Value::mkString()` and
`ExprAttrs::eval()`.

This PR also adds a bunch of accessor functions for value contents,
like `Value::integer()` to access the integer field in the union.
2024-03-25 19:21:25 +01:00
Théophane Hufschmitt 290072ca1e Merge pull request #10321 from Ericson2314/libstore-codeowner
Add @Ericson2314 as libstore codeowner
2024-03-25 18:27:20 +01:00
John Ericson bead1a1cde Add @Ericson2314 as libstore codeowner 2024-03-25 11:26:13 -04:00
Eelco Dolstra 6d90287f5a Merge pull request #10303 from hercules-ci/fix-empty-TMPDIR
fix: Treat empty TMPDIR as unset
2024-03-25 12:49:10 +01:00
Théophane Hufschmitt 45001c332d Remove accidental duplicate 2024-03-25 10:48:22 +01:00
Théophane Hufschmitt f78161bb8e Adress feedback from the PR
Trim down the proposal quite a bit, making it much closer to the previous text, just more explicit about what we support.
2024-03-25 10:47:19 +01:00
Tharun T 53c15336b0 derivation output selection test 2024-03-25 07:43:31 +05:30
Robert Hensing 641b0bd746 Merge pull request #10310 from obsidiansystems/fix-simple-git-hashing-test
Improve `git-hashing/simple.sh` test
2024-03-24 17:31:49 +01:00
John Ericson 513ba1977c Merge pull request #10307 from hercules-ci/doc-language-nix-dev
doc/language: Link to nix.dev introduction
2024-03-24 09:17:32 -04:00
John Ericson 2a44b11f55 Fix git-hashing/simple.sh
I realized it was checking NAR hashes before of added objects, which
makes little sense --- we don't really care about ancillary NAR hashes.

Now, the bottom `nix store add` tests compare the CA field with a git
hash to hashes calculated by Git. This matches top `nix hash path` ones
in using git as a source of truth.
2024-03-23 22:08:52 -04:00
Robert Hensing f891673460 Merge pull request #10293 from NixOS/fix-missing-format-arg
EvalCache: Fix missing format string argument
2024-03-24 02:28:05 +01:00
Robert Hensing fbf493758c doc/language: Link to nix.dev introduction
Closes #10283
2024-03-24 02:08:28 +01:00
Robert Hensing 9884018dfa baseNameOf(): Remove all trailing slashes 2024-03-24 01:38:22 +01:00
Robert Hensing 754a15e2db builtins.baseNameOf: Fork 2024-03-24 01:37:58 +01:00
Robert Hensing 175afc7106 Test and document builtins.baseNameOf 2024-03-24 01:26:17 +01:00
Robert Hensing 3b7f2bf997 git/dumpTree: Assert name not empty before back() 2024-03-24 00:58:08 +01:00
Robert Hensing 850c9a6caf HttpBinaryCacheStore: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing dd26f41379 local-derivation-goal.cc: Remove *all* trailing slashes 2024-03-24 00:52:14 +01:00
Robert Hensing fd31945742 local-derivation-goal.cc: Reuse defaultTempDir() 2024-03-24 00:45:15 +01:00
Robert Hensing b9e7f5aa2d fix: Treat empty XDG_RUNTIME_DIR as unset
See preceding commit. Not observed in the wild, but is sensible
and consistent with TMPDIR behavior.
2024-03-24 00:42:31 +01:00
Robert Hensing c3fb2aa1f9 fix: Treat empty TMPDIR as unset
Fixes an instance of

    nix: src/libutil/util.cc:139: nix::Path nix::canonPath(PathView, bool): Assertion `path != ""' failed.

... which I've been getting in one of my shells for some reason.
I have yet to find out why TMPDIR was empty, but it's no reason for
Nix to break.
2024-03-24 00:42:31 +01:00
detroyejr 50885b81c9 fix: correct remove-repl-flake.md 2024-03-23 06:31:49 -04:00
Théophane Hufschmitt 38ba96d7b0 Merge pull request #10296 from picnoir/pic/doc-lsp
Doc: explicitly specify which shell to use to get clangd
2024-03-23 08:46:13 +01:00
detroyejr b11dd58fe4 flakes: test to ensure we get an error if --file isn't used 2024-03-22 21:58:39 -04:00
detroyejr c625b45357 flakes: add remove-repl-flake.md note 2024-03-22 21:39:48 -04:00
detroyejr 717391731c flakes: remove experimental repl-flake 2024-03-22 19:30:03 -04:00
Picnoir cc29d85d06 Doc: explicitly specify which shell to use to get clangd
I was using by mistake the .#nix-clangStdenv shell to retrieve clangd.
This clangd is unusable with the project and constantly segfaults.
Let's explicitly state which shell the user should use in the docs.

I don't really understand the source of this segfault. I assume it's
related to a clang version incompatibility. (16.0.6 for
.#nix-clangStdenv 14.0.6 for .#native-clangStdenvPackages)
2024-03-22 22:36:55 +01:00
Théophane Hufschmitt 4ec2a7d3bf Merge pull request #10292 from dottharun/docBuildOutput
Docs: Documentation manual build output path correction
2024-03-22 21:45:20 +01:00
Eelco Dolstra bfd36402ac EvalCache: Fix missing format string argument
Fixes

  terminate called after throwing an instance of 'boost::wrapexcept<boost::io::too_few_args>'
    what():  boost::too_few_args: format-string referred to more arguments than were passed
  Aborted (core dumped)

for type errors in AttrCursor.
2024-03-22 18:11:24 +01:00
Tharun T 4e2f11b692 doc build output correction
Signed-off-by: Tharun T <tharun242424@gmail.com>
2024-03-22 18:34:55 +05:30
Théophane Hufschmitt c13a31f639 Update the release cycle
- Align the “frequent” release cycle with the calendar
  - The 6-month release cycle is hard to keep track of. A monthly
    release will make it much easier to remember the release date.
- Officialise the support for a stable version maintained for as long as NixOS stable
  - This is already the case in practice, it just happens that the
    “stable” Nixpkgs version is whichever version was deemed
    stable-enough at the time of the NixOS release.
    Officialise that by cutting a new major release alongside each NixOS one.

Note that this breaks whatever semver compatibility Nix might pretend to
have, but I don't think it makes sense any way.
2024-03-22 11:40:29 +01:00
Robert Hensing a223280664 Merge pull request #10286 from hercules-ci/various
Allocate a bit less
2024-03-22 11:19:58 +01:00
Théophane Hufschmitt 57d9d0d6e4 Merge pull request #10269 from andreabedini/make-compile-commands
Create compile-commands.json with Make
2024-03-21 17:22:35 +01:00
Théophane Hufschmitt d0824f661e Document the new compile_commands.json target 2024-03-21 16:54:28 +01:00
Théophane Hufschmitt f666eda94d Merge pull request #10274 from tweag/warn-on-untrusted-setting
Daemon: warn when an untrusted user cannot override a setting
2024-03-21 16:40:24 +01:00
Théophane Hufschmitt 039b3fbdfe Merge pull request #10271 from jonringer/document-flake-multiple-outputs
Document how to build many outputs of a flake package
2024-03-21 15:58:32 +01:00
Andrea Bedini 83fc988bec Create compile-commands.json with Make 2024-03-21 15:55:44 +08:00
Robert Hensing 1fcdd1640e functionArgs: Allocate bools only once 2024-03-20 23:25:28 +01:00
Robert Hensing 8c6e0df45f value.hh: Fix warning about {struct/class} Value 2024-03-20 23:25:28 +01:00
Robert Hensing a865049c4f tryEval: Allocate true and false once 2024-03-20 23:25:28 +01:00
Robert Hensing d71e74838a readDir: Allocate type strings only once 2024-03-20 23:25:28 +01:00
Jonathan RingerandThéophane Hufschmitt 4c8a33ce46 Update src/nix/build.md
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-20 14:42:44 -07:00
Robert Hensing 7d2ead50e5 Merge pull request #10083 from lf-/jade/refactor-repl-input
refactor: move the repl input code to its own file
2024-03-20 22:37:40 +01:00
Robert Hensing d16d7f5f31 Merge pull request #10251 from edolstra/list-builder
Add a ListBuilder helper for constructing list values
2024-03-20 22:30:38 +01:00
Robert Hensing d4b0b7f152 createBaseEnv: Use state.vNull 2024-03-20 21:34:23 +01:00
Robert Hensing 0b08dd45b0 prim_match: Use state.vNull 2024-03-20 21:28:38 +01:00
Félix Baylac-Jacqué 40a7929c8e Daemon: warn when an untrusted user cannot override a setting
In a daemon-based Nix setup, some options cannot be overridden by a
client unless the client's user is considered trusted.

Currently, if an untrusted user tries to override one of those
options, we are silently ignoring it.

This can be pretty confusing in certain situations.

e.g. a user thinks he disabled the sandbox when in reality he did not.

We are now sending a warning message letting know the user some options
have been ignored.

Related to #1761.

This is a cherry-pick of 9e0f5f803f.
The above commit has been reverted by
a59e77d9e5 to prevent spamming warnings
with experimental features, but these are now totally ignored on the
daemon side, so there's no reason for the revert any more.
2024-03-20 17:55:35 +01:00
Théophane Hufschmitt 346f92aa4c Merge pull request #10241 from bouk/resolve-ref-packed-refs
git fetcher: use resolveRef for getting revision of reference
2024-03-20 17:37:15 +01:00
Théophane Hufschmitt 4cb0847743 Merge pull request #10273 from jesses-canva/patch-1
Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in
2024-03-20 12:24:17 +01:00
Jesse Schalken c448636f7c Fix loop over $NIX_PROFILES in nix-profile-daemon.fish.in 2024-03-20 15:23:31 +11:00
Jonathan Ringer a5262fb880 Document how to build many outputs of a flake package 2024-03-19 17:37:04 -07:00
Théophane Hufschmitt 7c506432ab Merge pull request #10264 from lelgenio/document-pure-eval-store-path
Document builtins.storePath being disabled by pure-eval
2024-03-19 09:59:19 +01:00
John Ericson 18945e3f44 Merge branch 'master' into overlayfs-store 2024-03-18 16:43:33 -04:00
John EricsonandThéophane Hufschmitt cd35e00103 Adding missing tracking URL for local overlay store
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-18 16:41:16 -04:00
lelgenio ad217ddbbc Document builtins.storePath being disabled by pure-eval 2024-03-18 16:23:17 -03:00
Théophane Hufschmitt 5c8983b9e6 Merge pull request #10259 from ShamrockLee/doc-primops-fix
doc: builtins.addDrvOutputDependencies: fix link target
2024-03-17 21:10:52 +01:00
Yueh-Shun Li d2b512959c builtins.addDrvOutputDependencies: fix commentary 2024-03-18 02:38:31 +08:00
Yueh-Shun Li 39b0b8452f doc: builtins.addDrvOutputDependencies: fix link target 2024-03-17 06:13:14 +08:00
Eelco Dolstra 3e6730ee62 Mark Value pointers in Value::elems as const
This catches modification of finalized values (e.g. in prim_sort).
2024-03-15 18:26:37 +01:00
Eelco Dolstra fecff520d7 Add a ListBuilder helper for constructing list values
Previously, `state.mkList()` would set the type of the value to tList
and allocate the list vector, but it would not initialize the values
in the list. This has two problems:

* If an exception occurs, the list is left in an undefined state.

* More importantly, for multithreaded evaluation, if a value
  transitions from thunk to non-thunk, it should be final (i.e. other
  threads should be able to access the value safely).

To address this, there now is a `ListBuilder` class (analogous to
`BindingsBuilder`) to build the list vector prior to the call to
`Value::mkList()`. Typical usage:

   auto list = state.buildList(size);
   for (auto & v : list)
       v = ... set value ...;
   vRes.mkList(list);
2024-03-15 18:26:37 +01:00
Eelco Dolstra 0fb5024d8d Merge pull request #10243 from DimitarNestorov/patch-1
docs: update registry examples
2024-03-15 18:13:12 +01:00
Eelco Dolstra 650161b7ae Merge pull request #10242 from fricklerhandwerk/fix-index-redirects
adjust anchor redirects to point to new pages
2024-03-15 18:12:50 +01:00
Daniel Sidhion 49e9efeaaa doc: document SRI hash format for outputHash (#10230) 2024-03-15 07:09:47 +01:00
Dimitar Nestorov a50295425e docs: update registry examples
When you run `nix flake info` you get a deprecated message
2024-03-14 16:15:52 +02:00
Valentin Gagarin 3754614b9c adjust anchor redirects to point to new pages
a previous moving of files accounted for server-side redirects, but not
client-side redirects.
2024-03-14 15:01:47 +01:00
Bouke van der Bijl 60c2d15f5a git fetcher: use resolveRef for getting revision of reference
* Add regression test
* Fix 'no repo' test so it doesn't succeed if the data is still in cache
* Use git_revparse_single inside git-utils instead of reimplementing the same logic.
2024-03-14 14:04:51 +01:00
Robert Hensing c152c2767a Merge pull request #10239 from correabuscar/patch-1
typo consant->constant in context.cc
2024-03-13 22:45:36 +01:00
Emanuel Czirai e5840d5760 typo consant->constant in context.cc 2024-03-13 20:35:24 +01:00
Jade Lovelace ea31b8a117 refactor: repl prompts are now the job of the interacter 2024-03-12 18:25:35 -07:00
Jade Lovelace 70a6ce139b refactor: move readline stuff into its own file
This is in direct preparation for an automation mode of nix repl.
2024-03-12 18:25:35 -07:00
Jade Lovelace 76aced6915 finally.hh: delete copy constructor which is a bad idea 2024-03-12 18:25:35 -07:00
Eelco Dolstra bff5c94184 Merge pull request #10205 from 9999years/final-matchers
Make `Matcher` subclasses `final`
2024-03-12 07:23:15 +01:00
Eelco Dolstra 18b89ca58c Merge pull request #10227 from tmplt/doc/nix-copy
nix-copy: document --all --from local binary cache example
2024-03-12 07:22:28 +01:00
Eelco Dolstra e3cc701d86 Merge pull request #10231 from NixOS/dependabot/github_actions/cachix/install-nix-action-26
Bump cachix/install-nix-action from 25 to 26
2024-03-12 07:16:25 +01:00
dependabot[bot] aa121dc318 Bump cachix/install-nix-action from 25 to 26
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 25 to 26.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v25...v26)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-11 22:02:01 +00:00
Eelco Dolstra 90e86bf6d3 Merge pull request #10229 from NixOS/bump-2.22.0
Bump version
2024-03-11 21:51:52 +01:00
Eelco Dolstra 222c38370f Bump version 2024-03-11 21:16:10 +01:00
Théophane Hufschmitt 18b2ef8b20 Merge pull request #10209 from 9999years/rethrow-for-top-level-errors-in-repl
Print top-level errors normally in `nix repl`
2024-03-11 21:11:28 +01:00
Viktor Sonesten db36c9ca90 nix-copy: document --all --from local binary cache example 2024-03-11 19:17:45 +01:00
tomberek 25bf671015 Merge pull request #10200 from 9999years/fix-nix-eval-for-derivations
Fix `nix eval nixpkgs#bash` segfault
2024-03-11 13:56:49 -04:00
Eelco Dolstra 72e145e415 Merge pull request #10226 from edolstra/release-notes
Release notes
2024-03-11 18:32:51 +01:00
Rebecca Turner 7f45b1c8d8 Add release note 2024-03-11 10:03:01 -07:00
Eelco Dolstra 4c97a66b4c Tweak release notes 2024-03-11 18:00:10 +01:00
Eelco Dolstra b12dc76cfc release notes: 2.21.0 2024-03-11 16:22:29 +01:00
Eelco Dolstra 0ce58cd1c2 Merge pull request #10224 from tweag/fix-darwin-instaltests
Run preInstallCheck even when not building
2024-03-11 16:02:31 +01:00
Théophane Hufschmitt 74008d8215 Run preInstallCheck even when not building
Add `runHook preInstallCheck` to the overriden `installCheckPhase` used
for the non-build case.

In particular, this allow the fix from 2a34510776
to also apply there.
2024-03-11 15:34:23 +01:00
Robert Hensing 3539172fd2 Merge pull request #10208 from 9999years/print-strings-directly
`:print` strings directly in `nix repl`
2024-03-11 10:02:09 +01:00
Théophane Hufschmitt 83460fb64f Merge pull request #10216 from edolstra/fix-github-test
Fix GitHub test
2024-03-11 09:50:33 +01:00
Rebecca Turner db9bab2708 Matcher: Add virtual destructor 2024-03-10 12:56:07 -07:00
Eelco Dolstra 841fd78baa GitArchiveInputScheme: Support the narHash attribute
This is required to produce a locked flakeref.
2024-03-10 14:08:55 +01:00
Eelco Dolstra c1811c1eba Fix GitHub test
Cherry-picked from 03618bb85f.
2024-03-10 13:38:38 +01:00
Eelco Dolstra e65e9114d2 Merge pull request #10204 from 9999years/replace-foo-in-big-cxx-literal
Replace `foo` with `__NIX_STR` in `cxx-big-literal`
2024-03-10 13:11:36 +01:00
Rebecca Turner d13c63afa2 Print top-level errors normally in nix repl
Previously, errors while printing values in `nix repl` would be printed
in `«error: ...»` brackets rather than displayed normally:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
«error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.»
```

Now, errors will be displayed normally if they're emitted at the
top-level of an expression:

```
nix-repl> legacyPackages.aarch64-darwin.pythonPackages.APScheduler
error:
       … in the condition of the assert statement
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/lib/customisation.nix:268:17:
          267|     in commonAttrs // {
          268|       drvPath = assert condition; drv.drvPath;
             |                 ^
          269|       outPath = assert condition; drv.outPath;

       … in the left operand of the OR (||) operator
         at /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/passthrufun.nix:28:45:
           27|         if lib.isDerivation value then
           28|           lib.extendDerivation (valid value || throw "${name} should use `buildPythonPackage` or `toPythonModule` if it is to be part of the Python packages set.") {} value
             |                                             ^
           29|         else

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘python-2.7.18.7’ in /nix/store/6s0m1qc31zw3l3kq0q4wd5cp3lqpkq0q-source/pkgs/development/interpreters/python/cpython/2.7/default.nix:335 is marked as insecure, refusing to evaluate.
```

Errors emitted in nested structures (like e.g. when printing `nixpkgs`)
will still be printed in brackets.
2024-03-09 18:28:04 -08:00
Rebecca Turner d859d6c434 :print strings directly in nix repl
Strings are now printed directly when evaluated by `:print`, rather than
escaped. This makes it easier to debug multi-line strings or strings
containing quotes, like the results of `builtins.readFile`,
`lib.toShellArg`, and so on.

```
nix-repl> "cuppy\ndog\ncity"
"cuppy\ndog\ncity"

nix-repl> :p "cuppy\ndog\ncity"
cuppy
dog
city
```
2024-03-09 18:13:08 -08:00
Rebecca Turner 70e93c1e2b Make Matcher subclasses final
Fixes this very long warning, which I'll only include the first line of:

/nix/store/8wrjhrycpshhc3b41xmjwvgqr2m3yajq-libcxx-16.0.6-dev/include/c++/v1/__memory/construct_at.h:66:5: warning: destructor called on non-final 'RegexMatcher' that has virtual functions but non-virtual destructor [-Wdelete-non-abstract-non-virtual-dtor]
    __loc->~_Tp();
2024-03-09 17:07:52 -08:00
Rebecca Turner ea8faf8e9a Replace foo with __NIX_STR in cxx-big-literal
Looks a little nicer when you check the generated sources.
2024-03-09 17:05:56 -08:00
Valentin Gagarin ac730622e8 document where the value of builtins.nixPath comes from (#9113)
* document default values for `nix-path`

also note how it's overridden and note the effect of `restrict-eval`
2024-03-09 17:57:57 +00:00
Rebecca Turner 4910d74086 Print derivation paths in nix eval
`nix eval` forces values and prints derivations as attribute sets, so
commands that print derivations (e.g. `nix eval nixpkgs#bash`) will
infinitely loop and segfault.

Printing derivations as `.drv` paths makes `nix eval` complete as
expected. Further work is needed, but this is better than a segfault.
2024-03-08 21:43:54 -08:00
tomberek 3120fef01b Merge pull request #10184 from iFreilicht/patch-2
docs: Explain how to add release notes
2024-03-08 19:49:44 -05:00
Felix Uhl 35f2b07668 docs: Fix link to release note documentation 2024-03-08 20:04:55 +01:00
tomberek a200ee6bf7 Merge pull request #9874 from pennae/error-reports
improve error reports somewhat
2024-03-08 10:51:14 -05:00
Eelco Dolstra ae2bd460a8 Merge pull request #10166 from bobvanderlinden/profile-regex-all
profile: introduce --regex and --all
2024-03-08 15:59:24 +01:00
Jonathan Dickinson 1ffcbddf62 docs: add inherit to language overview (#10194)
* docs: add inherit to language overview

Adds a short summary about `inherit` to the language overview.
2024-03-08 14:24:44 +00:00
Théophane Hufschmitt ef5cb44608 Merge pull request #10197 from edolstra/github-treehash
GitHub fetcher: Don't emit treeHash yet
2024-03-08 14:14:51 +01:00
Eelco Dolstra 6d245182e8 GitHub fetcher: Don't emit treeHash yet
But do accept it if it's there, so we don't choke on future lock files
that do have the treeHash attribute.
2024-03-08 12:40:14 +01:00
Bob van der Linden 3d628d1704 profile: convert Matcher to abstract class 2024-03-08 10:47:51 +01:00
Eelco Dolstra 61b006ddf6 Merge pull request #10174 from tweag/disable-failing-hydra-jobs
flake: Remove the cross-compilation to freebsd13
2024-03-08 09:46:38 +01:00
Eelco Dolstra 4fda08d867 Merge pull request #10195 from tweag/fix-nix-env-qap
Update the golden hash of the evalNixpkgs test
2024-03-08 09:44:16 +01:00
Eelco Dolstra a7b3249d68 Merge pull request #10179 from edolstra/upload-release
maintainers/upload-release.pl: Handle 2.3 and 2.18 branches
2024-03-08 09:38:15 +01:00
Eelco Dolstra cb7ee1af89 Merge pull request #10175 from tweag/disable-i686-perl-bindings
flake: Disable the perl bindings on i686-linux
2024-03-08 09:21:00 +01:00
Théophane Hufschmitt ff74c081e9 flake: Remove the cross-compilation to freebsd13
`libc` is broken there: https://hydra.nixos.org/build/252347598.

We can reintroduce it once the base system is working
2024-03-08 08:58:37 +01:00
Théophane Hufschmitt 520a1df208 flake: Disable the perl bindings on i686-linux
Some perl dependencies are failing: https://hydra.nixos.org/build/252347639/nixlog/1

Since the support is only best-effort there, disable the perl bindings
2024-03-08 08:58:10 +01:00
Théophane Hufschmitt 201369dceb tests.evalNixpkgs: Update the golden hash
`nix-env -qaP`'s output has changed a bit because of https://github.com/NixOS/nix/issues/10132.

Although that's a bit annoying, it isn't nearly as problematic as the
evaluation changes that this test is supposed to catch. So it's find to
just update the hash for the time being and fix the issue later
(properly fixing the issue will very likely change the hash any way).
2024-03-08 08:50:27 +01:00
Théophane Hufschmitt 4354b37fc4 Add more logs to the evalNixpkgs test
Make it possible to understand a mimimum what's going on in case of a
failure
2024-03-08 08:48:53 +01:00
Bob van der Linden 4741d3e308 add release note 2024-03-07 21:07:03 +01:00
Bob van der Linden 91f068c193 profile: make --all exclusive 2024-03-07 21:07:03 +01:00
Bob van der Linden 7a4d5e89d3 profile: add --all option to match any package 2024-03-07 21:07:03 +01:00
Bob van der Linden fb391ebc77 profile: add tests for not matching any packages 2024-03-07 21:07:03 +01:00
Bob van der Linden 9fac62435c tests/functional: add assertStderr function
Currently there isn't a convenient way to check for multiline output. In
addition, these outputs will easily change and having a diff between the
expected an the actual output upon failures is convenient.
2024-03-07 21:07:03 +01:00
Bob van der Linden 87741dbd21 profile: add --regex option to match packages 2024-03-07 21:07:03 +01:00
Bob van der Linden d6f5da51d3 profile: match on package name instead of regex 2024-03-07 21:07:03 +01:00
Bob van der Linden 741a6bfad5 profile: allow different types of matchers 2024-03-07 21:07:03 +01:00
Eelco Dolstra b3b312827c Merge pull request #10180 from 9999years/rebeccat/flake-check-log
Restore "checking Hydra job" message in `nix flake check`
2024-03-07 17:48:37 +01:00
Rebecca Turner 4b4c71e239 Restore "checking Hydra job" message in nix flake check
Mistakenly removed in #8893, thanks @lf- for catching this!

https://github.com/NixOS/nix/commit/9404ce36e4edd1df12892089bdab1ceb7d4d7a97#r139485316
2024-03-07 08:15:53 -08:00
Eelco Dolstra 091f232896 maintainers/upload-release.pl: Handle 2.3 and 2.18 branches 2024-03-07 16:53:34 +01:00
Eelco Dolstra 751cb7cf51 Merge pull request #10178 from tweag/copy-built-outputs
Fix the outputs moving on macOS
2024-03-07 15:40:54 +01:00
Théophane Hufschmitt a3163b9eab Fix the outputs moving on macOS 2024-03-07 14:52:40 +01:00
Eelco Dolstra fb37e184a5 Merge pull request #10176 from NixOS/relative-path-fingerprint
PathInputScheme::getFingerprint(): Don't barf on relative paths
2024-03-07 14:23:57 +01:00
Eelco Dolstra 0282499e18 PathInputScheme::getFingerprint(): Don't barf on relative paths
This wasn't caught by CI because #10149 and #10152 pass
individually... It doesn't happen on lazy-trees either because we
never try to fetch relative path flakes (#10089).
2024-03-07 13:28:52 +01:00
Eelco Dolstra da62528487 Merge pull request from GHSA-2ffj-w4mj-pg37
Copy built outputs
2024-03-07 11:56:24 +01:00
Eelco Dolstra 454456209f Merge pull request #10165 from link2xt/patch-1
Make search.nixos.org link in quick start clickable
2024-03-07 11:05:48 +01:00
Eelco Dolstra 7d76bc8a39 Merge pull request #10152 from edolstra/fetcher-cleanups
Fetcher cleanups
2024-03-07 10:21:11 +01:00
Eelco Dolstra bd828b1937 Merge pull request #10150 from edolstra/always-upgrade-unlocked-flakerefs
nix profile upgrade: Always upgrade unlocked flakerefs
2024-03-07 10:20:37 +01:00
Eelco Dolstra 38b79da100 Merge pull request #10149 from edolstra/use-flake-fingerprint
LockedFlake::getFingerprint(): Use Input::getFingerprint()
2024-03-07 10:18:19 +01:00
Théophane Hufschmitt f175b3a4b7 Revert "install-multi-user.sh: _sudo: add proxy variables to sudo"
This reverts commit 24fd7e2755.
2024-03-07 10:06:33 +01:00
Théophane Hufschmitt 739f53aca4 Revert "Fix sudo in the darwin installer (#10128)"
This reverts commit 686405ef41.
2024-03-07 10:06:33 +01:00
Robert Hensing 73cf23275e Merge pull request #10107 from tweag/repl-file-argument
Properly fail on flakerefs that don't point to a directory
2024-03-07 10:04:35 +01:00
tomberek 0a11080c58 Merge pull request #10172 from lf-/jade/fix-9725
fix: bounds check result in getMaxCPU
2024-03-07 00:58:44 -05:00
Théophane Hufschmitt c4cfa18a33 Merge pull request #10164 from hercules-ci/objc-workaround
package.nix: Apply OBJC_DISABLE_INITIALIZE_FORK_SAFETY workaround
2024-03-07 06:26:15 +01:00
Jade Lovelace 9c64a09c70 fix: bounds check result in getMaxCPU
Fixes https://github.com/NixOS/nix/issues/9725
2024-03-06 20:57:31 -08:00
Robert Hensing 16fa5aa305 Merge pull request #10168 from hercules-ci/doc-glossary-file-system-object-anchor
doc/glossary: Fix file system object anchor
2024-03-07 00:40:36 +01:00
Robert Hensing e4500e539e doc/glossary: Fix file system object anchor
It was stealing the store object id. Browsers pick the first one.
It was confusing.
2024-03-07 00:04:30 +01:00
pennae 5d9fdab3de use byte indexed locations for PosIdx
we now keep not a table of all positions, but a table of all origins and
their sizes. position indices are now direct pointers into the virtual
concatenation of all parsed contents. this slightly reduces memory usage
and time spent in the parser, at the cost of not being able to report
positions if the total input size exceeds 4GiB. this limit is not unique
to nix though, rustc and clang also limit their input to 4GiB (although
at least clang refuses to process inputs that are larger, we will not).

this new 4GiB limit probably will not cause any problems for quite a
while, all of nixpkgs together is less than 100MiB in size and already
needs over 700MiB of memory and multiple seconds just to parse. 4GiB
worth of input will easily take multiple minutes and over 30GiB of
memory without even evaluating anything. if problems *do* arise we can
probably recover the old table-based system by adding some tracking to
Pos::Origin (or increasing the size of PosIdx outright), but for time
being this looks like more complexity than it's worth.

since we now need to read the entire input again to determine the
line/column of a position we'll make unsafeGetAttrPos slightly lazy:
mostly the set it returns is only used to determine the file of origin
of an attribute, not its exact location. the thunks do not add
measurable runtime overhead.

notably this change is necessary to allow changing the parser since
apparently nothing supports nix's very idiosyncratic line ending choice
of "anything goes", making it very hard to calculate line/column
positions in the parser (while byte offsets are very easy).
2024-03-06 23:48:42 +01:00
pennae 855fd5a1bb diagnose "unexpected EOF" at EOF
this needs a string comparison because there seems to be no other way to
get that information out of bison. usually the location info is going to
be correct (pointing at a bad token), but since EOF isn't a token as
such it'll be wrong in that this case.

this hasn't shown up much so far because a single line ending *is* a
token, so any file formatted in the usual manner (ie, ending in a line
ending) would have its EOF position reported correctly.
2024-03-06 23:11:12 +01:00
pennae 2be6b14328 match line endings used by parser and error reports
the parser treats a plain \r as a newline, error reports do not. this
can lead to interesting divergences if anything makes use of this
feature, with error reports pointing to wrong locations in the input (or
even outside the input altogether).
2024-03-06 23:11:12 +01:00
pennae 1edd6fada5 report inherit attr errors at the duplicate name
previously we reported the error at the beginning of the binding
block (for plain inherits) or the beginning of the attr list (for
inherit-from), effectively hiding where exactly the error happened.

this also carries over to runtime positions of attributes in sets as
reported by unsafeGetAttrPos. we're not worried about this changing
observable eval behavior because it *is* marked unsafe, and the new
behavior is much more useful.
2024-03-06 23:11:12 +01:00
pennae 4147ecfb1c normalize formal order on ExprLambda::show
we already normalize attr order to lexicographic, doing the same for
formals makes sense. doubly so because the order of formals would
otherwise depend on the context of the expression, which is not quite as
useful as one might expect.
2024-03-06 23:11:12 +01:00
pennae d384ecd553 keep copies of parser inputs that are in-memory only
the parser modifies its inputs, which means that sharing them between
the error context reporting system and the parser itself can confuse the
reporting system. usually this led to early truncation of error context
reports which, while not dangerous, can be quite confusing.
2024-03-06 23:11:12 +01:00
link2xt fe13d4a6e0 Make search.nixos.org link in quick start clickable 2024-03-06 21:55:02 +00:00
Théophane Hufschmitt 686405ef41 Fix sudo in the darwin installer (#10128) 2024-03-06 22:36:37 +01:00
Robert Hensing 2a34510776 package.nix: Apply OBJC_DISABLE_INITIALIZE_FORK_SAFETY workaround
This was previously already used in the launchd configuration for
nix-daemon.

(cherry picked from commit 855741aea5)
2024-03-06 22:07:05 +01:00
Théophane Hufschmitt fe42a0ead7 Documentation typo 2024-03-06 09:10:32 +01:00
Théophane Hufschmitt 67f95755d6 Merge pull request #10155 from NixOS/warn-against-changing-profile-version-number
Add a warning against carelessly changing the profile version number
2024-03-06 06:43:36 +01:00
Théophane Hufschmitt 0d26358bda Merge pull request #10151 from edolstra/fix-lstat-cache
PosixSourceAccessor::cachedLstat(): Use absolute path
2024-03-05 08:59:30 +01:00
Théophane Hufschmitt 7161ef14a2 Add a warning against carelessly changing the profile version number
Try and prevent the situation of https://github.com/NixOS/nix/issues/10109 to happen again in the future
2024-03-05 08:07:58 +01:00
Théophane Hufschmitt e164b39ee9 Merge pull request #9920 from 9999years/forbid-nested-debuggers
Forbid nested debuggers
2024-03-05 06:58:29 +01:00
Eelco Dolstra 0e07f81d2b Fetcher cleanups
* Convert all InputScheme::fetch() methods to getAccessor().

* Add checkLocks() method for checking lock attributes.

* Rename fetch() to fetchToStore().
2024-03-04 22:24:55 +01:00
Eelco Dolstra 4967c5ff6b Fix macOS build 2024-03-04 22:24:12 +01:00
Eelco Dolstra 9ee590e113 PosixSourceAccessor::cachedLstat(): Use absolute path
Using the relative path can cause collisions between cache entries for
PosixSourceAccessors with different roots.
2024-03-04 21:54:35 +01:00
Eelco Dolstra 6b976a1898 Merge pull request #10143 from fricklerhandwerk/test-help
add tests for showing help
2024-03-04 20:29:03 +01:00
Eelco Dolstra 32bf39c73a nix flake metadata: Don't show locked URL if it's not locked
This is the case for e.g. dirty Git workdirs, where we would get

  $ nix flake metadata
  Resolved URL:  git+file:///home/eelco/Dev/nix-master
  Locked URL:    git+file:///home/eelco/Dev/nix-master
2024-03-04 19:37:43 +01:00
Eelco Dolstra 2306e96767 nix profile upgrade: Always upgrade unlocked flakerefs
The "lockedRef" field is a misnomer, since it can be unlocked
(e.g. for a dirty Git workdir). In that case, `nix profile upgrade`
needs to assume that the package can have changed, and perform an
upgrade.
2024-03-04 19:30:38 +01:00
Valentin GagarinandThéophane Hufschmitt 8d23847571 fix indentation
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:44 +01:00
Valentin GagarinandThéophane Hufschmitt 8a6ef3bae5 less scary jq
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-03-04 19:25:28 +01:00
Eelco Dolstra 6558da45f5 LockedFlake::getFingerprint(): Use Input::getFingerprint() 2024-03-04 19:22:23 +01:00
Eelco Dolstra 29049d2653 Implement getFingerprint() for store paths 2024-03-04 19:21:31 +01:00
Rebecca Turner 2e8f4faa10 Fix build
Not sure why that was giving a duplicate symbol error, or why marking it
inline fixes it. Here it is!
2024-03-04 09:32:06 -08:00
Rebecca Turner 14b0356dc5 Forbid nested debuggers 2024-03-04 09:24:57 -08:00
Rebecca Turner 4ee5433919 Add release note 2024-03-04 09:24:56 -08:00
Valentin Gagarin 4b15ca2ffb add tests for showing help 2024-03-04 16:14:00 +01:00
Théophane Hufschmitt eaa6c26189 Merge pull request #10088 from edolstra/flake-source-path
Use SourcePath for reading flake.{nix,lock}
2024-03-04 14:40:40 +01:00
Théophane Hufschmitt 7764edf0e4 Merge pull request #10078 from szlend/fix-macos-local-network-sandbox
Fix `__darwinAllowLocalNetworking` sandbox
2024-03-04 14:34:57 +01:00
Eelco Dolstra 438855952b Merge pull request #10122 from edolstra/arg-from-file
Add --arg-from-file and --arg-from-stdin
2024-03-04 13:29:58 +01:00
Eelco Dolstra cbfd211b39 Fix build 2024-03-04 12:49:32 +01:00
Eelco Dolstra e6b9432542 Add release note 2024-03-04 12:48:21 +01:00
Théophane Hufschmitt 8a9eec375c Merge pull request #10073 from olmokramer/nix-flake-update-multiple-inputs
Accept multiple inputs in `nix flake update`
2024-03-04 09:13:26 +01:00
Eelco Dolstra 75c5e1a5cc Merge pull request #10126 from lf-/jade/kill-ndebug
Ban building Nix with NDEBUG
2024-03-03 20:01:12 +01:00
Eelco Dolstra fa17ec6b3e Merge pull request #10130 from tweag/remove-autoconf-gen-files
Remove and gitignore the autoreconf generated files
2024-03-03 19:58:22 +01:00
Eelco Dolstra dfb05f3957 Merge pull request #10129 from NixOS/fix-docker-push
Fix the docker push job
2024-03-03 19:57:23 +01:00
Olmo Kramer b1ad729add Add test for nix flake update with multiple inputs 2024-03-03 13:59:39 +01:00
Jade Lovelace 2625e9fb0a Ban building Nix with NDEBUG
When reviewing old PRs, I found that #9997 adds some code to ensure one
particular assert is always present. But, removing asserts isn't
something we do in our own release builds either in the flake here or in
nixpkgs, and is plainly a bad idea that increases support burden,
especially if other distros make bad choices of build flags in their Nix
packaging.

For context, the assert macro in the C standard is defined to do nothing
if NDEBUG is set.

There is no way in our build system to set -DNDEBUG without manually
adding it to CFLAGS, so this is simply a configuration we do not use.
Let's ban it at compile time.

I put this preprocessor directive in src/libutil.cc because it is not
obvious where else to put it, and it seems like the most logical file
since you are not getting a usable nix without it.
2024-03-02 11:22:46 -08:00
Ivan Shapovalov c6f0407103 libutil/terminal: cache isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 8c1eeb4681 treewide: shouldANSI() -> isTTY() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov d9fc4bf5c5 treewide: replace usages of isatty(STDERR_FILENO) with shouldANSI() 2024-03-02 16:32:04 +01:00
Ivan Shapovalov 950b6401f9 libmain/progress-bar: try harder to avoid escape sequences if !isTTY 2024-03-02 16:31:51 +01:00
Théophane Hufschmitt 2f0bc6373c Don't fail if a flakeref directly points to the flake.nix
Just warn and redirect it to the parent directory
2024-03-02 10:34:20 +01:00
Théophane Hufschmitt 11a1dcc43b Properly fail on flakerefs that don't point to a directory
Directly fail if a flakeref points to something that isn't a directory
instead of falling back to the logic of trying to look up the hierarchy
to find a valid flake root.

Fix https://github.com/NixOS/nix/issues/9868
2024-03-02 10:24:20 +01:00
Théophane Hufschmitt f8dc9bc563 Remove and gitignore the autoreconf generated files
No need to have them checked-in since we require running `autoreconf`
when building, and these are regenerated by the `autoreconf` script.
2024-03-02 10:18:47 +01:00
Théophane Hufschmitt 089d91ed4c Fix the docker push job
After https://github.com/NixOS/nix/pull/10071, the CI was trying to push
ghcr.io/nixos/nix:master for backwards-compatibility, but the image was
not tagged as such, causing the job to fail.

Fix this.
2024-03-02 09:00:42 +01:00
Théophane Hufschmitt 75101396dc Merge pull request #10117 from med8bra/doc-xp-feature-issues
doc(xp-feature): add issue url in experimental feature details
2024-03-02 08:59:12 +01:00
med8bra 1bc89b588b doc(xp-feature): add issue url in experimental feature details 2024-03-02 03:51:12 +01:00
Eelco Dolstra ba9b6b29b7 Merge pull request #10111 from obsidiansystems/git-objects
Support symlinks properly with `git-hashing` experimental feature
2024-03-01 15:00:43 +01:00
Eelco Dolstra 8ce1f6800b Add --arg-from-stdin to read an argument from stdin 2024-03-01 14:39:42 +01:00
Eelco Dolstra 291b10c607 Add --arg-from-file for reading a string from a file 2024-03-01 14:35:27 +01:00
Eelco Dolstra d72ee91d07 Clean up --arg processing 2024-03-01 14:14:14 +01:00
Théophane Hufschmitt cd9baa1809 Add release notes 2024-03-01 09:39:50 +01:00
Théophane HufschmittandValentin Gagarin 65b79c52c6 Fix a typo in a test comment
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt c3878f510e Copy the output of fixed-output derivations before registering them
It is possible to exfiltrate a file descriptor out of the build sandbox
of FODs, and use it to modify the store path after it has been
registered.
To avoid that issue, don't register the output of the build, but a copy
of it (that will be free of any leaked file descriptor).
2024-03-01 09:31:28 +01:00
Théophane Hufschmitt a55c6a0f47 Add a NixOS test for the sandbox escape
Test that we can't leverage abstract unix domain sockets to leak file
descriptors out of the sandbox and modify the path after it has been
registered.
2024-03-01 09:31:28 +01:00
John Ericson 5a2985431c Revert "Revert "Merge pull request #9546 from NixOS/nixos-23.11""
This reverts commit d6d7d2cb46.
2024-02-29 14:52:31 -05:00
John Ericson d6d7d2cb46 Revert "Merge pull request #9546 from NixOS/nixos-23.11"
This reverts commit 587c7dcb2b, reversing
changes made to 864fc85fc8.
2024-02-29 14:39:29 -05:00
John Ericson bcd6b33dbc Polish local overlay store docs 2024-02-29 11:58:53 -05:00
John Ericson 9b506ff0c1 Activate hermetic.nix variation only for new layered store tests 2024-02-29 10:06:53 -05:00
Théophane Hufschmitt bf48501194 Merge pull request #10112 from edolstra/fix-gcc12-warnings
Fix gcc 12 warnings
2024-02-29 15:55:59 +01:00
John Ericson cb4f85f11c Merge branch 'master' into overlayfs-store 2024-02-29 09:54:21 -05:00
Eelco Dolstra b1586a6799 Merge pull request #10090 from bobvanderlinden/profile-error-install-twice
profile install: warn on installing package twice
2024-02-29 10:04:08 +01:00
Bob van der Linden 14adff1711 profile install: skip and warn on installing package twice 2024-02-29 08:29:38 +01:00
tomberek ffe67c86a8 Merge pull request #9915 from 9999years/evaluating-attribute-position
Add position information to `while evaluating the attribute` errors in the debugger
2024-02-28 18:11:07 -05:00
Eelco Dolstra 65bb12ba78 Fix gcc 12 warnings 2024-02-28 22:59:20 +01:00
Théophane Hufschmitt 90ec015d61 Merge pull request #10070 from ramboman/fix-proxy-installer
`install-multi-user.sh`: `_sudo`: add proxy variables to sudo
2024-02-28 22:00:15 +01:00
Jade Lovelace f6158ea53b finally.hh: include works by itself; mark as nodiscard 2024-02-28 11:40:32 -08:00
Robert Hensing bd21b4b134 Merge pull request #10110 from hercules-ci/bump-actions-docker
actions docker_push_image: Update nix 2.13.3 -> 2.20.3
2024-02-28 20:27:54 +01:00
Robert Hensing 4d769e7a76 actions docker_push_image: Update nix 2.13.3 -> 2.20.3 2024-02-28 19:54:17 +01:00
John Ericson bcb5f235f9 Support symlinks properly with git-hashing experimental feature
Before, they would not be written to a file `FileSystemObjectSink`
correctly.
2024-02-28 12:24:45 -05:00
Robert Hensing 587c7dcb2b Merge pull request #9546 from NixOS/nixos-23.11
Update to nixos-23.11
2024-02-28 17:51:16 +01:00
Théophane Hufschmitt 864fc85fc8 Merge pull request #10010 from 9999years/fix-9941
Fix "Failed tcsetattr(TCSADRAIN)" when `nix repl` is not a TTY
2024-02-28 14:25:58 +01:00
Théophane Hufschmitt f6142cd0d1 unset NIX_HARDENING_ENABLE in fast build mode
`NIX_HARDENING_ENABLE` causes `_FORTIFY_SOURCE` to be defined.
This isn't compatible with `-O0`, and the compiler will happily remind
us about it at every call, spamming the terminal with warnings and stack
traces.

We don't really care hardening in that case, so just disable it if we
pass `OPTIMIZE=0`.
2024-02-28 08:07:51 +01:00
Théophane Hufschmitt da90be789d Fix a too smart implicit cast
Apparently gcc is able to implicitly cast from `FileIngestionMethod` to
`ContentAddressMethod`, but clang isn't. So explicit the cast
2024-02-28 08:00:17 +01:00
DavHau 358c26fd13 fetchTree: shallow git fetching by default
Motivation:
make git fetching more efficient for most repos by default
2024-02-28 13:27:22 +07:00
Théophane Hufschmitt 6147d27afb Bump the required daemon version for the git hashing tests
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:11:22 +01:00
Robert Hensing 8dc4b41c7f flake.lock: Strip out treeHash. Too soon...
I hate this.
We should have it, but for now we can't.
2024-02-28 07:08:21 +01:00
Robert Hensing 945940f2ef nixpkgs: nixos-23.11-small -> release-23.11
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/4dd376f7943c64b522224a548d9cab5627b4d9d6' (2024-02-26)
  → 'github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e' (2024-02-28)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt cf3ef060ff Disable the “static” darwin stdenvs
Don't evaluate, and probably not really useful (if at all)
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt a0cb75d96f Disable bear on all the things with darwin as hostPlatform
Just `stdenv.isDarwin` isn't enough because it doesn't apply to the
build platform, which mean that cross packages building from darwin to
another platform will have `isDarwin` set to false.
Replace it by `stdenv.buildPlatform.isDarwin`.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt bbef03872b Bump the required daemon version for the impure-env test
The required version check was a bit too lenient, and
`nixpkgs#nixUnstable` was considered valid while it didn't have the fix.
2024-02-28 07:08:21 +01:00
Théophane Hufschmitt 44f10f000a flake: Update to NixOS 23.11
About time :)

This required disabling `bear` on darwin as it's currently broken (fixed
on master, but not yet on 23.11).
2024-02-28 07:08:21 +01:00
John Ericson f489a6e42d Merge pull request #8918 from obsidiansystems/git-objects
Git object hashing in libstore
2024-02-27 19:02:22 -05:00
John EricsonandEelco Dolstra d4ad1fcf30 Avoid creating temporary store object for git over the wire
Instead, serialize as NAR and send that over, then rehash sever side.
This is alorithmically simpler, but comes at the cost of a newer
parameter to `Store::addToStoreFromDump`.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-02-27 11:27:34 -05:00
201551c937 Add Git object hashing to the store layer
Part of RFC 133

Extracted from our old IPFS branches.

Co-Authored-By: Matthew Bauer <mjbauer95@gmail.com>
Co-Authored-By: Carlo Nucera <carlo.nucera@protonmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Florian Klink <flokli@flokli.de>
2024-02-27 11:27:34 -05:00
John Ericson 04836c73e5 Merge nativeCheckInputs into nativeBuildInputs
They were getting skipped for the test-against checks.
2024-02-27 11:27:34 -05:00
Eelco Dolstra c3e9e3d0c3 Merge pull request #9767 from obsidiansystems/canon-path-split
Support Windows paths in `canonPath` and `absPath`
2024-02-27 17:12:29 +01:00
John Ericson 354ba27296 Merge pull request #10093 from NixOS/revert-10084-remove-dead-git-code
Revert "Remove dead Git code"
2024-02-27 09:54:25 -05:00
Eelco Dolstra 5b0d78ec73 Merge pull request #10091 from bobvanderlinden/default-package-name
Fix extraction of name for defaultPackage URLs
2024-02-27 15:14:20 +01:00
Bob van der Linden e5d9130a5b Fix extraction of name for defaultPackage URLs 2024-02-27 07:53:05 +01:00
Théophane Hufschmitt be0052b45f Revert "Remove dead Git code" 2024-02-27 06:39:30 +01:00
Bob van der Linden d28a240aa6 profile: extract getNameFromElement 2024-02-26 21:07:18 +01:00
Robert Hensing 4c7f0ef6ca Merge pull request #9847 from pennae/inherit-from-dedup
deduplicate inherit-from source expr work
2024-02-26 20:25:58 +01:00
Théophane Hufschmitt c10025d8ca Merge pull request #10084 from tweag/remove-dead-git-code
Remove dead Git code
2024-02-26 19:49:14 +01:00
pennaeandRobert Hensing f24e445bc0 add doc comment justifying ExprInheritFrom
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-26 19:07:08 +01:00
pennae 1cd87b7042 remove ExprAttrs::AttrDef::inherited
it's no longer widely used and has a rather confusing meaning now that
inherit-from is handled very differently.
2024-02-26 19:07:08 +01:00
pennae cefd0302b5 evaluate inherit (from) exprs only once per directive
desugaring inherit-from to syntactic duplication of the source expr also
duplicates side effects of the source expr (such as trace calls) and
expensive computations (such as derivationStrict).
2024-02-26 19:07:08 +01:00
Eelco Dolstra 598deb2b23 Use SourcePath for reading flake.{nix,lock}
Flakes still reside in the Nix store (so there shouldn't be any change
in behaviour), but they are now accessed via the rootFS
accessor. Since rootFS implements access checks, we no longer have to
worry about flake.{nix,lock} or their parents being symlinks that
escape from the flake.

Extracted from the lazy-trees branch.
2024-02-26 15:14:18 +01:00
John Ericson 62a13c8101 Merge pull request #10086 from ShamrockLee/hash-algorithm-rename
treewide: hash type -> hash algorithm
2024-02-26 08:37:09 -05:00
Théophane Hufschmitt cd2d8b6d4c Merge pull request #10085 from ShamrockLee/ignore-obsolete-testdir
.gitignore: ignore historical test binaries
2024-02-26 11:36:10 +01:00
Yueh-Shun Li fd47f76da9 treewide: hash type -> hash algorithm
"hash type" -> "hash algorithm" in all comments, documentation, and
messages.

ht -> ha, [Hh]ashType -> [HhashAlgo] for all local variables and
function arguments. No API change is made.

Continuation of 5334c9c792 and 837b889c41.
2024-02-26 18:09:06 +08:00
Théophane Hufschmitt 219705ff64 Remove dead code
Most of the code in `git.{cc,hh}` is dead, so get rid of it.
2024-02-26 11:07:47 +01:00
Yueh-Shun Li 8ac4542593 .gitignore: ignore historical test binaries
After commit 91b6833686 (" Move tests to separate directories, and
document"), previously-built test executables are now tracked by Git,
which is annoying for developers.

This patch add .gitignore rules to ignore the obsolete test directories
to solve such problem and enhance developer experience.
2024-02-26 18:05:07 +08:00
Théophane Hufschmitt 21282c3c20 Merge pull request #10074 from lf-/jade/ban-implicit-fallthrough
Warn on implicit switch case fallthrough
2024-02-26 10:48:07 +01:00
Eelco Dolstra 7d9ae5fd06 Merge pull request #10071 from zimbatm/ci-default-tag
ci default tag
2024-02-26 10:45:26 +01:00
Simon Žlender d60c3f7f7c Fix __darwinAllowLocalNetworking sandbox
The sandbox rule `(allow network* (local ip))` doesn't do what it
implies. Adding this rule permits all network traffic. We should be
matching on (remote ip "localhost:*")` instead.
2024-02-25 23:00:57 +01:00
Jade Lovelace a82aeedb5b Warn on implicit switch case fallthrough
This seems to have found one actual bug in fs-sink.cc: the symlink case
was falling into the regular file case, which can't possibly be
intentional, right?
2024-02-24 15:52:16 -08:00
Olmo Kramer 9f11b1b0c4 Accept multiple inputs in nix flake update 2024-02-24 20:58:44 +01:00
Johannes KirschbauerandValentin Gagarin d83008c3a7 documentation: clarify genericClosure (#10003)
* doc: clarify genericClosure documentation

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-02-24 12:34:53 +00:00
zimbatm 5598ce3e0f ci: fix docker default tag
Docker uses "latest" as the default label instead of "master".

This change will allow to docker run ghcr.io/nixos/nix without having to
specify the label.

It keeps the :master label on docker hub for back-compat.
2024-02-24 11:24:24 +01:00
ramboman 24fd7e2755 install-multi-user.sh: _sudo: add proxy variables to sudo 2024-02-24 01:00:13 +00:00
Robert Hensing 0b47783d0a Merge pull request #10066 from 9999years/print-all-frames
Do not skip any stack frames when `--show-trace` is given
2024-02-23 18:53:11 +01:00
Théophane Hufschmitt accae60e77 Merge pull request #10067 from ramboman/fix-proxy-nix
`nix`: Fix `haveInternet` to check for proxy
2024-02-23 11:06:36 +01:00
Théophane Hufschmitt bca737dcad c++-ize the proxy detection code
Just for consistency with the rest
2024-02-23 10:28:37 +01:00
ramboman d3bff699aa nix: Fix haveInternet to check for proxy 2024-02-23 01:05:25 -05:00
Rebecca Turner fe6408b5df Update snapshots 2024-02-22 17:58:55 -08:00
Rebecca Turner 91e89628fd Make addErrorTrace variadic 2024-02-22 17:18:27 -08:00
Rebecca Turner f05c13ecc2 Remove the concept of "skipped frames" 2024-02-22 17:14:55 -08:00
Rebecca Turner 040874e4db Print all stack frames 2024-02-22 17:14:33 -08:00
Théophane Hufschmitt 6a5210f48e Merge pull request #9815 from NixOS/nix-hash-path
`nix hash path`, text hashing for `nix store add`, and preparatory refactors
2024-02-22 17:15:34 +01:00
Eelco Dolstra f183eef4a0 Merge pull request #10060 from bobvanderlinden/pr-simple-profile-remove-suggestion
profile: suggest removal using profile entry name
2024-02-22 11:02:12 +01:00
Bob van der Linden 4ae5091716 nix profile: suggest removal using entry name
When a file conflict arises during a package install a suggestion is
made to remove the old entry. This was previously done using the
installable URLs of the old entry. These URLs are quite verbose and
often do not equal the URL of the existing entry.

This change uses the recently introduced profile entry name for the
suggestion, resulting in a simpler output.

The improvement is easily seen in the change to the functional test.
2024-02-21 21:58:36 +01:00
John Ericson 2080d89b87 Merge pull request #10038 from edolstra/tarball-git-cache
Use the Git cache for tarball flakes
2024-02-21 15:47:02 -05:00
Eelco Dolstra e391fc2101 Add comments 2024-02-21 21:15:28 +01:00
Théophane Hufschmitt 774e7ca584 Merge pull request #9914 from 9999years/debugger-on-trace
Enter debugger on `builtins.trace` with an option
2024-02-21 18:32:56 +01:00
Théophane Hufschmitt 36fa8b1bcc Merge pull request #9943 from 9999years/release-notes
Add release notes
2024-02-21 18:17:50 +01:00
John Ericson efd36b49e8 nix hash path, and preperatory refactors
- `nix store add` supports text hashing

  With functional test ensuring it matches `builtins.toFile`.

- Factored-out flags for both commands

- Move all common reusable flags to `libcmd`

  - They are not part of the *definition* of the CLI infra, just a usag
    of it.

  - The `libstore` flag couldn't go in `args.hh` in libutil anyways,
    would be awkward for it to live alone

- Shuffle around `Cmd*` hierarchy so flags for deprecated commands don't
  end up on the new ones
2024-02-21 12:11:25 -05:00
Rebecca Turner b111fba8cd Add documentation, rename to debugger-on-trace 2024-02-21 09:07:39 -08:00
Eelco Dolstra 3f5d7afe46 Merge pull request #10024 from edolstra/remove-locked-flag
Input: Replace 'locked' bool by isLocked() method
2024-02-21 16:19:15 +01:00
Eelco Dolstra 9ae665b9e1 Merge pull request #10054 from syvb/channel-unpack-count
Say how many channels were unpacked in nix-channel
2024-02-21 12:23:57 +01:00
Eelco Dolstra 09d76e512a GitArchiveInputScheme: Require a NAR hash 2024-02-21 12:08:18 +01:00
Eelco Dolstra ff4fa4dbd3 Merge pull request #10055 from GrahamDennis/gdennis/faster-flake-lock-parsing
Faster flake.lock parsing
2024-02-21 11:57:44 +01:00
Graham Dennis 7fd0de38c6 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.
2024-02-21 18:40:34 +11:00
syvb 60eeacc24a Say how many channels were unpacked 2024-02-20 19:17:18 -05:00
John Ericson a1b1070dbe Merge pull request #9916 from 9999years/quit-whole-program-from-debugger
`:quit` in the debugger should quit the whole program
2024-02-20 18:12:05 -05:00
Rebecca Turner 8e71883e3f Rename ProcessLineResult variants 2024-02-20 14:52:16 -08:00
Rebecca Turner 2a8fe9a938 :quit in the debugger should quit the whole program 2024-02-20 10:01:13 -08:00
Eelco Dolstra 071dd2b3a4 Input: Replace 'locked' bool by isLocked() method
It's better to just check whether the input has all the attributes
needed to consider itself locked (e.g. whether a Git input has an
'rev' attribute).

Also, the 'locked' field was actually incorrect for Git inputs: it
would be set to true even for dirty worktrees. As a result, we got
away with using fetchTree() internally even though fetchTree()
requires a locked input in pure mode. In particular, this allowed
'--override-input' to work by accident.

The fix is to pass a set of "overrides" to call-flake.nix for all the
unlocked inputs (i.e. the top-level flake and any --override-inputs).
2024-02-20 16:59:15 +01:00
John Ericson 78e7c98b02 Merge pull request #10049 from edolstra/dont-send-experimental-features
Don't send settings that depend on disabled experimental features to the daemon
2024-02-20 09:53:08 -05:00
Eelco Dolstra 0acd783190 Don't send settings that depend on disabled experimental features to the daemon
This fixes warnings like

   warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
   warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled

when using the daemon and the user didn't actually set those settings.

Note: this also hides those settings from `nix config show`, but that
seems a good thing.
2024-02-20 15:28:42 +01:00
Théophane Hufschmitt d2c6a93bd5 Merge pull request #10044 from edolstra/empty-git-repos
Handle empty Git repositories / workdirs
2024-02-20 14:01:23 +01:00
Eelco Dolstra cabee98152 Tarball fetcher: Use the content-addressed Git cache
Backported from the lazy-trees branch.
2024-02-20 12:57:36 +01:00
Eelco Dolstra 930b9c8269 PosixSourceAccessor: Support roots that are not directories
We have to support this for `fetchTree { type = "file" }` (and
probably other types of trees that can have a non-directory at the
root, like NARs).
2024-02-20 12:51:15 +01:00
Eelco Dolstra b00f412f81 Remove bad.tar.xz check, since libarchive doesn't care 2024-02-20 12:51:15 +01:00
Eelco Dolstra db012d1e63 tests/functional/tarball.sh: Fix invalid file:// URLs 2024-02-20 12:51:15 +01:00
Théophane Hufschmitt 6f4bb1b584 Merge pull request #10043 from edolstra/fix-readonly-fetchToStore
fetchToStore(): Don't always respect settings.readOnlyMode
2024-02-20 12:07:48 +01:00
Eelco Dolstra 7cb4d0c5b7 fetchToStore(): Don't always respect settings.readOnlyMode
It's now up to the caller whether readOnlyMode should be applied. In
some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:46:49 +01:00
Eelco Dolstra 9e762454cf Support empty Git repositories / workdirs
Fixes #10039.
2024-02-20 11:40:02 +01:00
Eelco Dolstra d52d91fe7a AllowListInputAccessor: Clarify that the "allowed paths" are actually allowed prefixes
E.g. adding "/" will allow access to the root and *everything below it*.
2024-02-20 11:23:26 +01:00
Eelco Dolstra 6162105675 Don't say "copying X to the store" in read-only mode 2024-02-20 10:47:30 +01:00
John Ericson 319ec6f84a Support Windows paths in canonPath and absPath
`canonPath` and `absPath` work on native paths, and so should switch
between supporting Unix paths and Windows paths accordingly.

The templating is because `CanonPath`, which shares the implementation,
should always be Unix style. It is the pure "nix-native" path type for
virtual file operations --- it is part of Nix's "business logic", and
should not vary with the host OS accordingly.
2024-02-16 10:31:36 -05:00
John Ericson 4531585275 Factor out the Unix-specific parts of canonPathInner
This prepares the code to also support Windows paths in the next commit.
2024-02-16 10:12:07 -05:00
John Ericson 60936f28e5 Merge pull request #9881 from obsidiansystems/purify-canon-path
Purify `CanonPath`
2024-02-16 10:11:11 -05:00
d17e1d9737 Purify CanonPath
The core `CanonPath` constructors were using `absPath`, but `absPath` in
some situations does IO which is not appropriate. It turns out that
these constructors avoided those situations, and thus were pure, but it
was far from obvious this was the case.

To remedy the situation, abstract the core algorithm from `canonPath` to
use separately in `CanonPath` without any IO. No we know by-construction
that those constructors are pure.

That leaves `CanonPath::fromCWD` as the only operation which uses IO /
is impure. Add docs on it, and `CanonPath` as a whole, explaining the
situation.

This is also necessary to support Windows paths on windows without
messing up `CanonPath`. But, I think it is good even without that.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-16 09:47:25 -05:00
John Ericson d53c8901ef Merge pull request #9985 from alois31/symlink-resolution
Restore `builtins.pathExists` behavior on broken symlinks
2024-02-16 09:24:03 -05:00
John Ericson e27b7e04bf Add note about this being a temp solution 2024-02-16 08:45:15 -05:00
Eelco Dolstra 06be819b89 Merge pull request #9485 from edolstra/tarball-cache
Add a Git-based content-addressed tarball cache
2024-02-15 22:37:57 +01:00
Eelco Dolstra 78b8db72b5 Remove tarball-cache.{hh,cc}
TarballInfo is only used in github.cc, and getTarballCache() is a bit
too trivial to have its own file.
2024-02-15 21:58:08 +01:00
Eelco Dolstra 9b04ce16fc Merge branch 'tarball-cache' of github.com:Ericson2314/nix into tarball-cache 2024-02-15 21:48:10 +01:00
Eelco Dolstra 4b40fc5b46 Merge remote-tracking branch 'origin/master' into tarball-cache 2024-02-15 21:46:36 +01:00
Eelco Dolstra 27b8d5d920 Merge pull request #10013 from edolstra/libgit2-lookup
GitInputAccessor: Speed up lookup()
2024-02-15 17:30:22 +01:00
John Ericson ed24baaec4 Finish separating concerns with tarball cache
There is no longer an `importTarball` method. Instead, there is a
`unpackTarfileToSink` function (back in libutil). The caller can use
thisw with the `getParseSink` method we added in the last commit easily
enough.

In addition, tarball cache functionality is separated from `git-utils`
and moved into `tarball-cache`. This ensures we are separating mechanism
and policy.
2024-02-15 10:34:01 -05:00
John Ericson ba6a5f06ee Split GitRepoImpl::importTarball
There is now a separation of:

1. A `FileSystemObjectSink` for writing to git repos

2. Adapting libarchive to use that parse sink.

The prepares a proper separation of concerns.
2024-02-15 10:27:54 -05:00
Théophane Hufschmitt bc9471d032 Merge pull request #9966 from NixOS/mention-OPTIMIZE-in-hacking-guide
Mention `OPTIMIZE=0` more prominently in the hacking guide
2024-02-15 09:33:15 +01:00
Eelco Dolstra 67a6d34448 GitInputAccessor: Speed up lookup()
A command like

  rm -rf ~/.cache/nix/tarball-cache/ ~/.cache/nix/fetcher-cache-v1.sqlite*; nix flake metadata 'git+file:///home/eelco/Dev/nixpkgs?rev=9463103069725474698139ab10f17a9d125da859'

was spending about 84% of its runtime in lookup(), specifically in
git_tree_entry_bypath(). (The reading of blobs is less than 3%.)

It appears libgit2 doesn't do a lot of caching of trees, so we now
make sure that when we look up a path, we add all its parents, and all
the immediate children of the parents (since we have them in memory
anyway), to our own cache.

This speed up the command above from 17.2s to 7.8s on my machine.

Fixes (or at least should improve a lot) #9684.
2024-02-14 19:25:23 +01:00
Rebecca Turner 6d2b446e2b Add release notes for "Pretty print values in nix repl" 2024-02-14 08:58:54 -08:00
Eelco Dolstra 54354eaecf Merge remote-tracking branch 'origin/master' into tarball-cache 2024-02-14 14:45:19 +01:00
Théophane Hufschmitt d857914e1a Merge pull request #9931 from 9999years/pretty-printer
Pretty-print values in the REPL
2024-02-14 13:32:58 +01:00
John Ericson ee9f6d022f Merge pull request #10001 from abathur/fix_macos_daemon_perms
install-darwin: fix symbolic perms for install cmd
2024-02-13 17:39:11 -05:00
John Ericson ef0b82a267 Merge pull request #9986 from nbelakovski/patch-1
Add a note about lists values.md
2024-02-13 17:10:32 -05:00
Eelco Dolstra 2b946aad8e Merge pull request #10006 from edolstra/fix-impure-fetchurl
<nix/fetchurl.nix>: Restore support for "impure = true"
2024-02-13 22:50:55 +01:00
Rebecca Turner a694cfb7bd Fix "Failed tcsetattr(TCSADRAIN)" when nix repl is not a TTY
Before:
```
$ echo builtins.nixVersion | nix repl
Welcome to Nix 2.18.1. Type :? for help.

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
"2.18.1"

Failed tcsetattr(TCSADRAIN): Inappropriate ioctl for device
```

After:
```
$ echo builtins.nixVersion | nix repl
Nix 2.21.0pre20240131_dirty
Type :? for help.
"2.21.0pre20240131_dirty"
```
2024-02-13 11:09:12 -08:00
John Ericson 5b26c66a8c Merge pull request #10009 from obsidiansystems/ca-type-names
Proper `parse` and `render` functions for `FileIngestionMethod` and `ContentAddressMethod`
2024-02-13 13:28:52 -05:00
John Ericson 9d64613dca Update src/libutil/file-content-address.cc 2024-02-13 12:50:10 -05:00
Alois Wohlschlager 89e21ab4bd Restore builtins.pathExists behavior on broken symlinks
Commit 83c067c0fa changed `builtins.pathExists`
to resolve symlinks before checking for existence. Consequently, if the path
refers to a symlink itself, existence of the target of the symlink (instead of
the symlink itself) was checked. Restore the previous behavior by skipping
symlink resolution in the last component.
2024-02-13 18:09:55 +01:00
John Ericson db41a0616a Use ContentAddressMethod::render in one more place
Good to deduplicate the code.
2024-02-13 11:14:49 -05:00
Eelco Dolstra 4254cdabf2 Merge pull request #10008 from edolstra/config-status-dep
*.in files: Depend on config.status
2024-02-13 16:49:41 +01:00
John Ericson f49efb32fb Merge pull request #10005 from fricklerhandwerk/local-store-glossary
update glossary entry on store types
2024-02-13 10:45:36 -05:00
John Ericson 41dd9857c7 Proper parse and render functions for FileIngestionMethod and ContentAddressMethod
No outward facing behavior is changed.

Older methods with same names that operate on on method + algo pair (for
old-style `<method>:algo`) are renamed to `*WithAlgo`.)

The functions are unit-tested in the same way the names for the hash
algorithms are tested.
2024-02-13 10:30:16 -05:00
Valentin Gagarin f27205f743 redirect local and chroot store to main page 2024-02-13 16:25:07 +01:00
John Ericson fb5a438dca Merge pull request #10007 from NixOS/split-test
Split `hash.sh` test in two
2024-02-13 10:22:12 -05:00
Eelco Dolstra bb21892351 *.in files: Depend on config.status 2024-02-13 16:10:56 +01:00
Valentin Gagarin 5b69409f6b only refer to the local store page 2024-02-13 16:08:44 +01:00
John Ericson 8bebf9607c Split hash.sh test in two
Converting hashes and hashing files are pretty separate tasks, and more
test parallelism is better.
2024-02-13 09:44:14 -05:00
Eelco Dolstra b439b1fc66 Merge pull request #9993 from edolstra/builtin-paths
builtin:{unpack-channel,buildenv}: Get output path from the derivation
2024-02-13 14:57:13 +01:00
Valentin Gagarin 39c353f6fa reword description of the cores setting (#9522)
* reword description of the `cores` setting

- be precise about the `builder` executable
- clearly distinguish between `builder` and job parallelism
- clarify the role of `mkDerivation` in the example
- remove prose for the default, it's shown programmatically
- mention relation to `max-jobs`
2024-02-13 13:52:51 +00:00
Eelco Dolstra bb63bd50e6 <nix/fetchurl.nix>: Restore support for "impure = true" 2024-02-13 14:14:20 +01:00
Valentin GagarinandJanik H fd82ba0985 extract reference documentation on remote builds (#9526)
- move all reference documentation to the `builders` configuration setting
- reword documentation on machine specification, add examples
- disable showing the default value, as it rendered as `@/dummy/machines`, which is wrong
- highlight the examples
- link to the configuration docs for distributed builds
- builder -> build machine

Co-authored-by: Janik H <janik@aq0.de>
2024-02-13 14:13:56 +01:00
Valentin Gagarin e37d502895 add instructions to wipe the substituter lookup cache (#9498)
* add instructions to wipe the substituter lookup cache
2024-02-13 13:34:51 +01:00
Valentin Gagarin ce19338f9f update glossary entry on store types
the interesting information is on the proper pages, and is now presented
a bit more prominently.

the paragraph was a bit confusing to read, also because an anchor link
to an inline definition was in the middle of the sentence. "local store"
now has its own glossary entry.
2024-02-13 13:30:17 +01:00
Théophane Hufschmitt 9d87f25673 Merge pull request #9997 from pg83/patch-1
Update nar-info-disk-cache.cc
2024-02-13 08:55:57 +01:00
Travis A. Everett 64a076fe06 install-darwin: fix symbolic perms for install cmd
The symbolic form in use here doesn't seem to have an effect
in either the BSD or coreutils install commands, leaving the
daemon plist with empty permissions. This seems to cause its
own problems.

I think I've got the right symbolic syntax now :)
2024-02-13 01:18:08 -06:00
Anton Samokhvalov 64cbd4c05a Update nar-info-disk-cache.cc
fix case when asserts are no-op, like in release build
2024-02-12 23:37:40 +03:00
Robert Hensing bdb6f56c90 Merge pull request #9295 from NixOS/store-path-complete-construction
Include store path exact spec in the docs
2024-02-12 19:02:03 +01:00
John Ericson f29d2a9d11 Small EBNF fix 2024-02-12 12:30:28 -05:00
John Ericson 898fd1e48d Update doc/manual/src/protocols/store-path.md 2024-02-12 12:04:37 -05:00
John EricsonandRobert Hensing 5169f5f4d9 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 12:01:54 -05:00
John EricsonandRobert Hensing c873a140d7 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 11:34:54 -05:00
John Ericson 4c3e4d6d71 Sections, EBNF tweaks 2024-02-12 11:33:06 -05:00
John Ericson 0862d7ce57 Move around non-terminals 2024-02-12 11:26:08 -05:00
John Ericson eb76b35efa Merge remote-tracking branch 'upstream/master' into store-path-complete-construction 2024-02-12 11:22:54 -05:00
John Ericson 30f6b0f9c5 pre -> fingerprint in store path grammar
As suggested by @roberth in
https://github.com/NixOS/nix/pull/9295#discussion_r1486402040.

Thanks!
2024-02-12 11:22:08 -05:00
John Ericson 95190e68ed Mention the use of Extended Backus–Naur form 2024-02-12 11:21:54 -05:00
John Ericson ac1301ddfd Convert store path "grammar" to EBNF 2024-02-12 11:16:53 -05:00
John EricsonandRobert Hensing 91557df4a7 Apply suggestions from code review
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-02-12 11:16:12 -05:00
Eelco Dolstra a9b69b2fff builtin:{unpack-channel,buildenv}: Get output path from the derivation
Similar to 1ee42c5b88, get the "out"
path from the derivation (and complain if it doesn't exist), rather
than getting it from the environment.
2024-02-12 16:34:59 +01:00
Eelco Dolstra c4ebb82da4 Merge pull request #9992 from edolstra/fix-warning
Fix "may be used uninitialized" warning
2024-02-12 16:02:21 +01:00
Eelco Dolstra 619ca631d0 Fix "may be used uninitialized" warning 2024-02-12 15:29:48 +01:00
Eelco Dolstra d3c1997127 Merge pull request #9976 from alois31/restore-manual-pages
Restore manual pages
2024-02-12 15:21:47 +01:00
Théophane Hufschmitt e352c66161 Merge pull request #9983 from BOHverkill/fix-string-interpolation-link
Fix link to derivation in string interpolation doc
2024-02-12 14:26:03 +01:00
Eelco Dolstra ec6ca6e42c Merge pull request #9948 from obsidiansystems/no-canon-path-from-cwd
Get rid of `CanonPath::fromCwd`
2024-02-12 14:04:01 +01:00
pennae ecf8b12d60 group inherit by source during Expr::show
for plain inherits this is really just a stylistic choice, but for
inherit-from it actually fixes an exponential size increase problem
during expr printing (as may happen during assertion failure reporting,
on during duplicate attr detection in the parser)
2024-02-12 13:58:29 +01:00
pennae 6c08fba533 use the same bindings print for ExprAttrs and ExprLet
this also has the effect of sorting let bindings lexicographically
rather than by symbol creation order as was previously done, giving a
better canonicalization in the process.
2024-02-12 13:35:00 +01:00
pennae 1f542adb3e add ExprAttrs::AttrDef::chooseByKind
in place of inherited() — not quite useful yet since we don't
distinguish plain and inheritFrom attr kinds so far.
2024-02-12 13:34:59 +01:00
pennae c66ee57edc preserve information about whether/how an attribute was inherited 2024-02-12 13:32:33 +01:00
pennae 73065a400d add test for inherit expr printing 2024-02-12 13:32:33 +01:00
pennae 8669c02468 add test for inherit-from semantics 2024-02-12 13:32:33 +01:00
nbelakovski 4496a4537b Update values.md
Link to elemAt
2024-02-11 22:52:49 -08:00
John Ericson 9d182e71fb Merge pull request #9984 from BOHverkill/fix-manual-link
Fix link to manual in CONTRIBUTING.md
2024-02-11 16:13:06 -05:00
nbelakovski f298159a2b Add a note about lists values.md
There's probably more that can be said, but I thought it might be helpful to put something here about how to access elements of a list for folks coming from more or less any other programming language. If this is rarely used, it might be nice to add to the documentation something about why it's rarely used.
2024-02-10 18:52:39 -08:00
BOHverkill fae8c15737 Fix link to manual in CONTRIBUTING.md 2024-02-10 17:44:33 +01:00
BOHverkill 53eecae525 Fix link to derivation in string interpolation doc
The reference link definition for it pointing to the glossary was
removed, so it is currently not displayed as a link.
2024-02-10 17:27:16 +01:00
Alois Wohlschlager 8f3253c6f4 Restore manual pages
Commit d536c57e87 inadvertedly broke build and
installation of all non-autogenerated manual pages (in particular, all the ones
documenting the stable CLI), by moving the definition of the man-pages variable
in doc/manual/local.mk after its usage in mk/lib.mk. Move including the former
earlier so that the correct order is restored.
2024-02-09 19:07:59 +01:00
Théophane Hufschmitt 28a988bdde Merge pull request #9974 from edolstra/less-chatty-post-build-hook
runPostBuildHook(): Be less chatty
2024-02-09 16:39:02 +01:00
Eelco Dolstra fb5a792280 runPostBuildHook(): Be less chatty
Don't spam the user with "running post-build-hook" messages. It's up
to the post-build hook if it has something interesting to say.
2024-02-09 15:55:24 +01:00
Eelco Dolstra 4487ebca66 Merge pull request #9973 from fricklerhandwerk/add-links-gc-confusion
add clickable anchor links
2024-02-09 13:41:18 +01:00
Eelco Dolstra 57ebcadb2f Merge pull request #9972 from NixOS/allow-unoptimized-builds
Don't hardcode the `-O2` compiler flag
2024-02-09 13:25:51 +01:00
Valentin Gagarin 60045f9c96 add clickable anchor links
how the different invocations relate to each other seems be
confusing, which is relatable because one has to wire it up in your head
while reading. an explicit reference should make it unambiguous and
easier to notice due to links being highlighted.
2024-02-09 10:41:03 +01:00
Théophane Hufschmitt 1fe7b01669 Don't hardcode the -O2 compiler flag
autoconf authors apparently decided that setting `-O2` by default was a good idea. I disagree, and Nix has its own way of deciding that (with `OPTIMIZE={0,1}`). Explicitly set `CFLAGS` and `CXXFLAGS` in the configure script to disable that behaviour.

Fix #9965
2024-02-09 06:27:24 +01:00
Théophane Hufschmitt 47a1dbb4b8 Merge pull request #9925 from 9999years/fmt-cleanup
Cleanup `fmt.hh`
2024-02-09 06:01:03 +01:00
Rebecca Turner 953eb0cba2 Fix tests 2024-02-08 15:55:20 -08:00
Rebecca Turner 4440eb54e7 Add release note 2024-02-08 12:07:38 -08:00
Rebecca Turner 8b7eb7400b Enter debugger on builtins.trace with an option 2024-02-08 12:07:37 -08:00
Rebecca Turner c0e7f50c1a Rename hintfmt to HintFmt 2024-02-08 11:58:25 -08:00
Rebecca Turner 149bd63afb Cleanup fmt.hh
When I started contributing to Nix, I found the mix of definitions and
names in `fmt.hh` to be rather confusing, especially the small
difference between `hintfmt` and `hintformat`. I've renamed many classes
and added documentation to most definitions.

- `formatHelper` is no longer exported.
- `fmt`'s documentation is now with `fmt` rather than (misleadingly)
  above `formatHelper`.
- `yellowtxt` is renamed to `Magenta`.

  `yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING`
  has been equal to `ANSI_MAGENTA` for a long time. Now the name is
  updated.
- `normaltxt` is renamed to `Uncolored`.
- `hintfmt` has been merged into `hintformat` as extra constructor
  functions.
- `hintformat` has been renamed to `hintfmt`.
- The single-argument `hintformat(std::string)` constructor has been
  renamed to a static member `hintformat::interpolate` to avoid pitfalls
  with using user-generated strings as format strings.
2024-02-08 11:51:03 -08:00
Théophane Hufschmitt 1ba9780cf5 Merge pull request #9834 from 9999years/structured-errors
Towards structured error classes
2024-02-08 20:00:25 +01:00
Rebecca Turner 403c90ddf5 Extract printSpace helper 2024-02-08 10:18:12 -08:00
Rebecca Turner 1c5f5d4291 prettyPrint -> shouldPrettyPrint 2024-02-08 10:18:10 -08:00
Rebecca TurnerandThéophane Hufschmitt a27651908f Add assertion for decreasing the indent
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-02-08 10:11:45 -08:00
Rebecca Turner 0f1269243b Revert "Add release notes for "Stack traces are more compact""
This reverts commit b2868acbdc.
2024-02-08 10:09:47 -08:00
Rebecca Turner b2868acbdc Add release notes for "Stack traces are more compact" 2024-02-08 10:08:48 -08:00
Rebecca Turner 7f8960d0f2 Add release notes for "Stack size is increased on macOS" 2024-02-08 10:08:48 -08:00
Rebecca Turner 837c350bcd Add release notes for "Cycle detection in nix repl is simpler and more reliable" 2024-02-08 10:08:48 -08:00
Rebecca Turner 4f0d43a397 Add release notes for "nix repl now respects Ctrl-C while printing values" 2024-02-08 10:08:48 -08:00
Rebecca Turner 24cdb81bb0 Add release notes for "Nix commands respect Ctrl-C" 2024-02-08 10:08:48 -08:00
Rebecca Turner abb5fef355 Add release notes for "Functions are printed with more detail" 2024-02-08 10:08:48 -08:00
Rebecca Turner 9a5d52262f Add release notes for "Nix no longer attempts to git add files that are .gitignored" 2024-02-08 10:08:48 -08:00
Rebecca Turner 754c35abfb Add release notes for "Debugger prints source position information" 2024-02-08 10:08:48 -08:00
John Ericson 4687beecef Get rid of CanonPath::fromCwd
As discussed in the last Nix team meeting (2024-02-95), this method
doesn't belong because `CanonPath` is a virtual/ideal absolute path
format, not used in file systems beyond the native OS format for which a
"current working directory" is defined.

Progress towards #9205
2024-02-08 11:01:41 -05:00
Eelco Dolstra 6563a58294 Merge pull request #9964 from fricklerhandwerk/merge-format-sections
move JSON section into Formats and Protocols
2024-02-08 16:53:59 +01:00
Théophane Hufschmitt a8050d9b83 Merge pull request #9928 from 9999years/error-messages-in-nix-repl
Improve error printing in `nix repl`
2024-02-08 16:21:13 +01:00
Théophane Hufschmitt 96686a2a29 Merge pull request #9963 from fricklerhandwerk/fold-sidebar
manual: fold sidebar sections
2024-02-08 16:18:27 +01:00
John Ericson f2f54cf087 Merge pull request #9863 from obsidiansystems/perl-open-other-store
Support arbitrary stores in Perl bindings
2024-02-08 09:30:18 -05:00
Théophane Hufschmitt 46a0625a40 Merge pull request #9929 from 9999years/dont-print-values-in-magenta
Don't print the first bracket in values in magenta in error messages
2024-02-08 10:56:18 +01:00
Théophane Hufschmitt 9b8b486091 Merge pull request #9933 from pennae/debugger-fix
fix debugger crashing while printing envs
2024-02-08 10:48:02 +01:00
Théophane Hufschmitt acef4f17a2 Merge pull request #9918 from 9999years/debugger-locals-for-let-expressions
Expose locals from `let` expressions to the debugger
2024-02-08 10:17:55 +01:00
Théophane Hufschmitt c4ed92fa6f Merge pull request #9917 from 9999years/enter-debugger-more-reliably
Enter debugger more reliably in `let` expressions and function calls
2024-02-08 10:09:54 +01:00
Théophane Hufschmitt fb78a99e04 Merge pull request #9924 from 9999years/rename-yellowtxt
Rename `yellowtxt` -> `magentatxt`
2024-02-08 10:01:30 +01:00
Théophane Hufschmitt 0dea16e686 Mention OPTIMIZE=0 more prominently in the hacking guide
This is a game-changer when developing, it shouldn't just be hidden amongst a list of more advanced variables.
2024-02-08 09:50:12 +01:00
Théophane Hufschmitt f388a6148d Merge pull request #9919 from 9999years/reduce-debugger-clutter
Reduce visual clutter in the debugger
2024-02-08 09:42:38 +01:00
Théophane Hufschmitt d24c8aa491 Simplify a conditional in the repl initialisation 2024-02-08 09:22:30 +01:00
Valentin Gagarin e486b76eef move JSON section into Formats and Protocols 2024-02-08 09:13:58 +01:00
Valentin Gagarin 140de3b278 manual: fold sidebar sections
the table of contents is very long now, and folded sections allow
for a better overview.
2024-02-08 09:00:00 +01:00
John Ericson bc08502249 Support arbitrary stores in Perl bindings
Fix #9859

It's a breaking change but that's fine; we can just update Hydra to use
the new bindings.
2024-02-07 22:31:58 -05:00
Eelco Dolstra 09a6e8e703 Merge pull request #9949 from fricklerhandwerk/fix-redirects-file
fix location of `_redirects` file
2024-02-07 11:34:35 +01:00
Rebecca Turner 9723f533d8 Add comment 2024-02-06 16:50:47 -08:00
Rebecca Turner 474fc4078a Add comments 2024-02-06 16:49:28 -08:00
Valentin Gagarin 2d74b56aee fix location of _redirects file
the Netlify `_redirects` file must be in the root directory [0] of the
files to serve, and mdBook copies all the files in `src` that aren't
`.md` to the output directory [1].

[0]: https://docs.netlify.com/routing/redirects/
[1]: https://rust-lang.github.io/mdBook/guide/creating.html#source-files
2024-02-06 23:22:34 +01:00
Rebecca Turner c0a15fb7d0 Pretty-print values in the REPL
Pretty-print values in the REPL by printing each item in a list or
attrset on a separate line. When possible, single-item lists and
attrsets are printed on one line, as long as they don't contain a nested
list, attrset, or thunk.

Before:
```
{ attrs = { a = { b = { c = { }; }; }; }; list = [ 1 ]; list' = [ 1 2 3 ]; }
```

After:
```
{
  attrs = {
    a = {
      b = {
        c = { };
      };
    };
  };
  list = [ 1 ];
  list' = [
    1
    2
    3
  ];
}
```
2024-02-05 13:23:38 -08:00
Rebecca Turner 657a607812 Add release note 2024-02-05 13:21:08 -08:00
Rebecca Turner 155bc761f6 Add release note 2024-02-05 13:16:39 -08:00
Rebecca Turner b63a8d7c46 Add release note 2024-02-05 13:15:29 -08:00
Rebecca Turner 601fc7d159 Add release note 2024-02-05 13:13:26 -08:00
Rebecca Turner 24205a8703 Add release note 2024-02-05 13:00:39 -08:00
Robert Hensing a31f2cb0cd Merge pull request #9939 from edolstra/slash-operator
CanonPath, SourcePath: Change operator + to /
2024-02-05 15:55:27 +01:00
Eelco Dolstra a6737b7e17 CanonPath, SourcePath: Change operator + to /
This is less confusing and makes it more similar to std::filesystem::path.
2024-02-05 15:17:39 +01:00
Eelco Dolstra c291d2d8dd Merge pull request #9927 from 9999years/catch-error-in-value-printer
Catch `Error`, not `BaseError` in `ValuePrinter`
2024-02-05 15:01:21 +01:00
Eelco Dolstra 6ec08b85f6 Merge pull request #9934 from nmeum/absPath-out-of-bounds
absPath: Explicitly check if path is empty before accessing it
2024-02-05 14:44:02 +01:00
Eelco Dolstra e190c20c33 Merge pull request #9926 from 9999years/fix-cycle-detection-in-nix-repl
Fix cycle detection in `nix repl`
2024-02-05 14:24:28 +01:00
Valentin Gagarin 8d4890c3f8 catch multiple use of link reference (#9937) 2024-02-04 21:45:10 +00:00
Valentin Gagarin 8b873edcca fix anchor link; less weird link texts (#9936) 2024-02-04 21:15:20 +00:00
Valentin Gagarin 721fddac2f use the right heading level (#9935) 2024-02-04 21:03:13 +00:00
Théophane Hufschmitt 25385a408e Merge pull request #9930 from rvl/print-dev-env-unbound-variables
print-dev-env: Avoid using unbound shellHook variable
2024-02-04 20:55:17 +01:00
pennae 5ccb06ee1b fix debugger crashing while printing envs
fixes #9932
2024-02-04 17:12:04 +01:00
Rodney Lorrimar e1131b5927 print-dev-env: Avoid using unbound shellHook variable
Some tools which consume the "nix print-dev-env" rc script (such as
"nix-direnv") are sensitive to the use of unbound variables. They use
"set -u".

The "nix print-dev-env" rc script initially unsets "shellHook", then
loads variables from the derivation, and then evaluates "shellHook".
However, most derivations don't have a "shellHook" attribute.

So users get the error "shellHook: unbound variable". This can be
demonstrated with the command:

    nix print-dev-env nixpkgs#hello | bash -u

This commit changes the rc script to provide an empty fallback value
for the "shellHook" variable.

Closes: #7951 #8253
2024-02-04 13:57:13 +08:00
Rebecca Turner 770d2bc779 Key repeated values on attribute binding pointers, not value pointers
Closes #8672
2024-02-03 21:23:06 -08:00
Rebecca Turner 9646d62b0c Don't print values in magenta
This fixes the opening bracket of lists/attrsets being printed in
magenta, unlike the closing bracket.

https://github.com/NixOS/nix/pull/9753#issuecomment-1904616088
2024-02-03 21:17:22 -08:00
Rebecca Turner c5d525cd84 Print error messages but not traces
This makes output of values that include errors much cleaner.

Before:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error:
       … while calling the 'throw' builtin
         at «string»:1:9:
            1| { err = builtins.throw "uh oh!"; }
             |         ^

       error: uh oh!»; }
```

After:
```
nix-repl> { err = builtins.throw "uh oh!"; }
{ err = «error: uh oh!»; }
```

But if the whole expression throws an error, source locations and (if
applicable) a stack trace are printed, like you'd expect:

```
nix-repl> builtins.throw "uh oh!"
error:
       … while calling the 'throw' builtin
         at «string»:1:1:
            1| builtins.throw "uh oh!"
             | ^

       error: uh oh!
```
2024-02-03 20:50:16 -08:00
Rebecca Turner a7927abdc1 Catch Error, not BaseError in ValuePrinter
`BaseError` includes `Interrupt`. We probably don't want the value
printer to tell you you pressed Ctrl-C while it was working.
2024-02-03 19:29:07 -08:00
Rebecca Turner a7939a6c2a Rename yellowtxt -> magentatxt
`yellowtxt` wraps its value with `ANSI_WARNING`, but `ANSI_WARNING` has
been equal to `ANSI_MAGENTA` for a long time. Now the name is updated.
2024-02-03 19:28:11 -08:00
Sören Tempel ec5cc1026d absPath: Explicitly check if path is empty before accessing it
It is entirely possible for the path to be an empty string and many
unit tests actually pass it as an empty string (e.g. both_roundrip or
turnsEmptyPathIntoCWD). In this case, without this patch, absPath will
perform a one-byte out-of-bounds access.

This was discovered while enabling the nix test suite on Alpine where
we compile all software with `-D_GLIBCXX_ASSERTIONS=1`, thus resulting
in a test failure on Alpine.
2024-02-04 00:47:47 +01:00
Eelco Dolstra 49cf090cb2 Merge pull request #9913 from 9999years/debugger-positions
Print positions in `--debugger`, instead of pointers
2024-02-03 09:27:09 +01:00
Rebecca Turner 6414cd259e Reduce visual clutter in the debugger 2024-02-02 19:58:35 -08:00
Rebecca Turner 36dfac7560 Expose locals from let expressions to the debugger 2024-02-02 19:31:18 -08:00
Rebecca Turner 0127d54d5e Enter debugger more reliably in let expressions and calls 2024-02-02 19:14:22 -08:00
Rebecca Turner 016db2d10f Add position information to while evaluating the attribute 2024-02-02 17:49:54 -08:00
Rebecca Turner 7d7483cafc Print positions in --debugger, instead of pointers 2024-02-02 17:38:46 -08:00
Eelco Dolstra 081dc5daa1 Merge pull request #9902 from NixOS/require-fixed-output-fetchurl
builtin:fetchurl: Ensure a fixed-output derivation
2024-02-02 14:00:34 +01:00
Eelco Dolstra e67458e5b8 Better test fix 2024-02-02 13:34:56 +01:00
Eelco Dolstra 05535be03a Fix test 2024-02-02 13:07:08 +01:00
Rebecca Turner faaccecbc8 Remove EXCEPTION_NEEDS_THROW_SPEC
We're on C++ 20 now, we don't need this
2024-02-01 16:39:38 -08:00
Rebecca Turner 87dc4bc7d1 Attach positions to errors in derivationStrict 2024-02-01 16:39:38 -08:00
Rebecca Turner c6a89c1a16 libexpr: Support structured error classes
While preparing PRs like #9753, I've had to change error messages in
dozens of code paths. It would be nice if instead of

    EvalError("expected 'boolean' but found '%1%'", showType(v))

we could write

    TypeError(v, "boolean")

or similar. Then, changing the error message could be a mechanical
refactor with the compiler pointing out places the constructor needs to
be changed, rather than the error-prone process of grepping through the
codebase. Structured errors would also help prevent the "same" error
from having multiple slightly different messages, and could be a first
step towards error codes / an error index.

This PR reworks the exception infrastructure in `libexpr` to
support exception types with different constructor signatures than
`BaseError`. Actually refactoring the exceptions to use structured data
will come in a future PR (this one is big enough already, as it has to
touch every exception in `libexpr`).

The core design is in `eval-error.hh`. Generally, errors like this:

    state.error("'%s' is not a string", getAttrPathStr())
      .debugThrow<TypeError>()

are transformed like this:

    state.error<TypeError>("'%s' is not a string", getAttrPathStr())
      .debugThrow()

The type annotation has moved from `ErrorBuilder::debugThrow` to
`EvalState::error`.
2024-02-01 16:39:38 -08:00
Rebecca Turner c62c21e29a Move PodIdx to pos-idx.hh and PosTable to pos-table.hh 2024-02-01 13:12:59 -08:00
Eelco Dolstra b8b739e484 builtin:fetchurl: Get output hash info from the drv 2024-02-01 22:01:02 +01:00
Eelco Dolstra 1ee42c5b88 builtin:fetchurl: Ensure a fixed-output derivation
Previously we didn't check that the derivation was fixed-output, so
you could use builtin:fetchurl to impurely fetch a file.
2024-02-01 21:46:01 +01:00
John Ericson dc439eaf23 Fill in missing markdown link dest 2024-02-01 11:20:19 -05:00
John Ericson 31881d651a Merge remote-tracking branch 'upstream/master' into overlayfs-store 2024-02-01 11:07:47 -05:00
Eelco Dolstra ef6d055ace Merge pull request #9896 from hercules-ci/tidy-tidy
Disable a slightly annoying clang-tidy check
2024-02-01 09:02:46 +01:00
Bryan Lai 8594f3cd5a libutil/url: fix git+file:./ parse error
Previously, the "file:./" prefix was not correctly recognized in
fixGitURL; instead, it was mistaken as a file path, which resulted in a
parsed url of the form "file://file:./".

This commit fixes the issue by properly detecting the "file:" prefix.
Note, however, that unlike "file://", the "file:./" URI is _not_
standardized, but has been widely used to referred to relative file
paths. In particular, the "git+file:./" did work for nix<=2.18, and was
broken since nix 2.19.0.

Finally, this commit fixes the issue completely for the 2.19 series, but
is still inadequate for the 2.20 series due to new behaviors from the
switch to libgit2. However, it does improve the correctness of parsing
even though it is not yet a complete solution.
2024-02-01 10:51:22 +08:00
Robert Hensing 58c26dd0f0 Add .clang-tidy 2024-02-01 01:01:39 +01:00
Robert Hensing 0f2e9e6bd2 Typo 2024-02-01 01:01:04 +01:00
Robert Hensing 4072a8fea0 Merge pull request #9867 from hercules-ci/issue-912
#912 allow leading period
2024-01-31 19:10:59 +01:00
Robert Hensing 7c9ed1b1a3 Merge pull request #9890 from obsidiansystems/remove-unneeded-toRealPath
Change an `allowPath` call to take a store path again
2024-01-31 18:45:19 +01:00
Robert Hensing f1b4663805 Disallow store path names that are . or .. (plus opt. -)
As discussed in the maintainer meeting on 2024-01-29.

Mainly this is to avoid a situation where the name is parsed and
treated as a file name, mostly to protect users.
.-* and ..-* are also considered invalid because they might strip
on that separator to remove versions. Doesn't really work, but that's
what we decided, and I won't argue with it, because .-* probably
doesn't seem to have a real world application anyway.
We do still permit a 1-character name that's just "-", which still
poses a similar risk in such a situation. We can't start disallowing
trailing -, because a non-zero number of users will need it and we've
seen how annoying and painful such a change is.

What matters most is preventing a situation where . or .. can be
injected, and to just get this done.
2024-01-31 18:35:19 +01:00
Robert Hensing 8406da2877 test: Generate distinct hashes
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing 69bbd5852a test: Generate distinct path names
Gen::just is the constant generator. Don't just return that!
2024-01-31 18:35:19 +01:00
Robert Hensing b13e6a76b4 parseStorePath: Support leading period 2024-01-31 18:35:19 +01:00
John Ericson caea7dcb7e Change an allowPath call to take a store path again
This looks like a revert of #5844, but is not.

That one was needed because
https://github.com/NixOS/nix/commit/d90f9d4b9994dc1f15b9d664ae313f06261d6058#diff-0f59bb6f197822ef9f19ceae9624989499d170c84dfdc1f486a8959bb4588cafR85
changed the type of the argument to `allowPath` from a `StorePath` to a
`Path`. But since
https://github.com/NixOS/nix/commit/caabc4f64889d5a4c47d6102b3aa1d3c80bbc107#diff-0f59bb6f197822ef9f19ceae9624989499d170c84dfdc1f486a8959bb4588cafL100-R92,
it is a `StorePath` again.

I think this is worth changing because we want to be very careful about
`toRealPath` and the evaluator --- ideally the choice of real path does
not affect evaluation at all. So using it fewer times is better.
2024-01-31 11:44:06 -05:00
John Ericson 75ebb90a70 Merge pull request #9884 from edolstra/resolve-symlinks
Resolve symlinks in a few more places
2024-01-30 10:42:54 -05:00
Eelco Dolstra d661a89398 Merge pull request #9880 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.4.1
build(deps): bump zeebe-io/backport-action from 2.4.0 to 2.4.1
2024-01-30 15:37:25 +01:00
Eelco Dolstra b36ff47e7c Resolve symlinks in a few more places
Fixes #9882.
2024-01-30 15:35:31 +01:00
Eelco Dolstra a7db7b6e44 Merge pull request #9879 from NixOS/bump-2.21.0
Bump version
2024-01-30 10:13:42 +01:00
dependabot[bot] a3aae7beef build(deps): bump zeebe-io/backport-action from 2.4.0 to 2.4.1
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-29 22:14:10 +00:00
Eelco Dolstra 2f3fb6c12e Bump version 2024-01-29 22:57:25 +01:00
Eelco Dolstra 9b8e0a7790 Merge pull request #9877 from edolstra/release-notes
Nix 2.20 release notes
2024-01-29 18:29:33 +01:00
Eelco Dolstra 652f334f87 Edit release notes 2024-01-29 17:51:21 +01:00
Eelco Dolstra 9465c8cca1 nix hash convert: Add manpage 2024-01-29 17:51:01 +01:00
Eelco Dolstra 6f86f87043 Fix formatting of hash args 2024-01-29 17:50:25 +01:00
Eelco Dolstra 3089bce41b release notes: 2.20.0 2024-01-29 17:14:17 +01:00
Eelco Dolstra 0070400809 maintainers/release-notes: Include changelog-d
Otherwise it quietly generates an empty rl-<version>.md
2024-01-29 17:13:48 +01:00
Eelco Dolstra 1ef6bbb16d Update release-process.md 2024-01-29 17:12:22 +01:00
Eelco Dolstra e9c8e859db Merge pull request #9876 from edolstra/store-settings
Don't include store docs in every manpage
2024-01-29 17:01:24 +01:00
Eelco Dolstra baff34d728 Don't include store docs in every manpage 2024-01-29 16:30:29 +01:00
Eelco Dolstra f6719032cf Shut up a gcc warning 2024-01-29 15:22:44 +01:00
Valentin Gagarin 44a0d04483 add missing link (#9869) 2024-01-29 05:56:19 +01:00
Robert Hensing 40254092dd Merge pull request #9770 from hercules-ci/refactor-rename-derivation-isPure
Refactor rename derivation type `isPure`
2024-01-27 11:24:20 +01:00
Robert Hensing 9ddd0f2af8 Revert "StorePath: reject names starting with '.'"
This reverts commit 24bda0c7b3.
2024-01-27 11:18:03 +01:00
Robert Hensing 6a99c18c30 doc/glossary: Define impure derivation 2024-01-27 11:00:10 +01:00
Robert Hensing 49b25ea85c refactor: Impure derivation type isPure -> isImpure
To quote the method doc:

Non-impure derivations can still behave impurely, to the degree permitted
by the sandbox. Hence why this method isn't `isPure`: impure derivations
are not the negation of pure derivations. Purity can not be ascertained
except by rather heavy tools.
2024-01-27 11:00:10 +01:00
John Ericson b83a2fb6dd Merge pull request #9776 from pennae/parser-refactor
Refactor the parser somewhat
2024-01-26 23:56:48 -05:00
John Ericson 365b831e6f Minor formatting tweaks 2024-01-26 23:11:31 -05:00
John Ericson 4a2444b3f3 Merge pull request #9864 from obsidiansystems/rlim_t-header
Add missing `#include` for `rlim_t`
2024-01-26 22:57:42 -05:00
John Ericson 1aec7771d4 Add missing #include for rlim_t
My local build in the shell was failing while CI was fine; not sure why
that is but having the include here is definitely more correct.

Per the POSIX spec, this is where it is supposed to be gotten
https://pubs.opengroup.org/onlinepubs/009695399/basedefs/sys/resource.h.html
2024-01-26 22:34:31 -05:00
John Ericson 1dc55c0f2f Merge pull request #9861 from 9999years/colored-diff-in-lang-tests
Color `diff` output in `tests/functional/lang` tests
2024-01-26 22:31:00 -05:00
John Ericson efb91d5979 Merge pull request #9860 from 9999years/set-stack-darwin
Increase stack size on macOS as well as Linux
2024-01-26 13:35:10 -05:00
Rebecca Turner 772897a1cd Color diff output in tests/functional/lang tests
Use `diff --color=always` to print colored output for language test
failures. I've also flipped the arguments so that expected lines missing
from the actual output will be marked with a red `-` and additional
lines found in the actual output will be marked with a green `+`.
Previously it was the other way around, which was very confusing.
2024-01-26 10:08:56 -08:00
Rebecca Turner 3a124d1e88 Increase stack size on macOS as well as Linux
The code works fine on macOS, but the default stack size we attempt to
set is larger than what my system will allow (Nix attempts to set the
stack size to 67108864, but the maximum allowed is 67092480), so I've
instead used the requested stack size or the maximum allowed, whichever
is smaller.

I've also added an error message if setting the stack size fails. It
looks like this:

> Failed to increase stack size from 8372224 to 67108864 (maximum
> allowed stack size: 67092480): Invalid argument
2024-01-26 09:40:41 -08:00
Valentin Gagarin 30bdee5c3b update docs on fetchGit shallow clone behavior (#9704) 2024-01-26 17:26:08 +00:00
Théophane Hufschmitt 8df68a213f Merge pull request #9849 from 13x1/patch-1
Fix typo in primops.cc (and therefore Nix docs)
2024-01-25 16:11:49 +01:00
Théophane Hufschmitt 5a9513cdba Merge pull request #9848 from obsidiansystems/default-system-features-static
Make `StoreConfig::getDefaultSystemFeatures` a static method
2024-01-25 16:05:57 +01:00
lexi 08f38a3a40 Fix typo in primops.cc (and therefore Nix docs)
This also fixes the typo in the Nix docs at https://nixos.org/manual/nix/unstable/language/builtins.
2024-01-25 15:30:51 +01:00
John Ericson a9e10a1dbd Make StoreConfig::getDefaultSystemFeatures a static method
This makes something in Hydra bit simpler. If someday the default
depends on the other config options, we can always change it back.
2024-01-24 21:34:31 -05:00
John Ericson 979b00bce9 Merge pull request #9844 from NixOS/pkg-config-gmock
Link both gmock and gtest, not just gtest
2024-01-24 19:33:54 -05:00
John Ericson c817305411 Link both gmock and gtest, not just gtest
GMock is not entirely header-only, we're finding.
2024-01-24 10:10:42 -05:00
Robert Hensing 212ba69e6f Merge pull request #9843 from hercules-ci/test-pr-9626
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
2024-01-24 13:55:41 +01:00
Robert Hensing 6532dd50fc tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message
Follow-up for https://github.com/NixOS/nix/pull/9626
176dcd5c61
2024-01-24 13:19:02 +01:00
Robert Hensing 5b7bfd2d6b Merge pull request #9754 from 9999years/print-value-when-coercion-fails
Print the value in `error: cannot coerce` messages
2024-01-24 12:48:39 +01:00
Théophane Hufschmitt 69d0ae27e3 Merge pull request #9841 from obsidiansystems/float-speed-factor
Convert `Machine::speedFactor` from a non-neg int to a non-neg float
2024-01-24 11:28:54 +01:00
John Ericson 1e24db6f9a Convert Machine::speedFactor from a non-neg int to a non-neg float
The short motivation is to match Hydra, so we can de-dup.

The long version is layed out in
https://github.com/NixOS/nix/issues/9840.
2024-01-24 01:03:07 -05:00
tomberek 1c260fa6d1 Merge pull request #9481 from iFreilicht/disallow-nix-search-without-search-terms
nix search: Disallow empty regex
2024-01-23 20:59:52 -05:00
tomberek 775d59f1fa Merge pull request #8893 from 9999years/fix-8882
Log what `nix flake check` does
2024-01-23 20:38:23 -05:00
Rebecca Turner 83bb494a30 Print the value in error: cannot coerce messages
This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.
2024-01-23 15:15:41 -08:00
Robert Hensing f1b0304153 Merge pull request #9838 from obsidiansystems/systemTypes-set
Make `Machine::systemTypes` a set not vector
2024-01-23 21:06:19 +01:00
John Ericson 0aa85088de Factor out Machine::systemSupported
There's just enough logic (the `"builtin"` special case) that makes this
worthy of its own method.
2024-01-23 12:53:51 -05:00
John Ericson 870acc2892 Add API docs to Machine methods 2024-01-23 12:50:48 -05:00
John Ericson 739032762a Make Machine::systemTypes a set not vector
This is more conceptually correct (the order does not matter), and also
matches what Hydra already does.

(Nix and Hydra matching is needed for dedup
https://github.com/NixOS/hydra/issues/1164)
2024-01-23 12:30:26 -05:00
Eelco Dolstra b6aee9a93f Merge pull request #9833 from obsidiansystems/ssh-arg-split
Make `SSHMaster::startCommand` work on an args list
2024-01-23 14:34:41 +01:00
Eelco Dolstra d210252f77 Merge pull request #9832 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.4.0
build(deps): bump zeebe-io/backport-action from 2.3.0 to 2.4.0
2024-01-23 14:32:16 +01:00
Robert Hensing 08bf2846df Merge pull request #9653 from obsidiansystems/improve-parse-sink
Improve the `ParseSink` interface
2024-01-23 01:04:57 +01:00
Robert Hensing 50e5d7b883 Merge pull request #9818 from Ma27/print-value-on-function-call-type-error
libexpr: print value of what is attempted to be called as function
2024-01-23 00:34:38 +01:00
John Ericson dcdc8db833 Merge pull request #9651 from NixOS/mention-assigned-review-in-handbook
maintainers: Mention the monthly “Assigned” column review
2024-01-22 18:33:05 -05:00
John EricsonandRobert Hensing 6365bbfa81 Improve the FileSystemObjectSink interface
More invariants are enforced in the type, and less state needs to be
stored in the main sink itself. The method here is roughly that known as
"session types".

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:21 -05:00
John EricsonandRobert Hensing 966d6fcd01 ParseSink -> FileSystemObjectSink
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 18:01:18 -05:00
John EricsonandAlexander Bantyev b71673109c Make SSHMaster::startCommand work on an args list
This avoids split-on-whitespace errors:

- No more `bash -c` needed

- No more `shellEscape` needed

- `remote-program` ssh store setting also cleanly supports args (e.g.
  `nix daemon`)

- `ssh` uses `--` to separate args for SSH from args for the command to
  run.

and will help with Hydra dedup.

Some code taken from #6628.

Co-Authored-By: Alexander Bantyev <balsoft@balsoft.ru>
2024-01-22 17:46:57 -05:00
dependabot[bot] 9a51209309 build(deps): bump zeebe-io/backport-action from 2.3.0 to 2.4.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.3.0...v2.4.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-22 22:40:01 +00:00
Maximilian Bosch 81499a0b93 libexpr: print value of what is attempted to be called as function
Low-hanging fruit in the spirit of #9753 and #9754 (means 9999years did
all the hard work already).

This basically prints out what was attempted to be called as function,
i.e.

  map (import <nixpkgs> {}) [ 1 2 3 ]

now gives the following error message:

    error:
           … while calling the 'map' builtin
             at «string»:1:1:
                1| map (import <nixpkgs> {}) [ 1 2 3 ]
                 | ^

           … while evaluating the first argument passed to builtins.map

           error: expected a function but found a set: { _type = "pkgs"; AAAAAASomeThingsFailToEvaluate = «thunk»; AMB-plugins = «thunk»; ArchiSteamFarm = «thunk»; BeatSaberModManager = «thunk»; CHOWTapeModel = «thunk»; ChowCentaur = «thunk»; ChowKick = «thunk»; ChowPhaser = «thunk»; CoinMP = «thunk»;  «18783 attributes elided»}
2024-01-22 22:41:42 +01:00
Robert Hensing 5f72a97092 Merge pull request #9753 from 9999years/print-value-on-type-error
Print the value in `value is X while a Y is expected` error
2024-01-22 22:18:16 +01:00
Robert Hensing 74534829f2 Merge pull request #9830 from hercules-ci/test-cross-version-remote-build
tests/nixos: Test remote build against older versions
2024-01-22 19:42:57 +01:00
John Ericson 71bf592381 Merge pull request #6134 from obsidiansystems/expose-proto-rawer
Factoring out parts of the serve protocol for Hydra to share
2024-01-22 13:38:50 -05:00
Robert Hensing c4d7c4a848 nixos/tests/remote-builds*: Format
nixpkgs-fmt
2024-01-22 18:47:59 +01:00
Robert Hensing 5167351efb tests/nixos/remote-builds*: Inline module + format 2024-01-22 18:44:16 +01:00
John EricsonandRobert Hensing 1fb2582969 Create unit tests for the serve proto handshake
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:11 -05:00
John EricsonandRobert Hensing e960b28230 Factor our ServeProto::BasicServerConnection::handshake
We'll need this for unit testing.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:11 -05:00
John Ericson 4a5ca576da Factor out ServeProto::BasicClientConnection::handshake Hydra to share 2024-01-22 12:43:11 -05:00
John Ericson 4580bed3e4 LegacySSHStore::openConnection move more logic inside catch block
Broader error handling logic is more robust.
2024-01-22 12:43:11 -05:00
John EricsonandRobert Hensing ce2f714e6d Start factoring out the serve protocol for Hydra to share
Factor out `ServeProto::BasicClientConnection` for Hydra to share

- `queryValidPaths`: Hydra uses the lock argument differently than Nix,
  so we un-hard-code it.

- `buildDerivationRequest`: Just the request half, as Hydra does some
  things between requesting and responding.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 12:43:03 -05:00
Robert Hensing e502d1cf94 tests/nixos: Test remote build against older versions 2024-01-22 18:39:15 +01:00
John Ericson 50ce3832dc Merge pull request #9512 from hercules-ci/release-notes-bdwgc-traceable-allocator-patch
prerequisites-source: Add bdwgc-traceable-allocator-patch
2024-01-22 11:58:43 -05:00
Rebecca Turner cb7fbd4d83 Print value on type error
Adds the failing value to `value is <TYPE> while a <TYPE> is expected`
error messages.
2024-01-22 08:56:02 -08:00
John Ericson b74ec56da9 Merge pull request #9829 from obsidiansystems/fix-if-if-else-ambiguity
If `if`...`if`...`else` ambiguity
2024-01-22 10:58:46 -05:00
John Ericson 316e50cc7c Fix if...if...else ambiguity
This can be parsed two ways. Add a pair of braces so it must be parsed
the intended way.
2024-01-22 10:33:40 -05:00
John Ericson 176dcd5c61 Merge pull request #9626 from jvns/locked-input
Make fetchTree locked input error message clearer
2024-01-22 09:31:59 -05:00
pennaeandJohn Ericson 80b84710b8 Update src/libexpr/eval.cc
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-01-22 15:15:53 +01:00
Eelco Dolstra 16ccca2e86 Merge pull request #9703 from fricklerhandwerk/upgrade-no-channel
don't show `nix-channel` in upgrade instructions
2024-01-22 14:46:47 +01:00
John EricsonandRobert Hensing 0bcdb4f5f0 Elaborate what the monthly assignments status check entails
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-01-22 08:38:52 -05:00
John Ericson b2066c3574 Merge pull request #9521 from fricklerhandwerk/conf-reword-max-jobs
reword description of the `max-jobs` setting
2024-01-22 08:29:12 -05:00
John Ericson a0b86b3160 Merge pull request #9760 from hercules-ci/doc-glossary-nix-expression
doc/glossary: Nix expression can be language expression
2024-01-22 08:26:01 -05:00
Théophane Hufschmitt c8d33bca8d Merge pull request #9822 from obsidiansystems/algo-vs-hash-algo
Start standardizing hash algo flags
2024-01-22 11:08:24 +01:00
Théophane Hufschmitt f51409cf98 Merge pull request #9805 from yshui/prefetch-unpack
Add --unpack to nix store prefetch-file
2024-01-22 10:51:50 +01:00
John Ericson da1aae2d06 Merge pull request #9821 from obsidiansystems/fix-typo
Fix typo in upcomming release notes
2024-01-20 18:03:25 -05:00
John Ericson 202c5e2afc Start standardizing hash algo flags
Do this if we want to do `--hash-algo` everywhere, and not `--algo` for
hash commands.

The new `nix hash convert` is updated. Deprecated new CLI commands are
left as-is (`nix hash path` needs to be redone and is also left as-is).
2024-01-20 17:29:35 -05:00
John Ericson a93b204c27 Merge pull request #9348 from obsidiansystems/json-formats
Document JSON formats
2024-01-20 17:22:39 -05:00
John Ericson 65294fe5fe Fix typo in upcomming release notes
Thanks @cole-h for finding in https://github.com/NixOS/nix/pull/9815#discussion_r1460604130
2024-01-20 17:07:21 -05:00
John EricsonandValentin Gagarin edf3ecc497 Document JSON formats
Good to document these formats separately from commands that happen to
use them.

Eventually I would like this and `builtins.derivation` to refer to a
store section on derivations that is authoritative, but that doesn't yet
exist, and will take some time to make. So I think we're just best off
merging this now as is.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-20 17:03:47 -05:00
Rebecca Turner 561a56cd13 Add release notes 2024-01-20 00:04:06 -08:00
Rebecca Turner d75a5f427a Print how many checks are run 2024-01-20 00:04:06 -08:00
Rebecca Turner 9404ce36e4 Print derivation paths
Also be more consistent with quotes around attribute paths
2024-01-20 00:04:05 -08:00
Rebecca Turner 49221493e2 Log what nix flake check does
There's still room for improvement, but this produces much more
informative output with `-v`:

```
$ nix flake check -v
evaluating flake...
checking flake output 'checks'...
checking derivation checks.aarch64-darwin.ghcid-ng-tests...
checking derivation checks.aarch64-darwin.ghcid-ng-clippy...
checking derivation checks.aarch64-darwin.ghcid-ng-doc...
checking derivation checks.aarch64-darwin.ghcid-ng-fmt...
checking derivation checks.aarch64-darwin.ghcid-ng-audit...
checking flake output 'packages'...
checking derivation packages.aarch64-darwin.ghcid-ng...
checking derivation packages.aarch64-darwin.ghcid-ng-tests...
checking derivation packages.aarch64-darwin.default...
checking flake output 'apps'...
checking flake output 'devShells'...
checking derivation devShells.aarch64-darwin.default...
running flake checks...
warning: The check omitted these incompatible systems: aarch64-linux, x86_64-darwin, x86_64-linux
Use '--all-systems' to check all.
```
2024-01-20 00:02:35 -08:00
John Ericson 9b896bf7e9 Merge pull request #9809 from obsidiansystems/nix-store-add-algo
Add missing `--hash-algo` flag to `nix store add`
2024-01-19 23:37:11 -05:00
John Ericson 356352c370 Add missing --hash-algo flag to nix store add 2024-01-19 23:11:18 -05:00
Robert Hensing 382fa51ff0 Merge pull request #9794 from hercules-ci/queryPathInfoFromClientCache
refactor: Extract `Store::queryPathInfoFromClientCache`
2024-01-19 17:33:52 +01:00
Robert Hensing d19627e8b4 refactor: Remove throw from queryPathInfoFromClientCache
Return a value instead of throwing.

Rather than the more trivial refactor of wrapping the return value in
another std::optional, we retain the meaning of the outer optional:
"we know at least something."

So we have changed:
return nullopt    -> return nullopt
throw InvalidPath -> return make_optional(nullptr)
return vpi        -> return make_optional(vpi)
2024-01-19 17:01:19 +01:00
Robert Hensing 8983ee8b2e refactor: Un-callback transform Store::queryPathInfoFromClientCache
This part of the code was not necessarily callback based.
Removing CPS is always nice; particularly if there's no loss of
functionality, like here.
2024-01-19 17:01:19 +01:00
Yuxuan Shui 75a6e6dd0e Add --unpack to nix store prefetch-file 2024-01-19 15:53:45 +00:00
Eelco Dolstra e6e160a075 Merge pull request #9632 from cole-h/nix-daemon-testing
Use `nix daemon` in the test suite
2024-01-19 16:21:51 +01:00
Théophane Hufschmitt 10165c7593 Merge pull request #9811 from DavHau/fix-git-auth
fetchTree/fetchGit: re-enable shallow fetching
2024-01-19 16:14:41 +01:00
John Ericson 57d6284eb7 Merge pull request #9812 from NixOS/remove-n-shorthand-for---mode
Remove a nonsensical shorthand flag in `nix store add`
2024-01-19 08:34:43 -05:00
DavHau bc00fa4647 fetchTree/fetchGit: re-enable shallow fetching
Add several tests for git fetching:
- shallow-cache-separation: can fetch the same repo shallowly and non-shallowly
- shallow-ignore-ref: ensure that ref gets ignored when shallow=true is set
- ssh-shallow: can fetch a git repo via ssh using shallow=1
2024-01-19 20:30:47 +07:00
Robert Hensing d762caff46 Merge pull request #9806 from hercules-ci/fix-git-auth
Fix git auth
2024-01-19 13:47:52 +01:00
Théophane Hufschmitt 28d7db249a Remove a nonsensical shorthand flag in nix store add
`-n` was an alias for `--mode`, but that seems to just be a copy-paste error as it doesn't make sense.
`--mode` probably doesn't need a shorthand flag at all, so remove it.

Noticed in https://github.com/NixOS/nix/pull/9809#issuecomment-1899890555
2024-01-19 10:10:00 +01:00
Eelco Dolstra 3b20cca962 Merge pull request #9804 from edolstra/missing-nar-crash
Fix crash when NAR is missing from binary cache
2024-01-19 09:38:53 +01:00
John Ericson a34ec0bd12 Include store path exact spec in the docs
This is niche, but deserves to be in the manual because it is describing
behavior visible to the outside world, not mere implementation details.
2024-01-18 22:19:14 -05:00
Robert Hensing 4e8483d09d Merge pull request #9792 from fricklerhandwerk/move-section
move section on make variables
2024-01-19 00:55:33 +01:00
Robert HensingandValentin Gagarin 14f470ec4e doc/hacking.md: Hint short option make -e
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-19 00:32:30 +01:00
Robert Hensing 346d513d86 tests/nixos/fetch-git: Add http-auth test 2024-01-18 22:34:38 +01:00
Robert HensingandEelco Dolstra 8d422c2fef Revert libgit2 fetching
libgit2 is not capable of using git-credentials helpers yet.
This prevents private repositories from being used.

Based on code that was replaced in https://github.com/NixOS/nix/pull/9240
(Introduce libgit2); hence:

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-01-18 22:29:26 +01:00
Robert Hensing 76a50b3a69 doc: GitRepoImpl::path 2024-01-18 22:29:26 +01:00
Robert Hensing ed975e953c tests/nixos/fetch-git: Testsupport for private repos 2024-01-18 22:29:26 +01:00
Robert Hensing 0bd9e10aea Merge pull request #9676 from DavHau/git-testsuite
initialize test suite for git fetchers
2024-01-18 21:51:48 +01:00
Robert Hensing 1254170405 tests/nixos/fetch-git: Make the store paths unique 2024-01-18 21:00:40 +01:00
Robert Hensing 94eba0ebbb tests/nixos/fetch-git: Memoize -> save
Memoization is for thunk-like behavior whereas this is executed eagerly.
2024-01-18 21:00:40 +01:00
Robert Hensing fd41979d78 tests/nixos/fetch-git: Factor out gitea repo module 2024-01-18 21:00:40 +01:00
Eelco Dolstra dca0a80240 copyStorePath(): Bail out early if the store path already exists
In rare cases (e.g. when using allowSubstitutes = false), it's
possible that we simultaneously have a DerivationGoal *and* a
SubstitutionGoal building the same path. So if a DerivationGoal
already built the path while the SubstitutionGoal was waiting for a
download slot, it saves us a superfluous download to exit early.
2024-01-18 17:19:15 +01:00
Eelco Dolstra a18d8d688a LocalStore::addToStore(): Ignore exceptions from parseDump()
In the "discard" case (i.e. when the store path already exists
locally), when we call parseDump() from a Finally and it throws an
exception (e.g. if the download of the NAR fails), Nix crashes:

   terminate called after throwing an instance of 'nix::SubstituteGone'
     what():  error: file 'nar/06br3254rx4gz4cvjzxlv028jrx80zg5i4jr62vjmn416dqihgr7.nar.xz' does not exist in binary cache 'http://localhost'
   Aborted (core dumped)
2024-01-18 17:19:15 +01:00
Eelco Dolstra ab786e22f1 Show what goal is waiting for a build slot 2024-01-18 17:19:15 +01:00
Théophane Hufschmitt e652322a61 Merge pull request #9799 from obsidiansystems/push-add-to-store-from-dump-unsupported-down
Push `addToStoreFromDump` `unsupported(...)` down `Store` class hierarchy
2024-01-18 16:54:44 +01:00
Eelco Dolstra 32706b14a7 Merge pull request #9798 from edolstra/remote-store-eof
Print a more helpful message if the daemon crashes
2024-01-18 15:47:59 +01:00
Eelco Dolstra 1fe8f54bd3 Use BackedStringView 2024-01-18 15:27:57 +01:00
Robert Hensing 78074bdea4 tests/nixos/fetch-git: Apply suggestions 2024-01-18 12:53:53 +01:00
Théophane Hufschmitt 28674247ec Merge pull request #9787 from obsidiansystems/bind-proc-syserror
`bind`: give same treatment as `connect` in #8544, dedup
2024-01-18 09:34:15 +01:00
John Ericson 574db83504 Push addToStoreFromDump unsupported(...) down Store class hierarchy
Instead of having it be the default method in `Store` itself, have it be
the implementation in `DummyStore` and `LegacySSHStore`. Then just the
implementations which fail to provide the method pay the "penalty" of
dealing with the icky `unimplemented` function for non-compliance.

Picks up where #8217. Getting close to no `unsupported` in the `Store`
interface itself!

More progress on issue #5729.
2024-01-17 23:46:03 -05:00
John Ericson 3016e67c21 bind: give same treatment as connect in #8544, dedup
It is good to propagate the underlying error so whether or not we use a
process to deal with path length issues is not observable.

Also, as these wrapper functions got more and more complex, the code
duplication got worse and worse. The new `bindConnectProcHelper`
function deduplicates them.
2024-01-17 23:35:08 -05:00
Eelco Dolstra a3cf27ca47 Print a more helpful message if the daemon crashes
Instead of

   error: unexpected end-of-file

you now get

   error: Nix daemon disconnected unexpectedly (maybe it crashed?)
2024-01-17 22:54:02 +01:00
John Ericson b5ed36e663 Merge pull request #9797 from edolstra/fix-error-message
Store::buildPaths(): Fix display of store paths
2024-01-17 16:11:21 -05:00
Eelco Dolstra 39ab50f9ee Store::buildPaths(): Fix display of store paths
This was broken in 7ac39ff05c.
2024-01-17 21:41:37 +01:00
Robert Hensing 1d89c7b02e Merge pull request #9796 from obsidiansystems/cleanup-dep-patches
Move dependency patches from top level into subdir
2024-01-17 20:06:27 +01:00
John Ericson 799f2d0699 Merge pull request #9795 from Ericson2314/flake-fix-indent-eror
Fix indentation error in `flake.nix`
2024-01-17 13:47:15 -05:00
John Ericson 1de8eed28a Move dependency patches from top level into subdir
Good to not clutter the top-level directory.
2024-01-17 13:13:54 -05:00
John Ericson e938912cff Fix indentation error in flake.nix 2024-01-17 13:08:03 -05:00
Robert Hensing d0a284284b refactor: Extract simply, awkwardly Store::queryPathInfoFromClientCache
This is useful for determining quickly which substituters to query.

An alternative would be for users to invoke the narinfo cache db directly,
so why do we need this change?

 - It is easier to use. I believe Nix itself should also use it.

 - This way, the narinfo cache db remains an implementation detail.

 - Callers get to use the in-memory cache as well.
2024-01-17 17:54:19 +01:00
Théophane Hufschmitt 2cb9c7c681 Merge pull request #9791 from fricklerhandwerk/typo
fix typo
2024-01-17 09:08:22 +01:00
Valentin Gagarin 28eb406834 reword section on make variables
- use one line per sentence
- use imperative for instructions
- add link to Make documentation
2024-01-17 04:39:26 +01:00
Valentin Gagarin f134dbdffb move section on make variables
it should be after the general build instructions, as it goes into more detail.
2024-01-17 04:23:16 +01:00
Valentin Gagarin 55da939424 fix typo 2024-01-17 02:57:11 +01:00
Robert Hensing 2a3c5e6b8b Merge pull request #9480 from NixOS/libfetchers-git-exportIgnore
libfetchers/git: Support export-ignore
2024-01-16 23:03:46 +01:00
Robert Hensing 01271f2996 Merge pull request #9786 from hercules-ci/package
DerivationInfo -> PackageInfo
2024-01-16 22:43:14 +01:00
Robert Hensing 51f524c629 Merge pull request #9755 from 9999years/printer-followup
Printer followup
2024-01-16 22:35:08 +01:00
John Ericson 799e662cbe Merge pull request #9279 from tfc/util-improv
Util improvements
2024-01-16 15:05:28 -05:00
Jacek Galowicz 8ae3aeec94 Don't use std::make_unique right before release 2024-01-16 17:00:46 +00:00
Jacek Galowicz 9d9f42cc38 Remove C-style casts 2024-01-16 17:00:46 +00:00
Jacek Galowicz d11d7849f7 Use ChildWrapperFunction type and make casts more explicit 2024-01-16 17:00:46 +00:00
Jacek Galowicz c924147c9d Drop parentheses from thunks 2024-01-16 17:00:46 +00:00
Jacek Galowicz 1885d579db Improve String Handling 2024-01-16 17:00:46 +00:00
Jacek Galowicz 0bc66e529f Use npos member variables instead of full type 2024-01-16 15:17:57 +00:00
Eelco Dolstra 7115edc85a Merge pull request #8544 from edolstra/handle-missing-gc-socket
LocalStore::addTempRoot(): Handle ENOENT
2024-01-16 16:05:58 +01:00
Robert Hensing ea6aa5ffd8 Package{,Info}: comments 2024-01-16 15:44:02 +01:00
Robert Hensing 85a1cd9bd5 Merge pull request #9762 from hercules-ci/doc-glossary-package
doc/glossary: Define package and package attribute set
2024-01-16 15:34:59 +01:00
Robert Hensing 65255edc9b DerivationInfo -> PackageInfo
This does not yet resolve the coupling between packages and
derivations, but it makes the code more consistent with the
terminology, and it accentuates places where the coupling is
obvious, such as

         auto drvPath = packageInfo.queryDrvPath();
         if (!drvPath)
             throw Error("'%s' is not a derivation", what());

... which isn't wrong, and in my opinion, doesn't even look
wrong, because it just reflects the current logic.
However, I do like that we can now start to see in the code that
this coupling is perhaps a bit arbitrary.
After this rename, we can bring the DerivingPath concept into type
and start to lift this limitation.
2024-01-16 15:28:03 +01:00
Eelco Dolstra d005bade7f connect(): Propagate errno from the child process
This is necessary on macOS since addTempRoot() relies on errno.
2024-01-16 15:26:17 +01:00
Eelco Dolstra 0b1d93d2ba Sleep a bit between attempts to connect to the root server 2024-01-16 15:26:17 +01:00
Robert HensingandValentin Gagarin baea5f42c6 doc/glossary: Simplify software package definition
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-16 14:50:53 +01:00
Eelco Dolstra 302625e83b Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-16 13:18:58 +01:00
Eelco Dolstra 86156d05dd Merge pull request #9785 from hercules-ci/test-substring-negative
tests/functional/lang: Test substring with negative length
2024-01-16 13:08:07 +01:00
Eelco Dolstra 7e072b6b1e Merge pull request #9781 from NixOS/dependabot/github_actions/cachix/cachix-action-14
Bump cachix/cachix-action from 13 to 14
2024-01-16 13:05:43 +01:00
Eelco Dolstra df1a931a45 Merge pull request #9780 from NixOS/dependabot/github_actions/cachix/install-nix-action-25
Bump cachix/install-nix-action from 24 to 25
2024-01-16 13:05:19 +01:00
Robert Hensing cbc319e9be tests/functional/lang: Test substring with negative length 2024-01-16 12:19:31 +01:00
Robert Hensing b2deff1947 Merge pull request #9747 from awakesecurity/mz/fix-quadratic-splitString
Fix performance of builtins.substring for empty substrings
2024-01-16 12:18:59 +01:00
dependabot[bot] bf7754c099 Bump cachix/cachix-action from 13 to 14
Bumps [cachix/cachix-action](https://github.com/cachix/cachix-action) from 13 to 14.
- [Release notes](https://github.com/cachix/cachix-action/releases)
- [Commits](https://github.com/cachix/cachix-action/compare/v13...v14)

---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-15 22:35:15 +00:00
dependabot[bot] e0a7643086 Bump cachix/install-nix-action from 24 to 25
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 24 to 25.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v24...v25)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-15 22:35:12 +00:00
pennae 09a1128d9e don't repeatedly look up ast internal symbols
these symbols are used a *lot*, so it makes sense to cache them. this
mostly increases clarity of the code (however clear one may wish to call
the parser desugaring here), but it also provides a small performance
benefit.
2024-01-15 16:52:18 +01:00
pennae b596cc9e79 decouple parser and EvalState
there's no reason the parser itself should be doing semantic analysis
like bindVars. split this bit apart (retaining the previous name in
EvalState) and have the parser really do *only* parsing, decoupled from
EvalState.
2024-01-15 16:52:18 +01:00
pennae e1aa585964 slim down parser.y
most EvalState and Expr members defined here could be elsewhere, where
they'd be easier to maintain (not being embedded in a file with arcane
syntax) and *somewhat* more faithfully placed according to the path of
the file they're defined in.
2024-01-15 16:52:18 +01:00
pennae 835a6c7bcf rename ParserState::{makeCurPos -> at}
most instances of this being used do not refer to the "current"
position, sometimes not even to one reasonably close by. it could also
be called `makePos` instead, but `at` seems clear in context.
2024-01-15 16:52:18 +01:00
pennae 0076056164 move ParseData to own header, rename to ParserState
ParserState better describes what this struct really is. the parser
really does modify its state (most notably position and symbol tables),
so calling it that rather than obliquely "data" (which implies being
input only) makes sense.
2024-01-15 16:52:18 +01:00
pennae 1b09b80afa make parser utility functions members of ParseData
all of them need access to parser state in some way. make them members
to allow this without fussing so much.
2024-01-15 16:52:18 +01:00
pennae e8d9de967f simplify parse error reporting
since nix doesn't use the bison `error` terminal anywhere any invocation
of yyerror will immediately cause a failure. since we're *already*
leaking tons of memory whatever little bit bison allocates internally
doesn't much matter any more, and we'll be replacing the parser soon anyway.

coincidentally this now also matches the error behavior of URIs when
they are disabled or ~/ paths in pure eval mode, duplicate attr
detection etc.
2024-01-15 16:52:18 +01:00
pennae f07388bf98 remove ParserFormals
this is a proper subset of Formals anyway, so let's just use those and
avoid the extra allocations and moves.
2024-01-15 16:52:18 +01:00
John Ericson 8a2da82220 Merge pull request #9759 from obsidiansystems/abs-path-string-view
`absPath`: just take a `std::string_view`
2024-01-15 10:22:20 -05:00
Robert Hensing eeaa188973 Merge pull request #9748 from obsidiansystems/no-rapidcheck-headers
Simplify RapidCheck configure
2024-01-15 14:52:14 +01:00
John EricsonandCole Helbling beed00c04e absPath: just take a std::string_view
1. Slightly more efficient

2. Easier to call

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2024-01-15 08:21:54 -05:00
John Ericson 9b9ecdee34 Simplify RapidCheck configure
No more `RAPIDCHECK_HEADERS`!
2024-01-15 08:05:05 -05:00
John Ericson dd42a4e3e9 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/86501af7f1d51915e6c335f90f2cab73d7704ef3' (2024-01-11)
  → 'github:NixOS/nixpkgs/a1982c92d8980a0114372973cbdfe0a307f1bdea' (2024-01-12)
2024-01-15 08:04:46 -05:00
Robert Hensing 268c49264a Merge pull request #9763 from L-as/avoid-unnecessary-copy
Avoid unnecessary copy of goal log
2024-01-15 08:32:03 +01:00
John Ericson 8d4b67912d Merge pull request #9768 from obsidiansystems/nix-profile-test-no-outlink
`tests/functional/nix-profile.sh`: Add missing `--no-link`
2024-01-14 16:11:42 -05:00
John Ericson 03a6ca9b25 tests/functional/nix-profile.sh: Add missing --no-link
Otherwise we get a stray `tests/functional/result`, which can cause
spurious failures later.

(I got a failure because the test temp dir effecting the store dir
changed. This caused a test later because Nix didn't want to remove the
old `result` because it wasn't pointing inside the new Nix store.)
2024-01-14 15:27:44 -05:00
John Ericson 7ab0a4c39d Merge pull request #9757 from obsidiansystems/parser-dodge-windows-macros
Avoid Windows macros in the parser and lexer
2024-01-14 13:36:13 -05:00
Las Safin f61d951909 Avoid unnecessary copy of goal log
The data was (accidentally?) copied into a std::string,
even though the string is immediately converted into a std::string_view.
The code has been changed to construct a std::string_view directly,
such that one copy less happens.
2024-01-13 19:32:37 +00:00
Cole Helbling bbcd9fcfc1 Arbitrarily bring back some nix-daemon calls
This means that both `nix daemon` and `nix-daemon` will be (somewhat)
tested.
2024-01-13 11:27:04 -08:00
Robert Hensing 7e5fa5c25c doc/glossary: Define package and package attribute set
A small step towards https://github.com/NixOS/nix/issues/6507

I believe this incomplete definition is one that can be agreed on.
It would be nice to define more, but considering that the issue
also proposes changes to the design, I believe we should hold off
on those.

As for the wording, we're dealing with some very general and vague
terms, that have to be treated with exactly the right amount of
vagueness to be effective.

I start out with a fairly abstract definition of package.
1. to establish a baseline so we know what we're talking about
2. so that we can go in and clarify that we have an extra, Nix-specific
   definition.

"Software" is notoriously ill-defined, so it makes a great qualifier
for package, which we don't really want to pin down either, because
that would just get us lost in discussion.
We can come back to this after we've done 6057 and a few years in a
desert cave.

Then comes the "package attribute set" definition.
I can already hear Valentin say "That's not even Nix's responsibility!"
and on some days I might even agree.
However, in our current reality, we have `nix-env`, `nix-build` and
`nix profile`, which query the `outputName` attribute - among others -
which just don't exist in the derivation.

For those who can't believe what they're reading:

    $ nix-build --expr 'with import ./. {}; bind // {outputName = "lib";}' --no-out-link
    this path will be fetched (1.16 MiB download, 3.72 MiB unpacked):
      /nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib
    copying path '/nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib' from 'https://cache.nixos.org'...
    /nix/store/rfk6klfx3z972gavxlw6iypnj6j806ma-bind-9.18.21-lib

and let me tell you that bind is not a library.

So anyway, that's also proof of why calling this a "derivation attrset" would be wrong, despite the type attribute.
2024-01-13 20:00:06 +01:00
Robert Hensing e838ac98d4 doc/glossary: Nix expression can be language expression 2024-01-13 19:42:05 +01:00
Qyriad cbd5553d57 doc: provide context in glossary definitions (#9378) 2024-01-13 12:20:08 +01:00
John Ericson e739a5002d Avoid Windows macros in the parser and lexer
`FLOAT`, `INT`, and `IN` are identifers taken by macros.

The name `IN_KW` is chosen to match `OR_KW`, which is presumably named
that way for the same reason of dodging macros.
2024-01-12 19:51:36 -05:00
John Ericson 381df7b9c9 Merge pull request #9756 from obsidiansystems/readline-on-windows
Newer Nixpkgs, get `readline` on Windows
2024-01-12 19:49:50 -05:00
John Ericson dd7e7b0a30 Newer Nixpkgs, get readline on Windows
Now `nix repl` an, in principle, work on that platform too.

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/2c9c58e98243930f8cb70387934daa4bc8b00373' (2023-12-31)
  → 'github:NixOS/nixpkgs/86501af7f1d51915e6c335f90f2cab73d7704ef3' (2024-01-11)
2024-01-12 19:19:21 -05:00
John Ericson 3bc1f8f22f Merge pull request #9752 from 9999years/hacking-filter-tests
Document unit tests in hacking.md
2024-01-12 19:00:19 -05:00
Robert Hensing 15f7bdaf27 CanonPath: Add rel_c_str()
Defensively because isRoot() is also defensive.
2024-01-12 22:55:37 +01:00
Rebecca Turner f73e50144f Clarify ambiguity in nix-instantiate man page 2024-01-12 13:53:45 -08:00
Rebecca Turner 2d96c7a51f Remove outdated reference to y in nix-instantiate man page 2024-01-12 11:27:31 -08:00
Rebecca Turner b29be1ff57 Document unit tests in hacking.md 2024-01-12 10:47:49 -08:00
John Ericson c58da62a06 Merge pull request #9737 from obsidiansystems/sys-error-split
Separate `SystemError` from `SysError`
2024-01-12 12:41:36 -05:00
Mel Zuser 1996105e91 added test for empty substring special case 2024-01-12 09:40:21 -08:00
John Ericson 6208ca7209 Separate SystemError from SysError
Most of this is a `catch SysError` -> `catch SystemError` sed. This
is a rather pure-churn change I would like to get out of the way. **The
intersting part is `src/libutil/error.hh`.**

On Unix, we will only throw the `SysError` concrete class, which has
the same constructors that `SystemError` used to have.

On Windows, we will throw `WinError` *and* `SysError`. `WinError`
(which will be created in a later PR), will use a `DWORD` instead of
`int` error value, and `GetLastError()`, which is the Windows equivalent
of the `errno` machinery. Windows will *also* use `SysError` because
Window's "libc" (MSVCRT) implements the POSIX interface, and we use it
too.

As the docs describe, while we *throw* one of the 3 choices above (2
concrete classes or the alias), we should always *catch* `SystemError`.
This ensures no matter how the implementation changes for Windows (e.g.
between `SysError` and `WinError`) the catching logic stays the same
and stays correct.

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Eugene Butler <eugene@eugene4.com>
2024-01-12 12:00:33 -05:00
Mel Zuser 25c889baac Fix performance of builtins.substring for empty substrings
When returning a 0-length substring, avoid calling coerceToString,
since it returns a string_view with the string's length, which is
expensive to compute for large strings.
2024-01-12 08:37:04 -08:00
Robert Hensing 274d887fee fetchTree/git: Cache export-ignore filter 2024-01-12 17:18:56 +01:00
Robert HensingandEelco Dolstra d80c582b78 libfetchers: Add CachingFilteringInputAccessor
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-01-12 17:16:59 +01:00
Eelco Dolstra 52f949bbf5 Merge pull request #9656 from edolstra/nix-profile-stable-names
Make profile element names stable
2024-01-12 16:59:36 +01:00
Eelco Dolstra 72560f7bbe Add profile migration test 2024-01-12 16:33:15 +01:00
Eelco Dolstra 8c7e2ed77c Update release notes 2024-01-12 16:21:07 +01:00
Robert Hensing f68ad5acbb fetchTree/git: Don't expose exportIgnore attr 2024-01-12 16:05:36 +01:00
Robert Hensing 469cf263c7 Format 2024-01-12 15:31:53 +01:00
Robert Hensing 692e9197bc fetchTree: Disallow combination of submodules and exportIgnore for now 2024-01-12 15:31:53 +01:00
Robert Hensing 71d08af15b rl-next: Add *general* note about git fetcher reimpl 2024-01-12 15:31:53 +01:00
Robert Hensing 99bd12f0b1 fetchGit/fetchTree: Improve exportIgnore, submodule interaction
Also fingerprint and some preparatory improvements.

Testing is still not up to scratch because lots of logic is duplicated
between the workdir and commit cases.
2024-01-12 15:31:53 +01:00
Robert Hensing 1bbe837184 fetchTree: Add isFetchGit exportIgnore 2024-01-12 15:31:53 +01:00
Robert Hensing 7774eff10e libfetchers/git: Move workdir accessor into GitRepo::getAccessor 2024-01-12 15:31:53 +01:00
Robert Hensing 8024b954d7 fetchTree: Recommend against exportIgnore 2024-01-12 15:31:53 +01:00
Robert Hensing 467c62a96e GitRepoImpl: Move exportIgnore into a filtering accessor 2024-01-12 15:31:53 +01:00
Robert Hensing cd5e752fa7 GitRepoImpl::getSubmodules: Access getSubmoduleRev without cast
This will be needed because the accessor will be wrapped, and therefore
not be an instance of GitInputAccessor anymore.
2024-01-12 15:31:53 +01:00
Robert Hensing f6b1d15580 MakeNotAllowedError: Touch up doc 2024-01-12 15:31:53 +01:00
Robert Hensing 1c6bb609af fetchTree: allow larger output attrsets
Intentionally dumb change ahead of architectural improvements.
2024-01-12 15:31:53 +01:00
Robert Hensing ce6d58a97c git fetcher: Add exportIgnore parameter
Enabled for fetchGit, which historically had this behavior,
among other behaviors we do not want in fetchGit.

fetchTree disables this parameter by default. It can choose the
simpler behavior, as it is still experimental.

I am not confident that the filtering implementation is future
proof. It should reuse a source filtering wrapper, which I believe
Eelco has already written, but not merged yet.
2024-01-12 15:31:52 +01:00
DavHau 4d0ecda33e fetchTree/fetchGit: add test for .gitattributes
...with the intention to prevent future regressions in fetchGit
2024-01-12 15:31:52 +01:00
Eelco Dolstra 0d55d660d5 Merge pull request #8043 from bobvanderlinden/pr-shell-env
nix: develop: use SHELL from rc script
2024-01-12 13:42:42 +01:00
Eelco Dolstra e21b3cf9db Merge remote-tracking branch 'origin/master' into nix-profile-stable-names 2024-01-12 13:36:27 +01:00
Robert Hensing be6c86038b Merge pull request #9606 from 9999years/printer
Support limiting attributes and list items in value printing to support detailed errors
2024-01-12 13:27:56 +01:00
Eelco Dolstra 7c6f093abc .data() -> .c_str() to be on the safe side 2024-01-12 13:00:53 +01:00
Eelco Dolstra 66bd1b0298 Merge remote-tracking branch 'origin/master' into pr-shell-env 2024-01-12 12:56:26 +01:00
Eelco Dolstra 3e23759834 gc-non-blocking.sh: Add explanation
Also name the _NIX_TEST_GC_SYNC environment variables logically.
2024-01-12 12:38:55 +01:00
Eelco Dolstra 5703c31325 Merge remote-tracking branch 'origin/master' into handle-missing-gc-socket 2024-01-12 12:26:25 +01:00
Eelco Dolstra cd5fd5cbf3 Merge pull request #9719 from jlesquembre/clang-format-config
Add clang format configuration
2024-01-12 12:19:00 +01:00
Eelco Dolstra 1acb1e0ca3 Merge pull request #9742 from obsidiansystems/mingw-makefiles
More makefile preparations for Windows
2024-01-12 12:11:48 +01:00
John Ericson 90fdbfc601 Build Windows DLLs with -Wl,--export-all-symbols
This is not the most elegant, but will match the SOs in exporting
everything for now. Later we can refine what is public/private to clean
up the interface.
2024-01-11 23:34:37 -05:00
John Ericson af0345df36 makefiles: Do some HOST_CYGWIN -> HOST_WINDOWS
These bits are not Cygwin-specific.
2024-01-11 23:28:11 -05:00
John Ericson 34bb6dcab1 makefiles: Support .exe executable prefix on Windows 2024-01-11 23:28:11 -05:00
Rebecca Turner df84dd4d8d Restore ambiguous value printer for nix-instantiate
The Nix team has requested that this output format remain unchanged.
I've added a warning to the man page explaining that `nix-instantiate
--eval` output will not parse correctly in many situations.
2024-01-11 16:34:36 -08:00
Rebecca Turner 0fa08b4516 Unify and refactor value printing
Previously, there were two mostly-identical value printers -- one in
`libexpr/eval.cc` (which didn't force values) and one in
`libcmd/repl.cc` (which did force values and also printed ANSI color
codes).

This PR unifies both of these printers into `print.cc` and provides a
`PrintOptions` struct for controlling the output, which allows for
toggling whether values are forced, whether repeated values are tracked,
and whether ANSI color codes are displayed.

Additionally, `PrintOptions` allows tuning the maximum number of
attributes, list items, and bytes in a string that will be displayed;
this makes it ideal for contexts where printing too much output (e.g.
all of Nixpkgs) is distracting. (As requested by @roberth in
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735)

Please read the tests for example output.

Future work:
- It would be nice to provide this function as a builtin, perhaps
  `builtins.toStringDebug` -- a printing function that never fails would
  be useful when debugging Nix code.
- It would be nice to support customizing `PrintOptions` members on the
  command line, e.g. `--option to-string-max-attrs 1000`.
2024-01-11 16:34:36 -08:00
Rebecca Turner c9125603a5 Unindent print.hh declarations 2024-01-11 16:07:01 -08:00
John Ericson a38e6b9087 Merge pull request #9745 from obsidiansystems/fix-install-unit-tests-cond
packages.nix: Fix `installUnitTests` condition
2024-01-11 17:11:27 -05:00
John Ericson a923444a94 packages.nix: Fix installUnitTests condition
The intent was we install the tests when we can *not* run them. Instead,
we were installing them when we can.
2024-01-11 16:39:56 -05:00
John Ericson 84502674f8 Merge pull request #9736 from obsidiansystems/mingw-makefiles
Some small Makefile improvements to prepare for Windows support
2024-01-11 10:58:55 -05:00
John Ericson 113499d16f Merge pull request #9634 from 9999years/combine-abstract-pos-and-pos
Combine `AbstractPos`, `PosAdapter`, and `Pos`
2024-01-11 10:47:27 -05:00
DavHau 0f95330fde fetchGit: add simple test for ssh fetching
Also move tests to separate files which are auto-imported. This should allow people adding tests concurrently without introducing merge conflicts
2024-01-11 15:46:26 +07:00
DavHau 813c113b9e initialize test suite for git fetchers
solves #9388

This utilizes nixos vm tests to allow:
- writing tests for fetchTree and fetchGit involving actual networking.
- writing small independent test cases by automating local and remote repository setup per test case.

This adds:
  - a gitea module setting up a gitea server
  - a setup module that simplifies writing test cases by automating the repo setup.
  - a simple git http test case

Other improvements:
For all nixos tests, add capability of overriding the nix version to test against.
This should make it easier to prevent regressions. If a new test is added it can simply be ran against any older nix version without having to backport the test.
For example, for running the container tests against nix 2.12.0:
`nix build "$(nix eval --raw .#hydraJobs.tests.containers --impure --apply 't: (t.forNix "2.12.0").drvPath')^*" -L`
2024-01-11 14:44:00 +07:00
John Ericson 423484ad26 Only link with -pthread on Unix
We don't want this with MinGW.
2024-01-10 20:38:39 -05:00
John Ericson f9e5eb5f0a Make indentation in makesfiles consistent
Tab (as required) for rules, two spaces for `if`...`endif`.
2024-01-10 20:26:34 -05:00
John Ericson 57dc4fc878 Make more expressive HOST_* macro system 2024-01-10 20:14:55 -05:00
John Ericson 75d509eb08 Merge pull request #9731 from obsidiansystems/better-test-no-gc
Improve the build without GC
2024-01-10 10:19:30 -05:00
John Ericson 0c3ce23754 Improve the build without GC
We don't just want to pass `--enable-gc=no`; we also want to make sure
boehmgc is not a dependency. Creating a nix-level configuration option
to do both, and then using that for the CI job, is more robust.
2024-01-09 17:48:50 -05:00
John Ericson 84aa8e9f19 Merge pull request #9728 from obsidiansystems/readline-support
Improve build sytem support for readline instead of editline
2024-01-09 17:23:02 -05:00
John EricsonandWeijia Wang 2cea88dbc8 Improve build sytem support for readline instead of editline
Changes:

- CPP variable is now `USE_READLINE` not `READLINE`

- `configure.ac` supports with new CLI flag

- `package.nix` supports with new configuration option

- `flake.nix` CIs this (along with no markdown)

Remove old Ubuntu 16.04 stop-gap too, as that is now quite old.

Motivation:

- editline does not build for Windows, but readline *should*. (I am
  still working on this in Nixpkgs at this time, however. So there will
  be a follow-up Nix PR removing the windows-only skipping of the
  readline library once I am done.)

- Per
  https://salsa.debian.org/debian/nix/-/blob/master/debian/rules?ref_type=heads#L27
  and #2551, Debian builds Nix with readline. Now we better support and
  CI that build configuration.

This is picking up where #2551 left off, ensuring we test a few more
things not merely have CPP for them.

Co-authored-by: Weijia Wang <9713184+wegank@users.noreply.github.com>
2024-01-09 16:56:55 -05:00
Eelco Dolstra b91c935c2f Merge pull request #9648 from cole-h/nix-shell-ordering
nix shell: reflect command line order in PATH order
2024-01-09 19:00:33 +01:00
Eelco Dolstra 47633ceec3 Merge pull request #9726 from DavHau/gitignore
gitignore: add result-*
2024-01-09 16:55:14 +01:00
DavHau 3d9e0c60e4 gitignore: add result-* 2024-01-09 18:36:09 +07:00
John Ericson 15820a0507 Merge pull request #9720 from NixOS/fix-internal-api-docs
Fix Internal API docs
2024-01-08 16:33:38 -05:00
John Ericson fa2a27d720 Merge pull request #9710 from wegank/disable-lowdown
Make lowdown optional
2024-01-08 15:13:45 -05:00
John Ericson 29eb5ed1dc Fix Internal API docs
Because of source filtering, they were empty.

Fixes #9694
2024-01-08 14:47:42 -05:00
Weijia WangandJohn Ericson bbd0a959e1 Make lowdown optional
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2024-01-08 20:37:42 +01:00
Rebecca Turner 4feb7d9f71 Combine AbstractPos, PosAdapter, and Pos
Also move `SourcePath` into `libutil`.

These changes allow `error.hh` and `error.cc` to access source path and
position information, which we can use to produce better error messages
(for example, we could consider omitting filenames when two or more
consecutive stack frames originate from the same file).
2024-01-08 10:59:41 -08:00
José Luis Lafuente 53fdcbca50 Add clang format configuration 2024-01-08 19:46:38 +01:00
Valentin Gagarin 6a243e5ed2 fix an old lost direct (#9458)
this part must have been moved quite a while ago, but apparently so far
no one noticed
2024-01-08 19:38:36 +01:00
John Ericson 3ecdda80be Merge pull request #9709 from wegank/configure-ac-doc-gen
Fix typo in configure.ac
2024-01-08 12:19:36 -05:00
Weijia Wang 605eba3829 Fix typo in configure.ac 2024-01-08 17:31:27 +01:00
John Ericson ff6de4a9ee Merge pull request #9662 from shlevy/flat-fixed-references-assert
Improve error message for fixed-outputs with references.
2024-01-08 10:46:37 -05:00
John Ericson c4c636284e Only test bug fix with new enough deamon 2024-01-08 10:17:28 -05:00
Eelco DolstraandJohn Ericson faf87b51f7 Show why GC socket connection was refused
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-01-08 14:14:36 +01:00
Shea Levy eeb2f083c5 Improve error message for fixed-outputs with references.
This codepath is possible, e.g. with a dockerTools.pullImage of an image with a Nix store.
2024-01-07 07:32:31 -05:00
Valentin Gagarin fe751fbde2 don't show channels in upgrade instructions
channels make everything more stateful, and therefore more complicated
and potentially confusing, but aren't needed for this task, so don't encourage their use.
2024-01-07 01:07:11 +01:00
Valentin GagarinandTravis A. Everett 8e865f3aba deduplicate installation instructions (#9507)
* deduplicate installation instructions

- reorder sections to present pinned installation more prominently
- remove outdated notes on the macOS installer rework
- update instructions to handle the installer tarball

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2024-01-06 22:45:25 +01:00
wiki-meandValentin Gagarin a4d33e816e Improve documentation around upgrading nix (#9679)
* Improve documentation around upgrading nix, add replacing nix channel with new one

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-06 18:01:10 +00:00
John Ericson 9651034dc2 Merge pull request #9698 from NixOS/no-boost-regex
Don't pull in libboost_regex
2024-01-05 09:49:13 -05:00
Eelco Dolstra 388c79d546 Don't pull in libboost_regex
We're not using and we don't want to pull in libicu (37 MiB).
2024-01-05 15:17:01 +01:00
Eelco Dolstra 359990dfdc Merge pull request #9324 from 9999years/fix-8854-take-2
Don't attempt to `git add` ignored files
2024-01-05 14:57:32 +01:00
Eelco Dolstra dedbbbb451 Merge pull request #9670 from DavHau/log-lines
saner default for log-lines: change to 25
2024-01-05 14:39:50 +01:00
Eelco Dolstra cbf99c71c6 Merge pull request #9692 from 9999years/update-clang-stdenv
Remove `clang11Stdenv`
2024-01-05 10:11:44 +01:00
Rebecca Turner d8a2b06e20 Remove clang11Stdenv
Clang 11 doesn't have support for three-way-comparisons (<=>, "spaceship
operator", "consistent comparisons") and is older than `clangStdenv`.

`clangStdenv` is currently 12 on FreeBSD and Android and 16 on other
platforms:

https://github.com/NixOS/nixpkgs/blob/32e718f00c26c811be0062dd0777066f02406940/pkgs/top-level/all-packages.nix#L16629-L16644

Let's start by removing Clang 11 from our distribution. Next we can
consider upgrading to Clang 17, which fully supports the spaceship
operator:

https://releases.llvm.org/17.0.1/tools/clang/docs/ReleaseNotes.html#what-s-new-in-clang-release
2024-01-04 11:43:56 -08:00
Robert Hensing 1ed245a606 Merge pull request #9690 from obsidiansystems/nixos-23.05-small-again
flake: Go back to regular `nixos-23.05-small`
2024-01-04 17:30:21 +01:00
Eelco Dolstra 965cfe9688 Merge pull request #9687 from edolstra/withFramedSink-ctrl-c-hang
withFramedSink(): Receive interrupts on the stderr thread
2024-01-04 17:05:18 +01:00
John Ericson 37ea1612c7 flake: Go back to regular nixos-23.05-small
Finally get off the ad-hoc staging commit!

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/36c4ac09e9bebcec1fa7b7539cddb0c9e837409c' (2023-11-30)
  → 'github:NixOS/nixpkgs/2c9c58e98243930f8cb70387934daa4bc8b00373' (2023-12-31)
2024-01-04 10:45:00 -05:00
John Ericson 65fad9b84c Merge pull request #9688 from obsidiansystems/signature-generalization-prep
Signer infrastructure: Prep for #9076
2024-01-03 17:25:37 -05:00
John EricsonandRaito Bezarius 12bb8cdd38 Signer infrastructure: Prep for #9076
This sets up infrastructure in libutil to allow for signing other than
by a secret key in memory. #9076 uses this to implement remote signing.

(Split from that PR to allow reviewing in smaller chunks.)

Co-Authored-By: Raito Bezarius <masterancpp@gmail.com>
2024-01-03 16:13:55 -05:00
Eelco Dolstra 295a2ff8bd Make some more threads receive interrupts
Shouldn't hurt to do this. In particular, this should speed up
shutting down the PathSubstitutionGoal thread if it's copying from a
remote store.
2024-01-03 19:30:02 +01:00
Eelco Dolstra 24e70489e5 withFramedSink(): Receive interrupts on the stderr thread
Otherwise Nix deadlocks when Ctrl-C is received in withFramedSink():
the parent thread will wait forever for the stderr thread to shut
down.

Fixes the hang reported in https://github.com/NixOS/nix/issues/7245#issuecomment-1770560923.
2024-01-03 19:14:50 +01:00
Eelco Dolstra 315aade89d Merge pull request #9681 from edolstra/eval-optimisations
Optimize empty list constants
2024-01-03 10:43:01 +01:00
Eelco Dolstra f5b97f1958 Merge pull request #9683 from obsidiansystems/fix-builds
Fix builds / CI jobs
2024-01-03 10:28:00 +01:00
Eelco Dolstra 484881f302 Move empty list constant 2024-01-03 10:23:27 +01:00
John Ericson 7b8af5f916 buildNoTests: Restore intent
The thing we wanted to test was that building Nix without building or
running tests, and without depending on libraries only needed by tests,
works.

But since 6c8f4ef350, we can also install
unit tests, and during the conversion to using `package.nix` this
started happening more often (they go to a separate output though, so
this should be fine).

This adds more `... = false` to restore the original intent: don't run
unit test or functional tests, and don't install unit tests.
2024-01-02 12:50:48 -05:00
John Ericson 86e9244437 Fix buildNoTest
`checkInputs` is not right for this because we don't just need these
deps when `doTest`, we also need them when `installUnitTests`.
2024-01-02 12:41:53 -05:00
John Ericson 2b20f36f95 Fix NetBSD build
There was still a mistake after my earlier
a7115a47ef and
e13fc0bbdb. This finally gets it right.
2024-01-02 12:33:51 -05:00
Eelco Dolstra 3f796514b3 Optimize empty list constants
This avoids a Value allocation for empty list constants. During a `nix
search nixpkgs`, about 82% of all thunked lists are empty, so this
removes about 3 million Value allocations.

Performance comparison on `nix search github:NixOS/nixpkgs/e1fa12d4f6c6fe19ccb59cac54b5b3f25e160870 --no-eval-cache`:

maximum RSS:        median = 3845432.0000  mean = 3845432.0000  stddev =      0.0000  min = 3845432.0000  max = 3845432.0000  [rejected?, p=0.00000, Δ=-70084.00000±0.00000]
soft page faults:   median = 965395.0000  mean = 965394.6667  stddev =      1.1181  min = 965392.0000  max = 965396.0000  [rejected?, p=0.00000, Δ=-17929.77778±38.59610]
system CPU time:    median =      1.8029  mean =      1.7702  stddev =      0.0621  min =      1.6749  max =      1.8417  [rejected, p=0.00064, Δ=-0.12873±0.09905]
user CPU time:      median =     14.1022  mean =     14.0633  stddev =      0.1869  min =     13.8118  max =     14.3190  [not rejected, p=0.03006, Δ=-0.18248±0.24928]
elapsed time:       median =     15.8205  mean =     15.8618  stddev =      0.2312  min =     15.5033  max =     16.1670  [not rejected, p=0.00558, Δ=-0.28963±0.29434]
2024-01-02 12:49:11 +01:00
Eelco Dolstra 3f834f5c64 Merge pull request #9657 from edolstra/fix-getNameFromURL
getNameFromURL(): Support uppercase characters in attribute names
2024-01-02 12:48:17 +01:00
Robert Hensing 83f5622545 Merge pull request #9658 from pennae/env-diet
reduce the size of Env by one pointer
2023-12-31 13:57:16 +01:00
Robert Hensing 3511430902 Merge pull request #9673 from pennae/drv-parse-opts
optimize derivation parsing
2023-12-31 13:49:03 +01:00
pennae 1fe66852ff reduce the size of Env by one pointer
since `up` and `values` are both pointer-aligned the type field will
also be pointer-aligned, wasting 48 bits of space on most machines. we
can get away with removing the type field altogether by encoding some
information into the `with` expr that created the env to begin with,
reducing the GC load for the absolutely massive amount of single-entry
envs we create for lambdas. this reduces memory usage of system eval by
quite a bit (reducing heap size of our system eval from 8.4GB to 8.23GB)
and gives similar savings in eval time.

running `nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'`

before:

  Time (mean ± σ):      5.576 s ±  0.003 s    [User: 5.197 s, System: 0.378 s]
  Range (min … max):    5.572 s …  5.581 s    10 runs

after:

  Time (mean ± σ):      5.408 s ±  0.002 s    [User: 5.019 s, System: 0.388 s]
  Range (min … max):    5.405 s …  5.411 s    10 runs
2023-12-30 18:55:13 +01:00
pennae c62686a95b reduce copies during drv parsing
many paths need not be heap-allocated, and derivation env name/valye
pairs can be moved into the map.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.883 s ±  0.016 s    [User: 5.250 s, System: 1.424 s]
  Range (min … max):    6.860 s …  6.905 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.868 s ±  0.027 s    [User: 5.194 s, System: 1.466 s]
  Range (min … max):    6.828 s …  6.913 s    10 runs
2023-12-30 18:44:15 +01:00
pennae 02c64abf1e use translation table for drv string parsing
the table is very small compared to cache sizes and a single indexed
load is faster than three comparisons.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.907 s ±  0.012 s    [User: 5.272 s, System: 1.429 s]
  Range (min … max):    6.893 s …  6.926 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.883 s ±  0.016 s    [User: 5.250 s, System: 1.424 s]
  Range (min … max):    6.860 s …  6.905 s    10 runs
2023-12-30 18:44:15 +01:00
pennae 79d3d412ca optimize derivation string parsing
a bunch of derivation strings contain no escape sequences. we can
optimize for this fact by first scanning for the end of a derivation
string and simply returning the contents unmodified if no escape
sequences were found. to make this even more efficient we can also use
BackedStringViews to avoid copies, avoiding heap allocations for
transient data.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.952 s ±  0.015 s    [User: 5.294 s, System: 1.452 s]
  Range (min … max):    6.926 s …  6.974 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.907 s ±  0.012 s    [User: 5.272 s, System: 1.429 s]
  Range (min … max):    6.893 s …  6.926 s    10 runs
2023-12-30 18:44:10 +01:00
Robert Hensing bf1b294bd8 Merge pull request #9617 from 9999years/stack-overflow-segfault
Fix segfault on infinite recursion in some cases
2023-12-30 10:40:25 +01:00
w 4e3dc5f925 tests: test with conflicting profile links 2023-12-30 06:24:06 +00:00
Rebecca Turner 7434caca05 Fix segfault on infinite recursion in some cases
This fixes a segfault on infinite function call recursion (rather than
infinite thunk recursion) by tracking the function call depth in
`EvalState`.

Additionally, to avoid printing extremely long stack traces, stack
frames are now deduplicated, with a `(19997 duplicate traces omitted)`
message. This should only really be triggered in infinite recursion
scenarios.

Before:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    Segmentation fault: 11

After:

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)'
    error: stack overflow

           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

    $ nix-instantiate --eval --expr '(x: x x) (x: x x)' --show-trace
    error:
           … from call site
             at «string»:1:1:
                1| (x: x x) (x: x x)
                 | ^

           … while calling anonymous lambda
             at «string»:1:2:
                1| (x: x x) (x: x x)
                 |  ^

           … from call site
             at «string»:1:5:
                1| (x: x x) (x: x x)
                 |     ^

           … while calling anonymous lambda
             at «string»:1:11:
                1| (x: x x) (x: x x)
                 |           ^

           … from call site
             at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^

           (19997 duplicate traces omitted)

           error: stack overflow
           at «string»:1:14:
                1| (x: x x) (x: x x)
                 |              ^
2023-12-29 22:16:44 -08:00
pennae 2cfc4ace35 malloc/memset even less
more buffers that can be uninitialized and on the stack. small
difference, but still worth doing.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.963 s ±  0.011 s    [User: 5.330 s, System: 1.421 s]
  Range (min … max):    6.943 s …  6.974 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.952 s ±  0.015 s    [User: 5.294 s, System: 1.452 s]
  Range (min … max):    6.926 s …  6.974 s    10 runs
2023-12-30 00:45:11 +01:00
pennae 99a691c8a1 don't use istreams in hot paths
istream sentry objects are very expensive for single-character
operations, and since we don't configure exception masks for the
istreams used here they don't even do anything. all we need is
end-of-string checks and an advancing position in an immutable memory
buffer, both of which can be had for much cheaper than istreams allow.

the effect of this change is most apparent on empty stores.

before:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      7.167 s ±  0.013 s    [User: 5.528 s, System: 1.431 s]
  Range (min … max):    7.147 s …  7.182 s    10 runs

after:

Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      6.963 s ±  0.011 s    [User: 5.330 s, System: 1.421 s]
  Range (min … max):    6.943 s …  6.974 s    10 runs
2023-12-30 00:45:10 +01:00
w 29eb4d354a nix-profile: add cross reference to installer test 2023-12-29 07:14:53 +00:00
wh0 5d0bdb1d3f nix-profile: fix both profile links detection 2023-12-29 05:43:22 +00:00
Eelco Dolstra a21c762dab Merge pull request #9666 from unblevable/dervation-typo
Fix "dervation" typo
2023-12-27 21:52:37 +01:00
DavHau b6313f64f7 saner default for log-lines: change to 25
This seems to be a much saner default. 10 lines are just not enough in so many cases.
2023-12-27 19:57:27 +07:00
Brian Le e2399fc949 Change "dervation" typos to "derivation" 2023-12-26 17:12:28 -05:00
John Ericson e23983db82 Merge pull request #9661 from shlevy/ifd-buildStore
Build IFD in the build store when using eval-store.
2023-12-24 01:26:14 -05:00
Shea Levy c3942ef85f Build IFD in the build store when using eval-store.
Previously, IFDs would be built within the eval store, even though one
is typically using `--eval-store` precisely to *avoid* local builds.

Because the resulting Nix expression must be copied back to the eval
store in order to be imported, this requires the eval store to trust
the build store's signatures.
2023-12-23 21:33:56 -05:00
Shea Levy 9cb287657b remote-store test: Break out IFD expression into a separate file 2023-12-23 17:15:09 -05:00
Robert Hensing ee439734e9 Merge pull request #9582 from pennae/misc-opts
a packet of small optimizations
2023-12-22 17:00:59 +01:00
Eelco Dolstra cf02b3335c Merge remote-tracking branch 'origin/master' into tarball-cache 2023-12-22 16:45:01 +01:00
Eelco Dolstra 936a364226 getNameFromURL(): Support uppercase characters in attribute names
In particular, this makes it handle 'legacyPackages' correctly.
2023-12-22 16:35:58 +01:00
Eelco Dolstra a748e88bf4 nix profile: Remove check for "name" attribute in manifests
AFAIK, we've never emitted this attribute.
2023-12-22 16:27:31 +01:00
John Ericson d65d56fa77 Merge pull request #9655 from iFreilicht/move-flake-url-name-util
Move url-name utility to libexpr/flake
2023-12-22 10:26:42 -05:00
Eelco Dolstra 6268a45b65 nix profile: Make profile element names stable
The profile manifest is now an object keyed on the name returned by
getNameFromURL() at installation time, instead of an array. This
ensures that the names of profile elements don't change when other
elements are added/removed.
2023-12-22 16:21:30 +01:00
Eelco Dolstra 3187bc9ac3 nix profile: Remove indices 2023-12-22 16:21:30 +01:00
Robert Hensing d2016c6b59 Merge pull request #9621 from blaggacao/fix/too-restrictive-branch-regex-master
fix: valid branch name
2023-12-22 16:02:25 +01:00
Felix Uhl 4b41118663 Move flakeref tests to new flake/ subdirectory 2023-12-22 09:38:13 +01:00
Felix Uhl 26d7b0c793 Move url-name utility to libexpr/flake 2023-12-22 09:33:02 +01:00
Robert Hensing 4f47152209 libutil/url-parts.hh: Fix regex
Regex syntax is awful.
2023-12-21 23:11:25 +01:00
Felix Uhl 397cf4e285 nix search: Disallow empty regex
Fixes #4739
Fixes #3553 in spirit IMO
2023-12-21 22:13:43 +01:00
Robert Hensing 8c4ea12f11 libutil/url-parts.hh: comment 2023-12-21 21:03:06 +01:00
Cole Helbling f4454aac9f nix shell: reflect command line order in PATH order
Prior to this change, Nix would prepend every installable to the PATH
list in order to ensure that installables appeared before the current
PATH from the ambient environment.

With this change, all the installables are still prepended to the PATH,
but in the same order as they appear on the command line. This means
that the first of two packages that expose an executable `hello` would
appear in the PATH first, and thus be executed first.

See the test in the prior commit for a more concrete example.
2023-12-21 10:56:19 -08:00
Cole Helbling 1fb43d1eee tests: add a test for command line ordering 2023-12-21 10:23:07 -08:00
Cole Helbling 5ed1884875 libcmd: Installable::toStorePaths -> Installable::toStorePathSet 2023-12-21 10:23:07 -08:00
Eelco Dolstra 17c202b3ea Merge pull request #8678 from iFreilicht/profile-names-instead-of-index
`nix profile`: Allow referring to elements by human-readable name
2023-12-21 17:14:11 +01:00
Eelco Dolstra 942d635102 Fix release notes 2023-12-21 16:38:25 +01:00
Eelco Dolstra 14508ade28 Typo 2023-12-21 16:25:55 +01:00
Eelco Dolstra d77a39a314 Fix indent 2023-12-21 16:22:34 +01:00
Eelco Dolstra 9d9d9ff0de Merge remote-tracking branch 'origin/master' into profile-names-instead-of-index 2023-12-21 16:21:26 +01:00
Théophane Hufschmitt e94a96893f maintainers: Mention the monthly “Assigned” column review
As decided during [the last team meeting](https://discourse.nixos.org/t/2023-12-18-nix-team-meeting-minutes-113/37050#improving-internal-and-external-communication-3), we want to regularly review the `Assigned` column in the team's board because it tends to turn into a graveyard of forgotten stuff.
So encode that in the handbook
2023-12-21 10:00:14 +01:00
Robert Hensing 8e64cd59b0 Merge pull request #9645 from tweag/undeprecate-isNull
Undeprecate isNull
2023-12-20 18:15:49 +01:00
Silvan MosbergerandRobert Hensing ea454d8687 Undeprecate isNull
There's no good reason to deprecate it:
- For consistency reasons it should continue to exist, such that all
primitive types have a corresponding `builtins.is*` primop.
- There's no implementation cost to continuing to have this function
- It costs users time to try to migrate away from it, e.g.
  https://github.com/NixOS/nixpkgs/pull/219747 and https://github.com/NixOS/nixpkgs/pull/275548
- Using it can give easier-to-read code like `all isNull list`

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-20 17:55:19 +01:00
Eelco Dolstra 75e10e42f3 Merge pull request #9631 from cole-h/fixup-check-warnings
Fix warnings when running checks
2023-12-20 17:47:30 +01:00
Eelco Dolstra 16a77767cd Merge pull request #9635 from 9999years/document-makefile-variables
Document more `Makefile` variables
2023-12-20 17:47:07 +01:00
Théophane Hufschmitt e3cdad4cf8 Merge pull request #9646 from obsidiansystems/fix-cross-build
Move down fallback targets in `Makefile`
2023-12-20 13:44:40 +01:00
John Ericson 26d60b837c Move down fallback targets in Makefile
This ensures `lib.mk` still defines `default` as the first target. This
fixes some builds.
2023-12-20 03:23:49 -05:00
pennae 2b0e95e7aa use singleton expr to generate black hole errors
this also reduces forceValue code size and removes the need for
hideInDiagnostics. coopting thunk forcing like this has the additional
benefit of clarifying how these errors can happen in the first place.
2023-12-19 19:32:16 +01:00
pennae f9db4de0f3 force-inline forceValue
forceValue is extremely hot. interestingly adding likeliness annotations
to the branches does not seem to make a difference.

before:

  Time (mean ± σ):      4.224 s ±  0.005 s    [User: 3.711 s, System: 0.512 s]
  Range (min … max):    4.218 s …  4.234 s    10 runs

after:

  Time (mean ± σ):      4.140 s ±  0.009 s    [User: 3.647 s, System: 0.492 s]
  Range (min … max):    4.130 s …  4.152 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 69ed4aee61 remove lazy-pos forceValue
almost all uses of this are interactive, except for deepSeq. deepSeq is
going to be expensive and rare enough to not care much about, and
Value::determinePos should usually be cheap enough to not be too much of
a burden in any case.
2023-12-19 19:32:16 +01:00
pennae f9aee2f2c4 don't malloc/memset posix accessor buffer
it's relatively small and fits on the stack nicely, and we don't need it
initialized either.
2023-12-19 19:32:16 +01:00
pennae b78e77b34c use custom location type in the parser
~1% parser speedup from not using TLS indirections, less on system eval.
this could have also gone in flex yyextra data, but that's significantly
slower for some reason (albeit still faster than thread locals).

before:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs

after:

  Time (mean ± σ):      4.224 s ±  0.005 s    [User: 3.711 s, System: 0.512 s]
  Range (min … max):    4.218 s …  4.234 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 2e0321912a use aligned flex tables
~2% speedup on parsing without eval, less (but still significant) on
system eval. having flex generate faster parsers leads to very strange
misparses. maybe re2c is worth investigating.

before:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs

after:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs
2023-12-19 19:32:16 +01:00
pennae cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 74c134914c compare string values with strcmp
string_view()ification calls strlen() first, which we don't need here.
2023-12-19 19:32:16 +01:00
pennae 78353deb02 encode black holes as tApp values
checking for isBlackhole in the forceValue hot path is rather more
expensive than necessary, and with a little bit of trickery we can move
such handling into the isApp case. small performance benefit, but under
some circumstances we've seen 2% improvement as well.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs

after:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs
2023-12-19 19:32:16 +01:00
pennae 0218e4e6c3 memset less in addToStoreFromDump
resizing a std::string clears the newly added bytes, which is not
necessary here and comes with a ~1.4% slowdown on our test nixos config.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.486 s ±  0.003 s    [User: 3.978 s, System: 0.507 s]
  Range (min … max):    4.482 s …  4.492 s    10 runs

after:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs
2023-12-19 19:32:16 +01:00
Eelco Dolstra 7cfd6c0efe Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Théophane Hufschmitt 83a32969ad Merge pull request #9633 from cole-h/fix-sysconfdir
package: don't set sysconfdir in devShells
2023-12-19 10:14:48 +01:00
Théophane Hufschmitt 94a03a810e Merge pull request #9637 from 9999years/fix-logging-test-on-macos
Fix `logging.sh` test on macOS
2023-12-19 10:13:08 +01:00
Théophane Hufschmitt b077d545ad Merge pull request #9638 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.3.0
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
2023-12-19 10:11:37 +01:00
Théophane Hufschmitt e85fd92816 Merge pull request #9639 from awakesecurity/mz/darwin-UID-override
installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
2023-12-19 10:11:24 +01:00
Théophane Hufschmitt c6d7013583 Merge pull request #9641 from amarshall/darwin-sandbox-stderr
Allow access to /dev/stderr in Darwin sandbox
2023-12-19 10:09:38 +01:00
Andrew Marshall 7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
Mel Zuser fa4bbe53e8 installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
because there are often already users in the 300 range and it's painful
to work around.

revives #6466
2023-12-18 15:02:26 -08:00
dependabot[bot] 23fb19cb18 build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-18 22:36:31 +00:00
Rebecca Turner 0cee56db1a Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Rebecca Turner 6f4930382b Document more Makefile variables 2023-12-18 13:34:08 -08:00
Cole Helbling ba0087316a package: don't set sysconfdir in devShells 2023-12-18 12:59:58 -08:00
Cole Helbling 1f7b62f123 Use nix daemon in the test suite
As part of the CLI stabilization effort, the last remaining checkbox (at
the moment) for `nix daemon` is that it "needs testing". This implements
the proposal of using `nix daemon` in place of `nix-daemon` in the test
suite.
2023-12-18 11:29:55 -08:00
Cole Helbling 123ef6a996 Fix warnings when running checks
`nix flake check` had these warnings:

    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/3m3hfpmbjdf4w39qfjami7ljhvhczay1-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
    trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
    trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
    trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
2023-12-18 10:20:52 -08:00
Robert Hensing 5d5b25f2e3 Merge pull request #9611 from obsidiansystems/fix-cross-configure
Split `--disable-tests`, fix cross builds
2023-12-18 17:25:19 +01:00
John EricsonandRobert Hensing 7feabf7d44 Split --disable-tests, fix cross builds
It might seem obnoxious to have yet more configure flags, but I found
controlling both the unit and functional tests with one flag was quite
confusing because they are so different:

- unit tests depending on building, functional tests don't (e.g. when
  we test already-built Nix)

- unit tests can be installed, functional tests cannot

- unit tests neeed extra libraries (GTest, RapidCheck), functional
  tests need extra executables (jq).

- unit tests are run by `make check`, functional tests are run by `make
  installcheck`

Really on a technical level, they seem wholly independent. Only on a
human level ("they are both are tests") do they have anything in common.

I had messed up the logic in cross builds because of this. Now I
split the flag in two (and cleaned up a few other inconsistencies), and
the logic fixed itself.

Co-Authored-By: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:47:50 -05:00
tomberekandValentin Gagarin b1c559eabc docs: add link to project board to PRs (#9630)
* docs: add link to project board to PRs

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

* fix wording

* add note on the process

---------

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-18 15:45:57 +00:00
John EricsonandRobert Hensing ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John EricsonandRobert Hensing dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Valentin Gagarin d19a667528 CODEOWNERS: unsubscribe fricklerhandwerk (#9614) 2023-12-18 13:54:40 +01:00
Théophane Hufschmitt ebfbc5a6a5 Merge pull request #9628 from fricklerhandwerk/add-links
add cross-reference
2023-12-18 13:07:39 +01:00
Robert Hensing cea8354436 Merge pull request #9619 from 9999years/remove-blank-lines-in-errors
Remove some blank lines from stack traces
2023-12-18 12:50:50 +01:00
Valentin Gagarin 4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
Julia Evans a47fabff0d use params.isFetchGit instead to check if it came from fetchGit 2023-12-17 12:14:55 -05:00
Julia Evans 06bed2eacd Make fetchTree locked input error message clearer 2023-12-17 12:04:59 -05:00
David Arnold c05d4fadd5 fix: valid branch name 2023-12-16 23:14:33 +01:00
Rebecca Turner 7f5ed330e4 Document Makefile variables in hacking.md (#9620) 2023-12-16 12:05:31 +00:00
Rebecca Turner 5cb98095ba Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson e7d95a1d80 Merge remote-tracking branch 'upstream/master' into tarball-cache 2023-12-15 09:37:13 -05:00
John Ericson 419a64f9e0 Merge pull request #9613 from jlesquembre/jl/clang-stdenv
Fix clang devshell
2023-12-15 09:08:41 -05:00
José Luis Lafuente 66d37b7338 Move clang dev deps to the nix devshell override 2023-12-15 12:41:38 +01:00
John Ericson 071dbbee33 Merge pull request #4093 from matthewbauer/eval-system
Add eval-system option
2023-12-14 20:41:29 -05:00
John Ericson 228e995cde Add release not for eval-system 2023-12-14 19:55:50 -05:00
John Ericson 70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
José Luis Lafuente 8d39c0c196 Fix clang devshell
Issue introduced in https://github.com/NixOS/nix/pull/9535
2023-12-14 23:14:59 +01:00
Eelco Dolstra fc0accf3dc Merge pull request #9567 from obsidiansystems/fix-sys-xattr-h
Fix `sys/xattr.h` check
2023-12-14 18:24:24 +01:00
John Ericson e13fc0bbdb Fix sys/xattr.h check
I wrote the `configure.ac` wrong, and so we just got no builds
supporting ACLs.

Also, it needs to be more precise because Darwin puts other stuff in
that same header, evidently.
2023-12-14 10:03:48 -05:00
Eelco Dolstra 06e106beff Disable GitHub tree hash mismatch warning 2023-12-14 13:38:10 +01:00
Eelco Dolstra 1fce12ec32 Merge remote-tracking branch 'origin/master' into tarball-cache 2023-12-14 13:31:29 +01:00
Eelco Dolstra 790cf13c26 Merge pull request #9608 from NixOS/default-lowdown
Remove custom lowdown
2023-12-14 13:18:32 +01:00
John Ericson e8ebcfd665 Merge pull request #9607 from 9999years/macos-boost
Use `--with-boost` on macOS to fix local builds
2023-12-13 23:15:44 -05:00
Rebecca Turner 6ed803737c Use --with-boost on macOS
`configureFlags` only included `--with-boost` on Linux, which makes
local builds as outlined in `doc/manual/src/contributing/hacking.md`
fail when performed on macOS.
2023-12-13 14:02:52 -08:00
Ramses 1e3d811840 worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
John Ericson 7b29b44d8e Remove custom lowdown
This was last upgraded in 788008385e, but
the version in Nixpkgs is a now a lot newer. I think the custom was
added to get ahead of Nixpkgs before, and so now that we are in fact
behind, it is no longer needed.
2023-12-13 16:25:18 -05:00
John Ericson e6515bd47b Merge pull request #9535 from tfc/package-nix
Simplify flake.nix by creating generic callPackage-able package.nix
2023-12-13 16:22:03 -05:00
John Ericson 2d24875fe4 package.nix: Avoid ${..} for conditional strings
Using `+` is Nixpkgs standard ideom for this, and helps avoid needless
rebuilds somewhat.
2023-12-13 20:49:31 +00:00
John Ericson ed93e51f4b Merge pull request #9604 from obsidiansystems/test-substitute-on-destination
Test `nix copy --substitute-on-destination`
2023-12-13 15:47:56 -05:00
John Ericson 28f2f3136d Delete stray install_name_tool call 2023-12-13 20:47:36 +00:00
John Ericson bf5804d46a flake.nix: Delete uneeded attrs0 binding 2023-12-13 20:41:41 +00:00
John Ericson f10f0f1b50 Move lowdown.nix to misc/ 2023-12-13 20:41:32 +00:00
John Ericson 19573f1b05 Restore comment 2023-12-13 15:33:15 -05:00
John Ericson d904f6483c Merge remote-tracking branch 'upstream/master' into package-nix 2023-12-13 15:30:59 -05:00
John Ericson e76df87814 Test nix copy --substitute-on-destination
It works with both `ssh://` and `ssh-ng://` now since #9600 (and
`ssh-ng:// didn't work before that).

Also, by making the two tests share code, we nudge ourselves towards
making sure there is feature parity.
2023-12-13 15:22:19 -05:00
Eelco Dolstra 1b7968ed86 Merge pull request #9547 from hercules-ci/allowed-scheme-without-slash
`allowed-uris`: match whole schemes without slashes
2023-12-13 20:23:33 +01:00
Théophane Hufschmitt b1c633c6bb Merge pull request #9600 from SharzyL/fix_nix_copy
fix: nix copy ssh-ng:// not respecting --substitute-on-destination
2023-12-13 18:08:38 +01:00
Eelco Dolstra 9dbfd186b1 Merge pull request #9602 from edolstra/unsafeGetAttrPos-test
Improve the unsafeGetAttrPos test
2023-12-13 15:52:43 +01:00
Eelco Dolstra 19ec1c9fd4 Improve the unsafeGetAttrPos test
We can use corepkgsFS->addFile() now to create a "real" position.
2023-12-13 15:15:30 +01:00
Eelco Dolstra 1f93fa2ed2 Merge pull request #9601 from edolstra/lazy-trees-backports
Lazy trees backports
2023-12-13 13:56:28 +01:00
Eelco Dolstra faa4cae9ae LibExprTest: Ignore $NIX_PATH
Otherwise a broken $NIX_PATH can cause the test suite to fail.
2023-12-13 13:27:39 +01:00
Eelco Dolstra 103ca0bde5 Improve SourcePath display 2023-12-13 13:27:29 +01:00
Eelco Dolstra cc3913e458 Remove unused variable 2023-12-13 13:27:23 +01:00
SharzyL 04f454f2a0 fix: nix copy ssh-ng:// not respecting --substitute-on-destination 2023-12-13 10:30:28 +08:00
tomberek 7026abfdde Merge pull request #9523 from fricklerhandwerk/conf-reword-always-allow-substitutes
reword documentation on settings and attributes related to substitution
2023-12-12 20:09:48 -05:00
tomberek 09041071bf Merge pull request #9525 from fricklerhandwerk/conf-reword-builders-use-substitutes
reword description of the `builders-use-substitutes` setting
2023-12-12 20:08:00 -05:00
Robert Hensing 0b87ba50c0 Revert "Add nix::isASCII*, locale-independent"
This reverts commit 79eb2920bb.

Not used at this time.
2023-12-12 17:46:34 +01:00
Robert Hensing 4eaeda6604 isValidSchemeName: Use regex
As requested by Eelco Dolstra. I think it used to be simpler.
2023-12-12 17:46:34 +01:00
Robert Hensing 2e451a663e schemeRegex -> schemeNameRegex
Scheme could be understood to include the typical `:` separator.
2023-12-12 17:25:20 +01:00
Théophane Hufschmitt 0dfa66d120 Merge pull request #9593 from B4dM4n/fix-path-like-flake-query
Fix query parsing for path-like flakes
2023-12-12 17:15:30 +01:00
John Ericson e687e86754 Merge pull request #9597 from tweag/fix-vm-tests
Fix the VM tests
2023-12-12 10:45:54 -05:00
Théophane Hufschmitt 589fb105f3 Fix the VM tests
Work around https://github.com/NixOS/nixpkgs/issues/271146 until we can
depend on a Nixpkgs version containing https://github.com/NixOS/nixpkgs/pull/271423
2023-12-12 16:05:32 +01:00
Théophane Hufschmitt dfc0cee702 Merge pull request #9589 from obsidiansystems/floating-content-addressing-derivations-eval-store
Fix building CA derivations with and eval store
2023-12-12 10:12:24 +01:00
John Ericson ff992f8b46 Merge remote-tracking branch 'upstream/master' into package-nix 2023-12-11 15:26:12 -05:00
John Ericson 5e649618be Merge pull request #9519 from nix-windows/mingw-shell
Allow getting in a MinGW dev shell
2023-12-11 15:01:56 -05:00
John Ericson eae2717e00 tests: Use cp -ar instead of tar-untar pipe 2023-12-11 13:56:18 -05:00
Ben Radford 4a2cee8e6c Document expected filesystem layout and OverlayFS mount command. 2023-12-11 18:55:39 +00:00
John Ericson 8d0a03b5a2 Fix tests after last rename (path -> pathInLowerStore) 2023-12-11 13:48:42 -05:00
John EricsonandThéophane Hufschmitt c90e46d3f0 Update tests/functional/local-overlay-store/common.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:45:46 -05:00
John Ericson b3bdd70ea2 Clarify toUpperPath docs
We're just mapping store paths to host OS paths, there is no checking
what is actually at this location.
2023-12-11 13:43:17 -05:00
John Ericson c93f78f6fa Fix test a bit from previous commit 2023-12-11 13:36:53 -05:00
John EricsonandThéophane Hufschmitt 6bb13358e6 Update tests/functional/local-overlay-store/redundant-add-inner.sh
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:34:09 -05:00
John EricsonandThéophane Hufschmitt bf0bf3d1be local-overlay store tests: storeDirs -> setupStoreDirs
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2023-12-11 13:30:40 -05:00
John Ericson b21ee60594 Get rid of verifyAllValidPaths boolean blindness 2023-12-11 13:28:40 -05:00
John EricsonandThéophane Hufschmitt c30b5d8a0b Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-11 13:18:34 -05:00
John Ericson 245af3ea02 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-12-11 13:12:09 -05:00
John Ericson b892161e31 flake.nix: Make a MinGW dev shell
This requires a `shellCrossSystems` for now, since Nix doesn't actually
build on Windows. This can be dropped once it does.
2023-12-11 12:31:27 -05:00
John Ericson 46b98a40a7 flake.nix: Make changes so a MinGW dev shell would work 2023-12-11 12:31:27 -05:00
John Ericson 78492cfde7 flake.nix: Use config not system for cross so we can be a bit more precise 2023-12-11 12:31:27 -05:00
John Ericson f60c2e8a5a flake.nix: installScriptFor take tarballs not strings
Trying to look up keys in multiple places is not nice, better for the
caller to be explicit.
2023-12-11 12:31:27 -05:00
John Ericson 0b81557e2c flake.nix: Put some list items on their own line
These things are about to become longer
2023-12-11 12:31:27 -05:00
John Ericson 9f39dda66c Fix building CA derivations with and eval store
I don't love the way this code looks. There are two larger problems:

- eval, build/scratch, destination stores (#5025) should have different
  types to reflect the fact that they are used for different purposes
  and those purposes correspond to different operations. It should be
  impossible to "use the wrong store" in my cases.

- Since drvs can end up in both the eval and build/scratch store, we
  should have some sort of union/layered store (not on the file sytem
  level, just conceptual level) that allows accessing both. This would
  get rid of the ugly "check both" boilerplate in this PR.

Still, it might be better to land this now / soon after minimal cleanup,
so we have a concrete idea of what problem better abstractions are
supposed to solve.
2023-12-11 12:17:36 -05:00
John Ericson 8cddda4f89 Merge pull request #9588 from obsidiansystems/queryDerivationOutputMap-evalStore
Give `Store::queryDerivationOutputMap` and `evalStore` argument
2023-12-11 11:16:18 -05:00
John Ericson 5f30c8acc7 Give Store::queryDerivationOutputMap and evalStore argument
Picking up where https://github.com/NixOS/nix/pull/9563 left off.
2023-12-11 10:39:08 -05:00
Fabian Möller 994f1b5c0d Add test cases for flake urls with fragments 2023-12-11 16:05:34 +01:00
Fabian Möller f45d2ee2b7 Fix query parsing for path-like flakes 2023-12-11 16:02:09 +01:00
John Ericson 2220a4a22c Merge remote-tracking branch 'upstream/master' into package-nix 2023-12-11 08:44:58 -05:00
Robert HensingandValentin Gagarin 89cf53648c Contributing branches and reverting (#9577)
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-11 12:26:31 +01:00
Robert Hensing da58c00ee0 Merge pull request #9586 from obsidiansystems/legacy-ssh-store-header
Create header for `LegacySSHStore`
2023-12-11 12:21:56 +01:00
Robert Hensing a05bc9eb92 allowed-uris: Match whole schemes also when scheme is not followed by slashes 2023-12-11 12:18:04 +01:00
Robert Hensing d3a85b6834 isValidSchemeName: Add function 2023-12-11 12:12:43 +01:00
Robert Hensing 79eb2920bb Add nix::isASCII*, locale-independent 2023-12-11 12:12:43 +01:00
Robert Hensing 1fa958dda1 isAllowedURI: Format 2023-12-11 12:12:43 +01:00
Robert Hensing 6cbba914a7 isAllowedURI: Remove incorrect note 2023-12-11 12:12:43 +01:00
Robert Hensing 91ba7b2307 isAllowedURI: Extract function and test 2023-12-11 12:12:42 +01:00
John Ericson 5c917c3204 Merge pull request #9587 from amjoseph-nixpkgs/pr/wopBuildDerivation/explain
libstore/daemon.cc: note trust model difference in readDerivation()s
2023-12-10 21:42:35 -05:00
Adam JosephandCole Helbling e43bb655fe libstore/daemon.cc: note trust model difference in readDerivation()s
Below the comment added by this commit is a much longer comment
followed by a trust check, both of which have confused me on at
least two occasions.  I figured it out once, forgot it, then had to
ask @Ericson2314 to explain it, at which point I understood it
again.  I think this might confuse other people too, or maybe I will
just forget it a third time.  So let's add a comment.

Farther down in the function is the following check:

```
if (!(drvType.isCA() || trusted))
  throw Error("you are not privileged to build input-addressed derivations");
```

This seems really strange at first.  A key property of Nix is that
you can compute the outpath of a derivation using the derivation
(and its references-closure) without trusting anybody!

The missing insight is that at this point in the code the builder
doesn't necessarily have the references-closure of the derivation
being built, and therefore needs to trust that the derivation's
outPath is honest.  It's incredibly easy to overlook this, because
the only difference between these two cases is which of these
identically-named functions we used:

- `readDerivation(Source,Store)`
- `Store::readDerivation()`

These functions have different trust models (except in the special
case where the first function is used on the local store).  We
should call the reader's attention to this fact.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-12-10 17:47:07 -08:00
John Ericson deadb3bfe9 Create header for LegacySSHStore
In https://github.com/NixOS/nix/pull/6134#issuecomment-1079199888,
@thuffschmitt proposed exposing `LegacySSHStore` in Nix for
deduplication with Hydra, at least temporarily. I think that is a good
idea.

Note that the diff will look bad unless one ignores whitespace! Also try
this locally:

```shell-session
git diff --ignore-all-space HEAD^:src/libstore/legacy-ssh-store.cc HEAD:src/libstore/legacy-ssh-store.cc
git diff --ignore-all-space HEAD^:src/libstore/legacy-ssh-store.cc HEAD:src/libstore/legacy-ssh-store.hh
```
2023-12-10 14:29:09 -05:00
John Ericson b7e016ab24 Merge pull request #9572 from obsidiansystems/serve-proto-build-options
Create `ServeProto::BuildOptions` and a serializer for it
2023-12-10 12:16:02 -05:00
3c200da242 document fetchTree (#9258)
* document `fetchTree`

* display experimental feature note at the top

we have to enable the new `fetchTree` experimental feature to render it
at all. this was a bug introduced when adding that new feature flag.

Co-authored-by: tomberek <tomberek@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Silvan Mosberger <github@infinisil.com>
2023-12-10 05:16:32 +00:00
Robert Hensing b1842a4f05 Merge pull request #9575 from hercules-ci/shell-on-hydra
flake.nix: Cache shell inputs through hydra
2023-12-10 04:42:29 +01:00
Robert Hensing a63be6578f flake.nix: Cache shell inputs through hydra 2023-12-09 21:22:20 +01:00
Robert Hensing 69b7876a08 Merge pull request #9573 from hercules-ci/rl-next-md-frontmatter
rl-next: Fix and support markdown frontmatter syntax
2023-12-09 21:18:04 +01:00
Robert Hensing a856f603ed Add checks.rl-next 2023-12-09 19:57:55 +01:00
Robert Hensing 3811b334c6 rl-next: Use markdown frontmatter syntax
The old syntax is still supported, as long as you don't use a {
in the description - the reason to migrate.
2023-12-09 19:57:55 +01:00
Robert Hensing 360f3b3a9e changelog-d: Use roberth fork with markdown frontmatter support 2023-12-09 19:50:33 +01:00
John Ericson 5417990e31 Create ServeProto::BuildOptions and a serializer for it
More tests, and more serializers for Hydra reuse.
2023-12-09 11:35:13 -05:00
Robert Hensing c8458bd731 Merge pull request #9555 from 9999years/positions-in-errors
Pass positions when evaluating
2023-12-09 03:55:58 +01:00
Robert Hensing 7cdc8786d9 Merge pull request #9568 from hercules-ci/revert-9553
Revert 9553
2023-12-09 03:49:53 +01:00
John Ericson 6aba2fdb4d Merge pull request #9306 from NixOS/accessor-no-follow-symlink
Clarify `SourceAccessor` methods should never implicitly follow symlinks
2023-12-08 21:22:37 -05:00
Robert Hensing 6e8d598314 tests/lang/eval-fail-bad-string-interpolation-4: init 2023-12-09 02:52:49 +01:00
Robert Hensing b9980b377e Update rl-next/source-positions-in-errors for Nix 2.19+ 2023-12-09 02:36:33 +01:00
Robert Hensing 9b7b7a7561 Revert "Print the value in error: cannot coerce messages (#9553)"
This reverts commit f0ac2a35d5.

The request from the sibling PR, which also applies here, was not addressed.
https://github.com/NixOS/nix/pull/9554#issuecomment-1845095735
2023-12-09 02:13:32 +01:00
John EricsonandCole Helbling ce4ca574d2 Clarify SourceAccessor methods should never implicitly follow symlinks
The code has already been fixed (yay!) so what is left of this commit is
just updating the API docs.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-12-08 20:00:47 -05:00
Robert Hensing d4f6b1d38b Merge pull request #9497 from edolstra/move-access-control
Move restricted/pure-eval access control out of the evaluator and into the accessor
2023-12-08 22:21:50 +01:00
Robert Hensing 005eaa1bd6 doc/prerequisites-source: Add bdwgc-traceable-allocator patch 2023-12-08 21:17:41 +01:00
John Ericson 762af72728 Merge pull request #9564 from NixOS/fix-clang-build
Avoid `std::strstream`, fix the clang build
2023-12-08 15:07:36 -05:00
Théophane Hufschmitt 36ca6adc60 Merge pull request #9563 from obsidiansystems/tryResolve-evalStore
Give `Derivation::tryResolve` an `evalStore` argument
2023-12-08 19:21:35 +01:00
John Ericson f9ee1bedcf Avoid std::strstream, fix the clang build
According https://en.cppreference.com/w/cpp/io/strstream, it has been
deprecated since C++98! The Clang + Linux build systems to not have it
at all, or at least be hiding it.

We can just use `std::stringstream` instead, I think.
2023-12-08 13:18:52 -05:00
Rebecca TurnerandValentin Gagarin f0ac2a35d5 Print the value in error: cannot coerce messages (#9553)
* Print the value in `error: cannot coerce` messages

This extends the `error: cannot coerce a TYPE to a string` message
to print the value that could not be coerced. This helps with debugging
by making it easier to track down where the value is being produced
from, especially in errors with deep or unhelpful stack traces.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-08 16:36:57 +00:00
John Ericson 139982997e Merge pull request #9560 from obsidiansystems/serve-proto-unkeyed-valid-path-info-serializer
Factor out `ServeProto::Serialiser<UnkeyedValidPathInfo>` and test
2023-12-08 11:33:24 -05:00
John Ericson 96dd757b0c Give Derivation::tryResolve an evalStore argument
This is needed for building CA deriations with a src store / dest store
split. In particular it is needed for Hydra.

https://github.com/NixOS/hydra/issues/838 currently puts realizations,
and thus build outputs, in the local store, but it should not.
2023-12-08 10:01:05 -05:00
Rebecca Turner 0b80935c22 Pass positions when evaluating
This includes position information in more places, making debugging
easier.

Before:

```
$ nix-instantiate --show-trace --eval tests/functional/lang/eval-fail-using-set-as-attr-name.nix
error:
       … while evaluating an attribute name

         at «none»:0: (source not available)

       error: value is a set while a string was expected
```

After:

```
error:
       … while evaluating an attribute name

         at /pwd/lang/eval-fail-using-set-as-attr-name.nix:5:10:

            4| in
            5|   attr.${key}
             |          ^
            6|

       error: value is a set while a string was expected
```
2023-12-07 10:27:21 -08:00
John Ericson a5521b7d94 Factor out ServeProto::Serialiser<UnkeyedValidPathInfo> and test
In the process, partially undo e89b5bd0bf
in that the ancient < 2.4 version is now supported again by the
serializer again. `LegacySSHStore`, instead of also asserting that the
version is at least 4, just checks that `narHash` is set.

This allows us to better test the serializer in isolation for both
versions (< 4 and >= 4).
2023-12-07 11:34:18 -05:00
Eelco Dolstra c3827ff634 Merge pull request #9557 from bryanhonof/bryanhonof.fix-apple-double-shenanigans
Add option to libarchive so it behaves correctly
2023-12-07 12:33:02 +01:00
Bryan Honof bf00d5ecef fix(libutil/tarfile): add option to libarchive so it behaves correctly with AppleDouble files
AppleDouble files were extracted differently on macOS machines than on other
UNIX's.
Setting `archive_read_set_format_option(this->archive, NULL ,"mac-ext",NULL)`
fixes this problem, since it just ignores the AppleDouble file and treats it as
a normal one.
This was a problem since it caused source archives to be different between macOS
and Linux.

Ref: nixos/nix#9290
2023-12-07 11:35:15 +01:00
tomberek 82449a455f Merge pull request #9452 from kolloch/feature/nix-hash-convert
Add nix hash convert
2023-12-06 19:47:53 -05:00
Peter Kolloch 9a1a3c43bf Store.xs: fix references to HashFormat::Nix32
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch e9a5365db6 hash.sh: Make failure tests more tolerant of additional output
"warning: you don'\''t have Internet access; disabling some network-dependent features" ...

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch bbba2055f0 Refactor concurrently added tests to use HashAlgorithm.
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter KollochandThéophane Hufschmitt d38ec12855 Update src/libexpr/primops.cc
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2023-12-06 23:43:42 +01:00
Peter Kolloch 8afeaf05c4 Add docs/rl-notes for nix hash convert / builtins.convertHash
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 7ff876b92b Add deprecation notice for old nix hash conversion subcommands.
(But not yet nix-hash since `nix hash` is still hidden behind a feature flag.)

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch fc6f29053a Renamed HashFormat::Base32 to HashFormat::Nix32
...and also adjusted parsing accordingly.

Also added CLI completion for HashFormats.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 837b889c41 Further HashType renaming + using mkHashAlgoOptFlag for new conversion
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 5334c9c792 HashType: Rename to HashAlgorithm
To be consistent with CLI, nix API
and many other references.

As part of this, we also converted it to a scoped enum.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 0c2d5f7673 nix hash convert: s/--type/--algo/ + more functional tests
https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:42 +01:00
Peter Kolloch 6bbd900d4f nix hash convert: added
This deviated from the proposal! See comments on the issue.

https://github.com/NixOS/nix/issues/8876
2023-12-06 23:43:34 +01:00
Peter Kolloch 156ea78d74 CmdHashBase: doc comment 2023-12-06 23:41:07 +01:00
Peter Kolloch e7abf60a0c hash.cc/hash.h: Minor C++ improvements 2023-12-06 23:41:07 +01:00
Eelco Dolstra 3dcb83409d Merge pull request #9509 from fricklerhandwerk/add-redirect
add redirect to new store page
2023-12-06 18:03:53 +01:00
Eelco Dolstra 9817788f6b Merge pull request #9518 from NixOS/fix-static-build
Update Nixpkgs to fix static build
2023-12-06 18:02:30 +01:00
Eelco Dolstra 34a69976d9 Merge pull request #9545 from NixOS/update-labeler-config
Fix the labeler.yml config file
2023-12-06 18:01:02 +01:00
Théophane Hufschmitt 7fff625e39 Improve the error message for “multicommands” commands (#9510)
* Factor out the default `MultiCommand` behavior

All the `MultiCommand`s had (nearly) the same behavior when called
without a subcommand.
Factor out this behavior into the `NixMultiCommand` class.

* Display the list of available subcommands when none is specified

Whenever a user runs a command that excepts a subcommand, add the list
of available subcommands to the error message.

* Print the multi-command lists as Markdown lists

This takes more screen real estate, but is also much more readable than
a comma-separated list
2023-12-06 13:13:45 +00:00
Eelco DolstraandRobert Hensing 2bd8322500 Update src/libfetchers/filtering-input-accessor.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-06 14:08:40 +01:00
Théophane Hufschmitt ee8540ae90 Fix the labeler.yml config file
labeler 5.0 changed the configuration file in a non-backwards-compatible way (https://github.com/actions/labeler/tree/main#breaking-changes-in-v5), so update our config file to match that (because all the CIs are red otherwise 😬 ).
2023-12-06 14:07:08 +01:00
Eelco Dolstra 53ab5d87c2 Use expectStderr 2023-12-06 14:05:32 +01:00
Eelco Dolstra 57246c4c38 PosixSourceAccessor: Cache lstat() calls
Since we're doing a lot of them in assertNoSymlinks().
2023-12-06 13:55:54 +01:00
Eelco Dolstra 504e4fc457 CanonPath: Support std::hash 2023-12-06 13:45:59 +01:00
John Ericson fbc855b3c3 Merge pull request #9542 from NixOS/pthread-fix-netbsd
Add missing `-pthread` for test support libraries
2023-12-05 18:44:25 -05:00
John Ericson b23273f6a2 Add missing -pthread for test support libraries
This is good in general (see how the other libraries also have long had
it, since 49fe9592a4) but in particular
needed to fix the NetBSD build.
2023-12-05 18:18:30 -05:00
John Ericson 96fdea3394 Merge pull request #9541 from obsidiansystems/config-buildprefix-fix
Including `config.h` also needs `$(buildprefix)`
2023-12-05 17:16:57 -05:00
Eelco Dolstra 83c067c0fa PosixSourceAccessor: Don't follow any symlinks
All path components must not be symlinks now (so the user needs to
call `resolveSymlinks()` when needed).
2023-12-05 23:02:59 +01:00
John Ericson 733333e87d Including config.h also needs $(buildprefix)
Per the instruction in the manual, we want to run configure in a
different directory so that we can configure + build for multiple
platforms. That means `config.h` will be in the build directory. This is
just like `Makefile.config`, which already is used with
`$(buildprefix)`.
2023-12-05 16:36:12 -05:00
Robert Hensing efc65ef122 Merge pull request #9505 from fricklerhandwerk/quick-start
shorten the quick start chapter
2023-12-05 19:52:03 +01:00
Eelco Dolstra 345f79d016 Check that we can't follow symlinks outside of the allowed paths 2023-12-05 16:33:02 +01:00
Eelco Dolstra d54f847682 Merge pull request #9520 from fricklerhandwerk/fix-links-stores-overview
fix links in stores overview
2023-12-05 14:46:54 +01:00
Eelco Dolstra 2f9bc1ecda Merge pull request #9536 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.2.0
Bump zeebe-io/backport-action from 2.1.1 to 2.2.0
2023-12-05 14:14:22 +01:00
Eelco Dolstra a4b3a63b1c Merge pull request #9537 from NixOS/dependabot/github_actions/cachix/install-nix-action-24
Bump cachix/install-nix-action from 23 to 24
2023-12-05 14:14:10 +01:00
Eelco Dolstra 4931697825 Merge pull request #9538 from NixOS/dependabot/github_actions/cachix/cachix-action-13
Bump cachix/cachix-action from 12 to 13
2023-12-05 14:13:59 +01:00
Eelco Dolstra 6278e264f7 Merge pull request #9539 from NixOS/dependabot/github_actions/actions/labeler-5
Bump actions/labeler from 4 to 5
2023-12-05 14:13:47 +01:00
dependabot[bot] e488a43f45 Bump actions/labeler from 4 to 5
Bumps [actions/labeler](https://github.com/actions/labeler) from 4 to 5.
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](https://github.com/actions/labeler/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-04 22:09:25 +00:00
dependabot[bot] e6a3cbfceb Bump cachix/cachix-action from 12 to 13
Bumps [cachix/cachix-action](https://github.com/cachix/cachix-action) from 12 to 13.
- [Release notes](https://github.com/cachix/cachix-action/releases)
- [Commits](https://github.com/cachix/cachix-action/compare/v12...v13)

---
updated-dependencies:
- dependency-name: cachix/cachix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-04 22:09:18 +00:00
dependabot[bot] c446e5294d Bump cachix/install-nix-action from 23 to 24
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 23 to 24.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/v23...v24)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-04 22:09:14 +00:00
dependabot[bot] 823512c1e7 Bump zeebe-io/backport-action from 2.1.1 to 2.2.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.1.1 to 2.2.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.1.1...v2.2.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-04 22:09:10 +00:00
Valentin Gagarin 5fe2accb75 fix up release note 2023-12-04 16:42:32 +01:00
Valentin Gagarin 3c310bde2e reword description for the fetch-tree experimental feature
without knowing a lot of context, it's not clear who "we" are in that
text. I'm also strongly opposed to adding procedural notes into
a reference manual; it just won't age well.

this change leaves a factual description of the experimental feature and
its purpose.
2023-12-04 16:42:32 +01:00
John Ericson 5910140f25 Merge pull request #9529 from wh0/patch-1
tests: avoid a chroot store without sandbox support
2023-12-04 08:57:20 -05:00
wh0 2e5abc0fd0 tests: avoid a chroot store without sandbox support 2023-12-03 17:18:58 -08:00
John Ericson 14c26d642e Clean up two comments 2023-12-03 18:57:16 -05:00
John Ericson 6e0656c66c Add another configure flag assertion 2023-12-03 18:53:05 -05:00
John Ericson 7e2b1cce6a Slap on perl-bindings in the caller
The Perl bindings are not part of Nix, but a downstream package, so they
don't belong in `package.nix`.

They don't really belong as an attribute on `nix` either, but we can
just leave that interface as is for now.
2023-12-03 18:48:26 -05:00
John Ericson a5a45e64e1 Don't expose file sets anymore 2023-12-03 18:45:15 -05:00
John Ericson f58615518c Add documenting comments to package.nix 2023-12-03 18:39:33 -05:00
John Ericson 77003a4f0c Factor out the installer script 2023-12-03 18:29:15 -05:00
John Ericson 60fe4ddaa1 Expose boehmgc-nix in overlay 2023-12-03 18:26:51 -05:00
John Ericson e275f0adfb Move binary-tarball.nix to scripts dir 2023-12-03 18:16:07 -05:00
John Ericson 7a7ad7c84b Remove uneeded file 2023-12-03 18:14:36 -05:00
John Ericson c160c62515 Fix underlying build system so --disable-build works better
- Internal API docs once again work

- configure skips checks for a bunch of things it doesn't need
2023-12-03 18:12:22 -05:00
John Ericson 7b51086d73 More fixes 2023-12-03 18:12:05 -05:00
John Ericson c71d987553 Fix incorrect flag name 2023-12-03 17:12:38 -05:00
John Ericson c9838bb9ee Merge remote-tracking branch 'upstream/master' into package-nix 2023-12-03 17:12:28 -05:00
John Ericson 3d47e02483 WIP 2023-12-03 16:48:50 -05:00
John Ericson ce598bae14 WIP 2023-12-03 14:10:09 -05:00
John Ericson 0ca49b0c86 Add installing unit test flags 2023-12-03 12:47:54 -05:00
Jacek Galowicz 19d41fb20a Fix stuff 2023-12-02 17:25:47 +00:00
Jacek Galowicz 118fa9689a Create internal-api-docs.nix 2023-12-02 16:42:01 +00:00
Jacek Galowicz ca59832808 Fix coverage.nix 2023-12-02 16:36:59 +00:00
Jacek Galowicz 2c3749a335 Fix cross builds 2023-12-02 16:08:06 +00:00
Théophane Hufschmitt cd0d4e4d47 Merge pull request #9524 from fricklerhandwerk/fix-docs-errors
fix random docs errors
2023-12-02 07:36:05 +01:00
Valentin Gagarin 5b281ddf50 reword description of the max-jobs setting
- remove prose for the default value, which is shown programmatically
- add note on how this relates to `cores`
- add link to mentioned derivation attribute
2023-12-02 04:06:26 +01:00
Valentin Gagarin 368fdb482d reword description of the builders-use-substitutes setting 2023-12-02 03:07:27 +01:00
Valentin Gagarin 24b781773f fix random docs errors
remove link to the contributing guide from user documentation.
it doesn't help here, and the target at first glance shows redundant
information.
2023-12-02 03:02:59 +01:00
Valentin Gagarin 51adfb9b27 reword documentation on settings and attributes related to substitution
- add links
- be more concise
- clarify the distinction between `preferLocalBuild` and `allowSubstitutes`
2023-12-02 02:56:25 +01:00
Valentin Gagarin 59c4c82aeb fix links in stores overview 2023-12-02 00:56:41 +01:00
John Ericson 7355a48b1a flake.lock: Update Nixpkgs to fix static build
The problem was since switching to use libgit2, we had a package in our
closure (`http-parser`) that was always trying to build as a shared
object.

Underlying Nixpkgs PR (a 23.05 backport)
https://github.com/NixOS/nixpkgs/pull/271202

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9ba29e2346bc542e9909d1021e8fd7d4b3f64db0' (2023-11-13)
  → 'github:NixOS/nixpkgs/36c4ac09e9bebcec1fa7b7539cddb0c9e837409c' (2023-11-30)
2023-12-01 15:59:41 -05:00
Robert Hensing 84fe429dfd Merge pull request #8886 from obsidiansystems/flatten-tests
Move unit tests to separate directories, and document
2023-12-01 17:14:38 +01:00
John EricsonandValentin Gagarin 91b6833686 Move tests to separate directories, and document
Today, with the tests inside a `tests` intermingled with the
corresponding library's source code, we have a few problems:

- We have to be careful that wildcards don't end up with tests being
  built as part of Nix proper, or test headers being installed as part
  of Nix proper.

- Tests in libraries but not executables is not right:

  - It means each executable runs the previous unit tests again, because
    it needs the libraries.

  - It doesn't work right on Windows, which doesn't want you to load a
    DLL just for the side global variable . It could be made to work
    with the dlopen equivalent, but that's gross!

This reorg solves these problems.

There is a remaining problem which is that sibbling headers (like
`hash.hh` the test header vs `hash.hh` the main `libnixutil` header) end
up shadowing each other. This PR doesn't solve that. That is left as
future work for a future PR.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-01 10:48:58 -05:00
Robert Hensing 77adb55ae4 Merge pull request #9511 from obsidiansystems/add-missing-includes
Add two missing `#include "nar-info.hh"`
2023-12-01 16:43:56 +01:00
John Ericson d59bdbe4fd Add two missing #include "nar-info.hh"
GitHub's racy CI caused this oversight to sneak through.
2023-12-01 10:20:19 -05:00
John Ericson ac4d2e7b85 Merge pull request #9478 from tweag/nix-config-check
Rename `nix doctor` to `nix config check`
2023-12-01 10:19:19 -05:00
Théophane Hufschmitt dfa219d03b Merge pull request #9494 from sellout/nix-run-execv
Don’t use `execvp` when we know the path
2023-12-01 16:07:16 +01:00
Robert Hensing fcf09813c6 Merge pull request #6236 from obsidiansystems/store-dir-config
Factor out `StoreDirConfig`
2023-12-01 15:38:14 +01:00
Théophane Hufschmitt ea2dd16623 Use a proper enum rather than a boolean in runProgramInStore
Makes the call-site much easier to understand.
2023-12-01 15:35:21 +01:00
John Ericson 82359eba6b Merge pull request #9233 from bouk/bouk/apply-config-inner
config: add included files into parsedContents before applying
2023-12-01 08:23:32 -05:00
Jacek Galowicz eff9b12bc2 Further changes 2023-12-01 11:25:22 +00:00
Robert Hensing 188c803ddb Merge pull request #9508 from infinisil/revert-7762-boost-regex
Revert "Switch from std::regex to boost::regex"
2023-12-01 02:08:58 +01:00
Valentin Gagarin d5e934fb73 add redirect to new store page 2023-12-01 01:54:48 +01:00
Silvan Mosberger 333ea684b0 Add boost::regex regression test 2023-12-01 01:39:52 +01:00
Valentin GagarinandJohn Ericson <John.Ericson@Obsidian.Systems 4781e7fa70 Document each store type on its own page
This makes for more useful manual table of contents, that displays the
information at a glance.

The `nix help-stores` command is kept as-is, even though it will show up
in the manual with the same information as these pages due to the way it
is written as a "`--help`-style" command. Deciding what to do with that
command is left for a later PR.

This change also lists all store types at the top of the respective overview page.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems
2023-12-01 01:27:52 +01:00
Silvan Mosberger 908a011a4a Revert "Switch from std::regex to boost::regex" 2023-12-01 00:50:20 +01:00
Valentin Gagarin 02d9cf2d30 shorten the quick start chapter
this focuses on `nix-shell -p` and refers to search.nixos.org for
package search, which is currently the easiest and most effective way to
find program names.
2023-12-01 00:42:34 +01:00
Jacek Galowicz f55ee7cf77 little refactoring 2023-11-30 22:53:46 +00:00
Jacek Galowicz c64190e650 Run statix 2023-11-30 22:53:46 +00:00
Jacek Galowicz c982198485 First step 2023-11-30 22:48:44 +00:00
Valentin Gagarin 0301b8fc73 reword the experimental feature notice
- put the highlight box around all the relevant instructions
- simplify the wording
- make the link more prominent by using the whole phrase for the link text
2023-11-30 23:24:53 +01:00
Valentin Gagarin 39de819eda rename debugging helper environment variable 2023-11-30 23:23:32 +01:00
Valentin Gagarin cab41025d8 mention renaming of nix doctor 2023-11-30 23:04:05 +01:00
John Ericson 2fe7e009bb Merge pull request #9502 from fricklerhandwerk/internal-lookup-paths
use lookup paths in helper expressions consistently
2023-11-30 16:23:24 -05:00
Eelco Dolstra 8cafc754d8 Move access control from FSInputAccessor to FilteringInputAccessor 2023-11-30 21:54:53 +01:00
Valentin Gagarin d5ffc94f33 use lookup paths in helper expressions consistently
this makes the files in question a bit more independent of source location.

to find where the value is set and how it's wired up:

    rg nix=doc/manual
2023-11-30 21:53:31 +01:00
John Ericson 7cc2aa799e Merge pull request #9500 from fricklerhandwerk/generated-docs
keep generated documentation in a separate directory
2023-11-30 15:51:01 -05:00
Valentin Gagarin 44d21f6ef9 keep generated documentation in a separate directory
- helps navigating the code as it highlights which files are generated
- makes it less error prone when working incrementally
  (although this should be just fixed by building out of tree)
2023-11-30 20:39:24 +01:00
Rebecca Turner be30c2ea8d Don't attempt to git add ignored files
This uses `git check-ignore` to determine if files are ignored before
attempting to add them in `putFile`.

We also add a condition to the `fetchFromWorkdir` filter to always add
the `flake.lock` file, even if it's not tracked. This is necessary to
resolve inputs.

This fixes #8854 without `git add --force`.
2023-11-30 10:26:13 -08:00
Théophane Hufschmitt 3d46fa85c8 Merge pull request #9489 from NixOS/doc-url
Avoid `<name>/<name>` in documentation URLs
2023-11-30 19:08:24 +01:00
Eelco Dolstra 43d9fb6cf1 Remove InputAccessor::root() 2023-11-30 16:44:54 +01:00
Eelco Dolstra 305939655a Remove superfluous use of hasAccessControl() 2023-11-30 16:28:33 +01:00
Eelco Dolstra ea95327e72 Move restricted/pure-eval access control out of the evaluator and into the accessor 2023-11-30 16:16:17 +01:00
Eelco Dolstra 3bebaefcd0 Merge pull request #9476 from alois31/restore-progress-bar
nix repl: Only hide the progress bar while waiting for user input
2023-11-30 16:07:28 +01:00
John Ericson f99e468640 Avoid <name>/<name> in documentation URLs
They are redundant and look weird.
2023-11-30 09:37:32 -05:00
Eelco Dolstra cb7f25869d Merge pull request #9430 from hercules-ci/remove-vlas
Fix stack overflow in `filter`
2023-11-30 15:31:44 +01:00
Eelco Dolstra b6a3fde6b7 Merge pull request #9465 from obsidiansystems/build-dir
Use `buildprefix` in a few more places
2023-11-30 15:28:37 +01:00
Eelco Dolstra eac0a62052 Merge pull request #9491 from obsidiansystems/fix-freebsd
Improve ACL clearing support (fixing FreeBSD build)
2023-11-30 15:27:24 +01:00
Greg Pfeil 743232bf04 Don’t use execvp when we know the path 2023-11-30 00:17:25 -07:00
John Ericson ca72e3e7e8 Merge pull request #5145 from fedepell/local_doc_build_5140
Docs build: depend on locally built nix executable and not installed one
2023-11-30 01:14:34 -05:00
Federico Pellegrin d536c57e87 Docs build: depend on locally built nix executable and not installed one
Previously many of the documentation targets were depending on
`$(bindir)/nix` which is the installed version. This meant that its
install rules would be triggered (which in chain would also trigger the
install of libraries, as reported in #5140). Therefore a build of the
documentation without an installation would not be possible (which apart
from doing unwanted operations it may also generate permission problems
for example).

The fix makes the rules depend on `$(nix_PATH)` instead, which is the
executable in the build tree.
2023-11-30 00:44:45 -05:00
John Ericson a7115a47ef Improve ACL clearing support (fixing FreeBSD build)
The problem was that f880469173 forgot
that the `#include <sys/xattr.h>` was guarded by an `#ifdef __linux__`.

However, the build failure was only on FreeBSD --- turns out other
platforms have this header too!

The fix therefore uses a new configure check so we properly clear ACLs
on more platforms.
2023-11-29 21:09:19 -05:00
Alex AmeenandJohn Ericson 02bd821f2e fix: nlohmann::adl_serializer for std::optional (#9147)
This allows templates such as `NLOHMANN_DEFINE_TYPE_*` templates and other generators with things like `std::vector<std::optional<T>>`.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2023-11-30 01:26:39 +00:00
John EricsonandValentin Gagarin 52e0911302 Use buildprefix in a few more places
`installcheck` doesn't yet work, but the rest of the build can now
happen mostly inside a separate build directory.

Progress on #9342

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-29 19:49:07 -05:00
Eelco Dolstra 043413bb59 boost::span -> std::span 2023-11-29 12:38:46 +01:00
Eelco Dolstra b36857ac8d Add a Git-based content-addressed tarball cache
GitArchiveInputScheme now streams tarballs into a Git repository. This
deduplicates data a lot, e.g. when you're fetching different revisions
of the Nixpkgs repo. It also warns if the tree hash returned by GitHub
doesn't match the tree hash of the imported tarball.
2023-11-29 12:37:03 +01:00
Théophane Hufschmitt a8fea5a54f Merge pull request #9477 from tweag/nix-config-show
Rename `nix show-config` to `nix config show`
2023-11-28 15:04:15 +01:00
Théophane Hufschmitt 6d1605818c Rename nix doctor to nix config check
Fix #7672
2023-11-28 14:28:19 +01:00
Théophane Hufschmitt f300e11b05 Rename nix show-config to nix config show
Part of #7672
2023-11-28 14:28:19 +01:00
Tom Bereknyei f66f498bd4 notes: document change in nix-shell behavior 2023-11-27 21:22:59 -05:00
Tom Bereknyei 9a4641146f tests: ensure nix-shell uses relative paths for expressions 2023-11-27 14:40:38 -05:00
Matthew Bauer 04023360ed Evaluate nix-shell -i args relative to script
When writing a shebang script, you expect your path to be relative to
the script, not the cwd. We previously handled this correctly for
relative file paths, but not for expressions.

This handles both -p & -E args. My understanding is this should be
what we want in any cases I can think of - people run scripts from
many different working directories. @edolstra is there any reason to
handle -p args differently in this case?

Fixes #4232
2023-11-27 14:40:38 -05:00
Alois Wohlschlager 20cd5eb2b3 nix repl: Only hide the progress bar while waiting for user input
In commit 0d2163c6dc, the progress bar was hidden
in nix repl because of a regression that caused it to interfere with user
input. Several users like(d) seeing the progress bar in the repl during builds.
Only hiding it while waiting for user input gives us the best of both worlds,
so do just that.
2023-11-27 19:25:15 +01:00
Robert Hensing f0180487a0 Merge pull request #9462 from trofi/eval.o-dependency
libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
2023-11-27 17:25:21 +01:00
Robert Hensing 928f0c1341 Merge pull request #9393 from hercules-ci/changelog-d
Automatically compile hand-written release notes with `changelog-d`
2023-11-27 16:29:16 +01:00
Robert Hensing 68c48756fe libexpr/local.mk: Make eval compile deps regular
Dependency is now entirely through the eval.cc rule.
All gen.hh deps are now there.
2023-11-27 15:52:24 +01:00
Robert HensingandValentin Gagarin f7bfec2806 maintainers/release-notes: Improve DATE check
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-27 15:18:29 +01:00
tomberek 0c6fe1be7d Merge pull request #9459 from fricklerhandwerk/page-redirects
add path based redirects
2023-11-27 08:28:27 -05:00
Valentin Gagarin 384ffb4443 add deprecation warnings in documentation
this is hacky, but can serve as a stopgap until we can do it
programmatically.
2023-11-27 14:07:32 +01:00
Théophane Hufschmitt d46230eff5 Merge pull request #7762 from yorickvP/boost-regex
Switch from std::regex to boost::regex
2023-11-27 10:59:22 +01:00
Théophane Hufschmitt e986d20bed Remove an obsolete comment 2023-11-27 10:25:18 +01:00
Sergei Trofimovich 75134b7513 libexpr: add missing dependency on 'flake/call-flake.nix.gen.hh'
Without the change build for `eval.o` fails occasionally as:

    $ make src/libexpr/eval.o
      GEN    Makefile.config
      GEN    src/libexpr/primops/derivation.nix.gen.hh
      GEN    src/libexpr/fetchurl.nix.gen.hh
      GEN    src/libexpr/parser-tab.cc
      GEN    src/libexpr/lexer-tab.cc
    src/libexpr/lexer.l:314: warning, -s option given but default rule can be matched
      CXX    src/libexpr/eval.o
    src/libexpr/eval.cc:519:18: fatal error: flake/call-flake.nix.gen.hh: No such file or directory
      519 |         #include "flake/call-flake.nix.gen.hh"
          |                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    make: *** [mk/patterns.mk:3: src/libexpr/eval.o] Error 1

Noticed in https://github.com/NixOS/nixpkgs/pull/269439
2023-11-27 08:56:24 +00:00
Moritz AngermannandCole Helbling f56401a114 nix flake update add deprecation warnings.
This builds on #8817, to add additional UX help for people with existing
muscle memory (or shell history) with --update-input and tries to gently
guide them towards the newly evolved CLI UI.

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2023-11-27 09:08:15 +01:00
Valentin Gagarin 2b7016cc56 add path based redirects
up to now, those were managed outside of this repo, which as
unsurprisingly a real hassle to deal with if one wanted to prevent URLs
from breaking when moving pages around. this change removes a large part
of the friction involved in moving content in the Nix manual.

possible next steps for further automation:
- check for content that moved and warn if it's not reachable from
  links that were valid prior to a change
- create redirect rules automatically based on this information
2023-11-27 08:33:03 +01:00
Bob van der Linden 06a745120b nix: develop: remove test for interactive shell executable 2023-11-26 21:27:46 +01:00
Robert Hensing e7e21aa0c8 flake.nix: Use top level changelog-d
It is about 2 MB now, as only it and libffi (tiny) are new in the
build or shell closures.
2023-11-26 21:21:55 +01:00
Robert Hensing c5d49ec7ab flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/decdf666c833a325cb4417041a90681499e06a41' (2023-11-18)
  → 'github:NixOS/nixpkgs/9ba29e2346bc542e9909d1021e8fd7d4b3f64db0' (2023-11-23)
2023-11-26 21:21:55 +01:00
Robert Hensing d63f72197c Don't run changelog-d in the build
This way we lose the preview of release notes on master, as well
as on https://nixos.org/manual/nix/unstable/release-notes/rl-next
but we can come back to this.
2023-11-26 21:21:55 +01:00
Jan Tojnar f25c06d7a3 docs: Fix broken link
Link target definitions need to be in a separate paragraph to be collected.

Fixup for https://github.com/NixOS/nix/commit/217d863f7a251a4d8a08ff3294944b45146c61c9
2023-11-26 19:32:28 +01:00
John Ericson fdac6e3818 Merge pull request #9451 from kolloch/feature/git-ignore-ds-store
gitignore: Also ignore .DS_Store
2023-11-25 11:47:49 -05:00
Peter Kolloch 213594721a gitignore: Also ignore .DS_Store
This is a file that Finder on Mac OS loves to add into various folders.
2023-11-25 17:30:03 +01:00
Yorick van Pelt d2f5e263e3 Switch from std::regex to boost::regex 2023-11-25 15:14:18 +01:00
John Ericson dda0e34ecf Merge pull request #9443 from ivan770/reproducibility
doc: fix machine-specific capabilities leaking
2023-11-24 22:59:38 -05:00
ivan770 54b6847655 doc: fix machine-specific capabilities leaking 2023-11-24 11:17:35 -05:00
Robert Hensing 857f9168f7 Migrate rl-next.md to doc/manual/rl-next directory 2023-11-24 16:53:33 +01:00
Robert Hensing 6971c4adc0 maintainers/release-notes <- scripts/release-notes 2023-11-24 16:53:33 +01:00
Robert Hensing 7c4ee5c813 scripts/release-notes: Avoid mutating variables 2023-11-24 16:53:33 +01:00
Robert Hensing b1ea30f21d scripts/release-notes: Support patch releases
This also fixes the broken case statement, which has globs, not regexes.
2023-11-24 16:53:33 +01:00
Robert Hensing 2a538c571b Add scripts/release-notes 2023-11-24 16:53:33 +01:00
Théophane Hufschmitt 7f626dba33 Merge pull request #9398 from Qyriad/fixes/flake-not-found
flakes: bare minimum fix the error message for untracked flake.nix
2023-11-24 15:33:08 +01:00
Valentin Gagarin 9aa63f70d7 fricklerhandwerk: subscribe to documentation changes (#9422)
* fricklerhandwerk: subscribe to documentation changes
2023-11-24 14:32:02 +00:00
Robert Hensing b26038c517 doc: Rename 2X.XX to "Upcoming release", and only generate if applicable 2023-11-24 15:13:23 +01:00
Robert Hensing b7982372d2 Compile hand-written release notes with changelog-d 2023-11-24 15:13:21 +01:00
Robert Hensing 8a52325d7d Merge pull request #9444 from phip1611/foldl-doc
doc: primops: fix typo
2023-11-24 13:36:18 +01:00
Robert Hensing 43fb39ca29 Merge pull request #9445 from NixOS/allow-input-in-git-commit
Allow user input in `git commit`
2023-11-24 13:34:59 +01:00
Théophane Hufschmitt 6a94755b12 Allow user input in git commit
We occasionnally commit to git repositories (like with `nix flake update --commit-lock-file`).
This shells out to `git commit`, which might wait for user input (for a signing key passphrase for instance).
Disable the progress bar while this is running to make sure that the
user can enter it.
2023-11-24 11:45:37 +01:00
Philipp Schuster 5be0e6b314 doc: primops: fix typo 2023-11-24 10:50:01 +01:00
Philipp SchusterandValentin Gagarin 2ce8c9650b doc: primops: add more info for foldl (#9254)
* doc: primops: add more info for foldl

From the existing doc it is not obvious whether the first or the
second argument is the accumulator. This is however relevant to
know, as for certain scenarios, this might change the behavior.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-11-23 21:02:20 +00:00
Théophane Hufschmitt bf13943206 Merge pull request #9173 from vkryachko/transitive-input-overrides
Fix transitive input locking.
2023-11-23 08:54:45 +01:00
Robert Hensing 030fb86dd1 Merge pull request #9394 from NixOS/separate-file-canonicalise
Put `canonicaliseTimestampAndPermissions` in its own header/file
2023-11-22 18:27:52 +01:00
Eelco Dolstra 38844943d0 Merge pull request #9413 from edolstra/fetchToStore-cache
Persistently cache `InputAccessor::fetchToStore()`
2023-11-22 12:19:25 +01:00
Eelco Dolstra 533eafc5f2 Merge pull request #9438 from R-VdP/fix_unused_variable_warning
Fix compile warning due to unused variable binding.
2023-11-22 12:09:10 +01:00
Eelco Dolstra b1ab592f28 Use the StorePath-based cache interface 2023-11-22 11:44:02 +01:00
r-vdp 5292f36426 Fix compile warning due to unused variable binding.
We still need the check, since we don't have narinfo for locally built store paths.
2023-11-22 11:34:53 +01:00
Eelco DolstraandJohn Ericson 61b76f5f34 Apply suggestion
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-11-22 11:26:12 +01:00
John Ericson 7155df091f Merge pull request #9431 from obsidiansystems/fix-rapidcheck-shared
Add missing `-lrapidcheck` fixing build with shared lib
2023-11-21 23:47:40 -05:00
John Ericson 46131567da Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).
2023-11-21 23:19:25 -05:00
Robert Hensing 4e790efade Use boost::container::small_vector in place of VLAs 2023-11-21 20:52:27 +01:00
John Ericson a6b315ae80 Merge pull request #7912 from mupdt/gcstore-add-perm-root
Mounted SSH Store
2023-11-21 14:04:45 -05:00
Matej Urbas 03c3af1bf9 mounted-ssh-ng store: integration tests 2023-11-21 13:35:12 -05:00
mupdt b32b20a6d7 release note entry for the mounted-ssh-ng:// store 2023-11-21 13:35:07 -05:00
Matej Urbas 06b8902562 MountedSSHStore: stores on shared filesystems 2023-11-21 13:34:01 -05:00
Matej Urbas 226b0f3956 Extend the worker protocol with wopAddPermRoot 2023-11-21 13:26:55 -05:00
Matej Urbas 9796ebd7ef Add --process-ops flag to nix-daemon 2023-11-21 13:18:19 -05:00
John Ericson 949f5841f8 Add the MountedSSHStore experimental feature
It will be implemented in the subsequent commits of this PR.
2023-11-21 13:16:46 -05:00
John Ericson f880469173 Put canonicaliseTimestampAndPermissions in its own header/file
It is not inherently tied to `LocalStore`, it could probably even go in
`libnixutil`. Functions not attached to `LocalStore` should not be
declared in `local-store.hh`.

I am moving it to facilitate experimenting for #9344. If
canonicalisation should be done client-side in client-side builds, there
wouldn't be a `LocalStore` at all so having to include that header to
get this freestanding function is cumbersome and wrong.

Perhaps canonicalisation should still be done server-side for security
reasons --- I don't mean to make that judgement call now --- but even if
so, this freestanding function still isn't connected to `LocalStore` so
while less urgent it is still better to move out of this header.
2023-11-21 12:57:59 -05:00
Robert Hensing 9cd69e1c39 Merge pull request #9411 from edolstra/path-display
Improve SourceAccessor path display
2023-11-21 18:12:01 +01:00
Lin Jian 4d8decbd13 doc: fix number of template attributes
This number is not updated when welcomeText is added[1][2].

[1]: f3a2940e70
[2]: https://github.com/NixOS/nix/pull/6103
2023-11-21 16:52:25 +01:00
Théophane Hufschmitt 9ad9e988b8 Merge pull request #9425 from iFreilicht/fix-unbound-variable-errors
Fix "unbound variable" errors in bash
2023-11-21 15:12:32 +01:00
Felix Uhl 64827360be Fix "unbound variable" errors in bash
Fixes #9414
2023-11-21 14:49:48 +01:00
Eelco Dolstra c185e45ed7 Merge pull request #9410 from edolstra/shallow
Some shallow git fetching improvements
2023-11-21 12:58:01 +01:00
John Ericson 516e7ddc41 Merge pull request #9257 from Artturin/nixenvjsondrvpath
`nix-env --query`: fix `--json` ignoring `--drv-path`
2023-11-20 16:32:35 -05:00
Eelco Dolstra 99d5204baa Persistently cache InputAccessor::fetchToStore()
This avoids repeated copying of the same source tree between Nix
invocations. It requires the accessor to have a "fingerprint" (e.g. a
Git revision) that uniquely determines its contents.
2023-11-20 20:04:37 +01:00
Eelco Dolstra a0162d5732 Improve SourceAccessor path display
Backported from lazy-trees. This allows SourceAccessors to show the
origin of the accessor. E.g. we now get

  copying '«git+https://github.com/blender/blender.git?ref=refs/heads/main&rev=4edc1389337dd3679ff66969c332d2aff52e1992»/' to the store

instead of

  copying '/' to the store
2023-11-20 18:54:36 +01:00
Eelco Dolstra 071f14a0bb Don't do shallow fetches over ssh 2023-11-20 17:25:40 +01:00
Eelco Dolstra e4066c0444 Fetch specific Git revisions
This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-20 17:25:40 +01:00
Eelco Dolstra f450c8773c Merge pull request #9408 from roblabla/fix-bad-format-string
Fix bad_format_string error when builder stdout contains %
2023-11-20 17:22:40 +01:00
Eelco Dolstra e29e952a25 Merge pull request #9386 from hercules-ci/flake-update
flake.nix: Update nixpkgs: release-23.05 -> nixos-23.05-small
2023-11-20 16:51:20 +01:00
Eelco Dolstra 065cd5bed7 Merge pull request #9401 from NixOS/bump-2.20.0
Bump version
2023-11-20 15:42:53 +01:00
roblabla e2b6821ca0 Fix bad_format_string error when builder stdout contains % 2023-11-20 15:41:38 +01:00
John Ericson e540d48c4f Merge pull request #9400 from hercules-ci/refactors-from-5e3986f59cb
Refactors from 5e3986f59c
2023-11-20 09:38:18 -05:00
Théophane Hufschmitt cfc6494d57 Merge pull request #9390 from DavHau/git-shallow-docs
fetchTree: clarify docs for shallow flag
2023-11-20 15:15:25 +01:00
Eelco Dolstra 2a96445d75 Bump version 2023-11-20 15:06:28 +01:00
John Ericson f9970fdc19 Merge pull request #9240 from edolstra/libgit2
Introduce libgit2
2023-11-20 09:00:27 -05:00
John Ericson 4ab27e5595 Merge remote-tracking branch 'upstream/master' into libgit2 2023-11-20 08:30:26 -05:00
Robert Hensing a5e51a9e02 refactor Worker::childStarted/Terminated: use switch
Preparation for RFC 92 dynamic derivations.
2023-11-20 13:32:48 +01:00
Robert Hensing 7ac39ff05c refactor Store::buildPaths: convert to string earlier
Preparation for RFC 92 dynamic derivations.
2023-11-20 13:11:58 +01:00
Qyriad 19993398a1 flakes: check for flake.nix before complaining that lstat on it fails
getFlake currently calls lstat (via isLink via canonPath) before it
performs the sanity check that a flake.nix exists in the first place.
This commit moves the check to before path canonicalization, so that
failed symlink check operations don't throw before the check does.
2023-11-20 03:37:02 -07:00
DavHau 796a7eb92d fetchTree: clarify docs for shallow flag 2023-11-19 20:32:23 +07:00
Robert Hensing fe4f573d49 flake.nix: Update nixpkgs: release-23.05 -> nixos-23.05-small
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9eb24edd6a0027fed010ccfe300a9734d029983c' (2023-11-01)
  → 'github:NixOS/nixpkgs/decdf666c833a325cb4417041a90681499e06a41' (2023-11-18)
2023-11-19 11:38:58 +01:00
John Ericson 87ac33f29a Merge branch 'master' into nixenvjsondrvpath 2023-11-18 13:47:14 -05:00
Bob van der Linden eaa3629532 fixup! nix: develop: add tests for interactive shell 2023-11-16 15:12:31 +01:00
Bob van der Linden ceab20d056 nix: develop: add tests for interactive shell 2023-11-16 09:37:49 +01:00
Bob van der Linden 8c54a01df5 nix: develop: always force SHELL to chosen shell
SHELL was inherited from the system environment. This resulted in a new
shell being started, but with SHELL still referring to the system shell
and not the one used by nix-develop.

Applications like make, use SHELL to run commands, which meant that
top-level commands are run inside the nix-develop-shell, but
sub-commands are ran inside the system shell.

This setenv forces SHELL to always be set to the shell used by
nix-develop.
2023-11-16 09:37:48 +01:00
Eelco Dolstra 7ab91e7238 Implement shallow fetching 2023-11-15 15:00:17 +01:00
Eelco Dolstra 5dd4ae8687 Remove unused cacheType field 2023-11-15 14:08:34 +01:00
Eelco Dolstra 2890999911 Show Git fetch progress 2023-11-15 13:57:20 +01:00
Eelco Dolstra 2964a9f562 Fix relative submodule handling
Tested on

  nix flake prefetch 'git+https://github.com/blender/blender.git?rev=4ed8a360e956daf2591add4d3c9ec0719e2628fe&submodules=1'
2023-11-14 16:00:21 +01:00
Eelco Dolstra 6ec6b8aa36 Improve git submodule error reporting 2023-11-14 15:52:18 +01:00
Eelco Dolstra c257c82447 Cleanup 2023-11-14 14:47:17 +01:00
Eelco Dolstra 7f576f5dfe Rename UnionInputAccessor to MountedInputAccessor 2023-11-14 14:02:57 +01:00
Eelco Dolstra 21140c987b Fix doxygen comments 2023-11-14 14:02:57 +01:00
Eelco Dolstra 4329bdf6a3 Move comment 2023-11-14 14:02:57 +01:00
Eelco DolstraandRobert Hensing 25cf8f1071 src/libfetchers/union-input-accessor.cc: Apply suggestion
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-14 13:57:24 +01:00
Eelco DolstraandRobert Hensing 38b07d6347 src/libfetchers/git.cc: Apply suggestion
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-11-14 13:38:03 +01:00
Eelco Dolstra d74d2fdaa7 Move statusCallbackTrampoline 2023-11-14 13:35:26 +01:00
Eelco Dolstra 21bb180547 Use libgit2 with ssh-exec support
See https://github.com/libgit2/libgit2/pull/6617. This ensures that we
get support for ~/.ssh/config, known_hosts etc.
2023-11-14 13:30:51 +01:00
Bouke van der Bijl d6898cd58b Move applyConfigFile to lambda inside libstore 2023-11-13 17:14:05 +01:00
Bouke van der Bijl e4cbdd26e0 Add TODO comment for include try/catch 2023-11-13 17:13:52 +01:00
Eelco Dolstra cf59ea83ec configure: Check for libgit2 2023-11-10 13:58:59 +01:00
Artturin a903f85f84 nix-env --query: fix --json ignoring --drv-path
```json
{
  "AMB-plugins": {
    "drvPath": "/nix/store/l99cb7h2hy8dg005arsjbd9kx0w05d3h-AMB-plugins-0.8.1.drv",
    "name": "AMB-plugins-0.8.1",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "AMB-plugins",
    "system": "x86_64-linux",
    "version": "0.8.1"
  },
  "ArchiSteamFarm": {
    "drvPath": "/nix/store/nhplgyjj34fz6hjmnyih25gxscfh8s7b-ArchiSteamFarm-5.4.12.5.drv",
    "name": "ArchiSteamFarm-5.4.12.5",
    "outputName": "out",
    "outputs": {
      "out": null
    },
    "pname": "ArchiSteamFarm",
    "system": "x86_64-linux",
    "version": "5.4.12.5"
  },
...
```
2023-11-09 22:31:32 +02:00
Eelco Dolstra 98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
Bob van der Linden 9c0a09f09f allow ^ in URLs
Users may select specific outputs using the ^output syntax or selecting
any output using ^*.

URL parsing currently doesn't support these kinds of output references:
parsing will fail.

Currently `queryRegex` was reused for URL fragments, which didn't
include support for ^. Now queryRegex has been split from fragmentRegex,
where only the fragmentRegex supports ^.
2023-11-06 21:21:20 +01:00
Felix Uhl 257b768436 Enable using human-readable name in nix profile 2023-11-06 21:21:20 +01:00
John Ericson dde1d86338 Restrict some code to StoreDirConfig
- part of eval cache
 - part of derivations
 - derived path
 - store path with outputs
 - serializers
2023-11-04 19:05:36 -04:00
John Ericson e97ac09abe Factor out StoreDirConfig
More progress on #5729.
2023-11-04 19:05:36 -04:00
Eelco Dolstra 39ea46abb1 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-02 16:16:37 +01:00
Eelco Dolstra 0c5eac9c45 Git fetcher: Handle submodules for workdirs 2023-10-31 15:59:25 +01:00
Eelco Dolstra 669b074f51 Cleanup 2023-10-31 14:52:21 +01:00
Eelco Dolstra d88106df24 Git fetcher: Improve submodule handling
Instead of making a complete copy of the repo, fetching the
submodules, and writing the result to the store (which is all
superexpensive), we now fetch the submodules recursively using the Git
fetcher, and return a union accessor that "mounts" the accessors for
the submodules on top of the root accessor.
2023-10-31 14:52:21 +01:00
Eelco Dolstra ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Eelco Dolstra 1d0e3d84b6 Provide a InputScheme::fetch() built on top of InputScheme::getAccessor()
This is for graceful migration to lazy-trees fetchers (which are all
accessor-based). Eventually fetch() will be removed.
2023-10-31 14:52:17 +01:00
Eelco Dolstra e1b8442fa1 Fetcher cache: Add support for caching facts not related to store paths 2023-10-31 14:51:39 +01:00
John Ericson 250c3541bb Use local-overlay:// not local-overlay for store URL
This is a bit uglier, but allows us to avoid an ad-hoc special case in
`store-api.cc`.
2023-10-25 15:29:11 -04:00
John Ericson 8434f23c97 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-10-25 14:23:20 -04:00
John Ericson 6f0a95897c Revert "Fix hard linking issue causing overlay fs copy-ups"
This reverts commit 9ef0a9e8aa.

Master now has a better solution.
2023-10-25 14:20:40 -04:00
Bouke van der Bijl 1d28d613b1 config: add included files into parsedContents before applying
Fixes #8719
2023-10-25 11:39:18 +02:00
John Ericson 5c1cb0b696 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-10-23 13:13:37 -04:00
Vladimir Kryachko 311e2ad024 Address review comments 2023-10-18 10:37:06 -04:00
Vladimir Kryachko a2f0ba6a6d Fix transitive input locking.
Fixes reproducibility issue described in #9143

Fixes #9143
2023-10-18 10:31:36 -04:00
Ben Radford c0e6466a1e Merge remote-tracking branch 'origin/master' into overlayfs-store 2023-08-08 13:39:18 +01:00
John Ericson d137002e94 Add API docs for all overridden local overlay methods
These docs explain the implementation relative to the local store
originals. The original declaration of virtual methods can still be
consulted for proper interface-level documentation.
2023-08-03 17:28:18 -04:00
John Ericson 7ad16c9d12 Add some docs for the local overlay store 2023-08-03 16:10:32 -04:00
John Ericson 4d99e407fd Remove FIXME on why something doesn't work
I now know it is due to
https://github.com/llvm/llvm-project/issues/64108. The workaround is
just fine and already in use in this codebase.
2023-08-03 12:12:54 -04:00
John Ericson 4f5b01f5cd Start to document the local-overlay store 2023-08-03 12:10:13 -04:00
John Ericson 6dc9030606 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-08-03 11:53:34 -04:00
John Ericson 2556c4d753 Rename test group overlay-local-store -> local-overlay-store
Makes it match the store name (`local-overlay`) and experimental feature
name (`local-overlay-store`)._
2023-08-02 20:32:45 -04:00
John Ericson 4b9a621812 Guard the local overlay store behind an experimental feature 2023-08-02 20:30:42 -04:00
John Ericson 3fc838c8a8 Merge remote-tracking branch 'layered-store/experimental-stores' into overlayfs-store 2023-08-02 19:03:34 -04:00
John Ericson 6b297e5895 Make verifyAllValidPaths more functional
return map rather than mutate one passed in by reference
2023-08-02 14:38:22 -04:00
John Ericson 73c9fc7ab1 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-08-02 14:14:11 -04:00
Ben Radford 19164cf727 Test that remounting fixes 'stale file handle' errors. 2023-08-01 15:05:33 +01:00
Ben Radford 8225b7a011 Merge branch 'master' into overlayfs-store 2023-08-01 12:52:53 +01:00
Ben Radford c712369ec5 Document remount-hook store parameter. 2023-08-01 12:48:02 +01:00
Ben Radford c409a753db Fix new lines in comment. 2023-08-01 12:48:02 +01:00
Ben RadfordandJohn Ericson 1255866e16 Update src/libstore/local-overlay-store.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-08-01 12:48:02 +01:00
Ben Radford 6a8de4c9dc Avoid enumerating entire overlay store dir upfront.
As an optimisation for LocalStore, we read all the store directory entries into
a set. Checking for membership of this set is much faster than a stat syscall.
However for LocalOverlayStore, the lower store directory is expected to contain
a vast number of entries and reading them all can take a very long time.

So instead of enumerating them all upfront, we call pathExists as needed. This
means making stat syscalls for each store path, but the upper layer is expected
to be relatively small compared to the lower store so that should be okay.
2023-08-01 12:48:02 +01:00
Ben Radford 50ce8d15eb Preparatory refactor of LocalStore::verifyStore. 2023-08-01 12:48:02 +01:00
John Ericson 31112fd26f Merge pull request #10 from NixLayeredStore/delete
Fix and test deleting when lower store references
2023-07-26 12:45:30 -04:00
John Ericson 621bdbdb31 Merge branch 'overlayfs-store' into delete 2023-07-26 12:01:08 -04:00
Ben Radford c2d54496a0 Forgot to check flag and early out. 2023-07-26 16:01:20 +01:00
Ben Radford 3a9fe1a085 Made remountRequired atomic to avoid concurrency issues. 2023-07-26 16:01:20 +01:00
Ben RadfordandJohn Ericson ca1a108dad Update tests/overlay-local-store/remount.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-07-26 16:01:20 +01:00
Ben Radford 5744a500d6 Use debug instead of writing directly to stderr. 2023-07-26 16:01:20 +01:00
Ben Radford 6da05c0a11 Rename test to delete-duplicate. 2023-07-26 16:01:20 +01:00
Ben Radford ed14286924 Invoke remount-hook program when necessary. 2023-07-26 16:01:20 +01:00
Ben Radford 33ebae75ca Reuse deletion logic for optimiseStore and rename method. 2023-07-26 16:01:20 +01:00
Ben Radford 11c493f8fa Avoid creating whiteout for duplicate store paths. 2023-07-26 16:01:20 +01:00
Ben Radford cc6f8aa91a Test that delete works for duplicate file edge case. 2023-07-26 16:01:20 +01:00
Ben Radford d9688ba708 Add new remount-hook store parameter. 2023-07-26 16:01:20 +01:00
John Ericson b0877ad3c9 Give test a more specific name 2023-07-26 09:50:31 -04:00
John Ericson 07b34edc44 Fix deletion test
Lower layer references are ignored for deleting just in the upper layer.
2023-07-25 18:09:02 -04:00
John Ericson 19c43c5d78 Write test for deleting objects referenced from below
Currently fails, as expected.
2023-07-25 18:09:02 -04:00
John Ericson a1a7f3156d Merge pull request #9 from NixLayeredStore/gc
Add a GC test, fix hardlinking issue
2023-07-25 11:44:45 -04:00
John Ericson 9ef0a9e8aa Fix hard linking issue causing overlay fs copy-ups 2023-07-25 10:28:11 -04:00
John Ericson 83e703e7a8 Merge branch 'overlayfs-store' into gc 2023-07-25 09:55:31 -04:00
John Ericson 272cfd6aed Merge branch 'overlayfs-store' of github.com:NixLayeredStore/nix into overlayfs-store 2023-07-25 09:54:50 -04:00
John Ericson 73a55e4005 Merge remote-tracking branch 'layered-store/overlayfs-store' into gc 2023-07-25 09:48:06 -04:00
Ben Radford 497464f494 Extend verify test to check that repair is supported. 2023-07-25 13:33:37 +01:00
John Ericson 3731208dc1 Adopt GC test for local-overlay store
Doesn't yet pass. Fixes are needed.
2023-07-24 18:14:12 -04:00
John Ericson 0e595a52a3 Remove trailing whitespace 2023-07-24 15:39:59 -04:00
John Ericson 2cabf85b53 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-07-24 15:39:36 -04:00
John Ericson ae0eb7424f Merge pull request #8 from NixLayeredStore/overlayfs-store-more-tests
Implement deduplication and add more test cases
2023-07-20 11:55:31 -04:00
Ben Radford 2fc00ec19f Fix unbound variable error in optimise test. 2023-07-20 11:27:41 +01:00
Ben Radford 2c66a093e0 Define storeBRoot variable distinct from storeB URI. 2023-07-20 11:03:41 +01:00
Ben Radford 878c84d5ee Fix errors about NIX_STORE_DIR being unset. 2023-07-20 10:42:40 +01:00
Ben Radford 9769a0ae7d Ensure all overlay tests use new tmpfs store paths. 2023-07-20 10:42:40 +01:00
Ben Radford 7fda19e2f1 Mount tmpfs first to ensure overlayfs works consistently. 2023-07-20 10:42:40 +01:00
Ben Radford 44f855d14e Missing addTextToStore function. 2023-07-20 10:42:39 +01:00
Ben Radford d1c77b201a Explicitly exec shell to fix ENOENT errors. 2023-07-20 10:42:39 +01:00
Ben Radford 8ddbcb736a Implement overlay store deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford a9510f9502 Implement test for store path deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford 614efc1240 Add test for store optimise path deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford d5cd74a401 Override verifyStore to always pass NoRepair for LocalOverlayStore. 2023-07-20 10:42:39 +01:00
Ben Radford 58085e4eff Have verify test exercise check-contents too. 2023-07-20 10:42:38 +01:00
Ben Radford 0ccf6382af Add test for verifying overlay store. 2023-07-20 10:42:38 +01:00
Ben Radford a33ee5c843 Paths added to lower store are accessible via overlay. 2023-07-20 10:42:38 +01:00
Ben Radford f66b65a30a Revert "Skip build-remote-trustless unless sandbox is supported."
This reverts commit fad0dd4afb.
2023-07-20 10:42:38 +01:00
Ben Radford 37598a13e8 Revert "Check _NIX_TEST_NO_SANDBOX when setting _canUseSandbox."
This reverts commit 7ed0ab2dab.
2023-07-20 10:42:38 +01:00
John Ericson 4107adcef8 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-07-19 13:29:34 -04:00
Ben Radford 21b9e15d25 Merge branch 'master' into overlayfs-store 2023-07-19 15:00:47 +01:00
cidkidnix 83cfa82e52 Add unset to NIX_STORE_DIR for local-overlay tests 2023-07-13 14:39:46 -05:00
cidkidnix c26fc96fed Merge remote-tracking branch 'layered/test-groups' into overlayfs-store 2023-07-13 14:36:49 -05:00
cidkidnix 04d5aa02e6 Merge branch 'best-effort-supplementary-groups' into overlayfs-store 2023-07-13 14:33:52 -05:00
John Ericson 735a672e1f Introduce notion of a test group, use for CA tests
Grouping our tests should make it easier to understand the intent than
one long poorly-arranged list. It also is convenient for running just
the tests for a specific component when working on that component.

We need at least one test group so this isn't dead code; I decided to
collect the tests for the `ca-derivations` and `dynamic-derivations`
experimental features in groups. Do
```bash
make ca.test-group -jN
```
and
```bash
make dyn-drv.test-group -jN
```
to try running just them.

I originally did this as part of #8397 for being able to just the local
overlay store alone. I am PRing it separately now so we can separate
general infra from new features.
2023-07-09 23:08:03 -04:00
John Ericson 2add230939 Fix build 2023-07-09 21:53:06 -04:00
John Ericson 28398e6d02 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-07-09 20:30:23 -04:00
Eelco Dolstra 3859b42597 Wait for pid 2023-06-21 16:17:21 +02:00
John Ericson 9c0473120f Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-06-21 10:15:13 -04:00
Eelco Dolstra c5fdbdae32 LocalStore::addTempRoot(): Handle ENOENT
If the garbage collector has acquired the global GC lock, but hasn't
created the GC socket yet, then a client attempting to connect would
get ENOENT. Note that this only happens when the GC runs for the first
time on a machine. Subsequently clients will get ECONNREFUSED which
was already handled.

Fixes #7370.
2023-06-20 11:14:55 +02:00
Ben Radford 71f3bad749 Merge branch 'read-only-local-store' into overlayfs-store 2023-06-19 16:12:04 +01:00
Ben RadfordandJohn Ericson 4e72b8483e Update src/libstore/sqlite.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-19 16:01:43 +01:00
Ben Radford 0b3fa995e5 Merge branch 'master' into read-only-local-store 2023-06-19 14:09:20 +01:00
Ben RadfordandValentin Gagarin ba492a98ba Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-19 14:07:31 +01:00
Ben Radford b09baa3bc3 Link to LocalStore section of nix3-help-stores section. 2023-06-19 13:57:10 +01:00
Ben Radford ef40448b1c Remove redundant description on experimental flag. 2023-06-19 13:52:32 +01:00
Ben RadfordandValentin Gagarin feb8d552ae Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-19 13:22:41 +01:00
Ben Radford abb3bb7133 Merge branch 'master' into read-only-local-store 2023-06-15 13:42:14 +01:00
Ben Radford 04d8f202a7 Merge branch 'read-only-local-store' into overlayfs-store 2023-06-15 13:37:57 +01:00
Ben Radford f5d83a8029 One line per sentence in markdown docs. 2023-06-15 13:36:28 +01:00
Ben Radford f2fe9822c1 Comment explaining what schema version 0 means. 2023-06-15 13:34:42 +01:00
Ben RadfordandValentin Gagarin 4642b60afe Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:33:26 +01:00
Ben RadfordandValentin Gagarin a7b1b92d81 Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:56 +01:00
Ben RadfordandValentin Gagarin 984b01924a Update src/libstore/local-store.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:35 +01:00
Ben RadfordandValentin Gagarin 78e2f931d0 Update src/libstore/local-store.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:16 +01:00
Ben RadfordandValentin Gagarin 7cdaa0b8a6 Update tests/read-only-store.sh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:25:15 +01:00
Ben Radford 264b644c53 More detail on why read-only mode disables locking. 2023-06-15 13:22:17 +01:00
Ben Radford fad0dd4afb Skip build-remote-trustless unless sandbox is supported. 2023-06-15 12:59:01 +01:00
Ben Radford 7ed0ab2dab Check _NIX_TEST_NO_SANDBOX when setting _canUseSandbox. 2023-06-15 12:48:08 +01:00
John Ericson 32147c8c7b Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-06-14 10:29:55 -04:00
Ben Radford ee1241da86 Remove unnecessary overrides of add methods. 2023-06-06 12:21:17 +01:00
Ben Radford 8a9baa0a30 More sensible to have deleteGCPath in LocalStore. 2023-06-06 12:21:17 +01:00
Ben Radford a48acfd684 Skip deletion of lower paths for overlay store GC. 2023-06-06 12:21:17 +01:00
Ben Radford 98edbb9686 Factor out GC path deletion so it can be overridden. 2023-06-06 12:21:17 +01:00
Ben Radford b852bdb3f8 Merge branch 'read-only-local-store' into overlayfs-store 2023-06-06 11:09:26 +01:00
Ben Radford c47f744e05 Also skip makeStoreWritable when read-only=true. 2023-06-06 11:07:55 +01:00
Ben Radford e484278e9b Merge branch 'master' into read-only-local-store 2023-06-06 11:07:36 +01:00
Ben Radford 8e4b15e6a0 Merge branch 'master' into read-only-local-store 2023-05-25 10:06:24 +01:00
Ben Radford ff12cf3b94 Merge branch 'read-only-local-store' into overlayfs-store 2023-05-23 09:52:41 +01:00
Ben Radford d6ea3b6a19 Need to enable read-only-local-store flag for test. 2023-05-22 12:14:10 +01:00
Ben Radford 32404292a3 Merge branch 'master' into read-only-local-store 2023-05-22 12:08:02 +01:00
Ben Radford 7251800086 Put read-only setting behind an experimental flag. 2023-05-22 11:38:37 +01:00
Ben Radford 0c36fe6c8c Update test to match new error message. 2023-05-18 14:36:24 +01:00
Ben Radford 8ffeb1c4e5 Throw error instead of silently skipping CA migration. 2023-05-18 13:51:21 +01:00
Ben Radford d55e38b98a Check earlier whether schema migration is required. 2023-05-18 13:51:21 +01:00
Ben Radford fe174d72a2 Fix spelling of 'immutable' in documentation. 2023-05-18 13:51:21 +01:00
Dylan GreenandJohn Ericson 60c0149721 Apply suggestions from code review
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-05-18 10:31:49 +01:00
Dylan GreenandJohn Ericson 9290af763a Update tests/read-only-store.sh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-05-18 10:31:49 +01:00
cidkidnix 85a2453052 Add tests for read-only local store
Make sure we don't go down the path of making temproots when doing
operations on a read-only store
2023-05-18 10:31:49 +01:00
Ben Radford 5966b76c97 Document the new read-only local store setting. 2023-05-17 09:04:52 +01:00
Ben Radford b1a7b26eef Rename ReadOnly to Immutable and clarify its purpose. 2023-05-17 09:04:48 +01:00
Ben Radford aa376f4ab1 Need to open database using immutable parameter.
This requires switching on SQLITE_OPEN_URI because there is no open flag to
make the database immutable. Without immutable, sqlite will still attempt to
create journal and wal files, even when the database is opened read-only.

https://www.sqlite.org/c3ref/open.html

The immutable parameter is a boolean query parameter that indicates that the
database file is stored on read-only media. When immutable is set, SQLite
assumes that the database file cannot be changed, even by a process with higher
privilege, and so the database is opened read-only and all locking and change
detection is disabled.
2023-05-17 08:55:04 +01:00
Ben Radford 78fdd6f24e Open sqlite database according to new modes. 2023-05-17 08:55:04 +01:00
Ben Radford afed9ccfad Add enum for intended sqlite database open modes. 2023-05-17 08:55:03 +01:00
Ben Radford 7f443e0428 Do not check for write access to database when read-only. 2023-05-17 08:55:03 +01:00
Ben Radford c22936ca6a Do not attempt to migrate to CA schema when read-only. 2023-05-17 08:55:03 +01:00
Ben Radford 50bbdc65c8 Do not attempt to acquire big-lock when read-only. 2023-05-17 08:55:03 +01:00
Ben Radford 79583c2d38 Do not attempt to chmod per-user dir when read-only. 2023-05-17 08:55:03 +01:00
Ben Radford de359da09a Add read-only setting to LocalStoreConfig. 2023-05-17 08:55:03 +01:00
Ben Radford 4d69bd034a More detailed explanation of check-mount setting. 2023-05-16 17:27:49 +01:00
John Ericson 0ec7f2fb3f Create local.mk for local-overlay-store tests 2023-05-15 23:11:29 -04:00
John Ericson 5d18120ba8 Split tests some more
Good for parallelism and easier reading.
2023-05-15 23:11:17 -04:00
John Ericson 97deb00cbc Create notion of "test group", use for local overlay store 2023-05-15 18:13:11 -04:00
John Ericson 1dcb782651 Merge branch 'best-effort-supplementary-groups' into overlayfs-store 2023-05-15 17:49:52 -04:00
John Ericson 3496a5ee86 Merge branch 'best-effort-supplementary-groups' into overlayfs-store 2023-05-15 17:46:15 -04:00
John Ericson b1fba1c2a1 Fix PS4 for heredocs 2023-05-15 16:44:36 -04:00
John Ericson 72bb9604f3 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-05-15 16:34:19 -04:00
cidkidnix 0979a374c5 Begin to split up overlay-local-store tests
The bad-uris tests are now in their own file.
"Outer" is a bad name, but it will be split up next.
2023-05-15 15:31:57 -05:00
cidkidnix b7e5aaf90d Add test for checking that we reject bad local overlay store uris 2023-05-15 13:41:38 -05:00
Ben Radford 0df37edb1c Make upper-layer a PathSetting instead of a Setting<string>. 2023-05-15 17:34:31 +01:00
Ben Radford b0989cb10b Support percent encoded URIs for lower store. 2023-05-15 17:34:31 +01:00
Ben Radford b5591ece4c Check that overlay store directory is mounted correctly.
Nix does not manage the overlayfs mount point itself, but the correct
functioning of the overlay store does depend on this mount point being set up
correctly. Rather than just assume this is the case, check that the lowerdir
and upperdir options are what we expect them to be. This check is on by
default, but can be disabled if needed.
2023-05-15 17:34:31 +01:00
John Ericson 4173743a3c Implement more queries 2023-05-09 17:20:58 -04:00
John Ericson 8339c170d7 More tests 2023-05-09 16:49:44 -04:00
John Ericson 5059be53b1 Fix recursive ingestion from lower store 2023-05-09 16:42:28 -04:00
John Ericson e7c3399ed2 Specialize LocalOverlayStore::queryPathFromHashPart
With test
2023-05-09 10:40:10 -04:00
John Ericson ddaf2750b5 Specialize more methods, fix tests 2023-05-09 10:22:38 -04:00
John Ericson b3d320c594 Convert more methods
Fixed one test, broke another
2023-05-08 18:50:16 -04:00
John Ericson 59a8099038 Fix LocalOverlayStore::queryPathInfoUncached, FIXME in test 2023-05-08 17:37:40 -04:00
John Ericson 5406256d78 Specialize LocalOverlayStore::queryPathInfoUncached 2023-05-08 17:30:17 -04:00
John Ericson 31e98ed0a0 Specialize LocalOverlayStore::registerDrvOutput 2023-05-08 16:48:55 -04:00
John Ericson 0193c2abcd Improve tests slightly 2023-05-08 16:03:32 -04:00
John Ericson d80fc2ac1b First round of testing, with todos 2023-05-08 16:01:36 -04:00
John Ericson 9c9f5f0d12 Merge branch 'best-effort-supplementary-groups' into overlayfs-store 2023-05-08 14:47:46 -04:00
John Ericson f08754a97a Progress on tests 2023-05-08 14:47:39 -04:00
John Ericson d5fe828d76 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-05-08 10:20:18 -04:00
John Ericson f0a176e2f1 Init local overlay store 2023-05-08 10:20:06 -04:00
John Ericson d62f6da81f Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-04-10 16:55:07 -04:00
John Ericson 111aeb9e92 Testing overlayfs stores 2023-03-21 10:53:23 -04:00
1680 changed files with 66975 additions and 40245 deletions

No files matched your search

+33
View File
@@ -0,0 +1,33 @@
BasedOnStyle: LLVM
IndentWidth: 4
BreakBeforeBraces: Custom
BraceWrapping:
AfterStruct: true
AfterClass: true
AfterFunction: true
AfterUnion: true
SplitEmptyRecord: false
PointerAlignment: Middle
FixNamespaceComments: false
SortIncludes: Never
#IndentPPDirectives: BeforeHash
SpaceAfterCStyleCast: true
SpaceAfterTemplateKeyword: false
AccessModifierOffset: -4
AlignAfterOpenBracket: AlwaysBreak
AlignEscapedNewlines: Left
ColumnLimit: 120
BreakStringLiterals: false
BitFieldColonSpacing: None
AllowShortFunctionsOnASingleLine: Empty
AlwaysBreakTemplateDeclarations: Yes
BinPackParameters: false
BreakConstructorInitializers: BeforeComma
EmptyLineAfterAccessModifier: Leave # change to always/never later?
EmptyLineBeforeAccessModifier: Leave
#PackConstructorInitializers: BinPack
BreakBeforeBinaryOperators: NonAssignment
AlwaysBreakBeforeMultilineStrings: true
IndentPPDirectives: AfterHash
PPIndentWidth: 2
BinPackArguments: false
+3
View File
@@ -0,0 +1,3 @@
# We use pointers to aggregates in a couple of places, intentionally.
# void * would look weird.
Checks: '-bugprone-sizeof-expression'
+4 -4
View File
@@ -4,20 +4,20 @@
# Top-most EditorConfig file
root = true
# Unix-style newlines with a newline ending every file, utf-8 charset
# Unix-style newlines with a newline ending every file, UTF-8 charset
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8
# Match nix files, set indent to spaces with width of two
# Match Nix files, set indent to spaces with width of two
[*.nix]
indent_style = space
indent_size = 2
# Match c++/shell/perl, set indent to spaces with width of four
[*.{hpp,cc,hh,sh,pl,xs}]
# Match C++/C/shell/Perl, set indent to spaces with width of four
[*.{hpp,cc,hh,c,h,sh,pl,xs}]
indent_style = space
indent_size = 4
+12 -4
View File
@@ -10,9 +10,17 @@
# This file
.github/CODEOWNERS @edolstra
# Public documentation
/doc @fricklerhandwerk
*.md @fricklerhandwerk
# Documentation of built-in functions
src/libexpr/primops.cc @roberth @fricklerhandwerk
# Documentation of settings
src/libexpr/eval-settings.hh @fricklerhandwerk
src/libstore/globals.hh @fricklerhandwerk
# Documentation
doc/manual @fricklerhandwerk
maintainers/*.md @fricklerhandwerk
src/**/*.md @fricklerhandwerk
# Libstore layer
/src/libstore @thufschmitt
/src/libstore @ericson2314
+3 -1
View File
@@ -10,6 +10,8 @@
<!-- Large change: Provide instructions to reviewers how to read the diff. -->
# Priorities
# Priorities and Process
Add :+1: to [pull requests you find important](https://github.com/NixOS/nix/pulls?q=is%3Aopen+sort%3Areactions-%2B1-desc).
The Nix maintainer team uses a [GitHub project board](https://github.com/orgs/NixOS/projects/19) to [schedule and track reviews](https://github.com/NixOS/nix/tree/master/maintainers#project-board-protocol).
+32 -12
View File
@@ -1,23 +1,43 @@
"c api":
- changed-files:
- any-glob-to-any-file: "src/lib*-c/**/*"
- any-glob-to-any-file: "test/unit/**/nix_api_*"
- any-glob-to-any-file: "doc/external-api/**/*"
"contributor-experience":
- changed-files:
- any-glob-to-any-file: "CONTRIBUTING.md"
- any-glob-to-any-file: ".github/ISSUE_TEMPLATE/*"
- any-glob-to-any-file: ".github/PULL_REQUEST_TEMPLATE.md"
- any-glob-to-any-file: "doc/manual/src/contributing/**"
"documentation":
- doc/manual/*
- src/nix/**/*.md
- changed-files:
- any-glob-to-any-file: "doc/manual/**/*"
- any-glob-to-any-file: "src/nix/**/*.md"
"store":
- src/libstore/store-api.*
- src/libstore/*-store.*
- changed-files:
- any-glob-to-any-file: "src/libstore/store-api.*"
- any-glob-to-any-file: "src/libstore/*-store.*"
"fetching":
- src/libfetchers/**/*
- changed-files:
- any-glob-to-any-file: "src/libfetchers/**/*"
"repl":
- src/libcmd/repl.*
- src/nix/repl.*
- changed-files:
- any-glob-to-any-file: "src/libcmd/repl.*"
- any-glob-to-any-file: "src/nix/repl.*"
"new-cli":
- src/nix/**/*
- changed-files:
- any-glob-to-any-file: "src/nix/**/*"
"with-tests":
# Unit tests
- src/*/tests/**/*
# Functional and integration tests
- tests/functional/**/*
- changed-files:
# Unit tests
- any-glob-to-any-file: "src/*/tests/**/*"
# Functional and integration tests
- any-glob-to-any-file: "tests/functional/**/*"
-32
View File
@@ -1,32 +0,0 @@
name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for zeebe-io/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v2.1.1
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
github_workspace: ${{ github.workspace }}
pull_description: |-
Automatic backport to `${target_branch}`, triggered by a label in #${pull_number}.
# should be kept in sync with `uses`
version: v0.0.5
+138 -82
View File
@@ -7,91 +7,84 @@ on:
permissions: read-all
jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
tests:
needs: [check_secrets]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v23
- uses: cachix/install-nix-action@V27
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v12
if: needs.check_secrets.outputs.cachix == 'true'
extra_nix_config: |
sandbox = true
max-jobs = 1
- uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'linux'
- run: scripts/build-checks
- run: scripts/prepare-installer-for-github-actions
- name: Upload installer tarball
uses: actions/upload-artifact@v4
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' flake check -L
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v23
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
- uses: cachix/cachix-action@v12
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- id: prepare-installer
run: scripts/prepare-installer-for-github-actions
name: installer-${{matrix.os}}
path: out/*
installer_test:
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
needs: [tests]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v23
- name: Download installer tarball
uses: actions/download-artifact@v4
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
name: installer-${{matrix.os}}
path: out
- name: Serving installer
id: serving_installer
run: ./scripts/serve-installer-for-github-actions
- uses: cachix/install-nix-action@v30
with:
install_url: 'http://localhost:8126/install'
install_options: "--tarball-url-prefix http://localhost:8126/"
- run: sudo apt install fish zsh
if: matrix.os == 'ubuntu-latest'
if: matrix.os == 'linux'
- run: brew install fish
if: matrix.os == 'macos-latest'
if: matrix.os == 'darwin'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
@@ -99,36 +92,54 @@ jobs:
- run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs"
- run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello"
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
docker_push_image:
needs: [check_secrets, tests]
needs: [tests, vm_tests, check_secrets]
permissions:
contents: read
packages: write
if: >-
needs.check_secrets.outputs.docker == 'true' &&
github.event_name == 'push' &&
github.ref_name == 'master' &&
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
github.ref_name == 'master'
runs-on: ubuntu-24.04
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v23
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.13.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#default.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v12
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION nixos/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION nixos/nix:master
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:master
# We'll deploy the newly built image to both Docker Hub and Github Container Registry.
#
# Push to Docker Hub first
@@ -137,8 +148,8 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- run: docker push nixos/nix:$NIX_VERSION
- run: docker push nixos/nix:master
- run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
- run: docker push ${{ secrets.DOCKERHUB_USERNAME }}/nix:master
# Push to GitHub Container Registry as well
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
@@ -153,6 +164,51 @@ jobs:
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag nix:$NIX_VERSION $IMAGE_ID:$NIX_VERSION
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker tag nix:$NIX_VERSION $IMAGE_ID:latest
docker push $IMAGE_ID:$NIX_VERSION
docker push $IMAGE_ID:latest
# deprecated 2024-02-24
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes .#hydraJobs.tests.functional_user
meson_build:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
# Only meson packages that don't have a tests.run derivation.
# Those that have it are already built and tested as part of nix flake check.
- run: nix build -L .#hydraJobs.build.{nix-cmd,nix-main}.$(nix-instantiate --eval --expr builtins.currentSystem | sed -e 's/"//g')
flake_regressions:
needs: vm_tests
runs-on: ubuntu-24.04
steps:
- name: Checkout nix
uses: actions/checkout@v4
- name: Checkout flake-regressions
uses: actions/checkout@v4
with:
repository: NixOS/flake-regressions
path: flake-regressions
- name: Checkout flake-regressions-data
uses: actions/checkout@v4
with:
repository: NixOS/flake-regressions-data
path: flake-regressions/tests
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build --out-link ./new-nix && PATH=$(pwd)/new-nix/bin:$PATH scripts/flake-regressions.sh
-20
View File
@@ -1,20 +0,0 @@
name: Hydra status
permissions: read-all
on:
schedule:
- cron: "12,42 * * * *"
workflow_dispatch:
jobs:
check_hydra_status:
name: Check Hydra status
if: github.repository_owner == 'NixOS'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: bash scripts/check-hydra-status.sh
+2 -2
View File
@@ -15,10 +15,10 @@ permissions:
jobs:
labels:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/labeler@v4
- uses: actions/labeler@v5
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
sync-labels: false
+29 -7
View File
@@ -10,6 +10,9 @@ perl/Makefile.config
/stamp-h1
/svn-revision
/libtool
/config/config.*
# Default meson build dir
/build
# /doc/manual/
/doc/manual/*.1
@@ -21,12 +24,16 @@ perl/Makefile.config
/doc/manual/language.json
/doc/manual/xp-features.json
/doc/manual/src/SUMMARY.md
/doc/manual/src/SUMMARY-rl-next.md
/doc/manual/src/store/types/*
!/doc/manual/src/store/types/index.md.in
/doc/manual/src/command-ref/new-cli
/doc/manual/src/command-ref/conf-file.md
/doc/manual/src/command-ref/experimental-features-shortlist.md
/doc/manual/src/contributing/experimental-feature-descriptions.md
/doc/manual/src/language/builtins.md
/doc/manual/src/language/builtin-constants.md
/doc/manual/src/release-notes/rl-next.md
# /scripts/
/scripts/nix-profile.sh
@@ -41,18 +48,27 @@ perl/Makefile.config
/src/libexpr/parser-tab.hh
/src/libexpr/parser-tab.output
/src/libexpr/nix.tbl
/src/libexpr/tests/libnixexpr-tests
/src/libexpr/tests
/tests/unit/libexpr/libnixexpr-tests
# /src/libfetchers
/tests/unit/libfetchers/libnixfetchers-tests
# /src/libflake
/tests/unit/libflake/libnixflake-tests
# /src/libstore/
*.gen.*
/src/libstore/tests/libnixstore-tests
/src/libstore/tests
/tests/unit/libstore/libnixstore-tests
# /src/libutil/
/src/libutil/tests/libnixutil-tests
/src/libutil/tests
/tests/unit/libutil/libnixutil-tests
/src/nix/nix
/src/nix/doc
/src/nix/generated-doc
# /src/nix-env/
/src/nix-env/nix-env
@@ -81,7 +97,7 @@ perl/Makefile.config
# /tests/functional/
/tests/functional/test-tmp
/tests/functional/common/vars-and-functions.sh
/tests/functional/common/subst-vars.sh
/tests/functional/result*
/tests/functional/restricted-innocent
/tests/functional/shell
@@ -90,6 +106,7 @@ perl/Makefile.config
/tests/functional/ca/config.nix
/tests/functional/dyn-drv/config.nix
/tests/functional/repl-result-out
/tests/functional/debugger-test-out
/tests/functional/test-libstoreconsumer/test-libstoreconsumer
# /tests/functional/lang/
@@ -106,8 +123,6 @@ perl/Makefile.config
/misc/systemd/nix-daemon.conf
/misc/upstart/nix-daemon.conf
/src/resolve-system-dependencies/resolve-system-dependencies
outputs/
*.a
@@ -133,14 +148,21 @@ GTAGS
# auto-generated compilation database
compile_commands.json
*.compile_commands.json
nix-rust/target
result
result-*
# IDE
.vscode/
.idea/
.pre-commit-config.yaml
# clangd and possibly more
.cache/
# Mac OS
.DS_Store
+94
View File
@@ -0,0 +1,94 @@
queue_rules:
- name: default
# all required tests need to go here
merge_conditions:
- check-success=installer
- check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest)
- check-success=tests on macos
- check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests
merge_method: rebase
batch_size: 5
pull_request_rules:
- name: merge using the merge queue
conditions:
- base=master
- label~=merge-queue|dependencies
actions:
queue: {}
# The rules below will first create backport pull requests and put those in a merge queue.
- name: backport patches to 2.18
conditions:
- label=backport 2.18-maintenance
actions:
backport:
branches:
- 2.18-maintenance
labels:
- merge-queue
- name: backport patches to 2.19
conditions:
- label=backport 2.19-maintenance
actions:
backport:
branches:
- 2.19-maintenance
labels:
- merge-queue
- name: backport patches to 2.20
conditions:
- label=backport 2.20-maintenance
actions:
backport:
branches:
- 2.20-maintenance
labels:
- merge-queue
- name: backport patches to 2.21
conditions:
- label=backport 2.21-maintenance
actions:
backport:
branches:
- 2.21-maintenance
labels:
- merge-queue
- name: backport patches to 2.22
conditions:
- label=backport 2.22-maintenance
actions:
backport:
branches:
- 2.22-maintenance
labels:
- merge-queue
- name: backport patches to 2.23
conditions:
- label=backport 2.23-maintenance
actions:
backport:
branches:
- 2.23-maintenance
labels:
- merge-queue
- name: backport patches to 2.24
conditions:
- label=backport 2.24-maintenance
actions:
backport:
branches:
- "2.24-maintenance"
labels:
- merge-queue
+4
View File
@@ -0,0 +1,4 @@
external-sources=true
source-path=SCRIPTDIR
# Hack for scripts in e.g. tests/functional/ca
source-path=SCRIPTDIR/..
+1 -1
View File
@@ -1 +1 @@
2.19.0
2.24.16
+42
View File
@@ -0,0 +1,42 @@
cff-version: 1.2.0
title: Nix
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Eelco
family-names: Dolstra
email: edolstra@gmail.com
- name: The Nix contributors
website: 'https://github.com/NixOS/nix'
references:
- title: The Purely Functional Software Deployment Model
authors:
- family-names: Dolstra
given-names: Eelco
year: 2006
type: thesis
thesis-type: PhD thesis
isbn: 90-393-4130-3
url: https://dspace.library.uu.nl/handle/1874/7540
database-provider: Utrecht University Repository
institution:
name: Utrecht University
keywords:
- configuration management
- software deployment
- purely functional
- component-based software engineering
repository-code: 'https://github.com/NixOS/nix'
url: 'https://nixos.org/'
abstract: >-
Nix, a purely functional package manager, is a powerful
package manager for Linux and other Unix systems that
makes package management reliable and reproducible.
keywords:
- reproducibility
- open-source
- c++
- functional
license: LGPL-2.1
+7 -5
View File
@@ -27,6 +27,8 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
1. Search for related issues that cover what you're going to work on.
It could help to mention there that you will work on the issue.
We strongly recommend first-time contributors not to propose new features but rather fix tightly-scoped problems in order to build trust and a working relationship with maintainers.
Issues labeled [good first issue](https://github.com/NixOS/nix/labels/good%20first%20issue) should be relatively easy to fix and are likely to get merged quickly.
Pull requests addressing issues labeled [idea approved](https://github.com/NixOS/nix/labels/idea%20approved) or [RFC](https://github.com/NixOS/nix/labels/RFC) are especially welcomed by maintainers and will receive prioritised review.
@@ -39,9 +41,9 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
There are many open pull requests that might already do what you intend to work on.
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
3. Check the [Nix reference manual](https://nix.dev/manual/nix/development/development/building.html) for information on building Nix and running its tests.
For contributions to the command line interface, please check the [CLI guidelines](https://nixos.org/manual/nix/unstable/contributing/cli-guideline.html).
For contributions to the command line interface, please check the [CLI guidelines](https://nix.dev/manual/nix/development/development/cli-guideline.html).
4. Make your change!
@@ -63,11 +65,11 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
- Functional tests – [`tests/functional/**.sh`](./tests/functional)
- Unit tests – [`src/*/tests`](./src/)
- Integration tests – [`tests/nixos/*`](./tests/nixos)
- [ ] User documentation in the [manual](..doc/manual/src)
- [ ] User documentation in the [manual](./doc/manual/src)
- [ ] API documentation in header files
- [ ] Code and comments are self-explanatory
- [ ] Commit message explains **why** the change was made
- [ ] New feature or incompatible change: updated [release notes](./doc/manual/src/release-notes/rl-next.md)
- [ ] New feature or incompatible change: [add a release note](https://nix.dev/manual/nix/development/development/contributing.html#add-a-release-note)
7. If you need additional feedback or help to getting pull request into shape, ask other contributors using [@mentions](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#mentioning-people-and-teams).
@@ -76,7 +78,7 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
The Nix reference manual is hosted on https://nixos.org/manual/nix.
The underlying source files are located in [`doc/manual/src`](./doc/manual/src).
For small changes you can [use GitHub to edit these files](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)
For larger changes see the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html).
For larger changes see the [Nix reference manual](https://nix.dev/manual/nix/development/development/contributing.html).
## Getting help
+86 -16
View File
@@ -1,5 +1,13 @@
-include Makefile.config
clean-files += Makefile.config
# External build directory support
include mk/build-dir.mk
-include $(buildprefix)Makefile.config
clean-files += $(buildprefix)Makefile.config
# List makefiles
include mk/platform.mk
ifeq ($(ENABLE_BUILD), yes)
makefiles = \
@@ -10,39 +18,60 @@ makefiles = \
src/libfetchers/local.mk \
src/libmain/local.mk \
src/libexpr/local.mk \
src/libflake/local.mk \
src/libcmd/local.mk \
src/nix/local.mk \
src/resolve-system-dependencies/local.mk \
src/libutil-c/local.mk \
src/libstore-c/local.mk \
src/libexpr-c/local.mk
ifdef HOST_UNIX
makefiles += \
scripts/local.mk \
maintainers/local.mk \
misc/bash/local.mk \
misc/fish/local.mk \
misc/zsh/local.mk \
misc/systemd/local.mk \
misc/launchd/local.mk \
misc/upstart/local.mk \
doc/manual/local.mk \
doc/internal-api/local.mk
misc/upstart/local.mk
endif
endif
ifeq ($(ENABLE_BUILD)_$(ENABLE_TESTS), yes_yes)
UNIT_TEST_ENV = _NIX_TEST_UNIT_DATA=unit-test-data
ifeq ($(ENABLE_UNIT_TESTS), yes)
makefiles += \
src/libutil/tests/local.mk \
src/libstore/tests/local.mk \
src/libexpr/tests/local.mk
tests/unit/libutil/local.mk \
tests/unit/libutil-support/local.mk \
tests/unit/libstore/local.mk \
tests/unit/libstore-support/local.mk \
tests/unit/libfetchers/local.mk \
tests/unit/libexpr/local.mk \
tests/unit/libexpr-support/local.mk \
tests/unit/libflake/local.mk
endif
ifeq ($(ENABLE_TESTS), yes)
ifeq ($(ENABLE_FUNCTIONAL_TESTS), yes)
ifdef HOST_UNIX
makefiles += \
tests/functional/local.mk \
tests/functional/flakes/local.mk \
tests/functional/ca/local.mk \
tests/functional/git-hashing/local.mk \
tests/functional/dyn-drv/local.mk \
tests/functional/local-overlay-store/local.mk \
tests/functional/test-libstoreconsumer/local.mk \
tests/functional/plugins/local.mk
else
makefiles += \
mk/disable-tests.mk
endif
endif
# Some makefiles require access to built programs and must be included late.
makefiles-late =
ifeq ($(ENABLE_DOC_GEN), yes)
makefiles-late += doc/manual/local.mk
endif
# Miscellaneous global Flags
OPTIMIZE = 1
@@ -51,8 +80,49 @@ ifeq ($(OPTIMIZE), 1)
GLOBAL_LDFLAGS += $(CXXLTO)
else
GLOBAL_CXXFLAGS += -O0 -U_FORTIFY_SOURCE
unexport NIX_HARDENING_ENABLE
endif
ifdef HOST_WINDOWS
# Windows DLLs are stricter about symbol visibility than Unix shared
# objects --- see https://gcc.gnu.org/wiki/Visibility for details.
# This is a temporary sledgehammer to export everything like on Unix,
# and not detail with this yet.
#
# TODO do not do this, and instead do fine-grained export annotations.
GLOBAL_LDFLAGS += -Wl,--export-all-symbols
endif
GLOBAL_CXXFLAGS += -g -Wall -Wdeprecated-copy -Wignored-qualifiers -Wimplicit-fallthrough -Werror=unused-result -Werror=suggest-override -include $(buildprefix)config.h -std=c++2a -I src
# Include the main lib, causing rules to be defined
include mk/lib.mk
GLOBAL_CXXFLAGS += -g -Wall -include config.h -std=c++2a -I src
# Fallback stub rules for better UX when things are disabled
#
# These must be defined after `mk/lib.mk`. Otherwise the first rule
# incorrectly becomes the default target.
ifneq ($(ENABLE_UNIT_TESTS), yes)
.PHONY: check
check:
@echo "Unit tests are disabled. Configure without '--disable-unit-tests', or avoid calling 'make check'."
@exit 1
endif
ifneq ($(ENABLE_FUNCTIONAL_TESTS), yes)
.PHONY: installcheck
installcheck:
@echo "Functional tests are disabled. Configure without '--disable-functional-tests', or avoid calling 'make installcheck'."
@exit 1
endif
# Documentation fallback stub rules.
ifneq ($(ENABLE_DOC_GEN), yes)
.PHONY: manual-html manpages
manual-html manpages:
@echo "Generated docs are disabled. Configure without '--disable-doc-gen', or avoid calling 'make manpages' and 'make manual-html'."
@exit 1
endif
+6 -6
View File
@@ -8,21 +8,26 @@ CXX = @CXX@
CXXFLAGS = @CXXFLAGS@
CXXLTO = @CXXLTO@
EDITLINE_LIBS = @EDITLINE_LIBS@
ENABLE_BUILD = @ENABLE_BUILD@
ENABLE_DOC_GEN = @ENABLE_DOC_GEN@
ENABLE_FUNCTIONAL_TESTS = @ENABLE_FUNCTIONAL_TESTS@
ENABLE_S3 = @ENABLE_S3@
ENABLE_UNIT_TESTS = @ENABLE_UNIT_TESTS@
GTEST_LIBS = @GTEST_LIBS@
HAVE_LIBCPUID = @HAVE_LIBCPUID@
HAVE_SECCOMP = @HAVE_SECCOMP@
HOST_OS = @host_os@
INSTALL_UNIT_TESTS = @INSTALL_UNIT_TESTS@
LDFLAGS = @LDFLAGS@
LIBARCHIVE_LIBS = @LIBARCHIVE_LIBS@
LIBBROTLI_LIBS = @LIBBROTLI_LIBS@
LIBCURL_LIBS = @LIBCURL_LIBS@
LIBGIT2_LIBS = @LIBGIT2_LIBS@
LIBSECCOMP_LIBS = @LIBSECCOMP_LIBS@
LOWDOWN_LIBS = @LOWDOWN_LIBS@
OPENSSL_LIBS = @OPENSSL_LIBS@
PACKAGE_NAME = @PACKAGE_NAME@
PACKAGE_VERSION = @PACKAGE_VERSION@
RAPIDCHECK_HEADERS = @RAPIDCHECK_HEADERS@
SHELL = @bash@
SODIUM_LIBS = @SODIUM_LIBS@
SQLITE3_LIBS = @SQLITE3_LIBS@
@@ -32,7 +37,6 @@ checkbindir = @checkbindir@
checklibdir = @checklibdir@
datadir = @datadir@
datarootdir = @datarootdir@
doc_generate = @doc_generate@
docdir = @docdir@
embedded_sandbox_shell = @embedded_sandbox_shell@
exec_prefix = @exec_prefix@
@@ -48,7 +52,3 @@ sandbox_shell = @sandbox_shell@
storedir = @storedir@
sysconfdir = @sysconfdir@
system = @system@
ENABLE_BUILD = @ENABLE_BUILD@
ENABLE_TESTS = @ENABLE_TESTS@
INSTALL_UNIT_TESTS = @INSTALL_UNIT_TESTS@
internal_api_docs = @internal_api_docs@
+14 -10
View File
@@ -4,30 +4,34 @@
[![Test](https://github.com/NixOS/nix/workflows/Test/badge.svg)](https://github.com/NixOS/nix/actions)
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nixos.org/nix/manual)
management reliable and reproducible. Please refer to the [Nix manual](https://nix.dev/reference/nix-manual)
for more details.
## Installation and first steps
Visit [nix.dev](https://nix.dev) for [installation instructions](https://nix.dev/tutorials/install-nix) and [beginner tutorials](https://nix.dev/tutorials/first-steps).
Full reference documentation can be found in the [Nix manual](https://nixos.org/nix/manual).
Full reference documentation can be found in the [Nix manual](https://nix.dev/reference/nix-manual).
## Building And Developing
## Building and developing
See our [Hacking guide](https://nixos.org/manual/nix/unstable/contributing/hacking.html) in our manual for instruction on how to
set up a development environment and build Nix from source.
Follow instructions in the Nix reference manual to [set up a development environment and build Nix from source](https://nix.dev/manual/nix/development/building.html).
## Contributing
Check the [contributing guide](./CONTRIBUTING.md) if you want to get involved with developing Nix.
## Additional Resources
## Additional resources
- [Nix manual](https://nixos.org/nix/manual)
- [Nix jobsets on hydra.nixos.org](https://hydra.nixos.org/project/nix)
- [NixOS Discourse](https://discourse.nixos.org/)
- [Matrix - #nix:nixos.org](https://matrix.to/#/#nix:nixos.org)
Nix was created by Eelco Dolstra and developed as the subject of his PhD thesis [The Purely Functional Software Deployment Model](https://edolstra.github.io/pubs/phd-thesis.pdf), published 2006.
Today, a world-wide developer community contributes to Nix and the ecosystem that has grown around it.
- [The Nix, Nixpkgs, NixOS Community on nixos.org](https://nixos.org/)
- [Official documentation on nix.dev](https://nix.dev)
- [Nixpkgs](https://github.com/NixOS/nixpkgs) is [the largest, most up-to-date free software repository in the world](https://repology.org/repositories/graphs)
- [NixOS](https://github.com/NixOS/nixpkgs/tree/master/nixos) is a Linux distribution that can be configured fully declaratively
- [Discourse](https://discourse.nixos.org/)
- [Matrix](https://matrix.to/#/#nix:nixos.org)
## License
-99
View File
@@ -1,99 +0,0 @@
diff --git a/darwin_stop_world.c b/darwin_stop_world.c
index 0468aaec..b348d869 100644
--- a/darwin_stop_world.c
+++ b/darwin_stop_world.c
@@ -356,6 +356,7 @@ GC_INNER void GC_push_all_stacks(void)
int nthreads = 0;
word total_size = 0;
mach_msg_type_number_t listcount = (mach_msg_type_number_t)THREAD_TABLE_SZ;
+ size_t stack_limit;
if (!EXPECT(GC_thr_initialized, TRUE))
GC_thr_init();
@@ -411,6 +412,19 @@ GC_INNER void GC_push_all_stacks(void)
GC_push_all_stack_sections(lo, hi, p->traced_stack_sect);
}
if (altstack_lo) {
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ stack_limit = pthread_get_stacksize_np(p->id);
+ if (altstack_lo >= altstack_hi || altstack_lo < altstack_hi - stack_limit) { // sp outside stack
+ altstack_lo = altstack_hi - stack_limit;
+ }
+
total_size += altstack_hi - altstack_lo;
GC_push_all_stack(altstack_lo, altstack_hi);
}
diff --git a/include/gc.h b/include/gc.h
index edab6c22..f2c61282 100644
--- a/include/gc.h
+++ b/include/gc.h
@@ -2172,6 +2172,11 @@ GC_API void GC_CALL GC_win32_free_heap(void);
(*GC_amiga_allocwrapper_do)(a,GC_malloc_atomic_ignore_off_page)
#endif /* _AMIGA && !GC_AMIGA_MAKINGLIB */
+#if !__APPLE__
+/* Patch doesn't work on apple */
+#define NIX_BOEHM_PATCH_VERSION 1
+#endif
+
#ifdef __cplusplus
} /* extern "C" */
#endif
diff --git a/pthread_stop_world.c b/pthread_stop_world.c
index b5d71e62..aed7b0bf 100644
--- a/pthread_stop_world.c
+++ b/pthread_stop_world.c
@@ -768,6 +768,8 @@ STATIC void GC_restart_handler(int sig)
/* world is stopped. Should not fail if it isn't. */
GC_INNER void GC_push_all_stacks(void)
{
+ size_t stack_limit;
+ pthread_attr_t pattr;
GC_bool found_me = FALSE;
size_t nthreads = 0;
int i;
@@ -851,6 +853,37 @@ GC_INNER void GC_push_all_stacks(void)
hi = p->altstack + p->altstack_size;
/* FIXME: Need to scan the normal stack too, but how ? */
/* FIXME: Assume stack grows down */
+ } else {
+#ifdef HAVE_PTHREAD_ATTR_GET_NP
+ if (!pthread_attr_init(&pattr)
+ || !pthread_attr_get_np(p->id, &pattr))
+#else /* HAVE_PTHREAD_GETATTR_NP */
+ if (pthread_getattr_np(p->id, &pattr))
+#endif
+ {
+ ABORT("GC_push_all_stacks: pthread_getattr_np failed!");
+ }
+ if (pthread_attr_getstacksize(&pattr, &stack_limit)) {
+ ABORT("GC_push_all_stacks: pthread_attr_getstacksize failed!");
+ }
+ if (pthread_attr_destroy(&pattr)) {
+ ABORT("GC_push_all_stacks: pthread_attr_destroy failed!");
+ }
+ // When a thread goes into a coroutine, we lose its original sp until
+ // control flow returns to the thread.
+ // While in the coroutine, the sp points outside the thread stack,
+ // so we can detect this and push the entire thread stack instead,
+ // as an approximation.
+ // We assume that the coroutine has similarly added its entire stack.
+ // This could be made accurate by cooperating with the application
+ // via new functions and/or callbacks.
+ #ifndef STACK_GROWS_UP
+ if (lo >= hi || lo < hi - stack_limit) { // sp outside stack
+ lo = hi - stack_limit;
+ }
+ #else
+ #error "STACK_GROWS_UP not supported in boost_coroutine2 (as of june 2021), so we don't support it in Nix."
+ #endif
}
GC_push_all_stack_sections(lo, hi, traced_stack_sect);
# ifdef STACK_GROWS_UP
+36
View File
@@ -0,0 +1,36 @@
# These are private dependencies with pkg-config files. What private
# means is that the dependencies are used by the library but they are
# *not* used (e.g. `#include`-ed) in any installed header file, and only
# in regular source code (`*.cc`) or private, uninstalled headers. They
# are thus part of the *implementation* of the library, but not its
# *interface*.
#
# See `man pkg-config` for some details.
deps_private = [ ]
# These are public dependencies with pkg-config files. Public is the
# opposite of private: these dependencies are used in installed header
# files. They are part of the interface (and implementation) of the
# library.
#
# N.B. This concept is mostly unrelated to our own concept of a public
# (stable) API, for consumption outside of the Nix repository.
# `libnixutil` is an unstable C++ library, whose public interface is
# likewise unstable. `libutilc` conversely is a hopefully-soon stable
# C library, whose public interface --- including public but not private
# dependencies --- will also likewise soon be stable.
#
# N.B. For distributions that care about "ABI" stability and not just
# "API" stability, the private dependencies also matter as they can
# potentially affect the public ABI.
deps_public = [ ]
# These are subproject deps (type == "internal"). They are other
# packages in `/src` in this repo. The private vs public distinction is
# the same as above.
deps_private_subproject = [ ]
deps_public_subproject = [ ]
# These are dependencencies without pkg-config files. Ideally they are
# just private, but they may also be public (e.g. boost).
deps_other = [ ]
+11
View File
@@ -0,0 +1,11 @@
add_project_arguments(
'-Wdeprecated-copy',
'-Werror=suggest-override',
'-Werror=switch',
'-Werror=switch-enum',
'-Werror=unused-result',
'-Wignored-qualifiers',
'-Wimplicit-fallthrough',
'-Wno-deprecated-declarations',
language : 'cpp',
)
@@ -0,0 +1,11 @@
if host_machine.system() == 'cygwin' or host_machine.system() == 'windows'
# Windows DLLs are stricter about symbol visibility than Unix shared
# objects --- see https://gcc.gnu.org/wiki/Visibility for details.
# This is a temporary sledgehammer to export everything like on Unix,
# and not detail with this yet.
#
# TODO do not do this, and instead do fine-grained export annotations.
linker_export_flags = ['-Wl,--export-all-symbols']
else
linker_export_flags = []
endif
+30
View File
@@ -0,0 +1,30 @@
requires_private = []
foreach dep : deps_private_subproject
requires_private += dep.name()
endforeach
requires_private += deps_private
requires_public = []
foreach dep : deps_public_subproject
requires_public += dep.name()
endforeach
requires_public += deps_public
import('pkgconfig').generate(
this_library,
filebase : meson.project_name(),
name : 'Nix',
description : 'Nix Package Manager',
subdirs : ['nix'],
extra_cflags : ['-std=c++2a'],
requires : requires_public,
requires_private : requires_private,
libraries_private : libraries_private,
)
meson.override_dependency(meson.project_name(), declare_dependency(
include_directories : include_dirs,
link_with : this_library,
compile_args : ['-std=c++2a'],
dependencies : deps_public_subproject + deps_public,
))
@@ -0,0 +1,7 @@
bash = find_program('bash', native: true)
gen_header = generator(
bash,
arguments : [ '-c', '{ echo \'R"__NIX_STR(\' && cat @INPUT@ && echo \')__NIX_STR"\'; } > "$1"', '_ignored_argv0', '@OUTPUT@' ],
output : '@PLAINNAME@.gen.hh',
)
+19
View File
@@ -0,0 +1,19 @@
foreach maybe_subproject_dep : deps_private_maybe_subproject
if maybe_subproject_dep.type_name() == 'internal'
deps_private_subproject += maybe_subproject_dep
# subproject sadly no good for pkg-config module
deps_other += maybe_subproject_dep
else
deps_private += maybe_subproject_dep
endif
endforeach
foreach maybe_subproject_dep : deps_public_maybe_subproject
if maybe_subproject_dep.type_name() == 'internal'
deps_public_subproject += maybe_subproject_dep
# subproject sadly no good for pkg-config module
deps_other += maybe_subproject_dep
else
deps_public += maybe_subproject_dep
endif
endforeach
+6
View File
@@ -0,0 +1,6 @@
# This is only conditional to work around
# https://github.com/mesonbuild/meson/issues/13293. It should be
# unconditional.
if not (host_machine.system() == 'windows' and cxx.get_id() == 'gcc')
deps_private += dependency('threads')
endif
-1700
View File
File diff suppressed because it is too large. Load diff
-1860
View File
File diff suppressed because it is too large. Load diff
+129 -70
View File
@@ -47,6 +47,10 @@ AC_DEFINE_UNQUOTED(SYSTEM, ["$system"], [platform identifier ('cpu-os')])
# State should be stored in /nix/var, unless the user overrides it explicitly.
test "$localstatedir" = '${prefix}/var' && localstatedir=/nix/var
# Assign a default value to C{,XX}FLAGS as the default configure script sets them
# to -O2 otherwise, which we don't want to have hardcoded
CFLAGS=${CFLAGS-""}
CXXFLAGS=${CXXFLAGS-""}
AC_PROG_CC
AC_PROG_CXX
@@ -58,13 +62,16 @@ AC_CHECK_TOOL([AR], [ar])
AC_SYS_LARGEFILE
# Solaris-specific stuff.
AC_STRUCT_DIRENT_D_TYPE
# OS-specific stuff.
case "$host_os" in
solaris*)
# Solaris requires -lsocket -lnsl for network functions
LDFLAGS="-lsocket -lnsl $LDFLAGS"
;;
darwin*)
# Need to link to libsandbox.
LDFLAGS="-lsandbox $LDFLAGS"
;;
esac
@@ -86,9 +93,10 @@ AC_LANG_POP(C++)
AC_CHECK_FUNCS([statvfs pipe2])
# Check for lutimes, optionally used for changing the mtime of
# symlinks.
AC_CHECK_FUNCS([lutimes])
# Check for lutimes and utimensat, optionally used for changing the
# mtime of symlinks.
AC_CHECK_DECLS([AT_SYMLINK_NOFOLLOW], [], [], [[#include <fcntl.h>]])
AC_CHECK_FUNCS([lutimes utimensat])
# Check whether the store optimiser can optimise symlinks.
@@ -122,7 +130,6 @@ AC_PATH_PROG(flex, flex, false)
AC_PATH_PROG(bison, bison, false)
AC_PATH_PROG(dot, dot)
AC_PATH_PROG(lsof, lsof, lsof)
NEED_PROG(jq, jq)
AC_SUBST(coreutils, [$(dirname $(type -p cat))])
@@ -133,6 +140,47 @@ AC_ARG_WITH(store-dir, AS_HELP_STRING([--with-store-dir=PATH],[path of the Nix s
AC_SUBST(storedir)
# Running the functional tests without building Nix is useful for testing
# different pre-built versions of Nix against each other.
AC_ARG_ENABLE(build, AS_HELP_STRING([--disable-build],[Do not build nix]),
ENABLE_BUILD=$enableval, ENABLE_BUILD=yes)
AC_SUBST(ENABLE_BUILD)
# Building without unit tests is useful for bootstrapping with a smaller footprint
# or running the tests in a separate derivation. Otherwise, we do compile and
# run them.
AC_ARG_ENABLE(unit-tests, AS_HELP_STRING([--disable-unit-tests],[Do not build the tests]),
ENABLE_UNIT_TESTS=$enableval, ENABLE_UNIT_TESTS=$ENABLE_BUILD)
AC_SUBST(ENABLE_UNIT_TESTS)
AS_IF(
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_UNIT_TESTS" == "yes"],
[AC_MSG_ERROR([Cannot enable unit tests when building overall is disabled. Please do not pass '--enable-unit-tests' or do not pass '--disable-build'.])])
AC_ARG_ENABLE(functional-tests, AS_HELP_STRING([--disable-functional-tests],[Do not build the tests]),
ENABLE_FUNCTIONAL_TESTS=$enableval, ENABLE_FUNCTIONAL_TESTS=yes)
AC_SUBST(ENABLE_FUNCTIONAL_TESTS)
# documentation generation switch
AC_ARG_ENABLE(doc-gen, AS_HELP_STRING([--disable-doc-gen],[disable documentation generation]),
ENABLE_DOC_GEN=$enableval, ENABLE_DOC_GEN=$ENABLE_BUILD)
AC_SUBST(ENABLE_DOC_GEN)
AS_IF(
[test "$ENABLE_BUILD" == "no" && test "$ENABLE_DOC_GEN" == "yes"],
[AC_MSG_ERROR([Cannot enable generated docs when building overall is disabled. Please do not pass '--enable-doc-gen' or do not pass '--disable-build'.])])
AS_IF(
[test "$ENABLE_FUNCTIONAL_TESTS" == "yes" || test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(jq, jq)])
AS_IF(
[test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(man, man)])
AS_IF([test "$ENABLE_BUILD" == "yes"],[
# Look for boost, a required dependency.
# Note that AX_BOOST_BASE only exports *CPP* BOOST_CPPFLAGS, no CXX flags,
# and CPPFLAGS are not passed to the C++ compiler automatically.
@@ -155,18 +203,6 @@ if test "x$GCC_ATOMIC_BUILTINS_NEED_LIBATOMIC" = xyes; then
LDFLAGS="-latomic $LDFLAGS"
fi
# Running the functional tests without building Nix is useful for testing
# different pre-built versions of Nix against each other.
AC_ARG_ENABLE(build, AS_HELP_STRING([--disable-build],[Do not build nix]),
ENABLE_BUILD=$enableval, ENABLE_BUILD=yes)
AC_SUBST(ENABLE_BUILD)
# Building without tests is useful for bootstrapping with a smaller footprint
# or running the tests in a separate derivation. Otherwise, we do compile and
# run them.
AC_ARG_ENABLE(tests, AS_HELP_STRING([--disable-tests],[Do not build the tests]),
ENABLE_TESTS=$enableval, ENABLE_TESTS=yes)
AC_SUBST(ENABLE_TESTS)
AC_ARG_ENABLE(install-unit-tests, AS_HELP_STRING([--enable-install-unit-tests],[Install the unit tests for running later (default no)]),
INSTALL_UNIT_TESTS=$enableval, INSTALL_UNIT_TESTS=no)
AC_SUBST(INSTALL_UNIT_TESTS)
@@ -179,11 +215,6 @@ AC_ARG_WITH(check-lib-dir, AS_HELP_STRING([--with-check-lib-dir=PATH],[path to i
checklibdir=$withval, checklibdir=$libdir)
AC_SUBST(checklibdir)
# Building without API docs is the default as Nix' C++ interfaces are internal and unstable.
AC_ARG_ENABLE(internal_api_docs, AS_HELP_STRING([--enable-internal-api-docs],[Build API docs for Nix's internal unstable C++ interfaces]),
internal_api_docs=$enableval, internal_api_docs=no)
AC_SUBST(internal_api_docs)
# LTO is currently broken with clang for unknown reasons; ld segfaults in the llvm plugin
AC_ARG_ENABLE(lto, AS_HELP_STRING([--enable-lto],[Enable LTO (only supported with GCC) [default=no]]),
lto=$enableval, lto=no)
@@ -227,17 +258,25 @@ PKG_CHECK_MODULES([SQLITE3], [sqlite3 >= 3.6.19], [CXXFLAGS="$SQLITE3_CFLAGS $CX
# Look for libcurl, a required dependency.
PKG_CHECK_MODULES([LIBCURL], [libcurl], [CXXFLAGS="$LIBCURL_CFLAGS $CXXFLAGS"])
# Look for editline, a required dependency.
# Look for editline or readline, a required dependency.
# The the libeditline.pc file was added only in libeditline >= 1.15.2,
# see https://github.com/troglobit/editline/commit/0a8f2ef4203c3a4a4726b9dd1336869cd0da8607,
# but e.g. Ubuntu 16.04 has an older version, so we fall back to searching for
# editline.h when the pkg-config approach fails.
PKG_CHECK_MODULES([EDITLINE], [libeditline], [CXXFLAGS="$EDITLINE_CFLAGS $CXXFLAGS"], [
AC_CHECK_HEADERS([editline.h], [true],
[AC_MSG_ERROR([Nix requires libeditline; it was found neither via pkg-config nor its normal header.])])
AC_SEARCH_LIBS([readline read_history], [editline], [],
[AC_MSG_ERROR([Nix requires libeditline; it was not found via pkg-config, but via its header, but required functions do not work. Maybe it is too old? >= 1.14 is required.])])
])
# Older versions are no longer supported.
AC_ARG_WITH(
[readline-flavor],
AS_HELP_STRING([--with-readline-flavor],[Which library to use for nice line editting with the Nix language REPL" [default=editline]]),
[readline_flavor=$withval],
[readline_flavor=editline])
AS_CASE(["$readline_flavor"],
[editline], [
readline_flavor_pc=libeditline
],
[readline], [
readline_flavor_pc=readline
AC_DEFINE([USE_READLINE], [1], [Use readline instead of editline])
],
[AC_MSG_ERROR([bad value "$readline_flavor" for --with-readline-flavor, must be one of: editline, readline])])
PKG_CHECK_MODULES([EDITLINE], [$readline_flavor_pc], [CXXFLAGS="$EDITLINE_CFLAGS $CXXFLAGS"])
# Look for libsodium.
PKG_CHECK_MODULES([SODIUM], [libsodium], [CXXFLAGS="$SODIUM_CFLAGS $CXXFLAGS"])
@@ -269,9 +308,20 @@ case "$host_os" in
]))
if test "x$enable_seccomp_sandboxing" != "xno"; then
PKG_CHECK_MODULES([LIBSECCOMP], [libseccomp],
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS"])
[CXXFLAGS="$LIBSECCOMP_CFLAGS $CXXFLAGS" CFLAGS="$LIBSECCOMP_CFLAGS $CFLAGS"])
have_seccomp=1
AC_DEFINE([HAVE_SECCOMP], [1], [Whether seccomp is available and should be used for sandboxing.])
AC_COMPILE_IFELSE([
AC_LANG_SOURCE([[
#include <seccomp.h>
#ifndef __SNR_fchmodat2
# error "Missing support for fchmodat2"
#endif
]])
], [], [
echo "libseccomp is missing __SNR_fchmodat2. Please provide libseccomp 2.5.5 or later"
exit 1
])
else
have_seccomp=
fi
@@ -282,6 +332,14 @@ case "$host_os" in
esac
AC_SUBST(HAVE_SECCOMP, [$have_seccomp])
# Optional dependencies for better normalizing file system data
AC_CHECK_HEADERS([sys/xattr.h])
AS_IF([test "$ac_cv_header_sys_xattr_h" = "yes"],[
AC_CHECK_FUNCS([llistxattr lremovexattr])
AS_IF([test "$ac_cv_func_llistxattr" = "yes" && test "$ac_cv_func_lremovexattr" = "yes"],[
AC_DEFINE([HAVE_ACL_SUPPORT], [1], [Define if we can manipulate file system Access Control Lists])
])
])
# Look for aws-cpp-sdk-s3.
AC_LANG_PUSH(C++)
@@ -291,13 +349,6 @@ AC_CHECK_HEADERS([aws/s3/S3Client.h],
AC_SUBST(ENABLE_S3, [$enable_s3])
AC_LANG_POP(C++)
if test -n "$enable_s3"; then
declare -a aws_version_tokens=($(printf '#include <aws/core/VersionConfig.h>\nAWS_SDK_VERSION_STRING' | $CPP $CPPFLAGS - | grep -v '^#.*' | sed 's/"//g' | tr '.' ' '))
AC_DEFINE_UNQUOTED([AWS_VERSION_MAJOR], ${aws_version_tokens@<:@0@:>@}, [Major version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_MINOR], ${aws_version_tokens@<:@1@:>@}, [Minor version of aws-sdk-cpp.])
AC_DEFINE_UNQUOTED([AWS_VERSION_PATCH], ${aws_version_tokens@<:@2@:>@}, [Patch version of aws-sdk-cpp.])
fi
# Whether to use the Boehm garbage collector.
AC_ARG_ENABLE(gc, AS_HELP_STRING([--enable-gc],[enable garbage collection in the Nix expression evaluator (requires Boehm GC) [default=yes]]),
@@ -306,49 +357,55 @@ if test "$gc" = yes; then
PKG_CHECK_MODULES([BDW_GC], [bdw-gc])
CXXFLAGS="$BDW_GC_CFLAGS $CXXFLAGS"
AC_DEFINE(HAVE_BOEHMGC, 1, [Whether to use the Boehm garbage collector.])
# See `fixupBoehmStackPointer`, for the integration between Boehm GC
# and Boost coroutines.
old_CFLAGS="$CFLAGS"
# Temporary set `-pthread` just for the next check
CFLAGS="$CFLAGS -pthread"
AC_CHECK_FUNCS([pthread_attr_get_np pthread_getattr_np])
CFLAGS="$old_CFLAGS"
fi
if test "$ENABLE_TESTS" = yes; then
AS_IF([test "$ENABLE_UNIT_TESTS" == "yes"],[
# Look for gtest.
PKG_CHECK_MODULES([GTEST], [gtest_main])
PKG_CHECK_MODULES([GTEST], [gtest_main gmock_main])
# Look for rapidcheck.
AC_ARG_VAR([RAPIDCHECK_HEADERS], [include path of gtest headers shipped by RAPIDCHECK])
# No pkg-config yet, https://github.com/emil-e/rapidcheck/issues/302
AC_LANG_PUSH(C++)
AC_SUBST(RAPIDCHECK_HEADERS)
[CXXFLAGS="-I $RAPIDCHECK_HEADERS $CXXFLAGS"]
[LIBS="-lrapidcheck -lgtest $LIBS"]
AC_CHECK_HEADERS([rapidcheck/gtest.h], [], [], [#include <gtest/gtest.h>])
dnl AC_CHECK_LIB doesn't work for C++ libs with mangled symbols
AC_LINK_IFELSE([
AC_LANG_PROGRAM([[
#include <gtest/gtest.h>
#include <rapidcheck/gtest.h>
]], [[
return RUN_ALL_TESTS();
]])
],
[],
[AC_MSG_ERROR([librapidcheck is not found.])])
AC_LANG_POP(C++)
PKG_CHECK_MODULES([RAPIDCHECK], [rapidcheck rapidcheck_gtest])
fi
])
# Look for nlohmann/json.
PKG_CHECK_MODULES([NLOHMANN_JSON], [nlohmann_json >= 3.9])
# documentation generation switch
AC_ARG_ENABLE(doc-gen, AS_HELP_STRING([--disable-doc-gen],[disable documentation generation]),
doc_generate=$enableval, doc_generate=yes)
AC_SUBST(doc_generate)
# Look for lowdown library.
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.9.0], [CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"])
AC_ARG_ENABLE([markdown], AS_HELP_STRING([--enable-markdown], [Enable Markdown rendering in the Nix binary (requires lowdown) [default=auto]]),
enable_markdown=$enableval, enable_markdown=auto)
AS_CASE(["$enable_markdown"],
[yes | auto], [
PKG_CHECK_MODULES([LOWDOWN], [lowdown >= 0.9.0], [
CXXFLAGS="$LOWDOWN_CFLAGS $CXXFLAGS"
have_lowdown=1
AC_DEFINE(HAVE_LOWDOWN, 1, [Whether lowdown is available and should be used for Markdown rendering.])
], [
AS_IF([test "x$enable_markdown" == "xyes"], [AC_MSG_ERROR([--enable-markdown was specified, but lowdown was not found.])])
])
],
[no], [have_lowdown=],
[AC_MSG_ERROR([bad value "$enable_markdown" for --enable-markdown, must be one of: yes, no, auto])])
# Look for libgit2.
PKG_CHECK_MODULES([LIBGIT2], [libgit2])
# Look for toml11, a required dependency.
AC_LANG_PUSH(C++)
AC_CHECK_HEADER([toml.hpp], [], [AC_MSG_ERROR([toml11 is not found.])])
AC_LANG_POP(C++)
# Setuid installations.
AC_CHECK_FUNCS([setresuid setreuid lchown])
@@ -380,6 +437,8 @@ if test "$embedded_sandbox_shell" = yes; then
AC_DEFINE(HAVE_EMBEDDED_SANDBOX_SHELL, 1, [Include the sandbox shell in the Nix binary.])
fi
])
# Expand all variables in config.status.
test "$prefix" = NONE && prefix=$ac_default_prefix
+9 -10
View File
@@ -1,10 +1,9 @@
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix
-19
View File
@@ -1,19 +0,0 @@
.PHONY: internal-api-html
ifeq ($(internal_api_docs), yes)
$(docdir)/internal-api/html/index.html $(docdir)/internal-api/latex: $(d)/doxygen.cfg
mkdir -p $(docdir)/internal-api
{ cat $< ; echo "OUTPUT_DIRECTORY=$(docdir)/internal-api" ; } | doxygen -
# Generate the HTML API docs for Nix's unstable internal interfaces.
internal-api-html: $(docdir)/internal-api/html/index.html
else
# Make a nicer error message
internal-api-html:
@echo "Internal API docs are disabled. Configure with '--enable-internal-api-docs', or avoid calling 'make internal-api-html'."
@exit 1
endif
+22
View File
@@ -1,3 +1,25 @@
:root {
--sidebar-width: 23em;
}
h1.menu-title::before {
content: "";
background-image: url("./favicon.svg");
padding: 1.25em;
background-position: center center;
background-size: 2em;
background-repeat: no-repeat;
}
h1.menu-title {
padding: 0.5em;
}
.sidebar .sidebar-scrollbox {
padding: 1em;
}
h1:not(:first-of-type) {
margin-top: 1.3em;
}
-31
View File
@@ -1,31 +0,0 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
in
builtinsInfo:
let
showBuiltin = name: { doc, type, impure-only }:
let
type' = optionalString (type != null) " (${type})";
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name}</code></a>${type'}
</dt>
<dd>
${doc}
${impureNotice}
</dd>
'';
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+32 -7
View File
@@ -1,28 +1,53 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
inherit (import <nix/utils.nix>) optionalString squash;
in
builtinsInfo:
let
showBuiltin = name: { doc, args, arity, experimental-feature }:
showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
let
type' = optionalString (type != null) " (${type})";
experimentalNotice = optionalString (experimental-feature != null) ''
This function is only available if the [${experimental-feature}](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) experimental feature is enabled.
> **Note**
>
> This function is only available if the [`${experimental-feature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimental-feature}) is enabled.
>
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimental-feature}
> ```
'';
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name} ${listArgs args}</code></a>
<a href="#builtins-${name}"><code>${name}${listArgs args}</code></a>${type'}
</dt>
<dd>
${doc}
${experimentalNotice}
${doc}
${impureNotice}
</dd>
'';
listArgs = args: concatStringsSep " " (map (s: "<var>${s}</var>") args);
listArgs = args: concatStringsSep "" (map (s: " <var>${s}</var>") args);
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+149 -70
View File
@@ -1,9 +1,29 @@
let
inherit (builtins)
attrNames attrValues fromJSON listToAttrs mapAttrs groupBy
concatStringsSep concatMap length lessThan replaceStrings sort;
inherit (import <nix/utils.nix>) attrsToList concatStrings optionalString filterAttrs trim squash unique;
showStoreDocs = import ./generate-store-info.nix;
attrNames
attrValues
concatMap
concatStringsSep
fromJSON
groupBy
length
lessThan
listToAttrs
mapAttrs
match
replaceStrings
sort
;
inherit (import <nix/utils.nix>)
attrsToList
concatStrings
filterAttrs
optionalString
squash
trim
unique
;
showStoreDocs = import <nix/generate-store-info.nix>;
in
inlineHTML: commandDump:
@@ -12,13 +32,19 @@ let
commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }:
showCommand =
{
command,
details,
filename,
toplevel,
}:
let
result = ''
> **Warning** \
> This program is
> [**experimental**](@docroot@/contributing/experimental-features.md#xp-feature-nix-command)
> [**experimental**](@docroot@/development/experimental-features.md#xp-feature-nix-command)
> and its interface is subject to change.
# Name
@@ -31,31 +57,32 @@ let
${maybeSubcommands}
${maybeStoreDocs}
${maybeProse}
${maybeOptions}
'';
showSynopsis = command: args:
showSynopsis =
command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in ''
in
''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories);
@@ -71,75 +98,125 @@ let
* [`${command} ${name}`](./${appendName filename name}.md) - ${subcmd.description}
'';
# FIXME: this is a hack.
# store parameters should not be part of command documentation to begin
# with, but instead be rendered on separate pages.
maybeStoreDocs = optionalString (details ? doc)
(replaceStrings [ "@stores@" ] [ (showStoreDocs inlineHTML commandInfo.stores) ] details.doc);
maybeProse =
# FIXME: this is a horrible hack to keep `nix help-stores` working.
let
help-stores = ''
${index}
maybeOptions = let
allVisibleOptions = filterAttrs
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
in optionalString (allVisibleOptions != {}) ''
# Options
${allStores}
'';
index =
replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview =
let
showEntry = store: "- [${store.name}](#${store.slug})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storesList = showStoreDocs {
storeInfo = commandInfo.stores;
inherit inlineHTML;
};
hasInfix =
infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in
optionalString (details ? doc) (
# An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc then help-stores else details.doc
);
${showOptions inlineHTML allVisibleOptions}
maybeOptions =
let
allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
in
optionalString (allVisibleOptions != { }) ''
# Options
> **Note**
>
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
${showOptions inlineHTML allVisibleOptions}
showOptions = inlineHTML: allOptions:
> **Note**
>
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions =
inlineHTML: allOptions:
let
showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
'';
showOption = name: option:
'';
showOption =
name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in result;
categories = mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy
(cmd: cmd.value.category)
(attrsToList allOptions));
in concatStrings (attrValues (mapAttrs showCategory categories));
in squash result;
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories =
mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy (cmd: cmd.value.category) (attrsToList allOptions));
in
concatStrings (attrValues (mapAttrs showCategory categories));
in
squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }:
processCommand =
{
command,
details,
filename,
toplevel,
}:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename toplevel; };
value = showCommand {
inherit
command
details
filename
toplevel
;
};
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand {
command = "nix";
@@ -148,9 +225,11 @@ let
toplevel = commandInfo.args;
};
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
tableOfContents =
let
showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in
concatStringsSep "\n" (map showEntry manpages) + "\n";
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+74 -41
View File
@@ -1,66 +1,99 @@
let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs;
inherit (import ./utils.nix) concatStrings indent optionalString squash;
inherit (builtins)
attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo:
{
prefix,
inlineHTML ? true,
}:
settingsInfo:
let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
let
result = squash ''
- ${item}
- ${item}
${indent " " body}
'';
item = if inlineHTML
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else "`${setting}`";
${indent " " body}
'';
item =
if inlineHTML then
''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
# separate body to cleanly handle indentation
body = ''
${description}
${experimentalFeatureNote}
${experimentalFeatureNote}
${description}
**Default:** ${showDefault documentDefault defaultValue}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
${showAliases aliases}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This setting is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
To change this setting, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
${setting} = ...
```
'';
showDefault = documentDefault: defaultValue:
showDefault =
documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
"*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else
"`${toString defaultValue}`"
else
"*machine-specific*";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
showAliases =
aliases:
optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result;
in
result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+66 -31
View File
@@ -1,45 +1,80 @@
let
inherit (builtins) attrValues mapAttrs;
inherit (import ./utils.nix) concatStrings optionalString;
showSettings = import ./generate-settings.nix;
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
showSettings = import <nix/generate-settings.nix>;
in
inlineHTML: storesInfo:
{
# data structure describing all stores and their parameters
storeInfo,
# whether to add inline HTML tags
# `lowdown` does not eat those for one of the output modes
inlineHTML,
}:
let
showStore = name: { settings, doc, experimentalFeature }:
showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
let
result = squash ''
# ${name}
result = ''
## ${name}
${experimentalFeatureNote}
${doc}
${doc}
${experimentalFeatureNote}
## Settings
### Settings
${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
'';
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
>
> This store is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To use this store, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ```
'';
in
result;
# markdown doesn't like spaces in URLs
slug = builtins.replaceStrings [ " " ] [ "-" ] name;
storesList = map (name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
}) (attrNames storeInfo);
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
> This store is part of an
> [experimental feature](@docroot@/contributing/experimental-features.md).
To use this store, you need to make sure the corresponding experimental feature,
[`${experimentalFeature}`](@docroot@/contributing/experimental-features.md#xp-feature-${experimentalFeature}),
is enabled.
For example, include the following in [`nix.conf`](#):
```
extra-experimental-features = ${experimentalFeature}
```
'';
in result;
in concatStrings (attrValues (mapAttrs showStore storesInfo))
in
storesList
+45
View File
@@ -0,0 +1,45 @@
let
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>;
in
storeInfo:
let
storesList = showStoreDocs {
inherit storeInfo;
inlineHTML = true;
};
index =
let
showEntry = store: "- [${store.name}](./${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings [ "@store-types@" ] [ index ] (readFile ./src/store/types/index.md.in);
tableOfContents =
let
showEntry = store: " - [${store.name}](store/types/${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents;
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
in
storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -1,9 +1,9 @@
with builtins;
with import ./utils.nix;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
''
- [`${name}`](@docroot@/contributing/experimental-features.md#xp-feature-${name})
'';
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+6 -3
View File
@@ -1,11 +1,14 @@
with builtins;
with import ./utils.nix;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
showExperimentalFeature =
name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
${doc}
'';
in xps: (concatStringsSep "\n" (attrValues (mapAttrs showExperimentalFeature xps)))
in
xps: (concatStringsSep "\n" (attrValues (mapAttrs showExperimentalFeature xps)))
+59 -27
View File
@@ -1,4 +1,7 @@
ifeq ($(doc_generate),yes)
# The version of Nix used to generate the doc. Can also be
# `$(nix_INSTALL_PATH)` or just `nix` (to grap ambient from the `PATH`),
# if one prefers.
doc_nix = $(nix_PATH)
MANUAL_SRCS := \
$(call rwildcard, $(d)/src, *.md) \
@@ -24,7 +27,7 @@ man-pages += $(foreach subcommand, \
clean-files += $(d)/*.1 $(d)/*.5 $(d)/*.8
# Provide a dummy environment for nix, so that it will not access files outside the macOS sandbox.
# Set cores to 0 because otherwise nix show-config resolves the cores based on the current machine
# Set cores to 0 because otherwise `nix config show` resolves the cores based on the current machine
dummy-env = env -i \
HOME=/dummy \
NIX_CONF_DIR=/dummy \
@@ -32,7 +35,7 @@ dummy-env = env -i \
NIX_STATE_DIR=/dummy \
NIX_CONFIG='cores = 0'
nix-eval = $(dummy-env) $(bindir)/nix eval --experimental-features nix-command -I nix=doc/manual --store dummy:// --impure --raw
nix-eval = $(dummy-env) $(doc_nix) eval --experimental-features nix-command -I nix=doc/manual --store dummy:// --impure --raw
# re-implement mdBook's include directive to make it usable for terminal output and for proper @docroot@ substitution
define process-includes
@@ -92,64 +95,95 @@ $(d)/nix-profiles.5: $(d)/src/command-ref/files/profiles.md
$(trace-gen) lowdown -sT man --nroff-nolinks -M section=5 $^.tmp -o $@
@rm $^.tmp
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md
$(d)/src/SUMMARY.md: $(d)/src/SUMMARY.md.in $(d)/src/SUMMARY-rl-next.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/development/experimental-feature-descriptions.md
@cp $< $@
@$(call process-includes,$@,$@)
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(d)/generate-settings.nix $(d)/generate-store-info.nix $(bindir)/nix
$(d)/src/store/types: $(d)/nix.json $(d)/utils.nix $(d)/generate-store-info.nix $(d)/generate-store-types.nix $(d)/src/store/types/index.md.in $(doc_nix)
@# FIXME: build out of tree!
@rm -rf $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-store-types.nix (builtins.fromJSON (builtins.readFile $<)).stores'
@# do not destroy existing contents
@mv $@.tmp/* $@/
$(d)/src/command-ref/new-cli: $(d)/nix.json $(d)/utils.nix $(d)/generate-manpage.nix $(d)/generate-settings.nix $(d)/generate-store-info.nix $(doc_nix)
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-manpage.nix true (builtins.readFile $<)'
@mv $@.tmp $@
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/utils.nix $(d)/generate-settings.nix $(d)/src/command-ref/conf-file-prefix.md $(d)/src/command-ref/experimental-features-shortlist.md $(bindir)/nix
$(d)/src/command-ref/conf-file.md: $(d)/conf-file.json $(d)/utils.nix $(d)/generate-settings.nix $(d)/src/command-ref/conf-file-prefix.md $(d)/src/command-ref/experimental-features-shortlist.md $(doc_nix)
@cat doc/manual/src/command-ref/conf-file-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-settings.nix { prefix = "conf"; } (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@mv $@.tmp $@
$(d)/nix.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix __dump-cli > $@.tmp
$(d)/nix.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) __dump-cli > $@.tmp
@mv $@.tmp $@
$(d)/conf-file.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix show-config --json --experimental-features nix-command > $@.tmp
$(d)/conf-file.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) config show --json --experimental-features nix-command > $@.tmp
@mv $@.tmp $@
$(d)/src/contributing/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(bindir)/nix
$(d)/src/development/experimental-feature-descriptions.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features.nix $(doc_nix)
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-xp-features.nix (builtins.fromJSON (builtins.readFile $<))'
@mv $@.tmp $@
$(d)/src/command-ref/experimental-features-shortlist.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features-shortlist.nix $(bindir)/nix
$(d)/src/command-ref/experimental-features-shortlist.md: $(d)/xp-features.json $(d)/utils.nix $(d)/generate-xp-features-shortlist.nix $(doc_nix)
@rm -rf $@ $@.tmp
$(trace-gen) $(nix-eval) --write-to $@.tmp --expr 'import doc/manual/generate-xp-features-shortlist.nix (builtins.fromJSON (builtins.readFile $<))'
@mv $@.tmp $@
$(d)/xp-features.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix __dump-xp-features > $@.tmp
$(d)/xp-features.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) __dump-xp-features > $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(bindir)/nix
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(doc_nix)
@cat doc/manual/src/language/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<)).builtins' >> $@.tmp;
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@cat doc/manual/src/language/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtin-constants.md: $(d)/language.json $(d)/generate-builtin-constants.nix $(d)/src/language/builtin-constants-prefix.md $(bindir)/nix
@cat doc/manual/src/language/builtin-constants-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtin-constants.nix (builtins.fromJSON (builtins.readFile $<)).constants' >> $@.tmp;
@cat doc/manual/src/language/builtin-constants-suffix.md >> $@.tmp
$(d)/language.json: $(doc_nix)
$(trace-gen) $(dummy-env) $(doc_nix) __dump-language > $@.tmp
@mv $@.tmp $@
$(d)/language.json: $(bindir)/nix
$(trace-gen) $(dummy-env) $(bindir)/nix __dump-language > $@.tmp
@mv $@.tmp $@
# Generate "Upcoming release" notes (or clear it and remove from menu)
$(d)/src/release-notes/rl-next.md: $(d)/rl-next $(d)/rl-next/*
@if type -p changelog-d > /dev/null; then \
echo " GEN " $@; \
changelog-d doc/manual/rl-next > $@; \
else \
echo " NULL " $@; \
true > $@; \
fi
$(d)/src/SUMMARY-rl-next.md: $(d)/src/release-notes/rl-next.md
$(trace-gen) true
@if [ -s $< ]; then \
echo ' - [Upcoming release](release-notes/rl-next.md)' > $@; \
else \
true > $@; \
fi
# Generate the HTML manual.
.PHONY: manual-html
manual-html: $(docdir)/manual/index.html
# Open the built HTML manual in the default browser.
manual-html-open: $(docdir)/manual/index.html
@echo " OPEN " $<; \
xdg-open $< \
|| open $< \
|| { \
echo "Could not open the manual in a browser. Please open '$<'" >&2; \
false; \
}
install: $(docdir)/manual/index.html
# Generate 'nix' manpages.
.PHONY: manpages
manpages: $(mandir)/man1/nix3-manpages
install: $(mandir)/man1/nix3-manpages
man: doc/manual/generated/man1/nix3-manpages
all: doc/manual/generated/man1/nix3-manpages
@@ -173,11 +207,11 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
done
@touch $@
# the `! -name 'contributing.md'` filter excludes the one place where
# the `! -name 'documentation.md'` filter excludes the one place where
# `@docroot@` is to be preserved for documenting the mechanism
# FIXME: maybe contributing guides should live right next to the code
# instead of in the manual
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/store/types $(d)/src/command-ref/new-cli $(d)/src/development/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/release-notes/rl-next.md $(d)/src/figures $(d)/src/favicon.png $(d)/src/favicon.svg
$(trace-gen) \
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
@@ -195,5 +229,3 @@ $(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/
@rm -rf $(DESTDIR)$(docdir)/manual
@mv $(DESTDIR)$(docdir)/manual.tmp/html $(DESTDIR)$(docdir)/manual
@rm -rf $(DESTDIR)$(docdir)/manual.tmp
endif
+39 -23
View File
@@ -1,7 +1,9 @@
// redirect rules for anchors ensure backwards compatibility of URLs.
// this must be done on the client side, as web servers do not see the anchor part of the URL.
// redirect rules for URL fragments (client-side) to prevent link rot.
// this must be done on the client side, as web servers do not see the fragment part of the URL.
// it will only work with JavaScript enabled in the browser, but this is the best we can do here.
// see src/_redirects for path redirects (server-side)
// redirections are declared as follows:
// redirects are declared as follows:
// each entry has as its key a path matching the requested URL path, relative to the mdBook document root.
//
// IMPORTANT: it must specify the full path with file name and suffix
@@ -12,14 +14,15 @@
const redirects = {
"index.html": {
"part-advanced-topics": "advanced-topics/advanced-topics.html",
"part-advanced-topics": "advanced-topics/index.html",
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
"chap-diff-hook": "advanced-topics/diff-hook.html",
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
"chap-distributed-builds": "advanced-topics/distributed-builds.html",
"chap-distributed-builds": "command-ref/conf-file.html#conf-builders",
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
"part-command-ref": "command-ref/command-ref.html",
"chap-writing-nix-expressions": "language/index.html",
"part-command-ref": "command-ref/index.html",
"conf-allow-import-from-derivation": "command-ref/conf-file.html#conf-allow-import-from-derivation",
"conf-allow-new-privileges": "command-ref/conf-file.html#conf-allow-new-privileges",
"conf-allowed-uris": "command-ref/conf-file.html#conf-allowed-uris",
@@ -140,7 +143,7 @@ const redirects = {
"opt-timeout": "command-ref/opt-common.html#opt-timeout",
"sec-common-options": "command-ref/opt-common.html",
"ch-utilities": "command-ref/utilities.html",
"chap-hacking": "contributing/hacking.html",
"chap-hacking": "development/building.html",
"adv-attr-allowSubstitutes": "language/advanced-attributes.html#adv-attr-allowSubstitutes",
"adv-attr-allowedReferences": "language/advanced-attributes.html#adv-attr-allowedReferences",
"adv-attr-allowedRequisites": "language/advanced-attributes.html#adv-attr-allowedRequisites",
@@ -235,12 +238,12 @@ const redirects = {
"attr-system": "language/derivations.html#attr-system",
"ssec-derivation": "language/derivations.html",
"ch-expression-language": "language/index.html",
"sec-constructs": "language/constructs.html",
"sect-let-language": "language/constructs.html#let-language",
"ss-functions": "language/constructs.html#functions",
"sec-constructs": "language/syntax.html",
"sect-let-language": "language/syntax.html#let-expressions",
"ss-functions": "language/syntax.html#functions",
"sec-language-operators": "language/operators.html",
"table-operators": "language/operators.html",
"ssec-values": "language/values.html",
"ssec-values": "language/types.html",
"gloss-closure": "glossary.html#gloss-closure",
"gloss-derivation": "glossary.html#gloss-derivation",
"gloss-deriver": "glossary.html#gloss-deriver",
@@ -258,7 +261,7 @@ const redirects = {
"sec-installer-proxy-settings": "installation/env-variables.html#proxy-environment-variables",
"sec-nix-ssl-cert-file": "installation/env-variables.html#nix_ssl_cert_file",
"sec-nix-ssl-cert-file-with-nix-daemon-and-macos": "installation/env-variables.html#nix_ssl_cert_file-with-macos-and-the-nix-daemon",
"chap-installation": "installation/installation.html",
"chap-installation": "installation/index.html",
"ch-installing-binary": "installation/installing-binary.html",
"sect-macos-installation": "installation/installing-binary.html#macos-installation",
"sect-macos-installation-change-store-prefix": "installation/installing-binary.html#macos-installation",
@@ -282,19 +285,19 @@ const redirects = {
"ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"sec-channels": "command-ref/nix-channel.html",
"ssec-copy-closure": "package-management/copy-closure.html",
"ssec-copy-closure": "command-ref/nix-copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
"chap-package-management": "package-management/package-management.html",
"chap-package-management": "package-management/index.html",
"sec-profiles": "package-management/profiles.html",
"ssec-s3-substituter": "package-management/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "package-management/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "package-management/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "package-management/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter": "store/types/s3-substituter.html",
"ssec-s3-substituter-anonymous-reads": "store/types/s3-substituter.html#anonymous-reads-to-your-s3-compatible-binary-cache",
"ssec-s3-substituter-authenticated-reads": "store/types/s3-substituter.html#authenticated-reads-to-your-s3-binary-cache",
"ssec-s3-substituter-authenticated-writes": "store/types/s3-substituter.html#authenticated-writes-to-your-s3-compatible-binary-cache",
"sec-sharing-packages": "package-management/sharing-packages.html",
"ssec-ssh-substituter": "package-management/ssh-substituter.html",
"chap-quick-start": "quick-start.html",
"sec-relnotes": "release-notes/release-notes.html",
"sec-relnotes": "release-notes/index.html",
"ch-relnotes-0.10.1": "release-notes/rl-0.10.1.html",
"ch-relnotes-0.10": "release-notes/rl-0.10.html",
"ssec-relnotes-0.11": "release-notes/rl-0.11.html",
@@ -332,18 +335,22 @@ const redirects = {
"ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html",
},
"language/values.html": {
"language/types.html": {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"attribute-sets": "#attribute-set",
"type-number": "#type-int",
},
"language/syntax.html": {
"scoping-rules": "scoping.html",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
},
"contributing/hacking.html": {
"development/building.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
"running-tests": "testing.html#running-tests",
@@ -354,8 +361,17 @@ const redirects = {
"installer-tests": "testing.html#installer-tests",
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
"characterization-testing": "#characterisation-testing-unit",
}
"characterization-testing": "testing.html#characterisation-testing-unit",
"add-a-release-note": "contributing.html#add-a-release-note",
"add-an-entry": "contributing.html#add-an-entry",
"build-process": "contributing.html#build-process",
"reverting": "contributing.html#reverting",
"branches": "contributing.html#branches",
},
"glossary.html": {
"gloss-local-store": "store/types/local-store.html",
"gloss-chroot-store": "store/types/local-store.html",
},
};
// the following code matches the current page's URL against the set of redirects.
+2
View File
@@ -0,0 +1,2 @@
organization: NixOS
repository: nix
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
@@ -0,0 +1,205 @@
# String literals
A *string literal* represents a [string](types.md#type-string) value.
> **Syntax**
>
> *expression* → *string*
>
> *string* → `"` ( *string_char*\* [*interpolation_element*][string interpolation] )* *string_char*\* `"`
>
> *string* → `''` ( *indented_string_char*\* [*interpolation_element*][string interpolation] )* *indented_string_char*\* `''`
>
> *string* → *uri*
>
> *string_char* ~ `[^"$\\]|\$(?!\{)|\\.`
>
> *indented_string_char* ~ `[^$']|\$\$|\$(?!\{)|''[$']|''\\.|'(?!')`
>
> *uri* ~ `[A-Za-z][+\-.0-9A-Za-z]*:[!$%&'*+,\-./0-9:=?@A-Z_a-z~]+`
Strings can be written in three ways.
The most common way is to enclose the string between double quotes, e.g., `"foo bar"`.
Strings can span multiple lines.
The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation].
[string interpolation]: ./string-interpolation.md
The following must be escaped to represent them within a string, by prefixing with a backslash (`\`):
- Double quote (`"`)
> **Example**
>
> ```nix
> "\""
> ```
>
> "\""
- Backslash (`\`)
> **Example**
>
> ```nix
> "\\"
> ```
>
> "\\"
- Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly"
> **Example**
>
> ```nix
> "\${"
> ```
>
> "\${"
The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> "$${"
> ```
>
> "$\${"
String values are output on the terminal with Nix-specific escaping.
Strings written to files will contain the characters encoded by the escaping.
The second way to write string literals is as an *indented string*, which is enclosed between pairs of *double single-quotes* (`''`), like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
> **Note**
>
> Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line.
> **Warning**
>
> Prefixed tab characters are not stripped.
>
> > **Example**
> >
> > The following indented string is prefixed with tabs:
> >
> > <pre><code class="nohighlight">''
> > all:
> > @echo hello
> > ''
> > </code></pre>
> >
> > "\tall:\n\t\t@echo hello\n"
Indented strings support [string interpolation].
The following must be escaped to represent them in an indented string:
- `$` is escaped by prefixing it with two single quotes (`''`)
> **Example**
>
> ```nix
> ''
> ''$
> ''
> ```
>
> "$\n"
- `''` is escaped by prefixing it with one single quote (`'`)
> **Example**
>
> ```nix
> ''
> '''
> ''
> ```
>
> "''\n"
These special characters are escaped as follows:
- Linefeed (`\n`): `''\n`
- Carriage return (`\r`): `''\r`
- Tab (`\t`): `''\t`
`''\` escapes any other character.
A "dollar-curly" (`${`) can be written as follows:
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
> **Note**
>
> This differs from the syntax for escaping a dollar-curly within double quotes (`"\${"`). Be aware of which one is needed at a given moment.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> ''
> $${
> ''
> ```
>
> "$\${\n"
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
+40 -23
View File
@@ -2,7 +2,7 @@
- [Introduction](introduction.md)
- [Quick Start](quick-start.md)
- [Installation](installation/installation.md)
- [Installation](installation/index.md)
- [Supported Platforms](installation/supported-platforms.md)
- [Installing a Binary Distribution](installation/installing-binary.md)
- [Installing Nix from Source](installation/installing-source.md)
@@ -18,34 +18,38 @@
- [Uninstalling Nix](installation/uninstall.md)
- [Nix Store](store/index.md)
- [File System Object](store/file-system-object.md)
- [Content-Addressing File System Objects](store/file-system-object/content-address.md)
- [Store Object](store/store-object.md)
- [Content-Addressing Store Objects](store/store-object/content-address.md)
- [Store Path](store/store-path.md)
- [Store Types](store/types/index.md)
{{#include ./store/types/SUMMARY.md}}
- [Nix Language](language/index.md)
- [Data Types](language/values.md)
- [Language Constructs](language/constructs.md)
- [Data Types](language/types.md)
- [String context](language/string-context.md)
- [Syntax and semantics](language/syntax.md)
- [Identifiers](language/identifiers.md)
- [Scoping rules](language/scope.md)
- [String interpolation](language/string-interpolation.md)
- [Lookup path](language/constructs/lookup-path.md)
- [Operators](language/operators.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Built-in Constants](language/builtin-constants.md)
- [Built-in Functions](language/builtins.md)
- [Package Management](package-management/package-management.md)
- [Built-ins](language/builtins.md)
- [Derivations](language/derivations.md)
- [Advanced Attributes](language/advanced-attributes.md)
- [Import From Derivation](language/import-from-derivation.md)
- [Package Management](package-management/index.md)
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Advanced Topics](advanced-topics/advanced-topics.md)
- [Advanced Topics](advanced-topics/index.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
- [Serving a Nix store via SSH](package-management/ssh-substituter.md)
- [Serving a Nix store via S3](package-management/s3-substituter.md)
- [Remote Builds](advanced-topics/distributed-builds.md)
- [Tuning Cores and Jobs](advanced-topics/cores-vs-jobs.md)
- [Verifying Build Reproducibility](advanced-topics/diff-hook.md)
- [Using the `post-build-hook`](advanced-topics/post-build-hook.md)
- [Command Reference](command-ref/command-ref.md)
- [Command Reference](command-ref/index.md)
- [Common Options](command-ref/opt-common.md)
- [Common Environment Variables](command-ref/env-common.md)
- [Main Commands](command-ref/main-commands.md)
@@ -102,19 +106,32 @@
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [Protocols](protocols/protocols.md)
- [Formats and Protocols](protocols/index.md)
- [JSON Formats](protocols/json/index.md)
- [Store Object Info](protocols/json/store-object-info.md)
- [Derivation](protocols/json/derivation.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Store Path Specification](protocols/store-path.md)
- [Nix Archive (NAR) Format](protocols/nix-archive.md)
- [Derivation "ATerm" file format](protocols/derivation-aterm.md)
- [C API](c-api.md)
- [Glossary](glossary.md)
- [Contributing](contributing/contributing.md)
- [Hacking](contributing/hacking.md)
- [Testing](contributing/testing.md)
- [Documentation](contributing/documentation.md)
- [Experimental Features](contributing/experimental-features.md)
- [CLI guideline](contributing/cli-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Release X.Y (202?-??-??)](release-notes/rl-next.md)
- [Development](development/index.md)
- [Building](development/building.md)
- [Testing](development/testing.md)
- [Documentation](development/documentation.md)
- [CLI guideline](development/cli-guideline.md)
- [JSON guideline](development/json-guideline.md)
- [C++ style guide](development/cxx.md)
- [Experimental Features](development/experimental-features.md)
- [Contributing](development/contributing.md)
- [Releases](release-notes/index.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.24 (2024-07-31)](release-notes/rl-2.24.md)
- [Release 2.23 (2024-06-03)](release-notes/rl-2.23.md)
- [Release 2.22 (2024-04-23)](release-notes/rl-2.22.md)
- [Release 2.21 (2024-03-11)](release-notes/rl-2.21.md)
- [Release 2.20 (2024-01-29)](release-notes/rl-2.20.md)
- [Release 2.19 (2023-11-17)](release-notes/rl-2.19.md)
- [Release 2.18 (2023-09-20)](release-notes/rl-2.18.md)
- [Release 2.17 (2023-07-24)](release-notes/rl-2.17.md)
+54
View File
@@ -0,0 +1,54 @@
# redirect rules for paths (server-side) to prevent link rot.
# see ../redirects.js for redirects based on URL fragments (client-side)
#
# concrete user story this supports:
# - user finds URL to the manual for Nix x.y
# - Nix x.z (z > y) is the most recent release
# - updating the version in the URL will show the right thing
#
# format documentation:
# - https://docs.netlify.com/routing/redirects/#syntax-for-the-redirects-file
# - https://docs.netlify.com/routing/redirects/redirect-options/
#
# conventions:
# - always force (<CODE>!) since this allows re-using file names
# - group related paths to ease readability
# - keep in alphabetical/wildcards-last order, which will reduce version control conflicts
# - redirects that should have been there but are missing can be inserted where they belong
/advanced-topics/advanced-topics /advanced-topics 301!
/command-ref/command-ref /command-ref 301!
/contributing/contributing /development 301!
/contributing /development 301!
/contributing/hacking /development/building 301!
/contributing/testing /development/testing 301!
/contributing/documentation /development/documentation 301!
/contributing/experimental-features /development/experimental-features 301!
/contributing/cli-guideline /development/cli-guideline 301!
/contributing/json-guideline /development/json-guideline 301!
/contributing/cxx /development/cxx 301!
/expressions/expression-language /language/ 301!
/expressions/language-constructs /language/constructs 301!
/expressions/language-operators /language/operators 301!
/expressions/language-values /language/values 301!
/expressions/* /language/:splat 301!
/language/values /language/types 301!
/language/constructs /language/syntax 301!
/language/builtin-constants /language/builtins 301!
/installation/installation /installation 301!
/package-management/basic-package-mgmt /command-ref/nix-env 301!
/package-management/channels /command-ref/nix-channel 301!
/package-management/package-management /package-management 301!
/package-management/s3-substituter /store/types/s3-binary-cache-store 301!
/protocols/protocols /protocols 301!
/json/* /protocols/json/:splat 301!
/release-notes/release-notes /release-notes 301!
/package-management/copy-closure /command-ref/nix-copy-closure 301!
@@ -12,14 +12,14 @@ machine is accessible via SSH and that it has Nix installed. You can
test whether connecting to the remote Nix instance works, e.g.
```console
$ nix store info --store ssh://mac
$ nix store ping --store ssh://mac
```
will try to connect to the machine named `mac`. It is possible to
specify an SSH identity file as part of the remote store URI, e.g.
```console
$ nix store info --store ssh://mac?ssh-key=/home/alice/my-key
$ nix store ping --store ssh://mac?ssh-key=/home/alice/my-key
```
Since builds should be non-interactive, the key should not have a
@@ -36,16 +36,8 @@ error: cannot connect to 'mac'
then you need to ensure that the `PATH` of non-interactive login shells
contains Nix.
> **Warning**
>
> If you are building via the Nix daemon, it is the Nix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
>
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
The list of remote machines can be specified on the command line or in
the Nix configuration file. The former is convenient for testing. For
example, the following command allows you to build a derivation for
`x86_64-darwin` on a Linux machine:
The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file.
For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine:
```console
$ uname
@@ -60,97 +52,20 @@ $ cat ./result
Darwin
```
It is possible to specify multiple builders separated by a semicolon or
a newline, e.g.
It is possible to specify multiple build machines separated by a semicolon or a newline, e.g.
```console
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
```
Each machine specification consists of the following elements, separated
by spaces. Only the first element is required. To leave a field at its
default, set it to `-`.
1. The URI of the remote store in the format
`ssh://[username@]hostname`, e.g. `ssh://nix@mac` or `ssh://mac`.
For backward compatibility, `ssh://` may be omitted. The hostname
may be an alias defined in your `~/.ssh/config`.
2. A comma-separated list of Nix platform type identifiers, such as
`x86_64-darwin`. It is possible for a machine to support multiple
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
defaults to the local platform type.
3. The SSH identity file to be used to log in to the remote machine. If
omitted, SSH will use its regular identities.
4. The maximum number of builds that Nix will execute in parallel on
the machine. Typically this should be equal to the number of CPU
cores. For instance, the machine `itchy` in the example will execute
up to 8 builds in parallel.
5. The “speed factor”, indicating the relative speed of the machine. If
there are multiple machines of the right type, Nix will prefer the
fastest, taking load into account.
6. A comma-separated list of *supported features*. If a derivation has
the `requiredSystemFeatures` attribute, then Nix will only perform
the derivation on a machine that has the specified features. For
instance, the attribute
```nix
requiredSystemFeatures = [ "kvm" ];
```
will cause the build to be performed on a machine that has the `kvm`
feature.
7. A comma-separated list of *mandatory features*. A machine will only
be used to build a derivation if all of the machine’s mandatory
features appear in the derivation’s `requiredSystemFeatures`
attribute.
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
will use its regular known-hosts file. Specifically, the field is calculated
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
For example, the machine specification
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
specifies several machines that can perform `i686-linux` builds.
However, `poochie` will only do builds that have the attribute
```nix
requiredSystemFeatures = [ "benchmark" ];
```
or
```nix
requiredSystemFeatures = [ "benchmark" "kvm" ];
```
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
such builds. For regular builds, `itchy` will be preferred over
`scratchy` because it has a higher speed factor.
Remote builders can also be configured in `nix.conf`, e.g.
Remote build machines can also be configured in [`nix.conf`](@docroot@/command-ref/conf-file.md), e.g.
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
Finally, remote builders can be configured in a separate configuration
file included in `builders` via the syntax `@file`. For example,
Finally, remote build machines can be configured in a separate configuration
file included in `builders` via the syntax `@/path/to/file`. For example,
builders = @/etc/nix/machines
causes the list of machines in `/etc/nix/machines` to be included. (This
is the default.)
If you want the builders to use caches, you likely want to set the
option `builders-use-substitutes` in your local `nix.conf`.
To build only on remote builders and disable building on the local
machine, you can use the option `--max-jobs 0`.
causes the list of machines in `/etc/nix/machines` to be included.
(This is the default.)
+2 -2
View File
@@ -52,7 +52,7 @@ The following [concept map] shows its main components (rectangles), the objects
'---------------'
```
At the top is the [command line interface](../command-ref/command-ref.md) that drives the underlying layers.
At the top is the [command line interface](../command-ref/index.md) that drives the underlying layers.
The [Nix language](../language/index.md) evaluator transforms Nix expressions into self-contained *build plans*, which are used to derive *build results* from referenced *build inputs*.
@@ -69,7 +69,7 @@ It can also execute build plans to produce new data, which are made available to
A build plan itself is a series of *build tasks*, together with their build inputs.
> **Important**
> A build task in Nix is called [derivation](../glossary.md#gloss-derivation).
> A build task in Nix is called [derivation](@docroot@/glossary.md#gloss-derivation).
Each build task has a special build input executed as *build instructions* in order to perform the build.
The result of a build task can be input to another build task.
+16
View File
@@ -0,0 +1,16 @@
# C API
Nix provides a C API with the intent of [_becoming_](https://github.com/NixOS/nix/milestone/52) a stable API, which it is currently not.
It is in development.
See:
- C API documentation for a recent build of master
- [Getting Started]
- [Index]
- [Matrix Room *Nix Bindings*](https://matrix.to/#/#nix-bindings:nixos.org) for discussion and questions.
- [Stabilisation Milestone](https://github.com/NixOS/nix/milestone/52)
- [Other C API PRs and issues](https://github.com/NixOS/nix/labels/c%20api)
- [Contributing C API Documentation](development/documentation.md#c-api-documentation), including how to build it locally.
[Getting Started]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs
[Index]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs/globals.html
@@ -66,5 +66,12 @@ Configuration options can be set on the command line, overriding the values set
The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`).
## Integer settings
Settings that have an integer type support the suffixes `K`, `M`, `G`
and `T`. These cause the specified value to be multiplied by 2^10,
2^20, 2^30 and 2^40, respectively. For instance, `--min-free 1M` is
equivalent to `--min-free 1048576`.
# Available settings
+15 -11
View File
@@ -9,22 +9,26 @@ Most Nix commands interpret the following environment variables:
- <span id="env-NIX_PATH">[`NIX_PATH`](#env-NIX_PATH)</span>
A colon-separated list of directories used to look up the location of Nix
expressions using [paths](@docroot@/language/values.md#type-path)
enclosed in angle brackets (i.e., `<path>`),
e.g. `/home/eelco/Dev:/etc/nixos`. It can be extended using the
[`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
A colon-separated list of search path entries used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
If `NIX_PATH` is not set at all, Nix will fall back to the following list in [impure](@docroot@/command-ref/conf-file.md#conf-pure-eval) and [unrestricted](@docroot@/command-ref/conf-file.md#conf-restrict-eval) evaluation mode:
This environment variable overrides the value of the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path).
1. `$HOME/.nix-defexpr/channels`
2. `nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs`
3. `/nix/var/nix/profiles/per-user/root/channels`
It can be extended using the [`-I` option](@docroot@/command-ref/opt-common.md#opt-I).
> **Example**
>
> ```bash
> $ export NIX_PATH=`/home/eelco/Dev:nixos-config=/etc/nixos
> ```
If `NIX_PATH` is set to an empty string, resolving search paths will always fail.
For example, attempting to use `<nixpkgs>` will produce:
error: file 'nixpkgs' was not found in the Nix search path
> **Example**
>
> ```bash
> $ NIX_PATH= nix-instantiate --eval '<nixpkgs>'
> error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
> ```
- <span id="env-NIX_IGNORE_SYMLINK_STORE">[`NIX_IGNORE_SYMLINK_STORE`](#env-NIX_IGNORE_SYMLINK_STORE)</span>
@@ -1,6 +1,6 @@
# Experimental Commands
This section lists [experimental commands](@docroot@/contributing/experimental-features.md#xp-feature-nix-command).
This section lists [experimental commands](@docroot@/development/experimental-features.md#xp-feature-nix-command).
> **Warning**
>
+10 -10
View File
@@ -41,7 +41,7 @@ expression to a low-level [store derivation]) and [`nix-store
--realise`](@docroot@/command-ref/nix-store/realise.md) (to build the store
derivation).
[store derivation]: ../glossary.md#gloss-store-derivation
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
> **Warning**
>
@@ -55,20 +55,20 @@ All options not listed here are passed to
[`nix-store --realise`](nix-store/realise.md),
except for `--arg` and `--attr` / `-A` which are passed to [`nix-instantiate`](nix-instantiate.md).
- <span id="opt-no-out-link">[`--no-out-link`](#opt-no-out-link)<span>
- <span id="opt-no-out-link">[`--no-out-link`](#opt-no-out-link)<span>
Do not create a symlink to the output path. Note that as a result
the output does not become a root of the garbage collector, and so
might be deleted by `nix-store --gc`.
Do not create a symlink to the output path. Note that as a result
the output does not become a root of the garbage collector, and so
might be deleted by `nix-store --gc`.
- <span id="opt-dry-run">[`--dry-run`](#opt-dry-run)</span>
- <span id="opt-dry-run">[`--dry-run`](#opt-dry-run)</span>
Show what store paths would be built or downloaded.
Show what store paths would be built or downloaded.
- <span id="opt-out-link">[`--out-link`](#opt-out-link)</span> / `-o` *outlink*
- <span id="opt-out-link">[`--out-link`](#opt-out-link)</span> / `-o` *outlink*
Change the name of the symlink to the output path created from
`result` to *outlink*.
Change the name of the symlink to the output path created from
`result` to *outlink*.
{{#include ./status-build-failure.md}}
+32 -26
View File
@@ -27,40 +27,46 @@ The moving parts of channels are:
This command has the following operations:
- `--add` *url* \[*name*\]\
Add a channel *name* located at *url* to the list of subscribed channels.
If *name* is omitted, default to the last component of *url*, with the suffixes `-stable` or `-unstable` removed.
- `--add` *url* \[*name*\]
> **Note**
>
> `--add` does not automatically perform an update.
> Use `--update` explicitly.
Add a channel *name* located at *url* to the list of subscribed channels.
If *name* is omitted, default to the last component of *url*, with the suffixes `-stable` or `-unstable` removed.
A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
> **Note**
>
> `--add` does not automatically perform an update.
> Use `--update` explicitly.
- `--remove` *name*\
Remove the channel *name* from the list of subscribed channels.
A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
- `--list`\
Print the names and URLs of all subscribed channels on standard output.
- `--remove` *name*
- `--update` \[*names*…\]\
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
Remove the channel *name* from the list of subscribed channels.
- `--list-generations`\
Prints a list of all the current existing generations for the
channel profile.
- `--list`
Works the same way as
```
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
```
Print the names and URLs of all subscribed channels on standard output.
- `--rollback` \[*generation*\]\
Revert channels to the state before the last call to `nix-channel --update`.
Optionally, you can specify a specific channel *generation* number to restore.
- `--update` \[*names*…\]
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
- `--list-generations`
Prints a list of all the current existing generations for the
channel profile.
Works the same way as
```
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
```
- `--rollback` \[*generation*\]
Revert channels to the state before the last call to `nix-channel --update`.
Optionally, you can specify a specific channel *generation* number to restore.
{{#include ./opt-common.md}}
@@ -48,18 +48,29 @@ Instead, it looks in a few locations, and acts on all profiles it finds there:
These options are for deleting old [profiles] prior to deleting unreachable [store objects].
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`\
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`
Delete all old generations of profiles.
This is the equivalent of invoking `nix-env --delete-generations old` on each found profile.
This is the equivalent of invoking [`nix-env --delete-generations old`](@docroot@/command-ref/nix-env/delete-generations.md#generations-old) on each found profile.
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*\
Delete all generations of profiles older than the specified amount (except for the generations that were active at that point in time).
*period* is a value such as `30d`, which would mean 30 days.
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
- <span id="opt-max-freed">[`--max-freed`](#opt-max-freed)</span> *bytes*
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
{{#include ./opt-common.md}}
{{#include ./env-common.md}}
@@ -74,4 +85,4 @@ $ nix-collect-garbage -d
```
[profiles]: @docroot@/command-ref/files/profiles.md
[store objects]: @docroot@/glossary.md#gloss-store-object
[store objects]: @docroot@/store/store-object.md
+56 -56
View File
@@ -1,91 +1,91 @@
# Name
`nix-copy-closure` - copy a closure to or from a remote machine via SSH
`nix-copy-closure` - copy store objects to or from a remote machine via SSH
# Synopsis
`nix-copy-closure`
[`--to` | `--from`]
[`--to` | `--from` ]
[`--gzip`]
[`--include-outputs`]
[`--use-substitutes` | `-s`]
[`-v`]
_user@machine_ _paths_
[_user_@]_machine_[:_port_] _paths_
# Description
`nix-copy-closure` gives you an easy and efficient way to exchange
software between machines. Given one or more Nix store _paths_ on the
local machine, `nix-copy-closure` computes the closure of those paths
(i.e. all their dependencies in the Nix store), and copies all paths
in the closure to the remote machine via the `ssh` (Secure Shell)
command. With the `--from` option, the direction is reversed: the
closure of _paths_ on a remote machine is copied to the Nix store on
the local machine.
Given _paths_ from one machine, `nix-copy-closure` computes the [closure](@docroot@/glossary.md#gloss-closure) of those paths (i.e. all their dependencies in the Nix store), and copies [store objects](@docroot@/glossary.md#gloss-store-object) in that closure to another machine via SSH.
It doesn’t copy store objects that are already present on the other machine.
This command is efficient because it only sends the store paths
that are missing on the target machine.
> **Note**
>
> While the Nix store to use on the local machine can be specified on the command line with the [`--store`](@docroot@/command-ref/conf-file.md#conf-store) option, the Nix store to be accessed on the remote machine can only be [configured statically](@docroot@/command-ref/conf-file.md#configuration-file) on that remote machine.
Since `nix-copy-closure` calls `ssh`, you may be asked to type in the
appropriate password or passphrase. In fact, you may be asked _twice_
because `nix-copy-closure` currently connects twice to the remote
machine, first to get the set of paths missing on the target machine,
and second to send the dump of those paths. When using public key
authentication, you can avoid typing the passphrase with `ssh-agent`.
Since `nix-copy-closure` calls `ssh`, you may need to authenticate with the remote machine.
In fact, you may be asked for authentication _twice_ because `nix-copy-closure` currently connects twice to the remote machine: first to get the set of paths missing on the target machine, and second to send the dump of those paths.
When using public key authentication, you can avoid typing the passphrase with `ssh-agent`.
# Options
- `--to`\
Copy the closure of _paths_ from the local Nix store to the Nix
store on _machine_. This is the default.
- `--to`
- `--from`\
Copy the closure of _paths_ from the Nix store on _machine_ to the
local Nix store.
Copy the closure of _paths_ from a Nix store accessible from the local machine to the Nix store on the remote _machine_.
This is the default behavior.
- `--gzip`\
Enable compression of the SSH connection.
- `--from`
- `--include-outputs`\
Also copy the outputs of [store derivation]s included in the closure.
Copy the closure of _paths_ from the Nix store on the remote _machine_ to the local machine's specified Nix store.
[store derivation]: ../glossary.md#gloss-store-derivation
- `--gzip`
- `--use-substitutes` / `-s`\
Attempt to download missing paths on the target machine using Nix’s
substitute mechanism. Any paths that cannot be substituted on the
target are still copied normally from the source. This is useful,
for instance, if the connection between the source and target
machine is slow, but the connection between the target machine and
`nixos.org` (the default binary cache server) is
fast.
Enable compression of the SSH connection.
- `-v`\
Show verbose output.
- `--include-outputs`
Also copy the outputs of [store derivation]s included in the closure.
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
- `--use-substitutes` / `-s`
Attempt to download missing store objects on the target from [substituters](@docroot@/command-ref/conf-file.md#conf-substituters).
Any store objects that cannot be substituted on the target are still copied normally from the source.
This is useful, for instance, if the connection between the source and target machine is slow, but the connection between the target machine and `cache.nixos.org` (the default binary cache server) is fast.
{{#include ./opt-common.md}}
# Environment variables
- `NIX_SSHOPTS`\
Additional options to be passed to `ssh` on the command
line.
- `NIX_SSHOPTS`
Additional options to be passed to `ssh` on the command line.
{{#include ./env-common.md}}
# Examples
Copy Firefox with all its dependencies to a remote machine:
> **Example**
>
> Copy GNU Hello with all its dependencies to a remote machine:
>
> ```shell-session
> $ storePath="$(nix-build '<nixpkgs>' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link)"
> $ nix-copy-closure --to alice@itchy.example.org "$storePath"
> copying 5 paths...
> copying path '/nix/store/nrwkk6ak3rgkrxbqhsscb01jpzmslf2r-xgcc-13.2.0-libgcc' to 'ssh://alice@itchy.example.org'...
> copying path '/nix/store/gm61h1y42pqyl6178g90x8zm22n6pyy5-libunistring-1.1' to 'ssh://alice@itchy.example.org'...
> copying path '/nix/store/ddfzjdykw67s20c35i7a6624by3iz5jv-libidn2-2.3.7' to 'ssh://alice@itchy.example.org'...
> copying path '/nix/store/apab5i73dqa09wx0q27b6fbhd1r18ihl-glibc-2.39-31' to 'ssh://alice@itchy.example.org'...
> copying path '/nix/store/g1n2vryg06amvcc1avb2mcq36faly0mh-hello-2.12.1' to 'ssh://alice@itchy.example.org'...
> ```
```console
$ nix-copy-closure --to alice@itchy.labs $(type -tP firefox)
```
Copy Subversion from a remote machine and then install it into a user
environment:
```console
$ nix-copy-closure --from alice@itchy.labs \
/nix/store/0dj0503hjxy5mbwlafv1rsbdiyx1gkdy-subversion-1.4.4
$ nix-env --install /nix/store/0dj0503hjxy5mbwlafv1rsbdiyx1gkdy-subversion-1.4.4
```
> **Example**
>
> Copy GNU Hello from a remote machine using a known store path, and run it:
>
> ```shell-session
> $ storePath="$(nix-instantiate --eval '<nixpkgs>' -I nixpkgs=channel:nixpkgs-unstable -A hello.outPath | tr -d '"')"
> $ nix-copy-closure --from alice@itchy.example.org "$storePath"
> $ "$storePath"/bin/hello
> Hello, world!
> ```
+69 -25
View File
@@ -47,39 +47,83 @@ These pages can be viewed offline:
Example: `nix-env --help --install`
# Package sources
`nix-env` can obtain packages from multiple sources:
- An attribute set of derivations from:
- The [default Nix expression](@docroot@/command-ref/files/default-nix-expression.md) (by default)
- A Nix file, specified via `--file`
- A [profile](@docroot@/command-ref/files/profiles.md), specified via `--from-profile`
- A Nix expression that is a function which takes default expression as argument, specified via `--from-expression`
- A [store path](@docroot@/store/store-path.md)
# Selectors
Several commands, such as `nix-env --query ` and `nix-env --install `, take a list of
arguments that specify the packages on which to operate. These are
extended regular expressions that must match the entire name of the
package. (For details on regular expressions, see **regex**(7).) The match is
case-sensitive. The regular expression can optionally be followed by a
dash and a version number; if omitted, any version of the package will
match. Here are some examples:
Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of *arguments* that specify the packages on which to operate.
- `firefox`\
Matches the package name `firefox` and any version.
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name):
- `firefox-32.0`\
Matches the package name `firefox` and version `32.0`.
- `name`: Everything up to but not including the first dash (`-`) that is *not* followed by a letter.
- `version`: The rest, excluding the separating dash.
- `gtk\\+`\
Matches the package name `gtk+`. The `+` character must be escaped
using a backslash to prevent it from being interpreted as a
quantifier, and the backslash must be escaped in turn with another
backslash to ensure that the shell passes it on.
> **Example**
>
> `nix-env` parses the symbolic derivation name `apache-httpd-2.0.48` as:
>
> ```json
> {
> "name": "apache-httpd",
> "version": "2.0.48"
> }
> ```
- `.\*`\
Matches any package name. This is the default for most commands.
> **Example**
>
> `nix-env` parses the symbolic derivation name `firefox.*` as:
>
> ```json
> {
> "name": "firefox.*",
> "version": ""
> }
> ```
- `'.*zip.*'`\
Matches any package name containing the string `zip`. Note the dots:
`'*zip*'` does not work, because in a regular expression, the
character `*` is interpreted as a quantifier.
The `name` parts of the *arguments* to `nix-env` are treated as extended regular expressions and matched against the `name` parts of derivation names in the package source.
The match is case-sensitive.
The regular expression can optionally be followed by a dash (`-`) and a version number; if omitted, any version of the package will match.
For details on regular expressions, see [**regex**(7)](https://linux.die.net/man/7/regex).
- `'.*(firefox|chromium).*'`\
Matches any package name containing the strings `firefox` or
`chromium`.
> **Example**
>
> Common patterns for finding package names with `nix-env`:
>
> - `firefox`
>
> Matches the package name `firefox` and any version.
>
> - `firefox-32.0`
>
> Matches the package name `firefox` and version `32.0`.
>
> - `gtk\\+`
>
> Matches the package name `gtk+`.
> The `+` character must be escaped using a backslash (`\`) to prevent it from being interpreted as a quantifier, and the backslash must be escaped in turn with another backslash to ensure that the shell passes it on.
>
> - `.\*`
>
> Matches any package name.
> This is the default for most commands.
>
> - `'.*zip.*'`
>
> Matches any package name containing the string `zip`.
> Note the dots: `'*zip*'` does not work, because in a regular expression, the character `*` is interpreted as a quantifier.
>
> - `'.*(firefox|chromium).*'`
>
> Matches any package name containing the strings `firefox` or `chromium`.
# Files
@@ -12,13 +12,14 @@ This operation deletes the specified generations of the current profile.
*generations* can be a one of the following:
- <span id="generations-list">`<number>...`</span>:\
- <span id="generations-list">[`<number>...`](#generations-list)</span>
A list of generation numbers, each one a separate command-line argument.
Delete exactly the profile generations given by their generation number.
Deleting the current generation is not allowed.
- The special value <span id="generations-old">`old`</span>
- <span id="generations-old">[The special value `old`](#generations-old)</span>
Delete all generations except the current one.
@@ -30,7 +31,8 @@ This operation deletes the specified generations of the current profile.
> Because one can roll back to a previous generation, it is possible to have generations newer than the current one.
> They will also be deleted.
- <span id="generations-time">`<number>d`</span>:\
- <span id="generations-time">[`<number>d`](#generations-time)</span>
The last *number* days
*Example*: `30d`
@@ -38,7 +40,8 @@ This operation deletes the specified generations of the current profile.
Delete all generations created more than *number* days ago, except the most recent one of them.
This allows rolling back to generations that were available within the specified period.
- <span id="generations-count">`+<number>`</span>:\
- <span id="generations-count">[`+<number>`](#generations-count)</span>
The last *number* generations up to the present
*Example*: `+5`
@@ -49,7 +52,7 @@ Periodically deleting old generations is important to make garbage collection
effective.
The is because profiles are also garbage collection roots — any [store object] reachable from a profile is "alive" and ineligible for deletion.
[store object]: @docroot@/glossary.md#gloss-store-object
[store object]: @docroot@/store/store-object.md
{{#include ./opt-common.md}}
@@ -1,6 +1,7 @@
# Environment variables
- `NIX_PROFILE`\
- `NIX_PROFILE`
Location of the Nix profile. Defaults to the target of the symlink
`~/.nix-profile`, if it exists, or `/nix/var/nix/profiles/default`
otherwise.
+94 -95
View File
@@ -14,133 +14,132 @@
# Description
The install operation creates a new user environment.
The `--install` operation creates a new user environment.
It is based on the current generation of the active [profile](@docroot@/command-ref/files/profiles.md), to which a set of [store paths] described by *args* is added.
[store paths]: @docroot@/glossary.md#gloss-store-path
[store paths]: @docroot@/store/store-path.md
The arguments *args* map to store paths in a number of possible ways:
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
- By default, *args* is a set of [derivation] names denoting derivations in the [default Nix expression].
These are [realised], and the resulting output paths are installed.
Currently installed derivations with a name equal to the name of a derivation being added are removed unless the option `--preserve-installed` is specified.
[derivation]: @docroot@/glossary.md#gloss-derivation
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise
[derivation]: @docroot@/glossary.md#gloss-derivation
[default Nix expression]: @docroot@/command-ref/files/default-nix-expression.md
[realised]: @docroot@/glossary.md#gloss-realise
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple derivations matching a name in *args* that
have the same name (e.g., `gcc-3.3.6` and `gcc-4.1.1`), then the
derivation with the highest *priority* is used. A derivation can
define a priority by declaring the `meta.priority` attribute. This
attribute should be a number, with a higher value denoting a lower
priority. The default priority is `5`.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
If there are multiple matching derivations with the same priority,
then the derivation with the highest version will be installed.
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
You can force the installation of multiple derivations with the same
name by being specific about the versions. For instance, `nix-env --install
gcc-3.3.6 gcc-4.1.1` will install both version of GCC (and will
probably cause a user environment conflict\!).
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
- If [`--attr`](#opt-attr) / `-A` is specified, the arguments are *attribute paths* that select attributes from the [default Nix expression].
This is faster than using derivation names and unambiguous.
Show the attribute paths of available packages with [`nix-env --query`](./query.md):
```console
nix-env --query --available --attr-path
```
```console
nix-env --query --available --attr-path`
```
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-profile` *path* is given, *args* is a set of names
denoting installed [store paths] in the profile *path*. This is an
easy way to copy user environment elements from one profile to
another.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/syntax.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If `--from-expression` is given, *args* are [Nix language functions](@docroot@/language/constructs.md#functions) that are called with the [default Nix expression] as their single argument.
The derivations returned by those function calls are installed.
This allows derivations to be specified in an unambiguous way, which is necessary if there are multiple derivations with the same name.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store derivations](@docroot@/glossary.md#gloss-store-derivation), then these are [realised], and the resulting output paths are installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- If *args* are [store paths] that are not store derivations, then these are [realised] and installed.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
- By default all [outputs](@docroot@/language/derivations.md#attr-outputs) are installed for each [derivation].
This can be overridden by adding a `meta.outputsToInstall` attribute on the derivation listing a subset of the output names.
Example:
Example:
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
The file `example.nix` defines a derivation with two outputs `foo` and `bar`, each containing a file.
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
```nix
# example.nix
let
pkgs = import <nixpkgs> {};
command = ''
${pkgs.coreutils}/bin/mkdir -p $foo $bar
echo foo > $foo/foo-file
echo bar > $bar/bar-file
'';
in
derivation {
name = "example";
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" command ];
outputs = [ "foo" "bar" ];
system = builtins.currentSystem;
}
```
Installing from this Nix expression will make files from both outputs appear in the current profile.
Installing from this Nix expression will make files from both outputs appear in the current profile.
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
```console
$ nix-env --install --file example.nix
installing 'example'
$ ls ~/.nix-profile
foo-file
bar-file
manifest.nix
```
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
Adding `meta.outputsToInstall` to that derivation will make `nix-env` only install files from the specified outputs.
```nix
# example-outputs.nix
import ./example.nix // { meta.outputsToInstall = [ "bar" ]; }
```
```nix
# example-outputs.nix
import ./example.nix // { meta.outputsToInstall = [ "bar" ]; }
```
```console
$ nix-env --install --file example-outputs.nix
installing 'example'
$ ls ~/.nix-profile
bar-file
manifest.nix
```
```console
$ nix-env --install --file example-outputs.nix
installing 'example'
$ ls ~/.nix-profile
bar-file
manifest.nix
```
# Options
- `--prebuilt-only` / `-b`
- `--prebuilt-only` / `-b`
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--preserve-installed` / `-P`
- `--preserve-installed` / `-P`
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
- `--remove-all` / `-r`
- `--remove-all` / `-r`
Remove all previously installed packages first. This is equivalent
to running `nix-env --uninstall '.*'` first, except that everything happens
in a single transaction.
Remove all previously installed packages first. This is equivalent
to running `nix-env --uninstall '.*'` first, except that everything happens
in a single transaction.
{{#include ./opt-common.md}}
@@ -2,34 +2,37 @@
The following options are allowed for all `nix-env` operations, but may not always have an effect.
- `--file` / `-f` *path*\
Specifies the Nix expression (designated below as the *active Nix
expression*) used by the `--install`, `--upgrade`, and `--query
--available` operations to obtain derivations. The default is
`~/.nix-defexpr`.
- `--file` / `-f` *path*
If the argument starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must include a single
top-level directory containing at least a file named `default.nix`.
Specifies the Nix expression (designated below as the *active Nix
expression*) used by the `--install`, `--upgrade`, and `--query
--available` operations to obtain derivations. The default is
`~/.nix-defexpr`.
- `--profile` / `-p` *path*\
Specifies the profile to be used by those operations that operate on
a profile (designated below as the *active profile*). A profile is a
sequence of user environments called *generations*, one of which is
the *current generation*.
If the argument starts with `http://` or `https://`, it is
interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must include a single
top-level directory containing at least a file named `default.nix`.
- `--dry-run`\
For the `--install`, `--upgrade`, `--uninstall`,
`--switch-generation`, `--delete-generations` and `--rollback`
operations, this flag will cause `nix-env` to print what *would* be
done if this flag had not been specified, without actually doing it.
- `--profile` / `-p` *path*
`--dry-run` also prints out which paths will be
[substituted](@docroot@/glossary.md) (i.e., downloaded) and which paths
will be built from source (because no substitute is available).
Specifies the profile to be used by those operations that operate on
a profile (designated below as the *active profile*). A profile is a
sequence of user environments called *generations*, one of which is
the *current generation*.
- `--system-filter` *system*\
By default, operations such as `--query
--available` show derivations matching any platform. This option
allows you to use derivations for the specified platform *system*.
- `--dry-run`
For the `--install`, `--upgrade`, `--uninstall`,
`--switch-generation`, `--delete-generations` and `--rollback`
operations, this flag will cause `nix-env` to print what *would* be
done if this flag had not been specified, without actually doing it.
`--dry-run` also prints out which paths will be
[substituted](@docroot@/glossary.md) (i.e., downloaded) and which paths
will be built from source (because no substitute is available).
- `--system-filter` *system*
By default, operations such as `--query --available` show derivations matching any platform. This option
allows you to use derivations for the specified platform *system*.
+36 -18
View File
@@ -35,11 +35,13 @@ The derivations are sorted by their `name` attributes.
The following flags specify the set of things on which the query
operates.
- `--installed`\
- `--installed`
The query operates on the store paths that are installed in the
current generation of the active profile. This is the default.
- `--available`; `-a`\
- `--available` / `-a`
The query operates on the derivations that are available in the
active Nix expression.
@@ -50,24 +52,28 @@ selected derivations. Multiple flags may be specified, in which case the
information is shown in the order given here. Note that the name of the
derivation is shown unless `--no-name` is specified.
- `--xml`\
- `--xml`
Print the result in an XML representation suitable for automatic
processing by other tools. The root element is called `items`, which
contains a `item` element for each available or installed
derivation. The fields discussed below are all stored in attributes
of the `item` elements.
- `--json`\
- `--json`
Print the result in a JSON representation suitable for automatic
processing by other tools.
- `--prebuilt-only` / `-b`\
- `--prebuilt-only` / `-b`
Show only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, this shows all packages that
probably can be installed quickly.
- `--status`; `-s`\
- `--status` / `-s`
Print the *status* of the derivation. The status consists of three
characters. The first is `I` or `-`, indicating whether the
derivation is currently installed in the current generation of the
@@ -78,49 +84,61 @@ derivation is shown unless `--no-name` is specified.
derivation to be built. The third is `S` or `-`, indicating whether
a substitute is available for the derivation.
- `--attr-path`; `-P`\
- `--attr-path` / `-P`
Print the *attribute path* of the derivation, which can be used to
unambiguously select it using the `--attr` option available in
commands that install derivations like `nix-env --install`. This
option only works together with `--available`
- `--no-name`\
- `--no-name`
Suppress printing of the `name` attribute of each derivation.
- `--compare-versions` / `-c`\
- `--compare-versions` / `-c`
Compare installed versions to available versions, or vice versa (if
`--available` is given). This is useful for quickly seeing whether
upgrades for installed packages are available in a Nix expression. A
column is added with the following meaning:
- `<` *version*\
- `<` *version*
A newer version of the package is available or installed.
- `=` *version*\
- `=` *version*
At most the same version of the package is available or
installed.
- `>` *version*\
- `>` *version*
Only older versions of the package are available or installed.
- `- ?`\
- `- ?`
No version of the package is available or installed.
- `--system`\
- `--system`
Print the `system` attribute of the derivation.
- `--drv-path`\
- `--drv-path`
Print the path of the [store derivation](@docroot@/glossary.md#gloss-store-derivation).
- `--out-path`\
- `--out-path`
Print the output path of the derivation.
- `--description`\
- `--description`
Print a short (one-line) description of the derivation, if
available. The description is taken from the `meta.description`
attribute of the derivation.
- `--meta`\
- `--meta`
Print all of the meta-attributes of the derivation. This option is
only available with `--xml` or `--json`.
+16 -16
View File
@@ -13,24 +13,24 @@ to be modified. There are several attributes that can be usefully
modified, because they affect the behaviour of `nix-env` or the user
environment build script:
- `priority` can be changed to resolve filename clashes. The user
environment build script uses the `meta.priority` attribute of
derivations to resolve filename collisions between packages. Lower
priority values denote a higher priority. For instance, the GCC
wrapper package and the Binutils package in Nixpkgs both have a file
`bin/ld`, so previously if you tried to install both you would get a
collision. Now, on the other hand, the GCC wrapper declares a higher
priority than Binutils, so the former’s `bin/ld` is symlinked in the
user environment.
- `priority` can be changed to resolve filename clashes. The user
environment build script uses the `meta.priority` attribute of
derivations to resolve filename collisions between packages. Lower
priority values denote a higher priority. For instance, the GCC
wrapper package and the Binutils package in Nixpkgs both have a file
`bin/ld`, so previously if you tried to install both you would get a
collision. Now, on the other hand, the GCC wrapper declares a higher
priority than Binutils, so the former’s `bin/ld` is symlinked in the
user environment.
- `keep` can be set to `true` to prevent the package from being
upgraded or replaced. This is useful if you want to hang on to an
older version of a package.
- `keep` can be set to `true` to prevent the package from being
upgraded or replaced. This is useful if you want to hang on to an
older version of a package.
- `active` can be set to `false` to “disable” the package. That is, no
symlinks will be generated to the files of the package, but it
remains part of the profile (so it won’t be garbage-collected). It
can be set back to `true` to re-enable the package.
- `active` can be set to `false` to “disable” the package. That is, no
symlinks will be generated to the files of the package, but it
remains part of the profile (so it won’t be garbage-collected). It
can be set back to `true` to re-enable the package.
{{#include ./opt-common.md}}
+37 -31
View File
@@ -28,42 +28,48 @@ version is installed.
# Flags
- `--lt`\
Only upgrade a derivation to newer versions. This is the default.
- `--lt`
- `--leq`\
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same version. Version are not a unique
identification of a derivation, so there may be many derivations
that have the same version. This flag may be useful to force
“synchronisation” between the installed and available derivations.
Only upgrade a derivation to newer versions. This is the default.
- `--eq`\
*Only* “upgrade” to derivations that have the same version. This may
not seem very useful, but it actually is, e.g., when there is a new
release of Nixpkgs and you want to replace installed applications
with the same versions built against newer dependencies (to reduce
the number of dependencies floating around on your system).
- `--leq`
- `--always`\
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same or a lower version. I.e., derivations
may actually be downgraded depending on what is available in the
active Nix expression.
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same version. Version are not a unique
identification of a derivation, so there may be many derivations
that have the same version. This flag may be useful to force
“synchronisation” between the installed and available derivations.
- `--prebuilt-only` / `-b`\
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--eq`
- `--preserve-installed` / `-P`\
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
*Only* “upgrade” to derivations that have the same version. This may
not seem very useful, but it actually is, e.g., when there is a new
release of Nixpkgs and you want to replace installed applications
with the same versions built against newer dependencies (to reduce
the number of dependencies floating around on your system).
- `--always`
In addition to upgrading to newer versions, also “upgrade” to
derivations that have the same or a lower version. I.e., derivations
may actually be downgraded depending on what is available in the
active Nix expression.
- `--prebuilt-only` / `-b`
Use only derivations for which a substitute is registered, i.e.,
there is a pre-built binary available that can be downloaded in lieu
of building the derivation. Thus, no packages will be built from
source.
- `--preserve-installed` / `-P`
Do not remove derivations with a name matching one of the
derivations being installed. Usually, trying to have two versions of
the same package installed in the same generation of a profile will
lead to an error in building the generation, due to file name
clashes between the two versions. However, this is not the case for
all packages.
{{#include ./opt-common.md}}
+52 -36
View File
@@ -20,58 +20,74 @@ an example.
The hash is computed over a *serialisation* of each path: a dump of
the file system tree rooted at the path. This allows directories and
symlinks to be hashed as well as regular files. The dump is in the
*NAR format* produced by [`nix-store
*[Nix Archive (NAR)][Nix Archive] format* produced by [`nix-store
--dump`](@docroot@/command-ref/nix-store/dump.md). Thus, `nix-hash path`
yields the same cryptographic hash as `nix-store --dump path |
md5sum`.
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
# Options
- `--flat`\
Print the cryptographic hash of the contents of each regular file
*path*. That is, do not compute the hash over the dump of *path*.
The result is identical to that produced by the GNU commands
`md5sum` and `sha1sum`.
- `--flat`
- `--base16`\
Print the hash in a hexadecimal representation (default).
Print the cryptographic hash of the contents of each regular file *path*.
That is, instead of computing
the hash of the [Nix Archive (NAR)](@docroot@/store/file-system-object/content-address.md#serial-nix-archive) of *path*,
just [directly hash]((@docroot@/store/file-system-object/content-address.md#serial-flat) *path* as is.
This requires *path* to resolve to a regular file rather than directory.
The result is identical to that produced by the GNU commands
`md5sum` and `sha1sum`.
- `--base32`\
Print the hash in a base-32 representation rather than hexadecimal.
This base-32 representation is more compact and can be used in Nix
expressions (such as in calls to `fetchurl`).
- `--base16`
- `--base64`\
Similar to --base32, but print the hash in a base-64 representation,
which is more compact than the base-32 one.
Print the hash in a hexadecimal representation (default).
- `--sri`\
Print the hash in SRI format with base-64 encoding.
The type of hash algorithm will be prepended to the hash string,
followed by a hyphen (-) and the base-64 hash body.
- `--base32`
- `--truncate`\
Truncate hashes longer than 160 bits (such as SHA-256) to 160 bits.
Print the hash in a base-32 representation rather than hexadecimal.
This base-32 representation is more compact and can be used in Nix
expressions (such as in calls to `fetchurl`).
- `--type` *hashAlgo*\
Use the specified cryptographic hash algorithm, which can be one of
`md5`, `sha1`, `sha256`, and `sha512`.
- `--base64`
- `--to-base16`\
Don’t hash anything, but convert the base-32 hash representation
*hash* to hexadecimal.
Similar to --base32, but print the hash in a base-64 representation,
which is more compact than the base-32 one.
- `--to-base32`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-32.
- `--sri`
- `--to-base64`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-64.
Print the hash in SRI format with base-64 encoding.
The type of hash algorithm will be prepended to the hash string,
followed by a hyphen (-) and the base-64 hash body.
- `--to-sri`\
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to SRI.
- `--truncate`
Truncate hashes longer than 160 bits (such as SHA-256) to 160 bits.
- `--type` *hashAlgo*
Use the specified cryptographic hash algorithm, which can be one of
`md5`, `sha1`, `sha256`, and `sha512`.
- `--to-base16`
Don’t hash anything, but convert the base-32 hash representation
*hash* to hexadecimal.
- `--to-base32`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-32.
- `--to-base64`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to base-64.
- `--to-sri`
Don’t hash anything, but convert the hexadecimal hash representation
*hash* to SRI.
# Examples
+103 -61
View File
@@ -23,58 +23,104 @@ It evaluates the Nix expressions in each of *files* (which defaults to
derivation, a list of derivations, or a set of derivations. The paths
of the resulting store derivations are printed on standard output.
[store derivation]: ../glossary.md#gloss-store-derivation
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
If *files* is the character `-`, then a Nix expression will be read from
standard input.
# Options
- `--add-root` *path*\
See the [corresponding option](nix-store.md) in `nix-store`.
- `--add-root` *path*
- `--parse`\
Just parse the input files, and print their abstract syntax trees on
standard output in ATerm format.
See the [corresponding option](nix-store.md) in `nix-store`.
- `--eval`\
Just parse and evaluate the input files, and print the resulting
values on standard output. No instantiation of store derivations
takes place.
- `--parse`
- `--find-file`\
Look up the given files in Nix’s search path (as specified by the
`NIX_PATH` environment variable). If found, print the corresponding
absolute paths on standard output. For instance, if `NIX_PATH` is
`nixpkgs=/home/alice/nixpkgs`, then `nix-instantiate --find-file
nixpkgs/default.nix` will print `/home/alice/nixpkgs/default.nix`.
Just parse the input files, and print their abstract syntax trees on
standard output as a Nix expression.
- `--strict`\
When used with `--eval`, recursively evaluate list elements and
attributes. Normally, such sub-expressions are left unevaluated
(since the Nix language is lazy).
- `--eval`
> **Warning**
>
> This option can cause non-termination, because lazy data
> structures can be infinitely large.
Just parse and evaluate the input files, and print the resulting
values on standard output. No instantiation of store derivations
takes place.
- `--json`\
When used with `--eval`, print the resulting value as an JSON
representation of the abstract syntax tree rather than as an ATerm.
> **Warning**
>
> This option produces output which can be parsed as a Nix expression which
> will produce a different result than the input expression when evaluated.
> For example, these two Nix expressions print the same result despite
> having different meaning:
>
> ```console
> $ nix-instantiate --eval --expr '{ a = {}; }'
> { a = <CODE>; }
> $ nix-instantiate --eval --expr '{ a = <CODE>; }'
> { a = <CODE>; }
> ```
>
> For human-readable output, `nix eval` (experimental) is more informative:
>
> ```console
> $ nix-instantiate --eval --expr 'a: a'
> <LAMBDA>
> $ nix eval --expr 'a: a'
> «lambda @ «string»:1:1»
> ```
>
> For machine-readable output, the `--xml` option produces unambiguous
> output:
>
> ```console
> $ nix-instantiate --eval --xml --expr '{ foo = <CODE>; }'
> <?xml version='1.0' encoding='utf-8'?>
> <expr>
> <attrs>
> <attr column="3" line="1" name="foo">
> <unevaluated />
> </attr>
> </attrs>
> </expr>
> ```
- `--xml`\
When used with `--eval`, print the resulting value as an XML
representation of the abstract syntax tree rather than as an ATerm.
The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
- `--find-file`
- `--read-write-mode`\
When used with `--eval`, perform evaluation in read/write mode so
nix language features that require it will still work (at the cost
of needing to do instantiation of every evaluated derivation). If
this option is not enabled, there may be uninstantiated store paths
in the final output.
Look up the given files in Nix’s search path (as specified by the
`NIX_PATH` environment variable). If found, print the corresponding
absolute paths on standard output. For instance, if `NIX_PATH` is
`nixpkgs=/home/alice/nixpkgs`, then `nix-instantiate --find-file
nixpkgs/default.nix` will print `/home/alice/nixpkgs/default.nix`.
- `--strict`
When used with `--eval`, recursively evaluate list elements and
attributes. Normally, such sub-expressions are left unevaluated
(since the Nix language is lazy).
> **Warning**
>
> This option can cause non-termination, because lazy data
> structures can be infinitely large.
- `--json`
When used with `--eval`, print the resulting value as an JSON
representation of the abstract syntax tree rather than as a Nix expression.
- `--xml`
When used with `--eval`, print the resulting value as an XML
representation of the abstract syntax tree rather than as a Nix expression.
The schema is the same as that used by the [`toXML`
built-in](../language/builtins.md).
- `--read-write-mode`
When used with `--eval`, perform evaluation in read/write mode so
nix language features that require it will still work (at the cost
of needing to do instantiation of every evaluated derivation). If
this option is not enabled, there may be uninstantiated store paths
in the final output.
{{#include ./opt-common.md}}
@@ -133,28 +179,24 @@ $ nix-instantiate --eval --xml --expr '1 + 2'
The difference between non-strict and strict evaluation:
```console
$ nix-instantiate --eval --xml --expr 'rec { x = "foo"; y = x; }'
...
<attr name="x">
<string value="foo" />
</attr>
<attr name="y">
<unevaluated />
</attr>
...
```
$ nix-instantiate --eval --xml --expr '{ x = {}; }'
<?xml version='1.0' encoding='utf-8'?>
<expr>
<attrs>
<attr column="3" line="1" name="x">
<unevaluated />
</attr>
</attrs>
</expr>
Note that `y` is left unevaluated (the XML representation doesn’t
attempt to show non-normal forms).
```console
$ nix-instantiate --eval --xml --strict --expr 'rec { x = "foo"; y = x; }'
...
<attr name="x">
<string value="foo" />
</attr>
<attr name="y">
<string value="foo" />
</attr>
...
$ nix-instantiate --eval --xml --strict --expr '{ x = {}; }'
<?xml version='1.0' encoding='utf-8'?>
<expr>
<attrs>
<attr column="3" line="1" name="x">
<attrs>
</attrs>
</attr>
</attrs>
</expr>
```
+22 -17
View File
@@ -39,27 +39,32 @@ the path of the downloaded file in the Nix store is also printed.
# Options
- `--type` *hashAlgo*\
Use the specified cryptographic hash algorithm,
which can be one of `md5`, `sha1`, `sha256`, and `sha512`.
The default is `sha256`.
- `--type` *hashAlgo*
- `--print-path`\
Print the store path of the downloaded file on standard output.
Use the specified cryptographic hash algorithm,
which can be one of `md5`, `sha1`, `sha256`, and `sha512`.
The default is `sha256`.
- `--unpack`\
Unpack the archive (which must be a tarball or zip file) and add the
result to the Nix store. The resulting hash can be used with
functions such as Nixpkgs’s `fetchzip` or `fetchFromGitHub`.
- `--print-path`
- `--executable`\
Set the executable bit on the downloaded file.
Print the store path of the downloaded file on standard output.
- `--name` *name*\
Override the name of the file in the Nix store. By default, this is
`hash-basename`, where *basename* is the last component of *url*.
Overriding the name is necessary when *basename* contains characters
that are not allowed in Nix store paths.
- `--unpack`
Unpack the archive (which must be a tarball or zip file) and add the
result to the Nix store. The resulting hash can be used with
functions such as Nixpkgs’s `fetchzip` or `fetchFromGitHub`.
- `--executable`
Set the executable bit on the downloaded file.
- `--name` *name*
Override the name of the file in the Nix store. By default, this is
`hash-basename`, where *basename* is the last component of *url*.
Overriding the name is necessary when *basename* contains characters
that are not allowed in Nix store paths.
# Examples
+53 -40
View File
@@ -60,55 +60,63 @@ All options not listed here are passed to `nix-store
--realise`, except for `--arg` and `--attr` / `-A` which are passed to
`nix-instantiate`.
- `--command` *cmd*\
In the environment of the derivation, run the shell command *cmd*.
This command is executed in an interactive shell. (Use `--run` to
use a non-interactive shell instead.) However, a call to `exit` is
implicitly added to the command, so the shell will exit after
running the command. To prevent this, add `return` at the end;
e.g. `--command "echo Hello; return"` will print `Hello` and then
drop you into the interactive shell. This can be useful for doing
any additional initialisation.
- `--command` *cmd*
- `--run` *cmd*\
Like `--command`, but executes the command in a non-interactive
shell. This means (among other things) that if you hit Ctrl-C while
the command is running, the shell exits.
In the environment of the derivation, run the shell command *cmd*.
This command is executed in an interactive shell. (Use `--run` to
use a non-interactive shell instead.) However, a call to `exit` is
implicitly added to the command, so the shell will exit after
running the command. To prevent this, add `return` at the end;
e.g. `--command "echo Hello; return"` will print `Hello` and then
drop you into the interactive shell. This can be useful for doing
any additional initialisation.
- `--exclude` *regexp*\
Do not build any dependencies whose store path matches the regular
expression *regexp*. This option may be specified multiple times.
- `--run` *cmd*
- `--pure`\
If this flag is specified, the environment is almost entirely
cleared before the interactive shell is started, so you get an
environment that more closely corresponds to the “real” Nix build. A
few variables, in particular `HOME`, `USER` and `DISPLAY`, are
retained.
Like `--command`, but executes the command in a non-interactive
shell. This means (among other things) that if you hit Ctrl-C while
the command is running, the shell exits.
- `--packages` / `-p` *packages*…\
Set up an environment in which the specified packages are present.
The command line arguments are interpreted as attribute names inside
the Nix Packages collection. Thus, `nix-shell --packages libjpeg openjdk`
will start a shell in which the packages denoted by the attribute
names `libjpeg` and `openjdk` are present.
- `--exclude` *regexp*
- `-i` *interpreter*\
The chained script interpreter to be invoked by `nix-shell`. Only
applicable in `#!`-scripts (described below).
Do not build any dependencies whose store path matches the regular
expression *regexp*. This option may be specified multiple times.
- `--keep` *name*\
When a `--pure` shell is started, keep the listed environment
variables.
- `--pure`
If this flag is specified, the environment is almost entirely
cleared before the interactive shell is started, so you get an
environment that more closely corresponds to the “real” Nix build. A
few variables, in particular `HOME`, `USER` and `DISPLAY`, are
retained.
- `--packages` / `-p` *packages*…
Set up an environment in which the specified packages are present.
The command line arguments are interpreted as attribute names inside
the Nix Packages collection. Thus, `nix-shell --packages libjpeg openjdk`
will start a shell in which the packages denoted by the attribute
names `libjpeg` and `openjdk` are present.
- `-i` *interpreter*
The chained script interpreter to be invoked by `nix-shell`. Only
applicable in `#!`-scripts (described below).
- `--keep` *name*
When a `--pure` shell is started, keep the listed environment
variables.
{{#include ./opt-common.md}}
# Environment variables
- `NIX_BUILD_SHELL`\
Shell used to start the interactive environment. Defaults to the
`bash` found in `<nixpkgs>`, falling back to the `bash` found in
`PATH` if not found.
- `NIX_BUILD_SHELL`
Shell used to start the interactive environment. Defaults to the
`bash` found in `<nixpkgs>`, falling back to the `bash` found in
`PATH` if not found.
{{#include ./env-common.md}}
@@ -202,14 +210,14 @@ For example, here is a Python script that depends on Python and the
```python
#! /usr/bin/env nix-shell
#! nix-shell -i python --packages python pythonPackages.prettytable
#! nix-shell -i python3 --packages python3 python3Packages.prettytable
import prettytable
# Print a simple table.
t = prettytable.PrettyTable(["N", "N^2"])
for n in range(1, 10): t.add_row([n, n * n])
print t
print(t)
```
Similarly, the following is a Perl script that specifies that it
@@ -289,3 +297,8 @@ with import <nixpkgs> {};
runCommand "dummy" { buildInputs = [ python pythonPackages.prettytable ]; } ""
```
The script's file name is passed as the first argument to the interpreter specified by the `-i` flag.
Aside from the very first line, which is a directive to the operating system, the additional `#! nix-shell` lines do not need to be at the beginning of the file.
This allows wrapping them in block comments for languages where `#` does not start a comment, such as ECMAScript, Erlang, PHP, or Ruby.
@@ -16,9 +16,13 @@ public url or broke since the download expression was written.
This operation has the following options:
- `--recursive`\
Use recursive instead of flat hashing mode, used when adding
directories to the store.
- `--recursive`
Use recursive instead of flat hashing mode, used when adding
directories to the store.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
@@ -11,6 +11,9 @@
The operation `--add` adds the specified paths to the Nix store. It
prints the resulting paths in the Nix store on standard output.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
+5 -4
View File
@@ -1,6 +1,6 @@
# Name
`nix-store --dump` - write a single path to a Nix Archive
`nix-store --dump` - write a single path to a [Nix Archive]
## Synopsis
@@ -8,7 +8,7 @@
## Description
The operation `--dump` produces a NAR (Nix ARchive) file containing the
The operation `--dump` produces a [Nix archive](@docroot@/glossary.md#gloss-nar) (NAR) file containing the
contents of the file system tree rooted at *path*. The archive is
written to standard output.
@@ -30,8 +30,9 @@ NAR archives support filenames of unlimited length and 64-bit file
sizes. They can contain regular files, directories, and symbolic links,
but not other types of files (such as device nodes).
A Nix archive can be unpacked using `nix-store
--restore`.
A Nix archive can be unpacked using [`nix-store --restore`](@docroot@/command-ref/nix-store/restore.md).
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
{{#include ./opt-common.md}}
+34 -22
View File
@@ -1,6 +1,6 @@
# Name
`nix-store --export` - export store paths to a Nix Archive
`nix-store --export` - export store paths to a [Nix Archive]
## Synopsis
@@ -8,16 +8,22 @@
## Description
The operation `--export` writes a serialisation of the specified store
paths to standard output in a format that can be imported into another
Nix store with `nix-store --import`. This is like `nix-store
--dump`, except that the NAR archive produced by that command doesn’t
contain the necessary meta-information to allow it to be imported into
another Nix store (namely, the set of references of the path).
The operation `--export` writes a serialisation of the given [store objects](@docroot@/glossary.md#gloss-store-object) to standard output in a format that can be imported into another [Nix store](@docroot@/store/index.md) with [`nix-store --import`](./import.md).
This command does not produce a *closure* of the specified paths, so if
a store path references other store paths that are missing in the target
Nix store, the import will fail.
> **Warning**
>
> This command *does not* produce a [closure](@docroot@/glossary.md#gloss-closure) of the specified store paths.
> Trying to import a store object that refers to store paths not available in the target Nix store will fail.
>
> Use [`nix-store --query`](@docroot@/command-ref/nix-store/query.md) to obtain the closure of a store path.
This command is different from [`nix-store --dump`](./dump.md), which produces a [Nix archive](@docroot@/glossary.md#gloss-nar) that *does not* contain the set of [references](@docroot@/glossary.md#gloss-reference) of a given store path.
> **Note**
>
> For efficient transfer of closures to remote machines over SSH, use [`nix-copy-closure`](@docroot@/command-ref/nix-copy-closure.md).
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
{{#include ./opt-common.md}}
@@ -27,15 +33,21 @@ Nix store, the import will fail.
# Examples
To copy a whole closure, do something
like:
```console
$ nix-store --export $(nix-store --query --requisites paths) > out
```
To import the whole closure again, run:
```console
$ nix-store --import < out
```
> **Example**
>
> Deploy GNU Hello to an airgapped machine via USB stick.
>
> Write the closure to the block device on a machine with internet connection:
>
> ```shell-session
> [alice@itchy]$ storePath=$(nix-build '<nixpkgs>' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link)
> [alice@itchy]$ nix-store --export $(nix-store --query --requisites $storePath) | sudo dd of=/dev/usb
> ```
>
> Read the closure from the block device on the machine without internet connection:
>
> ```shell-session
> [bob@scratchy]$ hello=$(sudo dd if=/dev/usb | nix-store --import | tail -1)
> [bob@scratchy]$ $hello/bin/hello
> Hello, world!
> ```
+22 -18
View File
@@ -14,30 +14,34 @@ reachable via file system references from a set of “roots”, are deleted.
The following suboperations may be specified:
- `--print-roots`\
This operation prints on standard output the set of roots used by
the garbage collector.
- `--print-roots`
- `--print-live`\
This operation prints on standard output the set of “live” store
paths, which are all the store paths reachable from the roots. Live
paths should never be deleted, since that would break consistency —
it would become possible that applications are installed that
reference things that are no longer present in the store.
This operation prints on standard output the set of roots used by
the garbage collector.
- `--print-dead`\
This operation prints out on standard output the set of “dead” store
paths, which is just the opposite of the set of live paths: any path
in the store that is not live (with respect to the roots) is dead.
- `--print-live`
This operation prints on standard output the set of “live” store
paths, which are all the store paths reachable from the roots. Live
paths should never be deleted, since that would break consistency —
it would become possible that applications are installed that
reference things that are no longer present in the store.
- `--print-dead`
This operation prints out on standard output the set of “dead” store
paths, which is just the opposite of the set of live paths: any path
in the store that is not live (with respect to the roots) is dead.
By default, all unreachable paths are deleted. The following options
control what gets deleted and in what order:
- `--max-freed` *bytes*\
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
- `--max-freed` *bytes*
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
The behaviour of the collector is also influenced by the
`keep-outputs` and `keep-derivations` settings in the Nix
+28 -6
View File
@@ -1,6 +1,8 @@
# Name
`nix-store --import` - import Nix Archive into the store
`nix-store --import` - import [Nix Archive] into the store
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
# Synopsis
@@ -8,14 +10,34 @@
# Description
The operation `--import` reads a serialisation of a set of store paths
produced by `nix-store --export` from standard input and adds those
store paths to the Nix store. Paths that already exist in the Nix store
are ignored. If a path refers to another path that doesn’t exist in the
Nix store, the import fails.
The operation `--import` reads a serialisation of a set of [store objects](@docroot@/glossary.md#gloss-store-object) produced by [`nix-store --export`](./export.md) from standard input, and adds those store objects to the specified [Nix store](@docroot@/store/index.md).
Paths that already exist in the target Nix store are ignored.
If a path [refers](@docroot@/glossary.md#gloss-reference) to another path that doesn’t exist in the target Nix store, the import fails.
> **Note**
>
> For efficient transfer of closures to remote machines over SSH, use [`nix-copy-closure`](@docroot@/command-ref/nix-copy-closure.md).
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
{{#include ../env-common.md}}
# Examples
> **Example**
>
> Given a closure of GNU Hello as a file:
>
> ```shell-session
> $ storePath="$(nix-build '<nixpkgs>' -I nixpkgs=channel:nixpkgs-unstable -A hello --no-out-link)"
> $ nix-store --export $(nix-store --query --requisites $storePath) > hello.closure
> ```
>
> Import the closure into a [remote SSH store](@docroot@/store/types/ssh-store.md) using the [`--store`](@docroot@/command-ref/conf-file.md#conf-store) option:
>
> ```console
> $ nix-store --import --store ssh://alice@itchy.example.org < hello.closure
> ```
@@ -12,7 +12,7 @@ The operation `--optimise` reduces Nix store disk space usage by finding
identical files in the store and hard-linking them to each other. It
typically reduces the size of the store by something like 25-35%. Only
regular files and symlinks are hard-linked in this manner. Files are
considered identical when they have the same NAR archive serialisation:
considered identical when they have the same [Nix Archive (NAR)][Nix Archive] serialisation:
that is, regular files must have the same contents and permission
(executable or non-executable), and symlinks must have the same
contents.
@@ -38,3 +38,4 @@ hashing files in `/nix/store/qhqx7l2f1kmwihc9bnxs7rc159hsxnf3-gcc-4.1.1'
there are 114486 files with equal contents out of 215894 files in total
```
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
+106 -91
View File
@@ -24,122 +24,138 @@ symlink.
# Common query options
- `--use-output`; `-u`\
For each argument to the query that is a [store derivation], apply the
query to the output path of the derivation instead.
- `--use-output` / `-u`
- `--force-realise`; `-f`\
Realise each argument to the query first (see [`nix-store --realise`](./realise.md)).
For each argument to the query that is a [store derivation], apply the
query to the output path of the derivation instead.
- `--force-realise` / `-f`
Realise each argument to the query first (see [`nix-store --realise`](./realise.md)).
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
# Queries
- `--outputs`\
Prints out the [output paths] of the store
derivations *paths*. These are the paths that will be produced when
the derivation is built.
- `--outputs`
[output paths]: ../../glossary.md#gloss-output-path
Prints out the [output paths] of the store
derivations *paths*. These are the paths that will be produced when
the derivation is built.
- `--requisites`; `-R`\
Prints out the [closure] of the store path *paths*.
[output paths]: @docroot@/glossary.md#gloss-output-path
[closure]: ../../glossary.md#gloss-closure
- `--requisites` / `-R`
This query has one option:
Prints out the [closure] of the store path *paths*.
- `--include-outputs`
Also include the existing output paths of [store derivation]s,
and their closures.
[closure]: @docroot@/glossary.md#gloss-closure
This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
store derivation. A *binary deployment* is obtained by distributing
the closure of an output path. A *cache deployment* (combined
source/binary deployment, including binaries of build-time-only
dependencies) is obtained by distributing the closure of a store
derivation and specifying the option `--include-outputs`.
This query has one option:
- `--references`\
Prints the set of [references] of the store paths
*paths*, that is, their immediate dependencies. (For *all*
dependencies, use `--requisites`.)
- `--include-outputs`
Also include the existing output paths of [store derivation]s,
and their closures.
[references]: ../../glossary.md#gloss-reference
This query can be used to implement various kinds of deployment. A
*source deployment* is obtained by distributing the closure of a
store derivation. A *binary deployment* is obtained by distributing
the closure of an output path. A *cache deployment* (combined
source/binary deployment, including binaries of build-time-only
dependencies) is obtained by distributing the closure of a store
derivation and specifying the option `--include-outputs`.
- `--referrers`\
Prints the set of *referrers* of the store paths *paths*, that is,
the store paths currently existing in the Nix store that refer to
one of *paths*. Note that contrary to the references, the set of
referrers is not constant; it can change as store paths are added or
removed.
- `--references`
- `--referrers-closure`\
Prints the closure of the set of store paths *paths* under the
referrers relation; that is, all store paths that directly or
indirectly refer to one of *paths*. These are all the path currently
in the Nix store that are dependent on *paths*.
Prints the set of [references] of the store paths
*paths*, that is, their immediate dependencies. (For *all*
dependencies, use `--requisites`.)
- `--deriver`; `-d`\
Prints the [deriver] that was used to build the store paths *paths*. If
the path has no deriver (e.g., if it is a source file), or if the
deriver is not known (e.g., in the case of a binary-only
deployment), the string `unknown-deriver` is printed.
The returned deriver is not guaranteed to exist in the local store, for
example when *paths* were substituted from a binary cache.
Use `--valid-derivers` instead to obtain valid paths only.
[references]: @docroot@/glossary.md#gloss-reference
[deriver]: ../../glossary.md#gloss-deriver
- `--referrers`
- `--valid-derivers`\
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths subsituted from a binary cache.
Prints the set of *referrers* of the store paths *paths*, that is,
the store paths currently existing in the Nix store that refer to
one of *paths*. Note that contrary to the references, the set of
referrers is not constant; it can change as store paths are added or
removed.
- `--graph`\
Prints the references graph of the store paths *paths* in the format
of the `dot` tool of AT\&T's [Graphviz
package](http://www.graphviz.org/). This can be used to visualise
dependency graphs. To obtain a build-time dependency graph, apply
this to a store derivation. To obtain a runtime dependency graph,
apply it to an output path.
- `--referrers-closure`
- `--tree`\
Prints the references graph of the store paths *paths* as a nested
ASCII tree. References are ordered by descending closure size; this
tends to flatten the tree, making it more readable. The query only
recurses into a store path when it is first encountered; this
prevents a blowup of the tree representation of the graph.
Prints the closure of the set of store paths *paths* under the
referrers relation; that is, all store paths that directly or
indirectly refer to one of *paths*. These are all the path currently
in the Nix store that are dependent on *paths*.
- `--graphml`\
Prints the references graph of the store paths *paths* in the
[GraphML](http://graphml.graphdrawing.org/) file format. This can be
used to visualise dependency graphs. To obtain a build-time
dependency graph, apply this to a [store derivation]. To obtain a
runtime dependency graph, apply it to an output path.
- `--deriver` / `-d`
- `--binding` *name*; `-b` *name*\
Prints the value of the attribute *name* (i.e., environment
variable) of the [store derivation]s *paths*. It is an error for a
derivation to not have the specified attribute.
Prints the [deriver] that was used to build the store paths *paths*. If
the path has no deriver (e.g., if it is a source file), or if the
deriver is not known (e.g., in the case of a binary-only
deployment), the string `unknown-deriver` is printed.
The returned deriver is not guaranteed to exist in the local store, for
example when *paths* were substituted from a binary cache.
Use `--valid-derivers` instead to obtain valid paths only.
- `--hash`\
Prints the SHA-256 hash of the contents of the store paths *paths*
(that is, the hash of the output of `nix-store --dump` on the given
paths). Since the hash is stored in the Nix database, this is a fast
operation.
[deriver]: @docroot@/glossary.md#gloss-deriver
- `--size`\
Prints the size in bytes of the contents of the store paths *paths*
— to be precise, the size of the output of `nix-store --dump` on
the given paths. Note that the actual disk space required by the
store paths may be higher, especially on filesystems with large
cluster sizes.
- `--valid-derivers`
- `--roots`\
Prints the garbage collector roots that point, directly or
indirectly, at the store paths *paths*.
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths substituted from a binary cache.
- `--graph`
Prints the references graph of the store paths *paths* in the format
of the `dot` tool of AT\&T's [Graphviz
package](http://www.graphviz.org/). This can be used to visualise
dependency graphs. To obtain a build-time dependency graph, apply
this to a store derivation. To obtain a runtime dependency graph,
apply it to an output path.
- `--tree`
Prints the references graph of the store paths *paths* as a nested
ASCII tree. References are ordered by descending closure size; this
tends to flatten the tree, making it more readable. The query only
recurses into a store path when it is first encountered; this
prevents a blowup of the tree representation of the graph.
- `--graphml`
Prints the references graph of the store paths *paths* in the
[GraphML](http://graphml.graphdrawing.org/) file format. This can be
used to visualise dependency graphs. To obtain a build-time
dependency graph, apply this to a [store derivation]. To obtain a
runtime dependency graph, apply it to an output path.
- `--binding` *name* / `-b` *name*
Prints the value of the attribute *name* (i.e., environment
variable) of the [store derivation]s *paths*. It is an error for a
derivation to not have the specified attribute.
- `--hash`
Prints the SHA-256 hash of the contents of the store paths *paths*
(that is, the hash of the output of `nix-store --dump` on the given
paths). Since the hash is stored in the Nix database, this is a fast
operation.
- `--size`
Prints the size in bytes of the contents of the store paths *paths*
— to be precise, the size of the output of `nix-store --dump` on
the given paths. Note that the actual disk space required by the
store paths may be higher, especially on filesystems with large
cluster sizes.
- `--roots`
Prints the garbage collector roots that point, directly or
indirectly, at the store paths *paths*.
{{#include ./opt-common.md}}
@@ -225,4 +241,3 @@ $ nix-store --query --roots $(which svn)
/nix/var/nix/profiles/default-82-link
/home/eelco/.local/state/nix/profiles/profile-97-link
```
+21 -18
View File
@@ -25,14 +25,14 @@ Each of *paths* is processed as follows:
If no substitutes are available and no store derivation is given, realisation fails.
[store paths]: @docroot@/glossary.md#gloss-store-path
[store paths]: @docroot@/store/store-path.md
[valid]: @docroot@/glossary.md#gloss-validity
[store derivation]: @docroot@/glossary.md#gloss-store-derivation
[output paths]: @docroot@/glossary.md#gloss-output-path
[store objects]: @docroot@/glossary.md#gloss-store-object
[store objects]: @docroot@/store/store-object.md
[closure]: @docroot@/glossary.md#gloss-closure
[substituters]: @docroot@/command-ref/conf-file.md#conf-substituters
[content-addressed derivations]: @docroot@/contributing/experimental-features.md#xp-feature-ca-derivations
[content-addressed derivations]: @docroot@/development/experimental-features.md#xp-feature-ca-derivations
[Nix database]: @docroot@/glossary.md#gloss-nix-database
The resulting paths are printed on standard output.
@@ -42,23 +42,26 @@ For non-derivation arguments, the argument itself is printed.
# Options
- `--dry-run`\
Print on standard error a description of what packages would be
built or downloaded, without actually performing the operation.
- `--dry-run`
- `--ignore-unknown`\
If a non-derivation path does not have a substitute, then silently
ignore it.
Print on standard error a description of what packages would be
built or downloaded, without actually performing the operation.
- `--check`\
This option allows you to check whether a derivation is
deterministic. It rebuilds the specified derivation and checks
whether the result is bitwise-identical with the existing outputs,
printing an error if that’s not the case. The outputs of the
specified derivation must already exist. When used with `-K`, if an
output path is not identical to the corresponding output from the
previous build, the new output path is left in
`/nix/store/name.check.`
- `--ignore-unknown`
If a non-derivation path does not have a substitute, then silently
ignore it.
- `--check`
This option allows you to check whether a derivation is
deterministic. It rebuilds the specified derivation and checks
whether the result is bitwise-identical with the existing outputs,
printing an error if that’s not the case. The outputs of the
specified derivation must already exist. When used with `-K`, if an
output path is not identical to the corresponding output from the
previous build, the new output path is left in
`/nix/store/name.check.`
{{#include ./opt-common.md}}
@@ -8,9 +8,11 @@
## Description
The operation `--restore` unpacks a NAR archive to *path*, which must
The operation `--restore` unpacks a [Nix Archive (NAR)][Nix Archive] to *path*, which must
not already exist. The archive is read from standard input.
[Nix Archive]: @docroot@/store/file-system-object/content-address.md#serial-nix-archive
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -14,10 +14,11 @@ access to a restricted ssh user.
The following flags are available:
- `--write`\
Allow the connected client to request the realization of
derivations. In effect, this can be used to make the host act as a
remote builder.
- `--write`
Allow the connected client to request the realization of
derivations. In effect, this can be used to make the host act as a
remote builder.
{{#include ./opt-common.md}}
+13 -11
View File
@@ -16,18 +16,20 @@ being modified by non-Nix tools, or of bugs in Nix itself.
This operation has the following options:
- `--check-contents`\
Checks that the contents of every valid store path has not been
altered by computing a SHA-256 hash of the contents and comparing it
with the hash stored in the Nix database at build time. Paths that
have been modified are printed out. For large stores,
`--check-contents` is obviously quite slow.
- `--check-contents`
- `--repair`\
If any valid path is missing from the store, or (if
`--check-contents` is given) the contents of a valid path has been
modified, then try to repair the path by redownloading it. See
`nix-store --repair-path` for details.
Checks that the contents of every valid store path has not been
altered by computing a SHA-256 hash of the contents and comparing it
with the hash stored in the Nix database at build time. Paths that
have been modified are printed out. For large stores,
`--check-contents` is obviously quite slow.
- `--repair`
If any valid path is missing from the store, or (if
`--check-contents` is given) the contents of a valid path has been
modified, then try to repair the path by redownloading it. See
`nix-store --repair-path` for details.
{{#include ./opt-common.md}}
+6 -5
View File
@@ -37,7 +37,7 @@ Most Nix commands accept the following command-line options:
Print even more informational messages.
- `4` “Debug”
Print debug information.
- `5` “Vomit”
@@ -143,7 +143,7 @@ Most Nix commands accept the following command-line options:
This option is accepted by `nix-env`, `nix-instantiate`, `nix-shell` and `nix-build`.
When evaluating Nix expressions, the expression evaluator will automatically try to call functions that it encounters.
It can automatically call functions for which every argument has a [default value](@docroot@/language/constructs.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
It can automatically call functions for which every argument has a [default value](@docroot@/language/syntax.md#functions) (e.g., `{ argName ? defaultValue }: ...`).
With `--arg`, you can also call functions that have arguments without a default value (or override a default value).
That is, if the evaluator encounters a function with an argument named *name*, it will call it with value *value*.
@@ -187,11 +187,12 @@ Most Nix commands accept the following command-line options:
For `nix-shell`, this option is commonly used to give you a shell in which you can build the packages returned by the expression.
If you want to get a shell which contain the *built* packages ready for use, give your expression to the `nix-shell --packages ` convenience flag instead.
- <span id="opt-I">[`-I`](#opt-I)</span> *path*
- <span id="opt-I">[`-I` / `--include`](#opt-I)</span> *path*
Add an entry to the [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path).
Add an entry to the list of search paths used to resolve [lookup paths](@docroot@/language/constructs/lookup-path.md).
This option may be given multiple times.
Paths added through `-I` take precedence over [`NIX_PATH`](@docroot@/command-ref/env-common.md#env-NIX_PATH).
Paths added through `-I` take precedence over the [`nix-path` configuration setting](@docroot@/command-ref/conf-file.md#conf-nix-path) and the [`NIX_PATH` environment variable](@docroot@/command-ref/env-common.md#env-NIX_PATH).
- <span id="opt-option">[`--option`](#opt-option)</span> *name* *value*
@@ -1,70 +1,18 @@
# Hacking
# Building Nix
This section provides some notes on how to hack on Nix. To get the
latest version of Nix from GitHub:
This section provides some notes on how to start hacking on Nix.
To get the latest version of Nix from GitHub:
```console
$ git clone https://github.com/NixOS/nix.git
$ cd nix
```
The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment. If you don't have it installed, follow the [installation instructions].
[installation instructions]: ../installation/installation.md
## Building Nix with flakes
This section assumes you are using Nix with the [`flakes`] and [`nix-command`] experimental features enabled.
See the [Building Nix](#building-nix) section for equivalent instructions using stable Nix interfaces.
[`flakes`]: @docroot@/contributing/experimental-features.md#xp-feature-flakes
[`nix-command`]: @docroot@/contributing/experimental-features.md#xp-nix-command
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
```console
$ nix develop
```
This shell also adds `./outputs/bin/nix` to your `$PATH` so you can run `nix` immediately after building it.
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix develop .#native-clang11StdenvPackages
```
> **Note**
>
> Use `ccacheStdenv` to drastically improve rebuild time.
> By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`.
> The following instructions assume you already have some version of Nix installed locally, so that you can use it to set up the development environment.
> If you don't have it installed, follow the [installation instructions](../installation/index.md).
To build Nix itself in this shell:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make -j $NIX_BUILD_CORES
```
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ make install
[nix-shell]$ make installcheck -j $NIX_BUILD_CORES
[nix-shell]$ nix --version
nix (Nix) 2.12
```
To build a release version of Nix for the current operating system and CPU architecture:
```console
$ nix build
```
You can also build Nix for one of the [supported platforms](#platforms).
## Building Nix
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
@@ -75,7 +23,7 @@ $ nix-shell
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix-shell --attr devShells.x86_64-linux.native-clang11StdenvPackages
$ nix-shell --attr devShells.x86_64-linux.native-clangStdenvPackages
```
> **Note**
@@ -108,6 +56,79 @@ $ nix-build
You can also build Nix for one of the [supported platforms](#platforms).
## Building Nix with flakes
This section assumes you are using Nix with the [`flakes`] and [`nix-command`] experimental features enabled.
[`flakes`]: @docroot@/development/experimental-features.md#xp-feature-flakes
[`nix-command`]: @docroot@/development/experimental-features.md#xp-nix-command
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
```console
$ nix develop
```
This shell also adds `./outputs/bin/nix` to your `$PATH` so you can run `nix` immediately after building it.
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix develop .#native-clangStdenvPackages
```
> **Note**
>
> Use `ccacheStdenv` to drastically improve rebuild time.
> By default, [ccache](https://ccache.dev) keeps artifacts in `~/.cache/ccache/`.
To build Nix itself in this shell:
```console
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make -j $NIX_BUILD_CORES OPTIMIZE=0
```
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ make install OPTIMIZE=0
[nix-shell]$ make installcheck check -j $NIX_BUILD_CORES
[nix-shell]$ nix --version
nix (Nix) 2.12
```
For more information on running and filtering tests, see
[`testing.md`](./testing.md).
To build a release version of Nix for the current operating system and CPU architecture:
```console
$ nix build
```
You can also build Nix for one of the [supported platforms](#platforms).
## Makefile variables
You may need `profiledir=$out/etc/profile.d` and `sysconfdir=$out/etc` to run `make install`.
Run `make` with [`-e` / `--environment-overrides`](https://www.gnu.org/software/make/manual/make.html#index-_002de) to allow environment variables to override `Makefile` variables:
- `ENABLE_BUILD=yes` to enable building the C++ code.
- `ENABLE_DOC_GEN=yes` to enable building the documentation (manual, man pages, etc.).
The docs can take a while to build, so you may want to disable this for local development.
- `ENABLE_FUNCTIONAL_TESTS=yes` to enable building the functional tests.
- `OPTIMIZE=1` to enable optimizations.
- `libraries=libutil programs=` to only build a specific library.
This will fail in the linking phase if the other libraries haven't been built, but is useful for checking types.
- `libraries= programs=nix` to only build a specific program.
This will not work in general, because the programs need the libraries.
## Platforms
Nix can be built for various platforms, as specified in [`flake.nix`]:
@@ -121,13 +142,14 @@ Nix can be built for various platforms, as specified in [`flake.nix`]:
- `aarch64-darwin`
- `armv6l-linux`
- `armv7l-linux`
- `riscv64-linux`
In order to build Nix for a different platform than the one you're currently
on, you need a way for your current Nix installation to build code for that
platform. Common solutions include [remote builders] and [binary format emulation]
platform. Common solutions include [remote build machines] and [binary format emulation]
(only supported on NixOS).
[remote builders]: ../advanced-topics/distributed-builds.md
[remote builders]: @docroot@/language/derivations.md#attr-builder
[binary format emulation]: https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems
Given such a setup, executing the build only requires selecting the respective attribute.
@@ -143,12 +165,40 @@ or for Nix with the [`flakes`] and [`nix-command`] experimental features enabled
$ nix build .#packages.aarch64-linux.default
```
Cross-compiled builds are available for ARMv6 (`armv6l-linux`) and ARMv7 (`armv7l-linux`).
Cross-compiled builds are available for:
- `armv6l-linux`
- `armv7l-linux`
- `riscv64-linux`
Add more [system types](#system-type) to `crossSystems` in `flake.nix` to bootstrap Nix on unsupported platforms.
### Building for multiple platforms at once
It is useful to perform multiple cross and native builds on the same source tree,
for example to ensure that better support for one platform doesn't break the build for another.
In order to facilitate this, Nix has some support for being built out of tree – that is, placing build artefacts in a different directory than the source code:
1. Create a directory for the build, e.g.
```bash
mkdir build
```
2. Run the configure script from that directory, e.g.
```bash
cd build
../configure <configure flags>
```
3. Run make from the source directory, but with the build directory specified, e.g.
```bash
make builddir=build <make flags>
```
## System type
Nix uses a string with he following format to identify the *system type* or *platform* it runs on:
Nix uses a string with the following format to identify the *system type* or *platform* it runs on:
```
<cpu>-<os>[-<abi>]
@@ -210,13 +260,36 @@ See [supported compilation environments](#compilation-environments) and instruct
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make clean && bear -- make -j$NIX_BUILD_CORES default check install
make compile_commands.json
```
Configure your editor to use the `clangd` from the shell, either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
Configure your editor to use the `clangd` from the `.#native-clangStdenvPackages` shell. You can do that either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
> **Note**
>
> For some editors (e.g. Visual Studio Code), you may need to install a [special extension](https://open-vsx.org/extension/llvm-vs-code-extensions/vscode-clangd) for the editor to interact with `clangd`.
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
## Formatting and pre-commit hooks
You may run the formatters as a one-off using:
```console
make format
```
If you'd like to run the formatters before every commit, install the hooks:
```
pre-commit-hooks-install
```
This installs [pre-commit](https://pre-commit.com) using [cachix/git-hooks.nix](https://github.com/cachix/git-hooks.nix).
When making a commit, pay attention to the console output.
If it fails, run `git add --patch` to approve the suggestions _and commit again_.
To refresh pre-commit hook's config file, do the following:
1. Exit the development shell and start it again by running `nix develop`.
2. If you also use the pre-commit hook, also run `pre-commit-hooks-install` again.
@@ -87,7 +87,7 @@ impacted the most by bad user experience.
and [aligning of text](#text-alignment).
- [Autocomplete](#shell-completion) of options.
Examples of such commands: `nix doctor`, `nix edit`, `nix eval`, ...
Examples of such commands: `nix edit`, `nix eval`, ...
- **Utility and scripting commands**
@@ -389,88 +389,6 @@ colors, no emojis and using ASCII instead of Unicode symbols). The same should
happen when TTY is not detected on STDERR. We should not display progress /
status section, but only print warnings and errors.
## Returning future proof JSON
The schema of JSON output should allow for backwards compatible extension. This section explains how to achieve this.
Two definitions are helpful here, because while JSON only defines one "key-value"
object type, we use it to cover two use cases:
- **dictionary**: a map from names to value that all have the same type. In
C++ this would be a `std::map` with string keys.
- **record**: a fixed set of attributes each with their own type. In C++, this
would be represented by a `struct`.
It is best not to mix these use cases, as that may lead to incompatibilities when the schema changes. For example, adding a record field to a dictionary breaks consumers that assume all JSON object fields to have the same meaning and type.
This leads to the following guidelines:
- The top-level (root) value must be a record.
Otherwise, one can not change the structure of a command's output.
- The value of a dictionary item must be a record.
Otherwise, the item type can not be extended.
- List items should be records.
Otherwise, one can not change the structure of the list items.
If the order of the items does not matter, and each item has a unique key that is a string, consider representing the list as a dictionary instead. If the order of the items needs to be preserved, return a list of records.
- Streaming JSON should return records.
An example of a streaming JSON format is [JSON lines](https://jsonlines.org/), where each line represents a JSON value. These JSON values can be considered top-level values or list items, and they must be records.
### Examples
This is bad, because all keys must be assumed to be store implementations:
```json
{
"local": { ... },
"remote": { ... },
"http": { ... }
}
```
This is good, because the it is extensible at the root, and is somewhat self-documenting:
```json
{
"storeTypes": { "local": { ... }, ... },
"pluginSupport": true
}
```
While the dictionary of store types seems like a very complete response at first, a use case may arise that warrants returning additional information.
For example, the presence of plugin support may be crucial information for a client to proceed when their desired store type is missing.
The following representation is bad because it is not extensible:
```json
{ "outputs": [ "out" "bin" ] }
```
However, simply converting everything to records is not enough, because the order of outputs must be preserved:
```json
{ "outputs": { "bin": {}, "out": {} } }
```
The first item is the default output. Deriving this information from the outputs ordering is not great, but this is how Nix currently happens to work.
While it is possible for a JSON parser to preserve the order of fields, we can not rely on this capability to be present in all JSON libraries.
This representation is extensible and preserves the ordering:
```json
{ "outputs": [ { "outputName": "out" }, { "outputName": "bin" } ] }
```
## Dialog with the user
CLIs don't always make it clear when an action has taken place. For every
@@ -0,0 +1,79 @@
# Contributing
## Add a release note
`doc/manual/rl-next` contains release notes entries for all unreleased changes.
User-visible changes should come with a release note.
### Add an entry
Here's what a complete entry looks like. The file name is not incorporated in the document.
```
---
synopsis: Basically a title
issues: 1234
prs: 1238
---
Here's one or more paragraphs that describe the change.
- It's markdown
- Add references to the manual using @docroot@
```
Significant changes should add the following header, which moves them to the top.
```
significance: significant
```
<!-- Keep an eye on https://codeberg.org/fgaz/changelog-d/issues/1 -->
See also the [format documentation](https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#changelog).
### Build process
Releases have a precomputed `rl-MAJOR.MINOR.md`, and no `rl-next.md`.
## Branches
- [`master`](https://github.com/NixOS/nix/commits/master)
The main development branch. All changes are approved and merged here.
When developing a change, create a branch based on the latest `master`.
Maintainers try to [keep it in a release-worthy state](#reverting).
- [`maintenance-*.*`](https://github.com/NixOS/nix/branches/all?query=maintenance)
These branches are the subject of backports only, and are
also [kept](#reverting) in a release-worthy state.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [`latest-release`](https://github.com/NixOS/nix/tree/latest-release)
The latest patch release of the latest minor version.
See [`maintainers/release-process.md`](https://github.com/NixOS/nix/blob/master/maintainers/release-process.md)
- [`backport-*-to-*`](https://github.com/NixOS/nix/branches/all?query=backport)
Generally branches created by the backport action.
See [`maintainers/backporting.md`](https://github.com/NixOS/nix/blob/master/maintainers/backporting.md)
- [_other_](https://github.com/NixOS/nix/branches/all)
Branches that do not conform to the above patterns should be feature branches.
## Reverting
If a change turns out to be merged by mistake, or contain a regression, it may be reverted.
A revert is not a rejection of the contribution, but merely part of an effective development process.
It makes sure that development keeps running smoothly, with minimal uncertainty, and less overhead.
If maintainers have to worry too much about avoiding reverts, they would not be able to merge as much.
By embracing reverts as a good part of the development process, everyone wins.
However, taking a step back may be frustrating, so maintainers will be extra supportive on the next try.
File renamed without changes.
@@ -24,14 +24,12 @@ nix build .#^doc
and open `./result-doc/share/doc/nix/manual/index.html`.
To build the manual incrementally, [enter the development shell](./hacking.md) and run:
To build the manual incrementally, [enter the development shell](./building.md) and run:
```console
make manual-html -j $NIX_BUILD_CORES
make manual-html-open -j $NIX_BUILD_CORES
```
and open `./outputs/out/share/doc/nix/manual/language/index.html`.
In order to reflect changes to the [Makefile for the manual], clear all generated files before re-building:
[Makefile for the manual]: https://github.com/NixOS/nix/blob/master/doc/manual/local.mk
@@ -149,7 +147,7 @@ Please observe these guidelines to ease reviews:
```
A [store object] contains a [file system object] and [references] to other store objects.
[store object]: @docroot@/glossary.md#gloss-store-object
[store object]: @docroot@/store/store-object.md
[file system object]: @docroot@/architecture/file-system-object.md
[references]: @docroot@/glossary.md#gloss-reference
```
@@ -172,7 +170,7 @@ Please observe these guidelines to ease reviews:
> ```
````
Highlight syntax definiions as such, using [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) notation:
Highlight syntax definitions as such, using [EBNF](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) notation:
````
> **Syntax**
@@ -198,13 +196,35 @@ You can also build and view it yourself:
[Doxygen API documentation]: https://hydra.nixos.org/job/nix/master/internal-api-docs/latest/download-by-type/doc/internal-api-docs
```console
# nix build .#hydraJobs.internal-api-docs
# xdg-open ./result/share/doc/nix/internal-api/html/index.html
$ nix build .#hydraJobs.internal-api-docs
$ xdg-open ./result/share/doc/nix/internal-api/html/index.html
```
or inside `nix-shell` or `nix develop`:
```console
$ mesonConfigurePhase
$ ninja src/internal-api-docs/html
$ xdg-open src/internal-api-docs/html/index.html
```
## C API documentation
Note that the C API is not yet stable.
[C API documentation] is available online.
You can also build and view it yourself:
[C API documentation]: https://hydra.nixos.org/job/nix/master/external-api-docs/latest/download-by-type/doc/external-api-docs
```console
$ nix build .#hydraJobs.external-api-docs
$ xdg-open ./result/share/doc/nix/external-api/html/index.html
```
or inside `nix-shell` or `nix develop`:
```
# make internal-api-html
# xdg-open ./outputs/doc/share/doc/nix/internal-api/html/index.html
$ mesonConfigurePhase
$ ninja src/external-api-docs/html
$ xdg-open src/external-api-docs/html/index.html
```
@@ -5,4 +5,4 @@ Check the [contributing guide](https://github.com/NixOS/nix/blob/master/CONTRIBU
This chapter is a collection of guides for making changes to the code and documentation.
If you're not sure where to start, try to [compile Nix from source](./hacking.md) and consider [making improvements to documentation](./documentation.md).
If you're not sure where to start, try to [compile Nix from source](./building.md) and consider [making improvements to documentation](./documentation.md).
@@ -0,0 +1,128 @@
# JSON guideline
Nix consumes and produces JSON in a variety of contexts.
These guidelines ensure consistent practices for all our JSON interfaces, for ease of use, and so that experience in one part carries over to another.
## Extensibility
The schema of JSON input and output should allow for backwards compatible extension.
This section explains how to achieve this.
Two definitions are helpful here, because while JSON only defines one "key-value" object type, we use it to cover two use cases:
- **dictionary**: a map from names to value that all have the same type.
In C++ this would be a `std::map` with string keys.
- **record**: a fixed set of attributes each with their own type.
In C++, this would be represented by a `struct`.
It is best not to mix these use cases, as that may lead to incompatibilities when the schema changes.
For example, adding a record field to a dictionary breaks consumers that assume all JSON object fields to have the same meaning and type, and dictionary items with a colliding name can not be represented anymore.
This leads to the following guidelines:
- The top-level (root) value must be a record.
Otherwise, one can not change the structure of a command's output.
- The value of a dictionary item must be a record.
Otherwise, the item type can not be extended.
- List items should be records.
Otherwise, one can not change the structure of the list items.
If the order of the items does not matter, and each item has a unique key that is a string, consider representing the list as a dictionary instead.
If the order of the items needs to be preserved, return a list of records.
- Streaming JSON should return records.
An example of a streaming JSON format is [JSON lines](https://jsonlines.org/), where each line represents a JSON value.
These JSON values can be considered top-level values or list items, and they must be records.
### Examples
This is bad, because all keys must be assumed to be store types:
```json
{
"local": { ... },
"remote": { ... },
"http": { ... }
}
```
This is good, because the it is extensible at the root, and is somewhat self-documenting:
```json
{
"storeTypes": { "local": { ... }, ... },
"pluginSupport": true
}
```
While the dictionary of store types seems like a very complete response at first, a use case may arise that warrants returning additional information.
For example, the presence of plugin support may be crucial information for a client to proceed when their desired store type is missing.
The following representation is bad because it is not extensible:
```json
{ "outputs": [ "out" "bin" ] }
```
However, simply converting everything to records is not enough, because the order of outputs must be preserved:
```json
{ "outputs": { "bin": {}, "out": {} } }
```
The first item is the default output. Deriving this information from the outputs ordering is not great, but this is how Nix currently happens to work.
While it is possible for a JSON parser to preserve the order of fields, we can not rely on this capability to be present in all JSON libraries.
This representation is extensible and preserves the ordering:
```json
{ "outputs": [ { "outputName": "out" }, { "outputName": "bin" } ] }
```
## Self-describing values
As described in the previous section, it's crucial that schemas can be extended with new fields without breaking compatibility.
However, that should *not* mean we use the presence/absence of fields to indicate optional information *within* a version of the schema.
Instead, always include the field, and use `null` to indicate the "nothing" case.
### Examples
Here are two JSON objects:
```json
{
"foo": {}
}
```
```json
{
"foo": {},
"bar": {}
}
```
Since they differ in which fields they contain, they should *not* both be valid values of the same schema.
At most, they can match two different schemas where the second (with `foo` and `bar`) is considered a newer version of the first (with just `foo`).
Within each version, all fields are mandatory (always `foo`, and always `foo` and `bar`).
Only *between* each version, `bar` gets added as a new mandatory field.
Here are another two JSON objects:
```json
{ "foo": null }
```
```json
{ "foo": { "bar": 1 } }
```
Since they both contain a `foo` field, they could be valid values of the same schema.
The schema would have `foo` has an optional field, which is either `null` or an object where `bar` is an integer.
@@ -20,6 +20,7 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
[googletest]: https://google.github.io/googletest/
[rapidcheck]: https://github.com/emil-e/rapidcheck
[property testing]: https://en.wikipedia.org/wiki/Property_testing
### Source and header layout
@@ -28,39 +29,72 @@ The unit tests are defined using the [googletest] and [rapidcheck] frameworks.
> ```
> src
> ├── libexpr
> │ ├── local.mk
> │ ├── value/context.hh
> │ ├── value/context.cc
> │ …
> │
> ├── tests
> │ │
> │ …
> └── tests
> │ ├── value/context.hh
> │ ├── value/context.cc
> │ └── unit
> │ ├── libutil
> │ │ ├── local.mk
> │ │ …
> │ │ └── data
> │ │ ├── git/tree.txt
> │ │ …
> │ │
> │ …
> │
> ├── unit-test-data
> │ ├── libstore
> │ │ ├── worker-protocol/content-address.bin
> │ │ …
> │ …
> │ ├── libexpr-support
> │ │ ├── local.mk
> │ │ └── tests
> │ │ ├── value/context.hh
> │ │ ├── value/context.cc
> │ │ …
> │ │
> │ ├── libexpr
> │ … ├── local.mk
> │ ├── value/context.cc
> │ …
> …
> ```
The unit tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_shortname}/tests` within the directory for the library (`src/${library_shortname}`).
The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/value/context.{hh,cc}`, we write tests for it in `src/nix-expr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/nix-expr-test-support/tests/value/context.{hh,cc}`.
The data is in `unit-test-data`, with one subdir per library, with the same name as where the code goes.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `unit-test-data/libstore`.
The path to the `unit-test-data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
Data for unit tests is stored in a `data` subdir of the directory for each unit test executable.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/nix-store-tests/data`.
The path to the `src/${library_name_without-nix}-test/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
Note that each executable only gets the data for its tests.
> **Note**
> Due to the way googletest works, downstream unit test executables will actually include and re-run upstream library tests.
> Therefore it is important that the same value for `_NIX_TEST_UNIT_DATA` be used with the tests for each library.
> That is why we have the test data nested within a single `unit-test-data` directory.
The unit test libraries are in `src/${library_name_without-nix}-test-support`.
All headers are in a `tests` subdirectory so they are included with `#include "tests/"`.
The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing.
But organizing the tests this way has one big benefit:
there is no risk of any build-system wildcards for the library accidentally picking up test code that should not built and installed as part of the library.
### Running tests
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`.
Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option, or the `GTEST_FILTER` environment variable.
You can run the whole testsuite with `meson test` from the Meson build directory, or the tests for a specific component with `meson test nix-store-tests`.
A environment variables that Google Test accepts are also worth knowing:
1. [`GTEST_FILTER`](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests)
This is used for finer-grained filtering of which tests to run.
2. [`GTEST_BRIEF`](https://google.github.io/googletest/advanced.html#suppressing-test-passes)
This is used to avoid logging passing tests.
Putting the two together, one might run
```bash
GTEST_BRIEF=1 GTEST_FILTER='ErrorTraceTest.*' meson test nix-expr-tests -v
```
for short but comprensive output.
### Characterisation testing { #characaterisation-testing-unit }
@@ -69,7 +103,7 @@ See [functional characterisation testing](#characterisation-testing-functional)
Like with the functional characterisation, `_NIX_TEST_ACCEPT=1` is also used.
For example:
```shell-session
$ _NIX_TEST_ACCEPT=1 make libstore-tests-exe_RUN
$ _NIX_TEST_ACCEPT=1 meson test nix-store-tests -v
...
[ SKIPPED ] WorkerProtoTest.string_read
[ SKIPPED ] WorkerProtoTest.string_write
@@ -80,11 +114,25 @@ $ _NIX_TEST_ACCEPT=1 make libstore-tests-exe_RUN
will regenerate the "golden master" expected result for the `libnixstore` characterisation tests.
The characterisation tests will mark themselves "skipped" since they regenerated the expected result instead of actually testing anything.
### Unit test support libraries
There are headers and code which are not just used to test the library in question, but also downstream libraries.
For example, we do [property testing] with the [rapidcheck] library.
This requires writing `Arbitrary` "instances", which are used to describe how to generate values of a given type for the sake of running property tests.
Because types contain other types, `Arbitrary` "instances" for some type are not just useful for testing that type, but also any other type that contains it.
Downstream types frequently contain upstream types, so it is very important that we share arbitrary instances so that downstream libraries' property tests can also use them.
It is important that these testing libraries don't contain any actual tests themselves.
On some platforms they would be run as part of every test executable that uses them, which is redundant.
On other platforms they wouldn't be run at all.
## Functional tests
The functional tests reside under the `tests/functional` directory and are listed in `tests/functional/local.mk`.
Each test is a bash script.
Functional tests are run during `installCheck` in the `nix` package build, as well as separately from the build, in VM tests.
### Running the whole test suite
The whole test suite can be run with:
@@ -133,14 +181,14 @@ ran test tests/functional/${testName}.sh... [PASS]
or without `make`:
```shell-session
$ ./mk/run-test.sh tests/functional/${testName}.sh tests/functional/init.sh
$ ./mk/run-test.sh tests/functional/${testName}.sh
ran test tests/functional/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/functional/${testName}.sh tests/functional/init.sh
$ ./mk/debug-test.sh tests/functional/${testName}.sh
+(${testName}.sh:1) foo
output from foo
+(${testName}.sh:2) bar
@@ -175,7 +223,7 @@ edit it like so:
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
```shell-session
$ ./mk/debug-test.sh tests/functional/${testName}.sh tests/functional/init.sh
$ ./mk/debug-test.sh tests/functional/${testName}.sh
...
+ gdb blash blub
GNU gdb (GDB) 12.1
@@ -223,13 +271,30 @@ Regressions are caught, and improvements always show up in code review.
To ensure that characterisation testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
### Running functional tests on NixOS
We run the functional tests not just in the build, but also in VM tests.
This helps us ensure that Nix works correctly on NixOS, and environments that have similar characteristics that are hard to reproduce in a build environment.
The recommended way to run these tests during development is:
```shell
nix build .#hydraJobs.tests.functional_user.quickBuild
```
The `quickBuild` attribute configures the test to use a `nix` package that's built without integration tests, so that you can iterate on the tests without performing recompilations due to the changed sources for `installCheck`.
Generally, this build is sufficient, but in nightly or CI we also test the attributes `functional_root` and `functional_trusted`, in which the test suite is run with different levels of authorization.
## Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`.
If you are testing a build of `nix` that you haven't compiled yet, you may iterate faster by appending the `quickBuild` attribute: `nix build .#hydraJobs.tests.{testName}.quickBuild`.
## Installer tests
@@ -243,7 +308,7 @@ Creating a Cachix cache for your installer tests and adding its authorisation to
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
- The `installer_test` job (which runs on `ubuntu-24.04` and `macos-14`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="587.11" height="516.604" viewBox="0 0 550.416 484.317"><defs><linearGradient id="a"><stop offset="0" style="stop-color:#699ad7;stop-opacity:1"/><stop offset=".243" style="stop-color:#7eb1dd;stop-opacity:1"/><stop offset="1" style="stop-color:#7ebae4;stop-opacity:1"/></linearGradient><linearGradient id="b"><stop offset="0" style="stop-color:#415e9a;stop-opacity:1"/><stop offset=".232" style="stop-color:#4a6baf;stop-opacity:1"/><stop offset="1" style="stop-color:#5277c3;stop-opacity:1"/></linearGradient><linearGradient xlink:href="#a" id="c" x1="200.597" x2="290.087" y1="351.411" y2="506.188" gradientTransform="translate(70.65 -1055.151)" gradientUnits="userSpaceOnUse"/><linearGradient xlink:href="#b" id="e" x1="-584.199" x2="-496.297" y1="782.336" y2="937.714" gradientTransform="translate(864.696 -1491.34)" gradientUnits="userSpaceOnUse"/></defs><g style="display:inline;opacity:1" transform="translate(-132.651 958.04)"><path id="d" d="m309.549-710.388 122.197 211.675-56.157.527-32.624-56.87-32.856 56.566-27.903-.011-14.29-24.69 46.81-80.49-33.23-57.826z" style="opacity:1;fill:url(#c);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:3;stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(60 407.112 -715.787)"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(-60 407.312 -715.7)"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(180 407.419 -715.756)"/><path id="f" d="m309.549-710.388 122.197 211.675-56.157.527-32.624-56.87-32.856 56.566-27.903-.011-14.29-24.69 46.81-80.49-33.23-57.826z" style="color:#000;clip-rule:nonzero;display:inline;overflow:visible;visibility:visible;opacity:1;isolation:auto;mix-blend-mode:normal;color-interpolation:sRGB;color-interpolation-filters:linearRGB;solid-color:#000;solid-opacity:1;fill:url(#e);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:3;stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;color-rendering:auto;image-rendering:auto;shape-rendering:auto;text-rendering:auto;enable-background:accumulate"/><use xlink:href="#f" width="100%" height="100%" style="display:inline" transform="rotate(120 407.34 -716.084)"/><use xlink:href="#f" width="100%" height="100%" style="display:inline" transform="rotate(-120 407.288 -715.87)"/></g></svg>

After

Width:  |  Height:  |  Size: 2.5 KiB

+111 -36
View File
@@ -1,12 +1,31 @@
# Glossary
- [content address]{#gloss-content-address}
A
[*content address*](https://en.wikipedia.org/wiki/Content-addressable_storage)
is a secure way to reference immutable data.
The reference is calculated directly from the content of the data being referenced, which means the reference is
[*tamper proof*](https://en.wikipedia.org/wiki/Tamperproofing)
--- variations of the data should always calculate to distinct content addresses.
For how Nix uses content addresses, see:
- [Content-Addressing File System Objects](@docroot@/store/file-system-object/content-address.md)
- [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md)
- [content-addressed derivation](#gloss-content-addressed-derivation)
Software Heritage's writing on [*Intrinsic and Extrinsic identifiers*](https://www.softwareheritage.org/2020/07/09/intrinsic-vs-extrinsic-identifiers) is also a good introduction to the value of content-addressing over other referencing schemes.
Besides content addressing, the Nix store also uses [input addressing](#gloss-input-addressed-store-object).
- [derivation]{#gloss-derivation}
A description of a build task. The result of a derivation is a
store object. Derivations are typically specified in Nix expressions
store object. Derivations declared in Nix expressions are specified
using the [`derivation` primitive](./language/derivations.md). These are
translated into low-level *store derivations* (implicitly by
`nix-env` and `nix-build`, or explicitly by `nix-instantiate`).
`nix-build`, or explicitly by `nix-instantiate`).
[derivation]: #gloss-derivation
@@ -14,6 +33,7 @@
A [derivation] represented as a `.drv` file in the [store].
It has a [store path], like any [store object].
It is the [instantiated][instantiate] form of a derivation.
Example: `/nix/store/g946hcz4c8mdvq2g8vxx42z51qb71rvp-git-2.38.1.drv`
@@ -23,9 +43,9 @@
- [instantiate]{#gloss-instantiate}, instantiation
Translate a [derivation] into a [store derivation].
Save an evaluated [derivation] as a [store derivation] in the Nix [store].
See [`nix-instantiate`](./command-ref/nix-instantiate.md).
See [`nix-instantiate`](./command-ref/nix-instantiate.md), which produces a store derivation from a Nix expression that evaluates to a derivation.
[instantiate]: #gloss-instantiate
@@ -36,7 +56,7 @@
This can be achieved by:
- Fetching a pre-built [store object] from a [substituter]
- Running the [`builder`](@docroot@/language/derivations.md#attr-builder) executable as specified in the corresponding [derivation]
- Delegating to a [remote builder](@docroot@/advanced-topics/distributed-builds.html) and retrieving the outputs
- Delegating to a [remote machine](@docroot@/command-ref/conf-file.md#conf-builders) and retrieving the outputs
<!-- TODO: link [running] to build process page, #8888 -->
See [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md) for a detailed description of the algorithm.
@@ -51,30 +71,18 @@
[`__contentAddressed`](./language/advanced-attributes.md#adv-attr-__contentAddressed)
attribute set to `true`.
- [fixed-output derivation]{#gloss-fixed-output-derivation}
- [fixed-output derivation]{#gloss-fixed-output-derivation} (FOD)
A derivation which includes the
[`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute.
A [derivation] where a cryptographic hash of the [output] is determined in advance using the [`outputHash`](./language/advanced-attributes.md#adv-attr-outputHash) attribute, and where the [`builder`](@docroot@/language/derivations.md#attr-builder) executable has access to the network.
- [store]{#gloss-store}
A collection of store objects, with operations to manipulate that collection.
See [Nix store](./store/index.md) for details.
A collection of [store objects][store object], with operations to manipulate that collection.
See [Nix Store](./store/index.md) for details.
There are many types of stores.
See [`nix help-stores`](@docroot@/command-ref/new-cli/nix3-help-stores.md) for a complete list.
From the perspective of the location where Nix is invoked, the Nix store can be referred to _local_ or _remote_.
Only a [local store]{#gloss-local-store} exposes a location in the file system of the machine where Nix is invoked that allows access to store objects, typically `/nix/store`.
Local stores can be used for building [derivations](#derivation).
See [Local Store](@docroot@/command-ref/new-cli/nix3-help-stores.md#local-store) for details.
There are many types of stores, see [Store Types](./store/types/index.md) for details.
[store]: #gloss-store
[local store]: #gloss-local-store
- [chroot store]{#gloss-chroot-store}
A [local store] whose canonical path is anything other than `/nix/store`.
- [binary cache]{#gloss-binary-cache}
@@ -86,7 +94,7 @@
- [store path]{#gloss-store-path}
The location of a [store object](@docroot@/store/index.md#store-object) in the file system, i.e., an immediate child of the Nix store directory.
The location of a [store object] in the file system, i.e., an immediate child of the Nix store directory.
> **Example**
>
@@ -96,7 +104,7 @@
[store path]: #gloss-store-path
- [file system object]{#gloss-store-object}
- [file system object]{#gloss-file-system-object}
The Nix data model for representing simplified file system data.
@@ -126,11 +134,14 @@
non-[fixed-output](#gloss-fixed-output-derivation)
derivation.
- [output-addressed store object]{#gloss-output-addressed-store-object}
- [content-addressed store object]{#gloss-content-addressed-store-object}
A [store object] whose [store path] is determined by its contents.
A [store object] which is [content-addressed](#gloss-content-address),
i.e. whose [store path] is determined by its contents.
This includes derivations, the outputs of [content-addressed derivations](#gloss-content-addressed-derivation), and the outputs of [fixed-output derivations](#gloss-fixed-output-derivation).
See [Content-Addressing Store Objects](@docroot@/store/store-object/content-address.md) for details.
- [substitute]{#gloss-substitute}
A substitute is a command invocation stored in the [Nix database] that
@@ -155,6 +166,11 @@
builder can rely on external inputs such as the network or the
system time) but the Nix model assumes it.
- [impure derivation]{#gloss-impure-derivation}
[An experimental feature](#@docroot@/development/experimental-features.md#xp-feature-impure-derivations) that allows derivations to be explicitly marked as impure,
so that they are always rebuilt, and their outputs not reused by subsequent calls to realise them.
- [Nix database]{#gloss-nix-database}
An SQlite database to track [reference]s between [store object]s.
@@ -166,11 +182,13 @@
- [Nix expression]{#gloss-nix-expression}
A high-level description of software packages and compositions
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions are translated to
derivations that are stored in the Nix store. These derivations can
then be built.
1. Commonly, a high-level description of software packages and compositions
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions specify [derivations][derivation],
which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
2. A syntactically valid use of the [Nix language]. For example, the contents of a `.nix` file form an expression.
- [reference]{#gloss-reference}
@@ -218,10 +236,27 @@
[output path]: #gloss-output-path
- [output closure]{#gloss-output-closure}\
The [closure] of an [output path]. It only contains what is [reachable] from the output.
- [deriving path]{#gloss-deriving-path}
Deriving paths are a way to refer to [store objects][store object] that ar not yet [realised][realise].
This is necessary because, in general and particularly for [content-addressed derivations][content-addressed derivation], the [output path] of an [output] is not known in advance.
There are two forms:
- *constant*: just a [store path]
It can be made [valid][validity] by copying it into the store: from the evaluator, command line interface or another store.
- *output*: a pair of a [store path] to a [derivation] and an [output] name.
- [deriver]{#gloss-deriver}
The [store derivation] that produced an [output path].
The deriver for an output path can be queried with the `--deriver` option to
[`nix-store --query`](@docroot@/command-ref/nix-store/query.md).
- [validity]{#gloss-validity}
A store path is valid if all [store object]s in its [closure] can be read from the [store].
@@ -232,6 +267,7 @@
- All paths in the store path's [closure] are valid.
[validity]: #gloss-validity
[local store]: @docroot@/store/types/local-store.md
- [user environment]{#gloss-user-env}
@@ -251,13 +287,15 @@
See [installables](./command-ref/new-cli/nix.md#installables) for [`nix` commands](./command-ref/new-cli/nix.md) (experimental) for details.
- [NAR]{#gloss-nar}
- [Nix Archive (NAR)]{#gloss-nar}
A *N*ix *AR*chive. This is a serialisation of a path in the Nix
store. It can contain regular files, directories and symbolic
links. NARs are generated and unpacked using `nix-store --dump`
and `nix-store --restore`.
See [Nix Archive](store/file-system-object/content-address.html#serial-nix-archive) for details.
- [`∅`]{#gloss-emtpy-set}
The empty set symbol. In the context of profile history, this denotes a package is not present in a particular version of the profile.
@@ -266,19 +304,56 @@
The epsilon symbol. In the context of a package, this means the version is empty. More precisely, the derivation does not have a version attribute.
- [package]{#package}
1. A software package; a collection of files and other data.
2. A [package attribute set].
- [package attribute set]{#package-attribute-set}
An [attribute set](@docroot@/language/types.md#attribute-set) containing the attribute `type = "derivation";` (derivation for historical reasons), as well as other attributes, such as
- attributes that refer to the files of a [package], typically in the form of [derivation outputs](#output),
- attributes that declare something about how the package is supposed to be installed or used,
- other metadata or arbitrary attributes.
[package attribute set]: #package-attribute-set
- [string interpolation]{#gloss-string-interpolation}
Expanding expressions enclosed in `${ }` within a [string], [path], or [attribute name].
See [String interpolation](./language/string-interpolation.md) for details.
[string]: ./language/values.md#type-string
[path]: ./language/values.md#type-path
[attribute name]: ./language/values.md#attribute-set
[string]: ./language/types.md#type-string
[path]: ./language/types.md#type-path
[attribute name]: ./language/types.md#attribute-set
- [base directory]{#gloss-base-directory}
The location from which relative paths are resolved.
- For expressions in a file, the base directory is the directory containing that file.
This is analogous to the directory of a [base URL](https://datatracker.ietf.org/doc/html/rfc1808#section-3.3).
<!-- which is sufficient for resolving non-empty URLs -->
<!--
The wording here may look awkward, but it's for these reasons:
* "with --expr": it's a flag, and not an option with an accompanying value
* "written in": the expression itself must be written as an argument,
whereas the more natural "passed as an argument" allows an interpretation
where the expression could be passed by file name.
-->
- For expressions written in command line arguments with [`--expr`](@docroot@/command-ref/opt-common.html#opt-expr), the base directory is the current working directory.
[base directory]: #gloss-base-directory
- [experimental feature]{#gloss-experimental-feature}
Not yet stabilized functionality guarded by named experimental feature flags.
These flags are enabled or disabled with the [`experimental-features`](./command-ref/conf-file.html#conf-experimental-features) setting.
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/contributing/experimental-features.md).
See the contribution guide on the [purpose and lifecycle of experimental feaures](@docroot@/development/experimental-features.md).
[Nix language]: ./language/index.md
+2 -1
View File
@@ -53,7 +53,8 @@ ssl-cert-file = /etc/ssl/my-certificate-bundle.crt
The Nix installer has special handling for these proxy-related
environment variables: `http_proxy`, `https_proxy`, `ftp_proxy`,
`no_proxy`, `HTTP_PROXY`, `HTTPS_PROXY`, `FTP_PROXY`, `NO_PROXY`.
`all_proxy`, `no_proxy`, `HTTP_PROXY`, `HTTPS_PROXY`, `FTP_PROXY`,
`ALL_PROXY`, `NO_PROXY`.
If any of these variables are set when running the Nix installer, then
the installer will create an override file at
@@ -14,6 +14,14 @@ This option requires either:
* Linux running systemd, with SELinux disabled
* MacOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
```console
$ bash <(curl -L https://nixos.org/nix/install) --daemon
```
@@ -1,89 +1,119 @@
# Installing a Binary Distribution
The easiest way to install Nix is to run the following command:
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
To install the latest version Nix, run the following command:
```console
$ curl -L https://nixos.org/nix/install | sh
```
This will run the installer interactively (causing it to explain what
it is doing more explicitly), and perform the default "type" of install
for your platform:
- single-user on Linux
- multi-user on macOS
This performs the default type of installation for your platform:
> **Notes on read-only filesystem root in macOS 10.15 Catalina +**
>
> - It took some time to support this cleanly. You may see posts,
> examples, and tutorials using obsolete workarounds.
> - Supporting it cleanly made macOS installs too complex to qualify
> as single-user, so this type is no longer supported on macOS.
- [Multi-user](#multi-user-installation):
- Linux with systemd and without SELinux
- macOS
- [Single-user](#single-user-installation):
- Linux without systemd
- Linux with SELinux
We recommend the multi-user install if it supports your platform and
you can authenticate with `sudo`.
We recommend the multi-user installation if it supports your platform and you can authenticate with `sudo`.
The installer can configured with various command line arguments and environment variables.
To show available command line flags:
```console
$ curl -L https://nixos.org/nix/install | sh -s -- --help
```
To check what it does and how it can be customised further, [download and edit the second-stage installation script](#installing-from-a-binary-tarball).
# Installing a pinned Nix version from a URL
Version-specific installation URLs for all Nix versions since 1.11.16 can be found at [releases.nixos.org](https://releases.nixos.org/?prefix=nix/).
The directory for each version contains the corresponding SHA-256 hash.
All installation scripts are invoked the same way:
```console
$ export VERSION=2.19.2
$ curl -L https://releases.nixos.org/nix/nix-$VERSION/install | sh
```
# Multi User Installation
The multi-user Nix installation creates system users and a system service for the Nix daemon.
Supported systems:
- Linux running systemd, with SELinux disabled
- macOS
To explicitly instruct the installer to perform a multi-user installation on your system:
```console
$ bash <(curl -L https://nixos.org/nix/install) --daemon
```
You can run this under your usual user account or `root`.
The script will invoke `sudo` as needed.
# Single User Installation
To explicitly select a single-user installation on your system:
```console
$ curl -L https://nixos.org/nix/install | sh -s -- --no-daemon
$ bash <(curl -L https://nixos.org/nix/install) --no-daemon
```
This will perform a single-user installation of Nix, meaning that `/nix`
is owned by the invoking user. You can run this under your usual user
account or root. The script will invoke `sudo` to create `/nix`
if it doesn’t already exist. If you don’t have `sudo`, you should
manually create `/nix` first as root, e.g.:
In a single-user installation, `/nix` is owned by the invoking user.
The script will invoke `sudo` to create `/nix` if it doesn’t already exist.
If you don’t have `sudo`, manually create `/nix` as `root`:
```console
$ mkdir /nix
$ chown alice /nix
$ su root
# mkdir /nix
# chown alice /nix
```
The install script will modify the first writable file from amongst
`.bash_profile`, `.bash_login` and `.profile` to source
`~/.nix-profile/etc/profile.d/nix.sh`. You can set the
`NIX_INSTALLER_NO_MODIFY_PROFILE` environment variable before executing
the install script to disable this behaviour.
# Installing from a binary tarball
# Multi User Installation
You can also download a binary tarball that contains Nix and all its dependencies:
- Choose a [version](https://releases.nixos.org/?prefix=nix/) and [system type](../development/building.md#platforms)
- Download and unpack the tarball
- Run the installer
The multi-user Nix installation creates system users, and a system
service for the Nix daemon.
**Supported Systems**
- Linux running systemd, with SELinux disabled
- macOS
You can instruct the installer to perform a multi-user installation on
your system:
```console
$ curl -L https://nixos.org/nix/install | sh -s -- --daemon
```
The multi-user installation of Nix will create build users between the
user IDs 30001 and 30032, and a group with the group ID 30000. You
can run this under your usual user account or root. The script
will invoke `sudo` as needed.
> **Note**
> **Example**
>
> If you need Nix to use a different group ID or user ID set, you will
> have to download the tarball manually and [edit the install
> script](#installing-from-a-binary-tarball).
> ```console
> $ pushd $(mktemp -d)
> $ export VERSION=2.19.2
> $ export SYSTEM=x86_64-linux
> $ curl -LO https://releases.nixos.org/nix/nix-$VERSION/nix-$VERSION-$SYSTEM.tar.xz
> $ tar xfj nix-$VERSION-$SYSTEM.tar.xz
> $ cd nix-$VERSION-$SYSTEM
> $ ./install
> $ popd
> ```
The installer will modify `/etc/bashrc`, and `/etc/zshrc` if they exist.
The installer will first back up these files with a `.backup-before-nix`
extension. The installer will also create `/etc/profile.d/nix.sh`.
The installer can be customised with the environment variables declared in the file named `install-multi-user`.
## Native packages for Linux distributions
The Nix community maintains installers for some Linux distributions in their native packaging format(https://nix-community.github.io/nix-installers/).
# macOS Installation
<!-- anchors to catch existing links -->
[]{#sect-macos-installation-change-store-prefix}[]{#sect-macos-installation-encrypted-volume}[]{#sect-macos-installation-symlink}[]{#sect-macos-installation-recommended-notes}
<!-- Note: anchors above to catch permalinks to old explanations -->
We believe we have ironed out how to cleanly support the read-only root
We believe we have ironed out how to cleanly support the read-only root file system
on modern macOS. New installs will do this automatically.
This section previously detailed the situation, options, and trade-offs,
@@ -126,33 +156,3 @@ this to run the installer, but it may help if you run into trouble:
boot process to avoid problems loading or restoring any programs that
need access to your Nix store
# Installing a pinned Nix version from a URL
Version-specific installation URLs for all Nix versions
since 1.11.16 can be found at [releases.nixos.org](https://releases.nixos.org/?prefix=nix/).
The corresponding SHA-256 hash can be found in the directory for the given version.
These install scripts can be used the same as usual:
```console
$ curl -L https://releases.nixos.org/nix/nix-<version>/install | sh
```
# Installing from a binary tarball
You can also download a binary tarball that contains Nix and all its
dependencies. (This is what the install script at
<https://nixos.org/nix/install> does automatically.) You should unpack
it somewhere (e.g. in `/tmp`), and then run the script named `install`
inside the binary tarball:
```console
$ cd /tmp
$ tar xfj nix-1.8-x86_64-darwin.tar.bz2
$ cd nix-1.8-x86_64-darwin
$ ./install
```
If you need to edit the multi-user installation script to use different
group ID or a different user ID range, modify the variables set in the
file named `install-multi-user`.
Loaded 100 of 1680 files, more files were not shown because too many files have changed in this diff. Show more