Compare commits

..
Author SHA1 Message Date
Jörg Thalheim c55a7e1423 Merge pull request #13744 from NixOS/mergify/bp/2.24-maintenance/pr-13741
libexpr: Canonicalize TOML timestamps for toml11 > 4.0 (backport #13741)
2025-08-29 07:36:10 +02:00
Sergei Zimmerman ae47fd6886 libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339

(cherry picked from commit dc769d72cb)
2025-08-13 23:13:08 +03:00
Sergei Zimmerman b78b79349e libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.

(cherry picked from commit df4e55ffc1)
2025-08-13 22:39:58 +03:00
Sergei Zimmerman 79b932277a libexpr: Format primops/fromTOML.cc 2025-08-13 22:37:03 +03:00
Sergei Zimmerman d52210fae3 tests/functional/lang: Add more tests for TOML timestamps
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.

(cherry picked from commit 7ed0229d1a)
2025-08-13 19:22:48 +00:00
Eelco Dolstra c2914c0d58 Bump version 2025-06-24 19:04:04 +02:00
Eelco Dolstra 148585470c Merge remote-tracking branch 'cve/fod-cves-2.24' into 2.24-maintenance 2025-06-24 16:08:50 +02:00
mergify[bot] ed56b2741d Merge pull request #13389 from NixOS/mergify/bp/2.24-maintenance/pr-13348
Make the repl test more robust (backport #13348)
2025-06-22 19:34:41 +00:00
Eelco Dolstra bcffa0ce59 Make the repl test more robust
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:

  nix-functional-tests> grep: repl_output: No such file or directory
  nix-functional-tests> +(repl.sh:174) cat repl_output

This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.

(cherry picked from commit 9eb46e9cc0)
2025-06-22 19:08:35 +00:00
Eelco Dolstra b0fab9f90b Fixes for GHSA-g948-229j-48j3
Squashed commit of the following:

commit 04fff3a637d455cbb1d75937a235950e43008db9
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:30:32 2025 +0200

    Chown structured attr files safely

commit 5417ad445e414c649d0cfc71a05661c7bf8f3ef5
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:14:04 2025 +0200

    Replace 'bool sync' with an enum for clarity

    And drop writeFileAndSync().

commit 7ae0141f328d8e8e1094be24665789c05f974ba6
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:35:28 2025 +0200

    Drop guessOrInventPathFromFD()

    No need to do hacky stuff like that when we already know the original path.

commit 45b05098bd019da7c57cd4227a89bfd0fa65bb08
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:15:58 2025 +0200

    Tweak comment

commit 0af15b31209d1b7ec8addfae9a1a6b60d8f35848
Author: Raito Bezarius <raito@lix.systems>
Date:   Thu Mar 27 12:22:26 2025 +0100

    libstore: ensure that temporary directory is always 0o000 before deletion

    In the case the deletion fails, we should ensure that the temporary
    directory cannot be used for nefarious purposes.

    Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 2c20fa37b15cfa03ac6a1a6a47cdb2ed66c0827e
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 12:42:55 2025 +0100

    libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds

    When calling `_deletePath` with a parent file descriptor, `openat` is
    made effective by using relative paths to the directory file descriptor.

    To avoid the problem, the signature is changed to resist misuse with an
    assert in the prologue of the function.

    Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit d3c370bbcae48bb825ce19fd0f73bb4eefd2c9ea
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:07:47 2025 +0100

    libstore: ensure that `passAsFile` is created in the original temp dir

    This ensures that `passAsFile` data is created inside the expected
    temporary build directory by `openat()` from the parent directory file
    descriptor.

    This avoids a TOCTOU which is part of the attack chain of CVE-????.

    Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 45d3598724f932d024ef6bc2ffb00c1bb90e6018
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:06:03 2025 +0100

    libutil: writeFile variant for file descriptors

    `writeFile` lose its `sync` boolean flag to make things simpler.

    A new `writeFileAndSync` function is created and all call sites are
    converted to it.

    Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 732bd9b98cabf4aaf95a01fd318923de303f9996
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:05:34 2025 +0100

    libstore: chown to builder variant for file descriptors

    We use it immediately for the build temporary directory.

    Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 962c65f8dcd5570dd92c72370a862c7b38942e0d
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:59 2025 +0100

    libstore: open build directory as a dirfd as well

    We now keep around a proper AutoCloseFD around the temporary directory
    which we plan to use for openat operations and avoiding the build
    directory being swapped out while we are doing something else.

    Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit c9b42462b75b5a37ee6564c2b53cff186c8323da
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:12 2025 +0100

    libutil: guess or invent a path from file descriptors

    This is useful for certain error recovery paths (no pun intended) that
    does not thread through the original path name.

    Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
    Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 16:55:43 +02:00
Jörg Thalheim 7d15dbf9d5 Merge pull request #12801 from pascalj/2.24-maintenance
Exclude 'dir' from the FlakeRef's URL
2025-05-18 23:35:53 +02:00
Pascal Jungblut fa2a3c6de8 Merge branch '2.24-maintenance' into 2.24-maintenance 2025-05-14 22:58:17 +02:00
mergify[bot] 3e2c0680b6 Merge pull request #13091 from NixOS/mergify/bp/2.24-maintenance/pr-13086
libutil: Fix invalid boost format string in infinite symlink recursion error (backport #13086)
2025-04-25 10:11:28 +00:00
Sergei Zimmerman 28c2c78716 libutil: Fix invalid boost format string in infinite symlink recursion error
Found while working on an automated migration to `std::format`.

(cherry picked from commit bfb357c40b)
2025-04-25 09:20:56 +00:00
Pascal Jungblut 72bf563191 Exclude 'dir' from the FlakeRef's URL
This fixes an issue where nix would try to check out invalid URLs,
because it would pass 'dir' to the HTTP endpoint.

For later versions this was fixed in
b2be6fed86. This is a backport of just the
relevant part.

See #12417
2025-04-15 13:34:48 +02:00
mergify[bot] 57190f9a39 Merge pull request #12988 from NixOS/mergify/bp/2.24-maintenance/pr-12986
Fix another machine config parsing bug (backport #12986)
2025-04-09 21:41:41 +00:00
mergify[bot] 89ea1a8758 Merge pull request #12987 from NixOS/mergify/bp/2.24-maintenance/pr-12984
Fix `;` and `#` bug in machine file parsing (backport #12984)
2025-04-09 20:47:38 +00:00
John Ericson 142f94b7e1 Fix another machine config parsing bug
We were ignorning the result of `trim`, and after my last change we were
also trimmming too early.

(cherry picked from commit b74b0f4e1c)
2025-04-09 16:20:07 -04:00
John Ericson 0c6a207916 Fix ; and # bug in machine file parsing
Comments go to the end of the line, not merely the next ; *or* \n. Fix
by splitting on `;` *within* lines, and test.

(cherry picked from commit f8b13cce19)
2025-04-09 20:06:25 +00:00
Eelco Dolstra a568447137 Merge pull request #12952 from NixOS/mergify/bp/2.24-maintenance/pr-12951
tests/functional/repl: fix race condition (backport #12951)
2025-04-07 19:51:27 +02:00
Jörg Thalheim bc88d09a08 tests/functional/repl: fix race condition
the sleep 1 is not enough in some circumstances. Switching to a fifo
helps.

(cherry picked from commit 1de951d31d)
2025-04-07 12:19:00 +00:00
Eelco Dolstra ccea5cce42 Bump version 2025-04-07 11:28:46 +02:00
mergify[bot] e949cdbc05 Merge pull request #12946 from NixOS/mergify/bp/2.24-maintenance/pr-12944
maintainers/release-notes: Let it fail (backport #12944)
2025-04-07 09:11:03 +00:00
Robert Hensing de18f5ac2d maintainers/release-notes: Let it fail
Fail when a command fails.

Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.

(cherry picked from commit da36c34db7)
2025-04-07 08:52:30 +00:00
Jörg Thalheim adbf23e818 Merge pull request #12922 from NixOS/mergify/bp/2.24-maintenance/pr-12911
create cache entry for paths already in the nix store (backport #12911)
2025-04-04 16:32:47 +02:00
Jörg Thalheim 2a4e2ecd89 create cache entry for paths already in the nix store
This allows path:/nix/store/* paths to not be copied twice to the nix
store.

(cherry picked from commit 61c6210dbf)
2025-04-04 16:01:17 +02:00
mergify[bot] b47bd02a6a Merge pull request #12854 from NixOS/mergify/bp/2.24-maintenance/pr-12765
Add various clang tidy fixes (backport #12765)
2025-04-01 14:35:45 +00:00
Jörg Thalheim 8abb67b842 libstore/daemon: make sure monitor is not considered "unused"
(cherry picked from commit 5c3682d7a1)
2025-04-01 13:37:38 +00:00
Jörg Thalheim 97f9f8e909 test/ca-fd-leak: fix clang-tidy lints
(cherry picked from commit b050db951b)
2025-04-01 13:37:37 +00:00
Jörg Thalheim 7ef04ba6dd git/getStringUntil: fix uninitialized stack variable
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.

(cherry picked from commit 7e540059a3)
2025-04-01 13:37:37 +00:00
Jörg Thalheim bc6e78402c libstore/local-store: fix linting warning about unused variable
(cherry picked from commit 05082ea1c5)
2025-04-01 13:37:37 +00:00
mergify[bot] 4687dc99f7 Merge pull request #12840 from NixOS/mergify/bp/2.24-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:47:08 +00:00
Thomas Miedema 8d99b6578a nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:47:25 +00:00
John Ericson e1dad7daa5 Merge pull request #12795 from NixOS/mergify/bp/2.24-maintenance/pr-12794
Fix windows build (backport #12794)
2025-03-28 21:54:39 -04:00
John Ericson d92b509bda Fix windows build
PR #12767 accidentally broke it.

(cherry picked from commit 99041b4d84)
2025-03-28 21:07:08 -04:00
mergify[bot] 8adc1f3011 Merge pull request #12774 from NixOS/mergify/bp/2.24-maintenance/pr-12767
use createDirs consistently everywhere (backport #12767)
2025-03-28 16:48:21 +00:00
Jörg Thalheim 076fa01a84 use createDirs consistently everywhere
(cherry picked from commit a8217f2642)
2025-03-28 15:54:56 +00:00
Eelco Dolstra 61f8fd3e9c Bump version 2025-03-24 19:35:50 +01:00
Eelco Dolstra 5ce8d922fd Merge pull request #12737 from NixOS/mergify/bp/2.24-maintenance/pr-12736
`MonitorFdHup::~MonitorFdHup`: use proper close method instead of lib… (backport #12736)
2025-03-24 14:00:41 +01:00
Jörg Thalheim 333479e6be MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)
2025-03-24 12:16:00 +00:00
John Ericson 71ab003a0c Merge pull request #12731 from NixOS/mergify/bp/2.24-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-23 20:39:08 -04:00
John Ericson 3fccc71492 MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-23 23:54:09 +00:00
Jade Lovelace 729d18fd4c daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-23 23:54:08 +00:00
Félix Baylac Jacqué 699f27c4ea MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-23 23:54:08 +00:00
John Ericson 2d792a1867 MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-23 23:54:08 +00:00
John Ericson da213e8ee5 MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-23 23:54:08 +00:00
Jörg Thalheim 787a14b173 MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-23 23:54:07 +00:00
John Ericson 01aace706a monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-23 23:54:07 +00:00
mergify[bot] 74a0ab3eea Merge pull request #12683 from NixOS/mergify/bp/2.24-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 21:07:12 +00:00
Pierre-Etienne Meunier 589da451e4 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:20:48 +00:00
mergify[bot] 56e113d67a Merge pull request #12653 from NixOS/mergify/bp/2.24-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-14 14:11:27 +00:00
Sergei Zimmerman aaf86cc0d5 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 6faf66d2f7 libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 3ab83f507c libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-14 13:22:43 +00:00
Sergei Zimmerman 0033cf4270 {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:43 +00:00
mergify[bot] 79828c12e0 Merge pull request #12535 from NixOS/mergify/bp/2.24-maintenance/pr-12391
Only try to chmod /nix/var/nix/profiles/per-user when necessary (backport #12391)
2025-02-20 16:55:38 +00:00
Sandro Jäckel 2a795bb793 Only try to chmod /nix/var/nix/profiles/per-user when necessary
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit dcbf4dcc09)
2025-02-20 16:11:26 +00:00
mergify[bot] 9238c892d3 Merge pull request #12528 from NixOS/mergify/bp/2.24-maintenance/pr-12439
Set FD_CLOEXEC on sockets created by curl (backport #12439)
2025-02-19 20:30:16 +00:00
MaxHearnden 5862686ce2 Set FD_CLOEXEC on sockets created by curl
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

(cherry picked from commit 12d2527276)
2025-02-19 19:47:40 +00:00
John Ericson feadb3178d Merge pull request #12508 from NixOS/mergify/bp/2.24-maintenance/pr-12499
startDaemon(): Detect if the daemon crashes before creating the socket (backport #12499)
2025-02-18 12:21:39 -05:00
Eelco Dolstra be35515ef7 startDaemon(): Detect if the daemon crashes before creating the socket
This avoids timeouts like those seen in
https://github.com/NixOS/nix/actions/runs/13376958708/job/37358120348?pr=6962.

(cherry picked from commit 11c42cb2e1)
2025-02-18 16:38:55 +00:00
Eelco Dolstra 71f93012fd Merge pull request #12487 from NixOS/mergify/bp/2.24-maintenance/pr-12481
resolveLookupPathPath(): Fix caching of negative lookups (backport #12481)
2025-02-18 17:27:52 +01:00
Eelco Dolstra b706642133 resolveLookupPathPath(): Fix caching of negative lookups
This avoids spamming in case the missing search path entry does not
exist (#12480).

(cherry picked from commit df08e1e204)
2025-02-18 16:58:07 +01:00
Eelco Dolstra 658392e029 getDefaultNixPath(): Don't add symlinks if the target doesn't exist
(cherry picked from commit 8ac49ea5de)
2025-02-17 17:08:45 +00:00
mergify[bot] d3a52c2ef4 Merge pull request #12451 from NixOS/mergify/bp/2.24-maintenance/pr-12448
copyPathToStore(): Preserve symlinks (backport #12448)
2025-02-11 22:21:12 +00:00
Eelco Dolstra 3d6a9c7410 copyPathToStore(): Preserve symlinks
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

(cherry picked from commit 26b87e78b5)

# Conflicts:
#	tests/functional/meson.build
2025-02-11 21:51:20 +00:00
mergify[bot] 0f5f24223d Merge pull request #12434 from NixOS/mergify/bp/2.24-maintenance/pr-12373
pre-commit/check-merge-conflicts-2: fix use outside dev shell (backport #12373)
2025-02-10 14:37:20 +00:00
Robert Hensing fcea626e49 test: Fix shellcheck by giving git-hashing scripts shebangs
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.

It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
2025-02-10 15:09:36 +01:00
Robert Hensing c6e09073ea pre-commit/check-merge-conflicts-2: fix use outside dev shell
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.

`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
2025-02-10 15:08:54 +01:00
mergify[bot] b9e6e00fe1 Merge pull request #12357 from NixOS/mergify/bp/2.24-maintenance/pr-12356
Improve "illegal path references in fixed output derivation" error (backport #12356)
2025-01-28 09:37:45 +00:00
mergify[bot] 5fb6551217 Merge pull request #12365 from NixOS/mergify/bp/2.24-maintenance/pr-12362
refactor: Extract EvalState::realiseString (backport #12362)
2025-01-28 08:51:24 +00:00
Robert Hensing 0f19bfe811 Fix conflict 2025-01-28 09:20:56 +01:00
Robert Hensing 865239c732 refactor: Extract EvalState::realiseString
(cherry picked from commit 7465fbe926)

# Conflicts:
#	src/libexpr/primops.cc
2025-01-27 15:50:52 +00:00
Ben Millwood 9c5482167f Improve "illegal path references in fixed output derivation" error
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.

(cherry picked from commit 4e5d1b281e)
2025-01-26 19:38:38 +00:00
Robert Hensing a28791a91e Merge pull request #12352 from NixOS/nixfmt-on-2.24
Apply nixfmt (backport 2.24)
2025-01-24 22:01:53 +01:00
Robert Hensing ae0bb7d71c test: Fix shifted source positions after formatting 2025-01-24 21:32:07 +01:00
Robert Hensing 2f1b70a529 Format .nix files
This does not include any automation for the release branch, but
is based on the configuration of https://github.com/NixOS/nix/pull/12349

    pre-commit run -a nixfmt-rfc-style
2025-01-24 20:30:07 +01:00
Eelco Dolstra 42b22fe3de Actually bump the version
Fixes 532092a812.
2025-01-24 16:47:16 +01:00
mergify[bot] 62e7144ba7 Merge pull request #12344 from NixOS/mergify/bp/2.24-maintenance/pr-12336
libstore: Fix progress bars (backport #12336)
2025-01-24 11:52:17 +00:00
Philipp Otterbein a106e33bab libstore: fix progress bars
(cherry picked from commit be97dc1efc)
2025-01-24 11:10:16 +00:00
Eelco Dolstra 7616bae342 Merge pull request #12332 from NixOS/mergify/bp/2.24-maintenance/pr-12331
GitRepo::fetch(): Ignore $GIT_DIR (backport #12331)
2025-01-22 22:49:18 +01:00
Eelco Dolstra 1a402db046 GitRepo::fetch(): Ignore $GIT_DIR
Fixes #12325.

(cherry picked from commit 41983dba8f)
2025-01-22 22:10:09 +01:00
Eelco Dolstra 4511c47ab7 GitRepo::fetch(): Cleanup
(cherry picked from commit bd10b859f7)

# Conflicts:
#	src/libfetchers/git-utils.cc
2025-01-22 22:10:09 +01:00
Eelco Dolstra 532092a812 Bump version 2025-01-22 11:20:50 +01:00
Robert Hensing f22359ba1a Merge pull request #12287 from bryango/2.24-maintenance
Backport git+file:./ fixes to 2.24 (#12107 + #12277)
2025-01-21 22:36:06 +01:00
mergify[bot] f366beaa78 Merge pull request #12318 from NixOS/mergify/bp/2.24-maintenance/pr-12315
tests: Wait for network *online* targets and/or multi-user targets (more) (backport #12315)
2025-01-20 18:29:53 +00:00
Robert Hensing 011084b127 tests: Wait for network *online* targets
More prevalent than I thought in 15073e86a

See also https://github.com/NixOS/nix/actions/runs/12872412321/job/35887830320?pr=12310
which is a failed github-flakes test without "Network is Online"

(cherry picked from commit 6f0bdd9ae7)

# Conflicts:
#	tests/nixos/nix-docker.nix
2025-01-20 18:16:42 +00:00
Eelco Dolstra d8872fce2c Merge pull request #12311 from NixOS/mergify/bp/2.24-maintenance/pr-12302
tests: Wait for network *online* and multi-user targets (backport #12302)
2025-01-20 18:27:50 +01:00
Robert Hensing 12c90a0ec0 tests: Wait for network *online* and multi-user targets
This should help prevent some test stalls.

By default, multi-user.target does not imply that the network is fully up.

(cherry picked from commit 15073e86a8)
2025-01-20 16:40:04 +00:00
Eelco Dolstra 296769742b Merge pull request #12307 from NixOS/mergify/bp/2.24-maintenance/pr-12294
processGraph(): Don't throw ThreadPoolShutDown if there is an exception (backport #12294)
2025-01-20 17:20:40 +01:00
Eelco Dolstra d31be32c4b Merge pull request #12299 from NixOS/mergify/bp/2.24-maintenance/pr-12296
addMultipleToStore(): Move pathsToCopy (backport #12296)
2025-01-20 16:25:17 +01:00
Eelco Dolstra a1b4bc7292 processGraph(): Don't throw ThreadPoolShutDown if there is an exception
Fixes

  $ nix copy --derivation --to /tmp/nix /nix/store/...
  error: cannot enqueue a work item while the thread pool is shutting down

The ThreadPoolShutDown exception was hiding the reason for the thread
pool shut down, e.g.

  error: cannot add path '/nix/store/03sl46khd8gmjpsad7223m32ma965vy9-fix-static.patch' because it lacks a signature by a trusted key

(cherry picked from commit a8c69cc907)
2025-01-20 15:24:44 +00:00
Eelco Dolstra ff91a777a6 Add comment
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 2669e4ac4f)
2025-01-20 14:17:51 +00:00
Eelco Dolstra b4418c5c1d addMultipleToStore(): Move pathsToCopy
This allows RemoteStore::addMultipleToStore() to free the Source
objects early (and in particular the associated sinkToSource()
buffers). This should fix #7359. For example, memory consumption of

  nix copy --derivation --to ssh-ng://localhost?remote-store=/tmp/nix --derivation --no-check-sigs \
    /nix/store/4p9xmfgnvclqpii8pxqcwcvl9bxqy2xf-nixos-system-...drv

went from 353 MB to 74 MB.

(cherry picked from commit cc838e8181)
2025-01-20 14:17:51 +00:00
Eelco Dolstra 876d724061 Add link to tracking issue
(cherry picked from commit 3197c19a31)
2025-01-19 15:41:06 +08:00
Eelco Dolstra f27e263f89 Warn against the use of relative 'git+file:' flake inputs
(cherry picked from commit 12e14956e2)
2025-01-19 15:38:05 +08:00
Eelco Dolstra d944cb7f58 Use isAbsolute()
(cherry picked from commit ff9d886f3c)
2025-01-19 14:58:36 +08:00
Robert Hensing b2acf4ab97 Clarify cd call in tests/functional/flakes/flakes.sh
(cherry picked from commit d9a50c0af2)
2025-01-19 14:56:13 +08:00
Bryan Lai d4f0e8f4e3 tests/flake-in-submodule: git+file:./* input
(cherry picked from commit 37ac18d1d9)
2025-01-19 14:56:11 +08:00
Bryan Lai 84e3f4ad79 tests/flakes: check git+file:./${submodule} protocol
Relative, local git repo used to work (for submodules), but it
fails after 3e0129ce3b.

This commit adds a test to prevent such failure in the future.

(cherry picked from commit 9d088fa502)
2025-01-19 14:55:51 +08:00
Bryan Lai 6e44bc2a89 fetchers/git: make path absolute for local repo
(cherry picked from commit 96bd9bad2f)
2025-01-19 14:55:49 +08:00
mergify[bot] 2009cc137a Merge pull request #12278 from NixOS/mergify/bp/2.24-maintenance/pr-12258
sequoia-nixbld-user-migration: nail down PATH (backport #12258)
2025-01-16 16:49:44 +00:00
Travis A. Everett 0cefe354c8 sequoia-nixbld-user-migration: nail down PATH
Fixes a user report of trouble with toybox grep and avoids
potential of same basic issue with other utils.

(cherry picked from commit 6a874c2865)

# Conflicts:
#	scripts/sequoia-nixbld-user-migration.sh
2025-01-16 16:43:56 +00:00
mergify[bot] c6f504d827 Merge pull request #12239 from NixOS/mergify/bp/2.24-maintenance/pr-12237
Reject merge conflicts (backport #12237)
2025-01-15 19:40:36 +00:00
mergify[bot] 6e718c7c5c Merge pull request #12259 from NixOS/mergify/bp/2.24-maintenance/pr-11922
gc: resume GC after a pathinuse error (backport #11922)
2025-01-15 16:42:29 +00:00
Picnoir f68cdf1801 gc: resume GC after a pathinuse error
First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.

This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.

Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.

(cherry picked from commit ced8d311a5)
2025-01-15 15:59:24 +00:00
Robert Hensing 1422832f7c Resolve conflict
Conflict introduced in 9ee4a85a99
2025-01-15 07:10:34 +01:00
Jörg Thalheim 7012e1ff8c Merge pull request #12208 from NixOS/mergify/bp/2.24-maintenance/pr-11305
[2.24] Document function application operator (backport #11305)
2025-01-13 09:56:16 +01:00
Jörg Thalheim e81be91738 Merge pull request #12242 from NixOS/mergify/bp/2.24-maintenance/pr-12235
libutil: thread-pool: ensure threads finished on error (backport #12235)
2025-01-13 09:49:15 +01:00
Robert Hensing 9cc5786201 doc: Edit language/operators
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 06b18cff20)
2025-01-13 09:07:22 +01:00
Robert Hensing acd43b2239 doc: Document function application operator
(cherry picked from commit e225b63062)
2025-01-13 09:07:22 +01:00
Dominique Martinet 3cd8c1e53d libutil: thread-pool: ensure threads finished on error
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.

Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.

It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.

By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.

[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]

Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
(cherry picked from commit afac093b34)
2025-01-13 08:03:02 +00:00
Robert Hensing 10438e6cfd Reject merge conflicts
They're usually found by other checks, but docs would remain
susceptible.

(cherry picked from commit 29a1a21ce4)
2025-01-12 21:22:43 +00:00
mergify[bot] c803fe0785 Merge pull request #12197 from NixOS/mergify/bp/2.24-maintenance/pr-11541
nix-profile.sh.in: fix envvar condition (backport #11541)
2025-01-12 13:09:04 +00:00
mergify[bot] 9a85784be2 Merge pull request #12209 from NixOS/mergify/bp/2.24-maintenance/pr-11294
parser-state: fix attribute merging (backport #11294)
2025-01-12 12:36:06 +00:00
Robert Hensing a69d32d194 Resolve conflict
Some of the code before https://github.com/NixOS/nix/pull/11351 did not
have the problem. What remains wasn't all that problematic, but is good
to have anyway.
2025-01-12 13:15:03 +01:00
Ryan Hendrickson e0e50fb01f parser-state: fix attribute merging
(cherry picked from commit 8034589d7e)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson 6ae5aa7fa0 parser.y: move attr doc setting into addAttr
(cherry picked from commit b0a8430e85)
2025-01-12 13:07:28 +01:00
Ryan Hendrickson cd169f14f0 parser.y: use names where I'll be refactoring
(cherry picked from commit 429a197d24)
2025-01-12 13:07:28 +01:00
Jörg Thalheim e3715053b1 Merge pull request #12183 from NixOS/mergify/bp/2.24-maintenance/pr-12051
Skip tests on systems with restricted usernamespaces (backport #12051)
2025-01-11 21:42:36 +01:00
Jörg Thalheim ada649ad7f functional-tests: skip tests if the kernel restricts unprivileged user namespaces
Update tests/functional/common/functions.sh

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: bryango <bryango@users.noreply.github.com>
(cherry picked from commit da7f7ba810)
2025-01-11 20:59:50 +01:00
mergify[bot] e1e219a299 Merge pull request #12211 from NixOS/mergify/bp/2.24-maintenance/pr-11240
fix: bash mangles flake ref completion (backport #11240)
2025-01-10 23:43:24 +00:00
mergify[bot] a658bafce7 Merge pull request #12207 from NixOS/mergify/bp/2.24-maintenance/pr-11343
withFramedSink(): Don't use a thread to monitor the other side (backport #11343)
2025-01-10 23:43:21 +00:00
mergify[bot] e76757083a Merge pull request #12206 from NixOS/mergify/bp/2.24-maintenance/pr-11381
Fix few duplicate word occurrences (backport #11381)
2025-01-10 23:43:18 +00:00
mergify[bot] c9dd4a7c28 Merge pull request #12205 from NixOS/mergify/bp/2.24-maintenance/pr-11407
add removal of ~root/.cache/nix to uninstall instructions (backport #11407)
2025-01-10 23:43:16 +00:00
mergify[bot] 01cb05a265 Merge pull request #12203 from NixOS/mergify/bp/2.24-maintenance/pr-11424
Pull fut.get() out of the lock (backport #11424)
2025-01-10 22:12:00 +00:00
mergify[bot] 9328fc7529 Merge pull request #12204 from NixOS/mergify/bp/2.24-maintenance/pr-11371
Fix diskutil deleteVolume command in Uninstall guide (backport #11371)
2025-01-10 22:11:57 +00:00
mergify[bot] 92e5f64068 Merge pull request #12202 from NixOS/mergify/bp/2.24-maintenance/pr-11514
nix derivation add: Remove reference to "installable" (backport #11514)
2025-01-10 22:11:54 +00:00
mergify[bot] 283249a972 Merge pull request #12201 from NixOS/mergify/bp/2.24-maintenance/pr-11611
doc, src: Fix various spelling typos (backport #11611)
2025-01-10 22:11:52 +00:00
mergify[bot] 4d2cc07b48 Merge pull request #12200 from NixOS/mergify/bp/2.24-maintenance/pr-11613
doc/nix fmt: Do not document behavior we do not control (backport #11613)
2025-01-10 22:11:49 +00:00
mergify[bot] 5c0f2b744a Merge pull request #12199 from NixOS/mergify/bp/2.24-maintenance/pr-11526
docs: Provide an example of updating multiple inputs (backport #11526)
2025-01-10 20:51:42 +00:00
mergify[bot] 0e95d8b467 Merge pull request #12198 from NixOS/mergify/bp/2.24-maintenance/pr-11530
add description + example for nix flake archive (backport #11530)
2025-01-10 20:51:38 +00:00
mergify[bot] 53bc8b543c Merge pull request #12196 from NixOS/mergify/bp/2.24-maintenance/pr-11621
Improve documentation of `nix flake lock` (backport #11621)
2025-01-10 20:51:35 +00:00
mergify[bot] 86da8bbbf3 Merge pull request #12194 from NixOS/mergify/bp/2.24-maintenance/pr-11684
libstore: Make our sandbox pivot_root directory accessible to ourself (backport #11684)
2025-01-10 20:51:33 +00:00
mergify[bot] fcdca4f8d1 Merge pull request #12195 from NixOS/mergify/bp/2.24-maintenance/pr-11629
doc/manual: Fix some broken fragments (backport #11629)
2025-01-10 20:51:30 +00:00
mergify[bot] f393260a5d Merge pull request #12192 from NixOS/mergify/bp/2.24-maintenance/pr-11717
docs: clarify syntax for escaping dollar curlies (backport #11717)
2025-01-10 19:35:02 +00:00
mergify[bot] d3cb938e07 Merge pull request #12191 from NixOS/mergify/bp/2.24-maintenance/pr-11723
doc: Explain why tryEval does not return the message (backport #11723)
2025-01-10 19:34:59 +00:00
mergify[bot] 51e41edbef Merge pull request #12190 from NixOS/mergify/bp/2.24-maintenance/pr-11813
fix(libstore-tests): remove use-after-free bug for `StringSource` (backport #11813)
2025-01-10 19:34:56 +00:00
mergify[bot] daf526f19d Merge pull request #12188 from NixOS/mergify/bp/2.24-maintenance/pr-11853
fix(libmain/common-args): do not exceed maximum allowed verbosity (backport #11853)
2025-01-10 19:34:54 +00:00
mergify[bot] d6217bc691 Merge pull request #12187 from NixOS/mergify/bp/2.24-maintenance/pr-11981
docker: Fix command "nix profile install", Don't require --impure (fix for 2.25) (backport #11981)
2025-01-10 19:34:51 +00:00
mergify[bot] 22c24a87a1 Merge pull request #12176 from NixOS/mergify/bp/2.24-maintenance/pr-12114
fix documentation of substring (backport #12114)
2025-01-10 17:21:50 +00:00
Eelco Dolstra f2bb7e043b Merge pull request #12185 from NixOS/mergify/bp/2.24-maintenance/pr-12016
Disable suid and atime on the /nix mount point on Darwin (backport #12016)
2025-01-10 15:16:44 +01:00
Eelco Dolstra 9a7196ce62 Merge pull request #12180 from NixOS/mergify/bp/2.24-maintenance/pr-12091
libstore: fixup unformatted uri when S3 getObject fails (backport #12091)
2025-01-10 15:11:08 +01:00
Eelco Dolstra 7a2b1cbd71 Merge pull request #12173 from NixOS/mergify/bp/2.24-maintenance/pr-12127
toJSON: re-throw serialization exception (backport #12127)
2025-01-10 13:32:42 +01:00
mergify[bot] 534473e90b Merge pull request #12170 from NixOS/mergify/bp/2.24-maintenance/pr-12155
Document `--max-freed` for `nix-collect-garbage` (backport #12155)
2025-01-10 11:05:01 +00:00
0x5a4 ce5c0c64af fix: bash mangles flake ref completion
(cherry picked from commit 088fa815d3)
2025-01-10 10:42:07 +00:00
Eelco Dolstra 931eb85f50 Add FIXME
(cherry picked from commit fac756fed4)
2025-01-10 10:36:06 +00:00
Eelco Dolstra a9020404cf select() -> poll() for Windows compat
(cherry picked from commit 270c8469d7)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 77a71c518f Update src/libutil/serialise.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit ce7cf4a2d3)
2025-01-10 10:36:06 +00:00
Eelco Dolstra 36f3fb72e9 withFramedSink(): Don't use a thread to monitor the other side
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.

  nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298

would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.

So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).

This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).

(cherry picked from commit 39daa4a0d3)
2025-01-10 10:36:05 +00:00
Jan Hrcek 5099768970 Fix few duplicate word occurrences (#11381)
(cherry picked from commit cb4b9be458)
2025-01-10 10:29:59 +00:00
Jeremy Kerfs 6f5684f6d1 add removal of ~root/.cache/nix to uninstall instructions (#11407)
(cherry picked from commit 4decd6f8b3)
2025-01-10 10:29:55 +00:00
Seggy Umboh d6488efeae Fix diskutil deleteVolume command in Uninstall guide (#11371)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
(cherry picked from commit 8e63dc4455)
2025-01-10 10:28:54 +00:00
Jeremy Kolb bc8611a29b Remote nullptr
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 8152c5c828)
2025-01-10 10:26:53 +00:00
Jeremy Kolb 1a21eb43af Pull fut.get() out of the lock
This is https://gerrit.lix.systems/c/lix/+/1462 by @jade_

see: https://git.lix.systems/lix-project/lix/issues/366
see: https://gerrit.lix.systems/c/lix/+/1462
(cherry picked from commit 46f6505865)
2025-01-10 10:26:53 +00:00
Eelco Dolstra bcf34e89c0 nix derivation add: Remove reference to "installable"
It doesn't operate on an installable, so don't mention it.

(cherry picked from commit 65f138f669)
2025-01-10 10:23:49 +00:00
Jordan Justen 0060a4ba5b doc, src: Fix various spelling typos
Signed-off-by: Jordan Justen <jljusten@gmail.com>
(cherry picked from commit 3b6cf350e7)
2025-01-10 10:22:51 +00:00
Robert Hensing 24e63b14e5 doc/nix fmt: Do not document behavior we do not control
This leads to confusion about what the command does.
E.g. https://github.com/NixOS/nix/issues/9359

- Move the description up

- Remove details about the individual formatters

(cherry picked from commit 8c4842de8c)
2025-01-10 10:22:50 +00:00
quatquatt 93e8e5b347 docs: Provide an example of updating multiple inputs with nix flake update
docs: Provide an example of updating multiple inputs with `nix flake update`
(cherry picked from commit 9e335ee3b9)
2025-01-10 10:22:45 +00:00
Jörg Thalheim 792099081c add description + example for nix flake archive
Update src/nix/flake-archive.md

Update src/nix/flake-archive.md

(cherry picked from commit f0a4f19087)
2025-01-10 10:21:45 +00:00
Noam Yorav-Raphael 971748ace9 nix-profile.sh.in: fix envvar condition
(cherry picked from commit 97fffd8765)

# Conflicts:
#	scripts/nix-profile.sh.in
2025-01-10 10:20:46 +00:00
Kevin Cox 0e421e7a35 Improve documentation of nix flake lock
The previous documentation was inaccurate, stating that it would not update existing inputs. However these inputs will be updated if they are outdated (for example the version of an existing input has been changed). The new text properly reflects this behaviour.

(cherry picked from commit d2f627dda9)
2025-01-10 10:18:03 +00:00
Robert Hensing 9ee4a85a99 doc/manual: Fix some broken fragments
(cherry picked from commit 071ddbed57)

# Conflicts:
#	doc/manual/src/language/operators.md
2025-01-10 10:17:08 +00:00
Geoffrey Thomas 24542a7b96 libstore: Make our sandbox pivot_root directory accessible to ourself
If you have the Nix store mounted from a nonlocal filesystem whose
exporter is not running as root, making the directory mode 000 makes it
inaccessible to that remote unprivileged user and therefore breaks the
build. (Specifically, I am running into this with a virtiofs mount using
Apple Virtualization.framework as a non-root user, but I expect the
same thing would happen with virtiofs in qemu on Linux as a non-root
user or with various userspace network file servers.)

Make the directory mode 500 (dr-x------) to make the sandbox work in
this use case, which explicitly conveys our intention to read and search
the directory.  The code only works because root can already bypass
directory checks, so this does not actually grant more permissions to
the directory owner / does not make the sandbox less secure.

(cherry picked from commit 5a794d9366)
2025-01-10 10:16:07 +00:00
Eman Resu 6304fa5cd1 docs: clarify syntax for escaping dollar curlies
(cherry picked from commit defff01a51)

# Conflicts:
#	doc/manual/source/language/string-literals.md
2025-01-10 10:14:55 +00:00
Robert Hensing 9c54fac5ff doc: Explain why tryEval does not return the message
(cherry picked from commit 90d257b771)
2025-01-10 10:12:37 +00:00
Sergei Zimmerman e863e6ab83 fix(libstore-tests): remove use-after-free bug for StringSource
Unfortunately `StringSource` class is very easy was very easy to misuse
because the ctor took a plain `std::string_view` which has a bad habit
of being implicitly convertible from an rvalue `std::string`. This lead
to unintentional use-after-free bugs.

This patch makes `StringSource` much harder to misuse by disabling the ctor
from a `std::string &&` (but `const std::string &` is ok).

Fix affected tests from libstore-tests.
Reformat those tests with clangd's range formatting since the diff is tiny
and it seems appropriate.

(cherry picked from commit 5bc8957c73)
2025-01-10 10:08:26 +00:00
Sergei Zimmerman dea80c4fec fix(libmain/common-args): do not exceed maximum allowed verbosity
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:

../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'

(cherry picked from commit b9f8c4af40)
2025-01-10 10:04:39 +00:00
h0nIg 26866eef0c docker: Fix command "nix profile install", Don't require --impure
(cherry picked from commit 61950c3375)
2025-01-10 09:58:36 +00:00
Graham Christensen 56332fda1c Disable suid and atime on the /nix mount point on Darwin
The Determinate Nix Installer has set nosuid and noatime in https://github.com/DeterminateSystems/nix-installer/pull/1338, and figured this perf and security improvement is worthy of upstreaming.

The /nix volume shouldn't have setuid binaries anyway, and filesystems seem to generally be noatime on macOS.
Further, the garbage collector doesn't use atime.

(cherry picked from commit 4137ead7a1)
2025-01-10 09:56:35 +00:00
Cole Helbling 4a7c7f12f5 libstore: fixup unformatted uri when S3 getObject fails
(cherry picked from commit b978fa8450)
2025-01-10 09:48:23 +00:00
Cole Helbling fed54f3550 tests/nixos/s3-binary-cache-store: disable default substituter so it runs faster
Since networking is disabled in these VMs, trying to talk to the default
cache.nixos.org slows the test down (since it can't resolve it).

(cherry picked from commit f0c1262d23)
2025-01-10 09:48:23 +00:00
Cole Helbling 6c1dfab347 tests/nixos/s3-binary-cache-store: test that "object does not exist" error message is properly formatted
(cherry picked from commit 535724fd79)
2025-01-10 09:48:22 +00:00
Philipp Otterbein 4b4b6377b2 fix documentation of substring
(cherry picked from commit 1e2cace5f1)
2025-01-10 09:44:14 +00:00
Philipp Otterbein 9936cb21ab toJSON: re-throw serialization exception
(cherry picked from commit 4a2310a3a0)
2025-01-10 09:42:14 +00:00
Domagoj Mišković 7c8276b799 Document --max-freed for nix-collect-garbage (#12155)
* Update nix-collect-garbage.md

Referencing issue at: https://github.com/NixOS/nix/issues/12132

Copied the description of `--max-freed` option from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44

(cherry picked from commit 5230d3ecc4)
2025-01-10 09:34:09 +00:00
mergify[bot] e0c8b0fc4f Merge pull request #12150 from NixOS/mergify/bp/2.24-maintenance/pr-12143
mergify: fix installer test name (backport #12143)
2025-01-07 19:42:11 +00:00
Jörg Thalheim 55dc2a9c9a mergify: fix installer test name
(cherry picked from commit 438a20427f)
2025-01-07 19:18:05 +00:00
Jörg Thalheim 112d0a7d85 Merge pull request #12144 from NixOS/mergify/bp/2.24-maintenance/pr-12046
CLI symlink fixes (backport #12046)
2025-01-07 08:06:58 +01:00
Jörg Thalheim b59b317e47 Merge pull request #12136 from NixOS/mergify/bp/2.24-maintenance/pr-11876
Filter OSC 8 correctly (backport #11876)
2025-01-07 07:22:37 +01:00
Jörg Thalheim 1eda38420f Merge pull request #11743 from NixOS/mergify/bp/2.24-maintenance/pr-11729
nix/tests: run test help.sh only if nix is built with documentation (backport #11729)
2025-01-07 07:21:51 +01:00
Robert Hensing caeec0404e test: Avoid regressing accidental use of weakly_canonical instead of makeParentCanonical
I'd messed up a rebase in my previous iteration, causing `weakly_canonical` to reappear,
but not trigger a test failure.

These two functions behave similarly when the argument is a path that points to a broken
symlink. `weakly_canonical` would not resolve it because the target doesn't exist, and
`makeParentCanonical` would not resolve it, because it never resolves the final path
element.
This new test case now also tests a valid symlink, "differentiating" the two.

(cherry picked from commit 4c74d679b6)
2025-01-07 07:20:55 +01:00
Jörg Thalheim f145dde10b makeParentCanonical: test case where parent is empty
(cherry picked from commit 5a5a86949a)

# Conflicts:
#	src/libutil-tests/file-system.cc
2025-01-07 07:20:55 +01:00
Robert Hensing 73dd3db039 test: Add hydraJobs.tests.functional_symlinked-home
(cherry picked from commit 628c11d237)
2025-01-07 07:20:55 +01:00
Robert Hensing e31d071793 fix: Resolve CLI parent symlinks before adding to store
Fixes https://github.com/NixOS/nix/issues/11941

(cherry picked from commit ddbbf53767)
2025-01-07 07:20:55 +01:00
Robert Hensing 42104d650a refactor: Don't re-construct SourcePath unnecessarily
(cherry picked from commit c0b64f3377)
2025-01-07 07:20:55 +01:00
Robert Hensing 129f8aaab2 doc: Document nix-store --add-fixed symlink behavior
Tested with

    nix run nix/2.3-maintenance#nix-store -- --add some_symlink
    nix run nix/2.3-maintenance#nix-store -- --add-fixed sha256 --recursive some_symlink

(cherry picked from commit 91e91f62fa)
2025-01-07 07:20:55 +01:00
Robert Hensing b9c30a9c87 fix: Handle symlinks and FIFOs in nix hash where possible
Fixes https://github.com/NixOS/nix/issues/11756
Fixes https://github.com/NixOS/nix/issues/11681

(cherry picked from commit 36563c69a4)
2025-01-07 07:20:55 +01:00
Robert Hensing fa3fd41063 Add makeParentCanonical()
(cherry picked from commit 69853c067c)
2025-01-07 07:20:55 +01:00
Marian Hammer e79d66ceef nix/tests: run test help.sh only if nix is built with documentation
tests/functional/help.sh calls nix-* commands with option --help
if nix is built without documentation the option --help throws an error
because the man page it wants to display is missing

(cherry picked from commit 85b0cd320a)
2025-01-07 05:55:13 +00:00
Jack Wilsdon e144f61ec0 Filter OSC 8 correctly
This allows Nix to use lowdown 1.2.0 which outputs OSC-8 links.

(cherry picked from commit 1301f8434d)
2025-01-07 05:33:19 +00:00
Jörg Thalheim 2c89c38fa1 Merge pull request #12074 from NixOS/mergify/bp/2.24-maintenance/pr-12059
installer: make sure we can always test the installer in ci and locally (backport #12059)
2025-01-07 00:27:16 +01:00
Robert Hensing 1a710e54dc .github/ci: Use fixed names
This lets us update "runs-on" without creating a mismatch with the
required checks that are configured for the repo in the github ui.
2025-01-07 00:04:32 +01:00
NAHO 365e0a9ab1 ci: lock macOS runner to macos-14 2025-01-07 00:04:32 +01:00
NAHO b3ea1e9b19 ci: update Ubuntu runner to ubuntu-24.04
Link: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
NAHO c78b818a0c ci: lock Ubuntu runner to ubuntu-22.04
Lock the Ubuntu runner to ubuntu-22.04 to avoid accidental updates [1]
and increase reproducibility.

[1]: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:04:32 +01:00
Jörg Thalheim 3109250faf mergify: updated required tests
(cherry picked from commit df3eb4f3f6)
2025-01-07 00:04:32 +01:00
Jörg Thalheim 2fbd0a943c installer: make sure we can always test the installer in ci and locally
Just now there is a dependency on cachix, which means we cannot test
the installer in CI if forks do not have the necessary secrets set up.
We replace this with a simple http server that serves the installer and
can be both used in CI and locally.

(cherry picked from commit 6162e1220e)
2025-01-07 00:04:32 +01:00
mergify[bot] bda59dee55 Merge pull request #12077 from NixOS/mergify/bp/2.24-maintenance/pr-12045
EvalState::realiseContext(): Allow access to the entire closure (backport #12045)
2024-12-31 15:33:23 +00:00
Robert Hensing 000db53f8f tests/functional: Add $config_nix 2024-12-31 15:14:56 +01:00
mergify[bot] e40d0352b2 Merge pull request #12118 from NixOS/mergify/bp/2.24-maintenance/pr-12115
libcmd: update to support lowdown-1.4 API (backport #12115)
2024-12-30 20:26:40 +00:00
Sergei Trofimovich c530b13321 libcmd: update to support lowdown-1.4 API
Upstream change
https://github.com/kristapsdz/lowdown/commit/bab1d75079ac3866fc14e1fa4825ae64a94a7c0e
moved a few fields from `lowdown_opts` toa  new `lowdown_opts_term`
struct. As a result the build started failing as:

    nix-cmd> [2/17] Compiling C++ object libnixcmd.so.p/markdown.cc.o
    nix-cmd> FAILED: libnixcmd.so.p/markdown.cc.o
    nix-cmd> g++ -Ilibnixcmd.so.p -I. -I.. -I/nix/store/b0bnrk5lacxbpgxgnc28r8q3wcazrgxj-nix-util-2.26.0pre-dev/include/nix -I/nix/store/cxnynq9ykyj4xxv6wf6dw7r0aw5x6n9k-libarchive-3.7.7-dev/include -I/nix/store/bfgjwkcb8snkizx578rzdahi75m8zyh4-nlohmann_json-3.11.3/include -I/nix/store/3sx8bq3sip6j2nv1m5xx4gbdp33v7iy6-nix-store-2.26.0pre-dev/include/nix -I/nix/store/sih2dgqzvsbv7p510lkfmas7s7wbsl4j-nix-fetchers-2.26.0pre-dev/include/nix -I/nix/store/68p8s20fsiiakj7nys7grbaixfnhsdzs-nix-expr-2.26.0pre-dev/include/nix -I/nix/store/gw7wknhzhfzzj9zww2kyi5xrzgf1ndki-boehm-gc-8.2.8-dev/include -I/nix/store/3jwb9j4vnsk5saq3wfyyp9il3mhs41l9-nix-flake-2.26.0pre-dev/include/nix -I/nix/store/8nwjvmq7m48v8g646jrxkikv6x47bc3m-nix-main-2.26.0pre-dev/include/nix -I/nix/store/rb0hzsw5wc1a7daizhpj824mbxlvijrq-lowdown-1.4.0-dev/include -I/nix/store/m388ywpk53fsp8r98brfd7nf1f5sskv0-editline-1.17.1-dev/include -fdiagnostics-color=always -D_GLIBCXX_ASSERTIONS=1 -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++2a -include config-util.hh -include config-store.hh -include config-expr.hh -include config-main.hh -include config-cmd.hh -Wdeprecated-copy -Werror=suggest-override -Werror=switch -Werror=switch-enum -Werror=unused-result -Wignored-qualifiers -Wimplicit-fallthrough -Wno-deprecated-declarations -O3 -fPIC -pthread -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -MD -MQ libnixcmd.so.p/markdown.cc.o -MF libnixcmd.so.p/markdown.cc.o.d -o libnixcmd.so.p/markdown.cc.o -c ../markdown.cc
    nix-cmd> ../markdown.cc: In function 'std::string nix::doRenderMarkdownToTerminal(std::string_view)':
    nix-cmd> ../markdown.cc:28:5: error: 'lowdown_opts' has no non-static data member named 'cols'
    nix-cmd>    28 |     };
    nix-cmd>       |     ^

The change adds version-based conditional to support both pre-1.4 and
1.4 forms of the initialization.

Closes: https://github.com/NixOS/nix/issues/12113
(cherry picked from commit edbfe863ce)
2024-12-30 19:16:43 +00:00
Eelco Dolstra a924db7d0b Resolve merge conflict 2024-12-16 18:01:52 +01:00
Eelco Dolstra f986f7e89b EvalState::realiseContext(): Allow access to the entire closure
Fixes #11030.

(cherry picked from commit 08361f031d)

# Conflicts:
#	src/libexpr/eval.cc
#	tests/functional/import-from-derivation.nix
#	tests/functional/import-from-derivation.sh
2024-12-16 15:36:40 +00:00
Jörg Thalheim 6a791e946f Merge pull request #11904 from NixOS/mergify/bp/2.24-maintenance/pr-11902
Fix issue #11892 (backport #11902)
2024-12-16 13:22:30 +01:00
mergify[bot] aa22f98b25 Merge pull request #12054 from NixOS/mergify/bp/2.24-maintenance/pr-12050
speed up ci by splitting off eval and build + fix ci with restricted namespaces (backport #12050)
2024-12-14 15:46:25 +00:00
Jörg Thalheim b3dc855569 ci: disable apparmor restrictions
For our tests we need to map the root user for some tests.
However ubuntu no longer allows this by default:
https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces

(cherry picked from commit 9f2b9256b6)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 7bafbfd50e binary-tarball: create tarball without listing files
This is producing quite a bit of log output in CI.

(cherry picked from commit 31b3a34e5b)
2024-12-14 15:24:48 +00:00
Jörg Thalheim 578f185daf speed up ci by splitting off eval and build
- This speeds up macOS builds from 30 minutes to 11 minutes (3x faster).
- Also improve error reporting e.g. printing out what actually failed to build.
- As a result we also no longer need swap.

(cherry picked from commit 691b1ea237)
2024-12-14 15:24:48 +00:00
Eelco Dolstra 628961d136 Bump version 2024-12-10 17:11:49 +01:00
Jörg Thalheim a4f978bd9b Merge pull request #12034 from NixOS/mergify/bp/2.24-maintenance/pr-11921
Make @nix json structured build log parsing warn instead of fail (backport #11921)
2024-12-10 05:37:28 +01:00
Robert Hensing 180031fb5a tests: Make unusual logging test conditional on fixed daemon version
(cherry picked from commit c783cd22ac)
2024-12-10 00:12:19 +01:00
Robert Hensing 29d91db99c Remove redundant warning: prefix from structured build log warning
(cherry picked from commit f3c722cab2)
2024-12-09 21:40:12 +00:00
Robert Hensing 6b2722fc84 test: Move unusual-logging to run only in logging test case
(cherry picked from commit 1421420e86)
2024-12-09 21:40:07 +00:00
Robert Hensing c951bb0689 Push log source description out of libutil and report build hook @nix warning correctly
(cherry picked from commit 03d4bfd852)
2024-12-09 21:40:04 +00:00
Robert Hensing 60eadfac7f tests/functional/dependencies.nix: Check that we tolerate syntax and type errors
(cherry picked from commit 1485937b89)
2024-12-09 21:40:00 +00:00
Linus Heckemann 9fed865d1d libutil: handle json builder log messages with unexpected format
Before this change, expressions like:

with import <nixpkgs> {};
runCommand "foo" {} ''
  echo '@nix {}' >&$NIX_LOG_FD
''

would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.

Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.

Fixes #544.

Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
(cherry picked from commit e55cd3beea710db727fd966f265a1b715b7285f3)
(cherry picked from commit ee03fd478e)
2024-12-09 21:39:58 +00:00
Robert Hensing 3cd4e9de1e tests/functional/dependencies.nix: Refactor, replace arcane let
(cherry picked from commit e82ff51726)
2024-12-09 21:39:55 +00:00
Eelco Dolstra 0f8fef49c3 Merge pull request #12017 from NixOS/mergify/bp/2.24-maintenance/pr-12013
nix hash convert: Support SRI hashes that lack trailing '=' characters (backport #12013)
2024-12-09 13:14:14 +01:00
Eelco Dolstra bd1b11ec66 Resolve conflict 2024-12-06 12:13:12 +01:00
Eelco Dolstra 6a8e9736e0 nix hash: Don't print 'nix hash' deprecation message
Fixes #11997.

(cherry picked from commit 408c2faf93)

# Conflicts:
#	src/nix/hash.cc
2024-12-05 22:30:24 +00:00
Eelco Dolstra dd79a490bf nix hash convert: Don't fail on uppercase base-16 hashes
(cherry picked from commit 33b645cedf)
2024-12-05 22:30:24 +00:00
Eelco Dolstra 00d0e63c61 nix hash convert: Support SRI hashes that lack trailing '=' characters
Fixes #11996.

(cherry picked from commit 52f1cd0595)
2024-12-05 22:30:24 +00:00
Jörg Thalheim fc11659148 Merge pull request #11932 from NixOS/mergify/bp/2.24-maintenance/pr-11915
Bump fetcher cache version (backport #11915)
2024-11-23 09:31:55 +01:00
Robert HensingandEelco Dolstra bf67741ff1 Bump fetcher cache version
We're getting more reports in https://github.com/NixOS/nix/issues/10985

It appears that something hasn't gone right process-wise.
I find this mistake not to be worth investigating, but rather something
to pay attention to going forward.

Let's nip this in the bud.

Closes https://github.com/NixOS/nix/issues/10985

(cherry picked from commit e948c8e033)

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-11-23 09:05:52 +01:00
Robert Hensing 403f388fdb Fix issue 11892
It seems that I copied the expression for baseDir thoughtlessly and
did not come back to it.

- `baseDir` was only used in the `fromArgs` branch.
- `fromArgs` is true when `packages` is true.

(cherry picked from commit c4b95dbdd1)
2024-11-18 21:21:13 +00:00
Eelco Dolstra 8e8a572d37 Merge pull request #11889 from NixOS/mergify/bp/2.24-maintenance/pr-11884
nix path-info: Don't write to std::cout directly (backport #11884)
2024-11-15 16:29:29 +01:00
Eelco Dolstra 3bfc35ba74 nix path-info: Don't write to std::cout directly
This interferes with the progress bar, resulting in output like

  evaluating derivation 'git+file:///home/eelco/Dev/nix-master#packages.x86_64-linux.default'/nix/store/zz8v96j5md952x0mxfix12xqnvq5qv5x-nix-2.26.0pre20241114_a95f6ea.drv

(cherry picked from commit 33a0fa882f)
2024-11-15 14:08:22 +00:00
Eelco Dolstra aaeaa4e133 Merge pull request #11879 from NixOS/mergify/bp/2.24-maintenance/pr-11878
Remove warning that interpretation of .drv has changed (backport #11878)
2024-11-13 18:23:23 +01:00
Robert Hensing 22e1613814 Remove warning that interpretation of .drv has changed
This was first tagged as 2.15.0, 1½ years ago; plenty of time for
everyone to catch up.

By now, the warning is causing more confusion than that it is helpful,
because passing a `.drv` or `drvPath` has legitimate use cases.

(cherry picked from commit f534a7a524)
2024-11-13 16:12:58 +00:00
Jörg Thalheim ef21dfa221 Merge pull request #11838 from NixOS/mergify/bp/2.24-maintenance/pr-11837
fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug (backport #11837)
2024-11-08 16:02:45 +01:00
Sergei Zimmerman c556c205a9 fix(libutil/posix-source-accessor.cc): get rid of use-after-move bug
Naming class member variables the same as constructor arguments is a very
slippery slope because of how member variable names get resolved. Compiler
is not very helpful here and we need static analysis to forbid this kind of
stuff.

The following example illustrates the cause quite well:

```cpp

struct B {
    B(int) {}
};

struct A {
    A(int b): b([&](){
        return b;
        static_assert(std::is_same_v<decltype(b), int>);
    }()) {
       static_assert(std::is_same_v<decltype(b), int>);
    }
    void member() {
        static_assert(std::is_same_v<decltype(b), B>);
    }
    B b;
};

int main() {
    A(1).member();
}
```

From N4861 6.5.1 Unqualified name lookup:

> In all the cases listed in [basic.lookup.unqual], the scopes are searched
> for a declaration in the order listed in each of the respective categories;
> name lookup ends as soon as a declaration is found for the name.
> If no declaration is found, the program is ill-formed.

In the affected code there was a use-after-move for all accesses in the constructor
body, but this UB wasn't triggered.

These types of errors are trivial to catch via clang-tidy's [clang-analyzer-cplusplus.Move].

(cherry picked from commit 3e0129ce3b)
2024-11-08 13:58:17 +00:00
Jörg Thalheim 6e095ddef9 Merge pull request #11823 from NixOS/mergify/bp/2.24-maintenance/pr-11821
initRepoAtomically: Catch directory_not_empty (backport #11821)
2024-11-07 08:03:52 +01:00
Robert Hensing c3e2419e73 initRepoAtomically: Catch directory_not_empty
(cherry picked from commit 388271e8ec)
2024-11-07 06:23:15 +00:00
Eelco Dolstra 960c288fa0 Merge pull request #11777 from NixOS/mergify/bp/2.24-maintenance/pr-11759
fix: make sure directory exists before using `ln` (backport #11759)
2024-11-01 16:56:25 +01:00
Valentin Gagarin d14b46f9ba Merge pull request #11783 from NixOS/mergify/bp/2.24-maintenance/pr-11775
doc/rl-2.19: add entry for always-allow-substitutes option (backport #11775)
2024-11-01 13:12:58 +01:00
Bjørn Forsman c9325fe343 doc/rl-2.19: add entry for always-allow-substitutes option (#11775)
* doc/rl-2.19: add entry for always-allow-substitutes option

Fixes https://github.com/NixOS/nix/issues/9427.

(cherry picked from commit 020dbac0e0)
2024-11-01 10:57:56 +00:00
Pol Dellaiera 2a2d484986 fix: make sure directory exists before using ln
(cherry picked from commit 5f71ebb956)
2024-11-01 10:19:42 +00:00
Eelco Dolstra 9728a4ac94 maintainers/upload-release.pl: Delete temporary directories when we're done
(cherry picked from commit 02bb633a58)
2024-10-31 12:12:17 +01:00
Eelco Dolstra 75b00e52ac 'build' is now 'build.nix'
(cherry picked from commit 617e711820)
2024-10-31 12:12:12 +01:00
Eelco Dolstra 6044907c13 Bump version 2024-10-31 12:11:43 +01:00
Eelco Dolstra 597fcc98e1 Merge remote-tracking branch 'nix-ghsa-wf4c-57rh-9pjg/advisory-fix-1-2.24' into 2.24-maintenance 2024-10-30 21:44:39 +01:00
Eelco Dolstra 82abed901f Merge pull request #11769 from DeterminateSystems/fix-release-notes
Fix 2.24 release notes
2024-10-30 16:12:43 +01:00
Eelco Dolstra f9180f12c4 release notes: 2.24.9 2024-10-30 15:32:37 +01:00
John Ericson baa7565710 Merge pull request #11763 from NixOS/mergify/bp/2.24-maintenance/pr-11751
Add support for `utimensat` as an alternative to `lutimes` (backport #11751)
2024-10-28 15:27:39 +01:00
John Ericson 7e6e75fd48 Merge pull request #11762 from NixOS/mergify/bp/2.24-maintenance/pr-11750
Fix building on OpenBSD (backport #11750)
2024-10-28 15:27:23 +01:00
Artemis Tosini 803943fce4 Add support for utimensat as an alternative to lutimes
OpenBSD doesn't support `lutimes`, but does support `utimensat` which
subsumes it. In fact, all the BSDs, Linux, and newer macOS all support
it. So lets make this our first choice for the implementation.

In addition, let's get rid of the `lutimes` `ENOSYS` special case. The
Linux manpage says

> ENOSYS
>
> The kernel does not support this call; Linux 2.6.22 or later is
> required.

which I think is the origin of this check, but that's a very old version
of Linux at this point. The code can be simplified a lot of we drop
support for it here (as we've done elsewhere, anyways).

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
(cherry picked from commit d023202811)
2024-10-28 10:27:39 +00:00
Artemis Tosini 0ae90918db package.nix: Disable GC on OpenBSD
Nix fails to build on OpenBSD with a linking error due to a non-found
symbol in boehm-gc. Just disable the GC until we can find a proper
workaround.

(cherry picked from commit fecc1ca205)
2024-10-28 10:27:37 +00:00
Artemis Tosini d6ece7e94a Fix OpenBSD build with Makefiles
OpenBSD dynamic libraries never link to libc directly.
Instead, they have undefined symbols for all libc functions they use
that ld.so resolves to the libc referred to in the main executable.

Thus, disallowing undefined symbols will always fail

(cherry picked from commit c49bff2434)
2024-10-28 10:27:37 +00:00
Robert Hensing 50f83e4bbd local-derivation-goal: Move builder preparation to non-builtin code path 2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt 047ee50db2 local-derivation-goal: Refactor
This works because the `builder` and `args` variables are only used
in the non-builtin code path.

Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert HensingandThéophane Hufschmitt ae7a2ea741 local-derivation-goal: Print sandbox error detail on darwin
Co-Authored-By: Théophane Hufschmitt <theophane.hufschmitt@tweag.io>
2024-10-22 17:56:06 +02:00
Robert Hensing f8a1a149c7 packaging: Add darwin -lsandbox in meson 2024-10-22 17:56:06 +02:00
Puck Meerburg 170242cf0c fix: Run all derivation builders inside the sandbox on macOS 2024-10-22 17:56:06 +02:00
Eelco Dolstra 7718688f52 Merge pull request #11730 from NixOS/mergify/bp/2.24-maintenance/pr-11713
fix env-vars beeing written to `/tmp` (backport #11713)
2024-10-22 15:01:56 +02:00
Jörg Thalheim 0e9b04a66e fix env-vars beeing written to /tmp
This overall seems like insecure tmp file handling to me. Because other
users could replace files in /tmp with a symlink and make the nix-shell
override other files.

fixes https://github.com/NixOS/nix/issues/11470

(cherry picked from commit 2105574702)
2024-10-22 12:13:47 +00:00
Eelco Dolstra 411ec33db3 Merge pull request #11692 from NixOS/mergify/bp/2.24-maintenance/pr-11690
Handle tarballs where directory entries are not contiguous (backport #11690)
2024-10-14 16:01:43 +02:00
Eelco Dolstra 31df105f45 Merge pull request #11691 from NixOS/mergify/bp/2.24-maintenance/pr-11677
builtins.fetchurl: Fix segfault on s3:// URLs (backport #11677)
2024-10-14 15:23:49 +02:00
Eelco Dolstra 57ace600af Add a test
(cherry picked from commit a7b9877da9)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 9da1300617 Handle tarballs where directory entries are not contiguous
I.e. when not all entries underneath a directory X follow eachother,
but there is some entry Y that isn't a child of X in between.

Fixes #11656.

(cherry picked from commit 4012954b59)
2024-10-14 12:51:03 +00:00
Eelco Dolstra 1294442c6c Add assert
(cherry picked from commit d2f4d07619)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 339236d32e Make S3 downloads slightly more interruptable
(cherry picked from commit d38f62f64d)
2024-10-14 14:44:28 +02:00
Eelco Dolstra 4912a9e7fd builtins.fetchurl: Fix segfault on s3:// URLs
Also, add an activity to show that we're downloading an s3:// file.

Fixes #11674.

(cherry picked from commit 0500fba56a)
2024-10-14 14:44:28 +02:00
Eelco Dolstra d80bf54e3b Add a VM test for S3BinaryCacheStore
Fixes #11238.

(cherry picked from commit 2950f9e18a)
2024-10-14 14:44:28 +02:00
Robert Hensing f1dc3b7d55 Merge pull request #11649 from NixOS/mergify/bp/2.24-maintenance/pr-11610
fix passing CA files into builtins:fetchurl sandbox (backport #11610)
2024-10-13 12:43:33 +02:00
Jörg Thalheim 5f1b132187 tests/nixos/fetchurl: drop unused variables
(cherry picked from commit 410853ddcf)
2024-10-07 12:45:04 +00:00
Puck Meerburg 742eb0f815 fix passing CA files into builtins:fetchurl sandbox
This patch has been manually adapted from
https://github.com/lix-project/lix/commit/14dc84ed03f1b7e5a41bb6fdce00916faab32b60

Tested with:

$ NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 16:57:50 after 1s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> error:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        … writing file '/nix/store/0zynn4n8yx59bczy1mgh1lq2rnprvvrc-google.com'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com>        error: unable to download 'https://google.com': Problem with the SSL CA cert (path? access rights?) (77) error setting certificate file: /nix/store/nlgbippbbgn38hynjkp1ghiybcq1dqhx-nss-cacert-3.101.1/etc/ssl/certs/ca-bundle.crt
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: builder for '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv' failed with exit code 1

Now returns:

nix-env % NIX_SSL_CERT_FILE=$(nix-build '<nixpkgs>' -A cacert)/etc/ssl/certs/ca-bundle.crt nix-build --store $(mktemp -d) -E 'import <nix/fetchurl.nix> { url = https://google.com; }'
Finished at 17:05:48 after 0s
warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
this derivation will be built:
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
  /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
google.com> building '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv'
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
nix-output-monitor error: DerivationReadError /nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv: openFile: does not exist (No such file or directory)
error: hash mismatch in fixed-output derivation '/nix/store/4qljhy0jj2b0abjzpsbyarpia1bqylwc-google.com.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

(cherry picked from commit c1ecf0bee9)
2024-10-07 12:45:03 +00:00
Robert Hensing 13e200df45 Merge pull request #11601 from Mic92/git-cache-backport
[2.24] create git caches atomically
2024-09-30 13:11:22 +02:00
Eelco Dolstra a1d841bf2c Bump version 2024-09-28 00:05:03 +02:00
Eelco Dolstra 048cfe51c9 Merge pull request #11604 from NixOS/mergify/bp/2.24-maintenance/pr-11600
HttpBinaryCacheStore::getFile(): Fix uncaught exception (backport #11600)
2024-09-27 13:26:21 +02:00
Eelco Dolstra 15a2b49115 HttpBinaryCacheStore::getFile(): Fix uncaught exception
This method is marked as `noexcept`, but `enqueueFileTransfer()` can
throw `Interrupted` if the user has hit Ctrl-C or if the `ThreadPool`
that the thread is a part of is shutting down.

(cherry picked from commit 4566854981)
2024-09-27 10:38:03 +00:00
Jörg Thalheim 34fd00accc create git caches atomically
When working on speeding up the CI,
I triggered a race condition in the creation of the tarball cache.
This code now instead will ensure that half-initialized repositories
are no longer visible to any other nix process.

This is the error message that I got before:

error: opening Git repository '"/Users/runner/.cache/nix/tarball-cache"': could not find repository at '/Users/runner/.cache/nix/tarball-cache'
(cherry picked from commit 12d5b2cfa1)
2024-09-27 10:06:58 +02:00
Eelco Dolstra b23812a59c Bump version 2024-09-26 03:25:40 +02:00
Eelco Dolstra 618a0cc987 Merge pull request #11592 from NixOS/mergify/bp/2.24-maintenance/pr-11585
builtin:fetchurl: Enable TLS verification (backport #11585)
2024-09-26 01:04:39 +02:00
Eelco Dolstra ba81598017 Resolve conflict 2024-09-26 00:17:03 +02:00
Eelco Dolstra e87be60055 Typo
(cherry picked from commit ef8987955b)
2024-09-26 00:16:17 +02:00
Eelco Dolstra 345a264a39 Add release note
(cherry picked from commit 7b39cd631e)
2024-09-25 21:55:36 +00:00
Eelco Dolstra ee6a5faf4b Add a test for builtin:fetchurl cert verification
(cherry picked from commit f2f47fa725)

# Conflicts:
#	tests/nixos/default.nix
2024-09-25 21:55:36 +00:00
Eelco Dolstra d4824c8ff7 builtin:fetchurl: Enable TLS verification
This is better for privacy and to avoid leaking netrc credentials in a
MITM attack, but also the assumption that we check the hash no longer
holds in some cases (in particular for impure derivations).

Partially reverts https://github.com/NixOS/nix/commit/5db358d4d78aea7204a8f22c5bf2a309267ee038.

(cherry picked from commit c04bc17a5a)
2024-09-25 21:55:36 +00:00
Eelco Dolstra b4fcd27590 Merge pull request #11578 from Mic92/key-backport
[2.24-maintainence] Ensure error messages don't leak private key
2024-09-24 13:45:43 +02:00
John Ericson 082f6bb35d Ensure error messages don't leak private key
Since #8766, invalid base64 is rendered in errors, but we don't actually
want to show this in the case of an invalid private keys.

Co-Authored-By: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit 2b6b03d8df)
2024-09-24 06:39:03 +02:00
John Ericson 1e03ea386b Revert "base64Decode: clearer error message when an invalid character is detected"
We have a safer way of doing this.

This reverts commit dc3ccf02bf.

(cherry picked from commit d0c351bf43)
2024-09-24 06:31:50 +02:00
John Ericson b523e4de34 Merge pull request #11571 from NixOS/mergify/bp/2.24-maintenance/pr-11390
Don't refer to public keys as secret keys in error (backport #11390)
2024-09-23 18:50:03 -04:00
Alyssa Ross 563dedcf64 Don't refer to public keys as secret keys in error
This constructor is used for public keys as well.

(cherry picked from commit 9cc550d652)
2024-09-23 22:00:10 +00:00
Eelco Dolstra a7fdef6858 Bump version 2024-09-20 01:19:15 +02:00
Eelco Dolstra b5154deba3 Merge pull request #11553 from NixOS/mergify/bp/2.24-maintenance/pr-11548
Fix missing GC root in zipAttrsWith (backport #11548)
2024-09-19 22:09:56 +02:00
Eelco Dolstra ecd83dc155 Use HAVE_BOEHMGC
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 4449b0da74)
2024-09-19 19:04:17 +00:00
Eelco Dolstra 5b5e1920eb Fix missing GC root in zipAttrsWith
My SNAFU was that I assumed that all the `Value *`s we put in
`attrsSeen` are already reachable (which they are), but I forgot about
the `elems` pointer in `ListBuilder`.

Fixes #11547.

(cherry picked from commit 0c2fdd2f3c)
2024-09-19 19:04:17 +00:00
Valentin Gagarin fc1d6b2f03 Merge pull request #11521 from NixOS/mergify/bp/2.24-maintenance/pr-8766
base64Decode: clearer error message when an invalid character is detected (backport #8766)
2024-09-19 11:29:21 +02:00
Brian McGee 9941f620c4 base64Decode: clearer error message when an invalid character is detected
Output the offending string in its entirety to provide context.

Closes #8479

(cherry picked from commit dc3ccf02bf)
2024-09-17 14:03:19 +00:00
Valentin Gagarin 5b2a8c223e Merge pull request #11497 from Mic92/mergify/bp/2.24-maintenance/pr-11378 2024-09-17 07:09:32 +02:00
mergify[bot]andTravis A. Everett 1b076b4f84 doc: add admonitions for macOS 15 Sequoia update (#11487) (#11509)
The impending release of macOS 15 Sequoia will break many existing nix
installs on macOS, which may lead to an increased number of people who
are looking to try to reinstall Nix without noticing the open/pinned
issue (#10892) that explains the problem and outlines how to migrate
existing installs.

These admonitions are a short-term measure until we are over the hump
and support volumes dwindle.

(cherry picked from commit 48477d4a3e)

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2024-09-16 16:03:05 +02:00
Robert Hensing d9ef3dd012 Merge pull request #11484 from NixOS/mergify/bp/2.24-maintenance/pr-11473
Fix making the build directory kept by `keep-failed` readable (backport #11473)
2024-09-16 12:46:23 +02:00
Robert Hensing f9714bac34 Merge pull request #11456 from NixOS/mergify/bp/2.24-maintenance/pr-11321
replace backport github action with mergify (backport #11321)
2024-09-16 12:41:14 +02:00
Jörg Thalheim 684a690480 update filesystem-errors changelog to 2.24 release 2024-09-13 14:20:34 +02:00
John Ericson 4354d90384 tweak unpack channel built-in, std::filesystem::path for tarball
(cherry picked from commit 193dc49097)
2024-09-13 14:11:36 +02:00
Jörg Thalheim 60001b1936 add release notes for filesystem fixes
Update doc/manual/rl-next/filesystem-errors.md

Co-authored-by: John Ericson <git@JohnEricson.me>
(cherry picked from commit 04ce0e648a)
2024-09-13 14:11:36 +02:00
Jörg Thalheim c84fc0120f builtins.unpackChannel: wrap filesystem errors and sanitize channelName
Otherwise these errors are not caught correctly

(cherry picked from commit 70c52d72f4)
2024-09-13 14:11:36 +02:00
Jörg Thalheim cd97688bce builtins.readDir: fix nix error trace on filesystem errors
Before:

nix-env % ./src/nix/nix eval --impure --expr 'let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f' --show-trace
error: filesystem error: directory iterator cannot open directory: No such file or directory [/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo]

After:

error:
       … while calling the 'readDir' builtin
         at «string»:1:9:
            1| let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f
             |         ^

       error: reading directory '/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo': No such file or directory

(cherry picked from commit 22ba4dc78d)
2024-09-13 11:56:41 +00:00
Eelco Dolstra 07909de6ed Merge pull request #11493 from NixOS/mergify/bp/2.24-maintenance/pr-11423
Git fetcher: Ignore .gitmodules entries that are not submodules (backport #11423)
2024-09-12 20:30:45 +02:00
Eelco Dolstra 751907dc8a Git fetcher: Ignore .gitmodules entries that are not submodules
Fixes #10739.

(cherry picked from commit 9d24080090)
2024-09-12 16:16:36 +00:00
Eelco Dolstra d9dd6c62d6 Merge pull request #11486 from NixOS/mergify/bp/2.24-maintenance/pr-11466
Git fetcher: Don't update mtime of ref file if fetching by rev (backport #11466)
2024-09-11 19:59:10 +02:00
Eelco Dolstra 97c5ac5752 Git fetcher: Don't update mtime of ref file if fetching by rev
This fixes the warning

  $ nix eval --store /tmp/nix --expr 'builtins.fetchTree { type = "git"; url = "https://github.com/DeterminateSystems/attic"; ref = "fixups-for-magic-nix-cache"; rev = "635753a2069d4b8228e846dc5c09ad361c75cd1a"; }'
  warning: could not update mtime for file '/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache': error: changing modification time of '"/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache"': No such file or directory

When we're fetching by rev, that file doesn't necessarily exist, and we
don't care about it anyway.

(cherry picked from commit b80b091bac)
2024-09-11 15:37:36 +00:00
Artturin 40461a8e0e Fix making the build directory kept by keep-failed readable
Caused by 1d3696f0fb

Without this fix the kept build directory is readable only by root

```
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5
drwx------ root root 60 B Wed Sep 11 00:09:48 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/

$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build
drwxr-xr-x nixbld1 nixbld 80 B Wed Sep 11 00:09:58 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build/
```

(cherry picked from commit ebebe626ff)
2024-09-11 12:56:18 +00:00
Eelco Dolstra 0f825b38f4 Bump version 2024-09-10 13:45:04 +02:00
Eelco Dolstra eb11c14998 Merge commit from fork
Fix unsafe NAR unpacking (2.24)
2024-09-10 12:42:55 +02:00
Eelco Dolstra a6ad5565ef Improve use-case-hack description slightly
(cherry picked from commit 5ca2f58798)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 2e1cb495c1 Typo
(cherry picked from commit 4cfa59fdb3)
2024-09-10 09:51:50 +02:00
Eelco Dolstra e25410c788 Test that deserializing regular files / symlinks is exclusive
(cherry picked from commit 52ba3cc5ea)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 25510ba66f RestoreSink::createDirectory(): Use append()
On macOS, `mkdir("x/')` behaves differently than `mkdir("x")` if `x` is
a dangling symlink (the formed succeed while the latter fails). So make
sure we always strip the trailing slash.

(cherry picked from commit 9fcb588dd8)
2024-09-10 09:51:50 +02:00
Eelco Dolstra d9b60b3902 Fix test on macOS
(cherry picked from commit 21dcbd7e83)
2024-09-10 09:51:50 +02:00
Eelco Dolstra a041688133 Test that deserializing NARs with names with equal Unicode normal forms fails on macOS
The test is based on the one by @puckipedia but with the file names
swapped to make them sorted.

(cherry picked from commit 7a765a6aaf)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 1288970496 Detect NAR directory entries that collide with another path after case-hacking
The test was made by @puckipedia.

(cherry picked from commit 3557587381)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 0cfc9bf133 More tests
(cherry picked from commit 77c090cdbd)
2024-09-10 09:51:50 +02:00
Eelco Dolstra f160d3ac68 Test that nix-store --restore fails if the output already exists
This restores the behaviour from before the std::filesystem
refactorings.

(cherry picked from commit da1ad28912)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 6187ee468f Add test case for NARs with duplicate directory entries
This test was made by @puckipedia.

(cherry picked from commit 83d5b32803)
2024-09-10 09:51:50 +02:00
Eelco Dolstra 12fa019ae5 NAR parser: Fix check for duplicate / incorrectly sorted entries
"prevName" was always empty because it was declared in the wrong scope.

(cherry picked from commit 495d32e1b8)
2024-09-10 09:51:50 +02:00
Jörg Thalheim f0cffa7300 replace backport github action with mergify
The current backport action cannot automerge because
the github action bot does not trigger github CI actions.
Mergify instead does not have this limitation and can also
use a merge queue.

On top we have now a declarative configuration to allow
contributers to add new tests to required without having access
to the github org.

An example pull request and backport can be seen here:

https://github.com/Mic92/nix-1/pull/4

and here:

https://github.com/Mic92/nix-1/pull/5

To complete the setup the mergify app must be enabled for this repository.
It's already installed in the nixos organization for nixos-hardware and
other repositories.

(cherry picked from commit 80f20fa4cb)
2024-09-09 16:12:24 +00:00
Robert Hensing ae486b2910 Merge pull request #11446 from NixOS/mergify/bp/2.24-maintenance/pr-11285
fetchers::downloadTarball(): Return a cacheable accessor (backport #11285)
2024-09-09 18:05:32 +02:00
Eelco Dolstra 437f7a0042 fetchers::downloadTarball(): Return a cacheable accessor
downloadTarball() is used by `-I foo=<url>` etc. fetchToStore() needs
the accessor to have a fingerprint to enable caching.

Fixes #11271.

(cherry picked from commit 9f6ee93f48)
2024-09-09 12:46:07 +00:00
Eelco Dolstra b53b07b30b Merge pull request #11421 from NixOS/mergify/bp/2.24-maintenance/pr-10919
install-darwin: fix _nixbld uids for macOS sequoia (backport #10919)
2024-09-04 18:55:18 +02:00
Emily 8d0414d682 install-darwin: increment base UID by 1 (#15)
(cherry picked from commit 11cf29b15c)
2024-09-03 23:59:16 +00:00
Travis A. Everett c5a0e624d9 install-darwin: move nixbld gid to match first UID
(cherry picked from commit 75567423fb)
2024-09-03 23:59:15 +00:00
Travis A. Everett 0679505d8c install-darwin: fix _nixbld uids for macOS sequoia
Starting in macOS 15 Sequoia, macOS daemon UIDs are encroaching on our
default UIDs of 301-332. This commit relocates our range up to avoid
clashing with the current UIDs of 301-304 and buy us a little time
while still leaving headroom for people installing more than 32 users.

(cherry picked from commit df36ff0d1e)
2024-09-03 23:59:15 +00:00
Eelco Dolstra ea7abb58b5 Bump version 2024-09-03 16:51:36 +02:00
Eelco Dolstra f1ab41b2bc Merge pull request #11408 from NixOS/mergify/bp/2.24-maintenance/pr-11402
Respect max-substitution-jobs again (backport #11402)
2024-09-03 14:23:49 +02:00
Eelco Dolstra c21f664e82 "unsigned" -> size_t
Slight cleanup.

(cherry picked from commit b7acd1c414)
2024-09-03 11:36:40 +00:00
Eelco Dolstra 0c25bea7cc Respect max-substitution-jobs again
This broke in #11005. Any number of PathSubstitutionGoals would
be woken up by a single build slot becoming available. If there
are a lot of substitution goals active, this could lead to us
running out of file descriptors (especially on macOS where the
default limit is 256).

(cherry picked from commit a33cb8af56)
2024-09-03 11:36:39 +00:00
Eelco Dolstra 9d8669b14a Bump version 2024-08-23 16:15:11 +02:00
Eelco Dolstra cb0439f0c2 Merge pull request #11360 from DeterminateSystems/backport-11152
[Backport 2.24-maintenance] addToStore(): Evaluate on the main stack
2024-08-23 15:08:56 +02:00
Eelco Dolstra 30a57328d2 Backport https://github.com/NixOS/nix/pull/11152 2024-08-23 13:15:30 +02:00
Robert Hensing 79a43160ca Merge pull request #11334 from NixOS/backport-11329-to-2.24-maintenance
[Backport 2.24-maintenance] fix: check to see if there are any lines before
2024-08-19 16:38:44 +02:00
Tom Bereknyei 7befd60c01 fix: check to see if there are any lines before
(cherry picked from commit 59db8fd62b)
2024-08-19 13:40:38 +00:00
Eelco Dolstra 90fb4e8890 Bump version 2024-08-19 12:46:17 +02:00
tomberek 3ac5d736e2 Merge pull request #11319 from NixOS/backport-11270-to-2.24-maintenance
[Backport 2.24-maintenance] libstore: fix port binding in __darwinAllowLocalNetworking sandbox
2024-08-17 03:17:29 -04:00
Andrew Marshall 4e707b8e57 libstore: fix port binding in __darwinAllowLocalNetworking sandbox
In d60c3f7f7c, this was changed to close a
hole in the sandbox. Unfortunately, this was too restrictive such that it
made local port binding fail, thus making derivations that needed
`__darwinAllowLocalNetworking` gain nearly nothing, and thus largely
fail (as the primary use for it is to enable port binding).

This unfortunately does mean that a sandboxed build process can, in
coordination with an actor outside the sandbox, escape the sandbox by
binding a port and connecting to it externally to send data. I do not
see a way around this with my experimentation and understanding of the
(quite undocumented) macOS sandbox profile API. Notably it seems not
possible to use the sandbox to do any of:

- Restrict the remote IP of inbound network requests
- Restrict the address being bound to

As such, the `(local ip "*:*")` here appears to be functionally no
different than `(local ip "localhost:*")` (however it *should* be
different than removing the filter entirely, as that would make it also
apply to non-IP networking). Doing `(allow network-inbound (require-all
(local ip "localhost:*") (remote ip "localhost:*")))` causes listening
to fail.

Note that `network-inbound` implies `network-bind`.

(cherry picked from commit 00f6db36fd)
2024-08-17 03:17:45 +00:00
Eelco Dolstra 20cae372f4 Merge pull request #11314 from cole-h/backport-2.24-update-nixpkgs-input-fix-darwin
[2.24] Update nixpkgs input to fix darwin ccache evaluation, have CI check that all outputs on all systems evaluate
2024-08-16 20:48:49 +02:00
Cole Helbling d550139191 ci: check that all outputs for all systems can evaluate
(cherry picked from commit aa3d35c1f4)
2024-08-16 11:46:29 -07:00
Cole Helbling 5b62a1dbd6 flake.lock: Update
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/63d37ccd2d178d54e7fb691d7ec76000740ea24a?narHash=sha256-7cCC8%2BTdq1%2B3OPyc3%2BgVo9dzUNkNIQfwSDJ2HSi2u3o%3D' (2024-07-21)
  → 'github:NixOS/nixpkgs/c3d4ac725177c030b1e289015989da2ad9d56af0?narHash=sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz%2BNG82pbdg%3D' (2024-08-15)

(cherry picked from commit 8866d2cd83)
2024-08-16 11:46:24 -07:00
Eelco Dolstra 450252c92c Bump version 2024-08-08 17:21:00 +02:00
Eelco Dolstra 4036c3aafb Bump version 2024-08-08 15:02:48 +02:00
Robert Hensing 935bf1157d Merge pull request #11267 from NixOS/backport-11244-to-2.24-maintenance
[Backport 2.24-maintenance] allow to c api with older c versions
2024-08-08 01:43:24 +02:00
Jörg Thalheim b1941c9f8a allow to c api with older c versions
In the FFI world we have many tools that are not gcc/clang and therefore
not always support the latest C standard. This fixes support with cffi
i.e. used in https://github.com/tweag/python-nix

(cherry picked from commit 739418504c)
2024-08-07 23:12:17 +00:00
Eelco Dolstra 40832b0a95 Merge pull request #11262 from NixOS/backport-11257-to-2.24-maintenance
[Backport 2.24-maintenance] PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
2024-08-06 11:20:31 +02:00
Eelco Dolstra fa78d7f72f PathSubstitutionGoal: Fix spurious "failed" count in the progress bar
It is not an error if queryPathInfo() indicates that a path does not
exist in the substituter.

Fixes #11198. This was broken in 846869da0e.

(cherry picked from commit 0a00bd07b2)
2024-08-06 08:33:46 +00:00
Eelco Dolstra 2382a52c84 Merge pull request #11239 from NixOS/backport-11237-to-2.24-maintenance
[Backport 2.24-maintenance] Fix the S3 store
2024-08-01 17:56:18 +02:00
Eelco Dolstra fe6a7c805c Fix the S3 store
It was failing with:

   error: AWS error fetching 'nix-cache-info': The specified bucket does not exist

because `S3BinaryCacheStoreImpl` had a `bucketName` field that
shadowed the inherited `bucketName from `S3BinaryCacheStoreConfig`.

(cherry picked from commit 9b5b7b7963)
2024-08-01 15:46:45 +00:00
Eelco Dolstra 0a167ffd1f Bump version 2024-08-01 10:41:11 +02:00
Eelco Dolstra 206e32e2d7 Mark release 2024-07-31 23:37:43 +02:00
699 changed files with 12548 additions and 12856 deletions

No files matched your search

-1
View File
@@ -31,4 +31,3 @@ AlwaysBreakBeforeMultilineStrings: true
IndentPPDirectives: AfterHash
PPIndentWidth: 2
BinPackArguments: false
BreakBeforeTernaryOperators: true
-19
View File
@@ -1,22 +1,3 @@
<!--
IMPORTANT
Nix is a non-trivial project, so for your contribution to be successful,
it really is important to follow the contributing guidelines:
https://github.com/NixOS/nix/blob/master/CONTRIBUTING.md
Even if you've contributed to open source before, take a moment to read it,
so you understand the process and the expectations.
- what information to include in commit messages
- proper attribution
- volunteering contributions effectively
- how to get help and our review process.
-->
# Motivation
<!-- Briefly explain what the change is about and why it is desirable. -->
+102 -105
View File
@@ -7,119 +7,84 @@ on:
permissions: read-all
jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
tests:
needs: [check_secrets]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V28
- uses: cachix/install-nix-action@V27
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
extra_nix_config: |
sandbox = true
max-jobs = 1
- uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'linux'
- run: scripts/build-checks
- run: scripts/prepare-installer-for-github-actions
- name: Upload installer tarball
uses: actions/upload-artifact@v4
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- if: matrix.os == 'ubuntu-latest'
run: |
free -h
swapon --show
swap=$(swapon --show --noheadings | head -n 1 | awk '{print $1}')
echo "Found swap: $swap"
sudo swapoff $swap
# resize it (fallocate)
sudo fallocate -l 10G $swap
sudo mkswap $swap
sudo swapon $swap
free -h
(
while sleep 60; do
free -h
done
) &
- run: nix --experimental-features 'nix-command flakes' flake check -L
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
# Steps to test CI automation in your own fork.
# Cachix:
# 1. Sign-up for https://www.cachix.org/
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
# Dockerhub:
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V28
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v15
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
cachixArgs: '-v'
- id: prepare-installer
run: scripts/prepare-installer-for-github-actions
name: installer-${{matrix.os}}
path: out/*
installer_test:
needs: [installer, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
needs: [tests]
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
include:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V28
- name: Download installer tarball
uses: actions/download-artifact@v4
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
name: installer-${{matrix.os}}
path: out
- name: Serving installer
id: serving_installer
run: ./scripts/serve-installer-for-github-actions
- uses: cachix/install-nix-action@v30
with:
install_url: 'http://localhost:8126/install'
install_options: "--tarball-url-prefix http://localhost:8126/"
- run: sudo apt install fish zsh
if: matrix.os == 'ubuntu-latest'
if: matrix.os == 'linux'
- run: brew install fish
if: matrix.os == 'macos-latest'
if: matrix.os == 'darwin'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
@@ -127,32 +92,50 @@ jobs:
- run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs"
- run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello"
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
docker_push_image:
needs: [check_secrets, tests]
needs: [tests, vm_tests, check_secrets]
permissions:
contents: read
packages: write
if: >-
needs.check_secrets.outputs.docker == 'true' &&
github.event_name == 'push' &&
github.ref_name == 'master' &&
needs.check_secrets.outputs.cachix == 'true' &&
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
github.ref_name == 'master'
runs-on: ubuntu-24.04
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V28
- uses: cachix/install-nix-action@V27
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
@@ -189,16 +172,30 @@ jobs:
docker push $IMAGE_ID:master
vm_tests:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: nix build -L .#hydraJobs.tests.githubFlakes .#hydraJobs.tests.tarballFlakes .#hydraJobs.tests.functional_user
meson_build:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
# Only meson packages that don't have a tests.run derivation.
# Those that have it are already built and tested as part of nix flake check.
- run: nix build -L .#hydraJobs.build.{nix-cmd,nix-main}.$(nix-instantiate --eval --expr builtins.currentSystem | sed -e 's/"//g')
flake_regressions:
needs: vm_tests
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
steps:
- name: Checkout nix
uses: actions/checkout@v4
+1 -1
View File
@@ -15,7 +15,7 @@ permissions:
jobs:
labels:
runs-on: ubuntu-latest
runs-on: ubuntu-24.04
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/labeler@v5
+4 -2
View File
@@ -5,8 +5,10 @@ queue_rules:
- check-success=installer
- check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest)
- check-success=tests (macos-latest)
- check-success=tests (ubuntu-latest)
- check-success=tests on macos
- check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests
merge_method: rebase
batch_size: 5
+1 -1
View File
@@ -1 +1 @@
2.25.0
2.24.16
+1 -15
View File
@@ -52,20 +52,6 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
Link related issues to inform interested parties and future contributors about your change.
If your pull request closes one or multiple issues, mention that in the description using `Closes: #<number>`, as it will then happen automatically when your change is merged.
* Credit original authors when you're reusing or building on their work.
* Link to relevant changes in other projects, so that others can understand the full context of the change in the future when you or someone else will change or troubleshoot the code.
This is especially important when your change is based on work done in other repositories.
Example:
```
This is based on the work of @user in <url>.
This solution took inspiration from <url>.
Co-authored-by: User Name <user@example.com>
```
When cherry-picking from a different repository, use the `-x` flag, and then amend the commits to turn the hashes into URLs.
* Make sure to have [a clean history of commits on your branch by using rebase](https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request).
* [Mark the pull request as draft](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request) if you're not done with the changes.
@@ -97,4 +83,4 @@ For larger changes see the [Nix reference manual](https://nix.dev/manual/nix/dev
## Getting help
Whenever you're stuck or do not know how to proceed, you can always ask for help.
We invite you to use our [Matrix room](https://matrix.to/#/#nix-dev:nixos.org) to ask questions.
The appropriate channels to do so can be found on the [NixOS Community](https://nixos.org/community/) page.
-1
View File
@@ -1 +0,0 @@
doc/manual/src/development/building.md
+2 -2
View File
@@ -1,7 +1,7 @@
# Nix
[![Open Collective supporters](https://opencollective.com/nixos/tiers/supporter/badge.svg?label=Supporters&color=brightgreen)](https://opencollective.com/nixos)
[![CI](https://github.com/NixOS/nix/workflows/CI/badge.svg)](https://github.com/NixOS/nix/actions/workflows/ci.yml)
[![Test](https://github.com/NixOS/nix/workflows/Test/badge.svg)](https://github.com/NixOS/nix/actions)
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nix.dev/reference/nix-manual)
@@ -15,7 +15,7 @@ Full reference documentation can be found in the [Nix manual](https://nix.dev/re
## Building and developing
Follow instructions in the Nix reference manual to [set up a development environment and build Nix from source](https://nix.dev/manual/nix/development/development/building.html).
Follow instructions in the Nix reference manual to [set up a development environment and build Nix from source](https://nix.dev/manual/nix/development/building.html).
## Contributing
-3
View File
@@ -10,7 +10,6 @@ foreach dep : deps_public_subproject
endforeach
requires_public += deps_public
extra_pkg_config_variables = get_variable('extra_pkg_config_variables', {})
import('pkgconfig').generate(
this_library,
filebase : meson.project_name(),
@@ -21,7 +20,6 @@ import('pkgconfig').generate(
requires : requires_public,
requires_private : requires_private,
libraries_private : libraries_private,
variables : extra_pkg_config_variables,
)
meson.override_dependency(meson.project_name(), declare_dependency(
@@ -29,5 +27,4 @@ meson.override_dependency(meson.project_name(), declare_dependency(
link_with : this_library,
compile_args : ['-std=c++2a'],
dependencies : deps_public_subproject + deps_public,
variables : extra_pkg_config_variables,
))
+13 -4
View File
@@ -62,12 +62,16 @@ AC_CHECK_TOOL([AR], [ar])
AC_SYS_LARGEFILE
# Solaris-specific stuff.
# OS-specific stuff.
case "$host_os" in
solaris*)
# Solaris requires -lsocket -lnsl for network functions
LDFLAGS="-lsocket -lnsl $LDFLAGS"
;;
darwin*)
# Need to link to libsandbox.
LDFLAGS="-lsandbox $LDFLAGS"
;;
esac
@@ -89,9 +93,10 @@ AC_LANG_POP(C++)
AC_CHECK_FUNCS([statvfs pipe2])
# Check for lutimes, optionally used for changing the mtime of
# symlinks.
AC_CHECK_FUNCS([lutimes])
# Check for lutimes and utimensat, optionally used for changing the
# mtime of symlinks.
AC_CHECK_DECLS([AT_SYMLINK_NOFOLLOW], [], [], [[#include <fcntl.h>]])
AC_CHECK_FUNCS([lutimes utimensat])
# Check whether the store optimiser can optimise symlinks.
@@ -170,6 +175,10 @@ AS_IF(
[test "$ENABLE_FUNCTIONAL_TESTS" == "yes" || test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(jq, jq)])
AS_IF(
[test "$ENABLE_DOC_GEN" == "yes"],
[NEED_PROG(man, man)])
AS_IF([test "$ENABLE_BUILD" == "yes"],[
# Look for boost, a required dependency.
+9 -10
View File
@@ -1,10 +1,9 @@
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) { src = ./.; }).defaultNix
+2 -2
View File
@@ -12,8 +12,8 @@ h1.menu-title::before {
}
.menu-bar {
padding: 0.5em 0em;
h1.menu-title {
padding: 0.5em;
}
.sidebar .sidebar-scrollbox {
+9 -1
View File
@@ -5,7 +5,15 @@ in
builtinsInfo:
let
showBuiltin = name: { doc, type ? null, args ? [ ], experimental-feature ? null, impure-only ? false }:
showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
let
type' = optionalString (type != null) " (${type})";
+90 -62
View File
@@ -32,7 +32,13 @@ let
commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }:
showCommand =
{
command,
details,
filename,
toplevel,
}:
let
result = ''
@@ -56,26 +62,27 @@ let
${maybeOptions}
'';
showSynopsis = command: args:
showSynopsis =
command: args:
let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "...";
showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args);
in ''
in
''
`${command}` [*option*...] ${arguments}
'';
maybeSubcommands = optionalString (details ? commands && details.commands != {})
''
where *subcommand* is one of the following:
maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
where *subcommand* is one of the following:
${subcommands}
'';
${subcommands}
'';
subcommands = if length categories > 1
then listCategories
else listSubcommands details.commands;
subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands)));
categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories);
@@ -99,38 +106,39 @@ let
${allStores}
'';
index = replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
index =
replaceStrings
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
[ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview =
let
showEntry = store:
"- [${store.name}](#${store.slug})";
showEntry = store: "- [${store.name}](#${store.slug})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storesList = showStoreDocs {
storeInfo = commandInfo.stores;
inherit inlineHTML;
};
hasInfix = infix: content:
hasInfix =
infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in
optionalString (details ? doc) (
# An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc
then help-stores
else details.doc
if hasInfix "@store-types@" details.doc then help-stores else details.doc
);
maybeOptions =
let
allVisibleOptions = filterAttrs
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
in
optionalString (allVisibleOptions != { }) ''
# Options
@@ -142,55 +150,73 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
'';
showOptions = inlineHTML: allOptions:
showOptions =
inlineHTML: allOptions:
let
showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
'';
showOption = name: option:
showOption =
name: option:
let
result = trim ''
- ${item}
${option.description}
'';
item = if inlineHTML
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else "`--${name}` ${shortName} ${labels}";
shortName = optionalString
(option ? shortName)
("/ `-${option.shortName}`");
labels = optionalString
(option ? labels)
(concatStringsSep " " (map (s: "*${s}*") option.labels));
in result;
categories = mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy
(cmd: cmd.value.category)
(attrsToList allOptions));
in concatStrings (attrValues (mapAttrs showCategory categories));
in squash result;
item =
if inlineHTML then
''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
else
"`--${name}` ${shortName} ${labels}";
shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
in
result;
categories =
mapAttrs
# Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs)
(groupBy (cmd: cmd.value.category) (attrsToList allOptions));
in
concatStrings (attrValues (mapAttrs showCategory categories));
in
squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }:
processCommand =
{
command,
details,
filename,
toplevel,
}:
let
cmd = {
inherit command;
name = filename + ".md";
value = showCommand { inherit command details filename toplevel; };
value = showCommand {
inherit
command
details
filename
toplevel
;
};
};
subcommand = subCmd: processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {});
subcommand =
subCmd:
processCommand {
command = command + " " + subCmd;
details = details.commands.${subCmd};
filename = appendName filename subCmd;
inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand {
command = "nix";
@@ -199,9 +225,11 @@ let
toplevel = commandInfo.args;
};
tableOfContents = let
showEntry = page:
" - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n";
tableOfContents =
let
showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in
concatStringsSep "\n" (map showEntry manpages) + "\n";
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+74 -42
View File
@@ -1,67 +1,99 @@
let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs;
inherit (import <nix/utils.nix>) concatStrings indent optionalString squash;
inherit (builtins)
attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo:
{
prefix,
inlineHTML ? true,
}:
settingsInfo:
let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }:
showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
let
result = squash ''
- ${item}
- ${item}
${indent " " body}
'';
item = if inlineHTML
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else "`${setting}`";
${indent " " body}
'';
item =
if inlineHTML then
''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
# separate body to cleanly handle indentation
body = ''
${experimentalFeatureNote}
${experimentalFeatureNote}
${description}
${description}
**Default:** ${showDefault documentDefault defaultValue}
**Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases}
'';
${showAliases aliases}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
> **Warning**
>
> This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md).
>
> To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
>
> ```
> extra-experimental-features = ${experimentalFeature}
> ${setting} = ...
> ```
'';
showDefault = documentDefault: defaultValue:
showDefault =
documentDefault: defaultValue:
if documentDefault then
# a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue
then "*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`"
else "*machine-specific*";
if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
"*empty*"
else if isBool defaultValue then
if defaultValue then "`true`" else "`false`"
else
"`${toString defaultValue}`"
else
"*machine-specific*";
showAliases = aliases:
optionalString (aliases != [])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
showAliases =
aliases:
optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result;
in
result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+37 -14
View File
@@ -1,6 +1,20 @@
let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
inherit (import <nix/utils.nix>) optionalString filterAttrs trim squash toLower unique indent;
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
showSettings = import <nix/generate-settings.nix>;
in
@@ -14,7 +28,13 @@ in
let
showStore = { name, slug }: { settings, doc, experimentalFeature }:
showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
let
result = squash ''
# ${name}
@@ -25,7 +45,10 @@ let
## Settings
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings}
${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
'';
experimentalFeatureNote = optionalString (experimentalFeature != null) ''
@@ -43,15 +66,15 @@ let
> extra-experimental-features = ${experimentalFeature}
> ```
'';
in result;
in
result;
storesList = map
(name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
})
(attrNames storeInfo);
storesList = map (name: rec {
inherit name;
slug = replaceStrings [ " " ] [ "-" ] (toLower name);
filename = "${slug}.md";
page = showStore { inherit name slug; } storeInfo.${name};
}) (attrNames storeInfo);
in storesList
in
storesList
+16 -10
View File
@@ -1,5 +1,11 @@
let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings;
inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>;
in
@@ -14,26 +20,26 @@ let
index =
let
showEntry = store:
"- [${store.name}](./${store.filename})";
showEntry = store: "- [${store.name}](./${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings
[ "@store-types@" ] [ index ]
(readFile ./src/store/types/index.md.in);
"index.md" = replaceStrings [ "@store-types@" ] [ index ] (readFile ./src/store/types/index.md.in);
tableOfContents =
let
showEntry = store:
" - [${store.name}](store/types/${store.filename})";
showEntry = store: " - [${store.name}](store/types/${store.filename})";
in
concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents;
storePages = listToAttrs
(map (s: { name = s.filename; value = s.page; }) storesList);
storePages = listToAttrs (
map (s: {
name = s.filename;
value = s.page;
}) storesList
);
in
storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -2,8 +2,8 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
showExperimentalFeature = name: doc: ''
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
'';
in
xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+2 -1
View File
@@ -2,7 +2,8 @@ with builtins;
with import <nix/utils.nix>;
let
showExperimentalFeature = name: doc:
showExperimentalFeature =
name: doc:
squash ''
## [`${name}`]{#xp-feature-${name}}
-1
View File
@@ -344,7 +344,6 @@ const redirects = {
},
"language/syntax.html": {
"scoping-rules": "scoping.html",
"string-literal": "string-literals.html",
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: Use envvars NIX_CACHE_HOME, NIX_CONFIG_HOME, NIX_DATA_HOME, NIX_STATE_HOME if defined
prs: [11351]
---
Added new environment variables:
- `NIX_CACHE_HOME`
- `NIX_CONFIG_HOME`
- `NIX_DATA_HOME`
- `NIX_STATE_HOME`
Each, if defined, takes precedence over the corresponding [XDG environment variable](@docroot@/command-ref/env-common.md#xdg-base-directories).
This provides more fine-grained control over where Nix looks for files, and allows to have a stand-alone Nix environment, which only uses files in a specific directory, and doesn't interfere with the user environment.
@@ -1,21 +0,0 @@
---
synopsis: Define integer overflow in the Nix language as an error
issues: [10968]
prs: [11188]
---
Previously, integer overflow in the Nix language invoked C++ level signed overflow, which was undefined behaviour, but *usually* manifested as wrapping around on overflow.
Since prior to the public release of Lix, Lix had C++ signed overflow defined to crash the process and nobody noticed this having accidentally removed overflow from the Nix language for three months until it was caught by fiddling around.
Given the significant body of actual Nix code that has been evaluated by Lix in that time, it does not appear that nixpkgs or much of importance depends on integer overflow, so it appears safe to turn into an error.
Some other overflows were fixed:
- `builtins.fromJSON` of values greater than the maximum representable value in a signed 64-bit integer will generate an error.
- `nixConfig` in flakes will no longer accept negative values for configuration options.
Integer overflow now looks like the following:
```
$ nix eval --expr '9223372036854775807 + 1'
error: integer overflow in adding 9223372036854775807 + 1
```
-22
View File
@@ -1,22 +0,0 @@
---
synopsis: |-
The `build-hook` setting's default is less useful when using `libnixstore` as a library
prs:
- 11178
---
*This is an obscure issue that only affects usage of the `libnixstore` library outside of the Nix executable.*
As part the ongoing [rewrite of the build system](https://github.com/NixOS/nix/issues/2503) to use [Meson](https://mesonbuild.com/), we are also switching to packaging individual Nix components separately (and building them in separate derivations).
This means that when building `libnixstore` we do not know where the Nix binaries will be installed --- `libnixstore` doesn't know about downstream consumers like the Nix binaries at all.
*This is also unrelated to the _`post`_-`build-hook`*, which is often used for pushing to a cache.*
This has a small adverse affect on remote building --- the `build-remote` executable that is specified from the [`build-hook`](@docroot@/command-ref/conf-file.md#conf-build-hook) setting will not be gotten from the (presumed) installation location, but instead looked up on the `PATH`.
This means that other applications linking `libnixstore` that wish to use remote building must arrange for the `nix` command to be on the PATH (or manually overriding `build-hook`) in order for that to work.
Long term we don't envision this being a downside, because we plan to [get rid of `build-remote` and the build hook setting entirely](https://github.com/NixOS/nix/issues/1221).
There is simply no need to add a second layer of remote-procedure-calling when we want to connect to a remote builder.
The build hook protocol did in principle support custom ways of remote building, but that can also be accomplished with a custom service for the ssh or daemon/ssh-ng protocols, or with a custom [store type](@docroot@/store/types/index.md) i.e. `Store` subclass. <!-- we normally don't mention classes, but consider that this release note is about a library use case -->
The Perl bindings no longer expose `getBinDir` either, since they libraries those bindings wrap no longer know the location of installed binaries as described above.
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
-14
View File
@@ -1,14 +0,0 @@
---
synopsis: wrap filesystem exceptions more correctly
issues: []
prs: [11378]
---
With the switch to `std::filesystem` in different places, Nix started to throw `std::filesystem::filesystem_error` in many places instead of its own exceptions.
This lead to no longer generating error traces, for example when listing a non-existing directory, and can also lead to crashes inside the Nix REPL.
This version catches these types of exception correctly and wrap them into Nix's own exeception type.
Author: [**@Mic92**](https://github.com/Mic92)
-9
View File
@@ -1,9 +0,0 @@
---
synopsis: Add setting `fsync-store-paths`
issues: [1218]
prs: [7126]
---
Nix now has a setting `fsync-store-paths` that ensures that new store paths are durably written to disk before they are registered as "valid" in Nix's database. This can prevent Nix store corruption if the system crashes or there is a power loss. This setting defaults to `false`.
Author: [**@squalus**](https://github.com/squalus)
@@ -1,25 +0,0 @@
---
synopsis: Show package descriptions with `nix flake show`
issues: [10977]
prs: [10980]
---
`nix flake show` will now display a package's `meta.description` if it exists. If the description does not fit in the terminal it will be truncated to fit the terminal width. If the size of the terminal width is unknown the description will be capped at 80 characters.
```
$ nix flake show
└───packages
└───x86_64-linux
├───builderImage: package 'docker-image-ara-builder-image.tar.gz' - 'Docker image hosting the nix build environment'
└───runnerImage: package 'docker-image-gitlab-runner.tar.gz' - 'Docker image hosting the gitlab-runner executable'
```
In a narrower terminal:
```
$ nix flake show
└───packages
└───x86_64-linux
├───builderImage: package 'docker-image-ara-builder-image.tar.gz' - 'Docker image hosting the nix b...
└───runnerImage: package 'docker-image-gitlab-runner.tar.gz' - 'Docker image hosting the gitlab-run...
```
@@ -1,17 +0,0 @@
---
synopsis: Removing the default argument passed to the `nix fmt` formatter
issues: []
prs: [11438]
---
The underlying formatter no longer receives the ". " default argument when `nix fmt` is called with no arguments.
This change was necessary as the formatter wasn't able to distinguish between
a user wanting to format the current folder with `nix fmt .` or the generic
`nix fmt`.
The default behaviour is now the responsibility of the formatter itself, and
allows tools such as treefmt to format the whole tree instead of only the
current directory and below.
Author: [**@zimbatm**](https://github.com/zimbatm)
@@ -1,8 +0,0 @@
---
synopsis: Flakes are no longer substituted
prs: [10612]
---
Nix will no longer attempt to substitute the source code of flakes from a binary cache. This functionality was broken because it could lead to different evaluation results depending on whether the flake was available in the binary cache, or even depending on whether the flake was already in the local store.
Author: [**@edolstra**](https://github.com/edolstra)
@@ -150,6 +150,21 @@ These special characters are escaped as follows:
`''\` escapes any other character.
A "dollar-curly" (`${`) can be written as follows:
> **Example**
>
> ```nix
> ''
> echo ''${PATH}
> ''
> ```
>
> "echo ${PATH}\n"
> **Note**
>
> This differs from the syntax for escaping a dollar-curly within double quotes (`"\${"`). Be aware of which one is needed at a given moment.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
-2
View File
@@ -28,8 +28,6 @@
- [Data Types](language/types.md)
- [String context](language/string-context.md)
- [Syntax and semantics](language/syntax.md)
- [Variables](language/variables.md)
- [String literals](language/string-literals.md)
- [Identifiers](language/identifiers.md)
- [Scoping rules](language/scope.md)
- [String interpolation](language/string-interpolation.md)
+1 -14
View File
@@ -138,19 +138,6 @@ The following environment variables are used to determine locations of various s
- [`XDG_STATE_HOME`]{#env-XDG_STATE_HOME} (default `~/.local/state`)
- [`XDG_CACHE_HOME`]{#env-XDG_CACHE_HOME} (default `~/.cache`)
[XDG Base Directory Specification]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
In addition, setting the following environment variables overrides the XDG base directories:
- [`NIX_CONFIG_HOME`]{#env-NIX_CONFIG_HOME} (default `$XDG_CONFIG_HOME/nix`)
- [`NIX_STATE_HOME`]{#env-NIX_STATE_HOME} (default `$XDG_STATE_HOME/nix`)
- [`NIX_CACHE_HOME`]{#env-NIX_CACHE_HOME} (default `$XDG_CACHE_HOME/nix`)
When [`use-xdg-base-directories`] is enabled, the configuration directory is:
1. `$NIX_CONFIG_HOME`, if it is defined
2. Otherwise, `$XDG_CONFIG_HOME/nix`, if `XDG_CONFIG_HOME` is defined
3. Otherwise, `~/.config/nix`.
Likewise for the state and cache directories.
@@ -1,6 +1,6 @@
## Default Nix expression
The source for the [Nix expressions](@docroot@/glossary.md#gloss-nix-expression) used by [`nix-env`] by default:
The source for the default [Nix expressions](@docroot@/language/index.md) used by [`nix-env`]:
- `~/.nix-defexpr`
- `$XDG_STATE_HOME/nix/defexpr` if [`use-xdg-base-directories`] is set to `true`.
@@ -18,25 +18,24 @@ Then, the resulting expression is interpreted like this:
- If the expression is an attribute set, it is used as the default Nix expression.
- If the expression is a function, an empty set is passed as argument and the return value is used as the default Nix expression.
> **Example**
>
> If the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
>
> ```nix
> {
> foo = import ~/.nix-defexpr/foo.nix;
> bar = import ~/.nix-defexpr/bar.nix;
> }
> ```
For example, if the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
```nix
{
foo = import ~/.nix-defexpr/foo.nix;
bar = import ~/.nix-defexpr/bar.nix;
}
```
The file [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) is always ignored.
The command [`nix-channel`] places a symlink to the current user's [channels] in this directory, the [user channel link](#user-channel-link).
The command [`nix-channel`] places a symlink to the user's current [channels profile](@docroot@/command-ref/files/channels.md) in this directory.
This makes all subscribed channels available as attributes in the default expression.
## User channel link
A symlink that ensures that [`nix-env`] can find the current user's [channels]:
A symlink that ensures that [`nix-env`] can find your channels:
- `~/.nix-defexpr/channels`
- `$XDG_STATE_HOME/defexpr/channels` if [`use-xdg-base-directories`] is set to `true`.
@@ -46,9 +45,8 @@ This symlink points to:
- `$XDG_STATE_HOME/profiles/channels` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.
In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.[`nix-env`]: ../nix-env.md
[`nix-channel`]: @docroot@/command-ref/nix-channel.md
[`nix-env`]: @docroot@/command-ref/nix-env.md
[`nix-channel`]: @docroot@/command-ref/nix-channel.md
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
[channels]: @docroot@/command-ref/files/channels.md
@@ -62,6 +62,15 @@ These options are for deleting old [profiles] prior to deleting unreachable [sto
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
- <span id="opt-max-freed">[`--max-freed`](#opt-max-freed)</span> *bytes*
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
{{#include ./opt-common.md}}
{{#include ./env-common.md}}
+1 -1
View File
@@ -62,7 +62,7 @@ These pages can be viewed offline:
Several operations, such as [`nix-env --query`](./nix-env/query.md) and [`nix-env --install`](./nix-env/install.md), take a list of *arguments* that specify the packages on which to operate.
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-names):
Packages are identified based on a `name` part and a `version` part of a [symbolic derivation name](@docroot@/language/derivations.md#attr-name):
- `name`: Everything up to but not including the first dash (`-`) that is *not* followed by a letter.
- `version`: The rest, excluding the separating dash.
@@ -21,6 +21,9 @@ This operation has the following options:
Use recursive instead of flat hashing mode, used when adding
directories to the store.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -11,6 +11,9 @@
The operation `--add` adds the specified paths to the Nix store. It
prints the resulting paths in the Nix store on standard output.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}}
{{#include ../opt-common.md}}
@@ -104,7 +104,7 @@ symlink.
Prints a set of derivation files (`.drv`) which are supposed produce
said paths when realized. Might print nothing, for example for source paths
or paths subsituted from a binary cache.
or paths substituted from a binary cache.
- `--graph`
@@ -241,4 +241,3 @@ $ nix-store --query --roots $(which svn)
/nix/var/nix/profiles/default-82-link
/home/eelco/.local/state/nix/profiles/profile-97-link
```
+45 -43
View File
@@ -34,21 +34,16 @@ $ nix-shell --attr devShells.x86_64-linux.native-clangStdenvPackages
To build Nix itself in this shell:
```console
[nix-shell]$ mesonFlags+=" --prefix=$(pwd)/outputs/out"
[nix-shell]$ dontAddPrefix=1 mesonConfigurePhase
[nix-shell]$ ninjaBuildPhase
[nix-shell]$ autoreconfPhase
[nix-shell]$ ./configure $configureFlags --prefix=$(pwd)/outputs/out
[nix-shell]$ make -j $NIX_BUILD_CORES
```
To test it:
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ mesonCheckPhase
```
To install it in `$(pwd)/outputs`:
```console
[nix-shell]$ ninjaInstallPhase
[nix-shell]$ make install
[nix-shell]$ make installcheck -j $NIX_BUILD_CORES
[nix-shell]$ ./outputs/out/bin/nix --version
nix (Nix) 2.12
```
@@ -90,20 +85,16 @@ $ nix develop .#native-clangStdenvPackages
To build Nix itself in this shell:
```console
[nix-shell]$ mesonConfigurePhase
[nix-shell]$ ninjaBuildPhase
[nix-shell]$ autoreconfPhase
[nix-shell]$ configurePhase
[nix-shell]$ make -j $NIX_BUILD_CORES OPTIMIZE=0
```
To test it:
To install it in `$(pwd)/outputs` and test it:
```console
[nix-shell]$ mesonCheckPhase
```
To install it in `$(pwd)/outputs`:
```console
[nix-shell]$ ninjaInstallPhase
[nix-shell]$ make install OPTIMIZE=0
[nix-shell]$ make installcheck check -j $NIX_BUILD_CORES
[nix-shell]$ nix --version
nix (Nix) 2.12
```
@@ -119,6 +110,25 @@ $ nix build
You can also build Nix for one of the [supported platforms](#platforms).
## Makefile variables
You may need `profiledir=$out/etc/profile.d` and `sysconfdir=$out/etc` to run `make install`.
Run `make` with [`-e` / `--environment-overrides`](https://www.gnu.org/software/make/manual/make.html#index-_002de) to allow environment variables to override `Makefile` variables:
- `ENABLE_BUILD=yes` to enable building the C++ code.
- `ENABLE_DOC_GEN=yes` to enable building the documentation (manual, man pages, etc.).
The docs can take a while to build, so you may want to disable this for local development.
- `ENABLE_FUNCTIONAL_TESTS=yes` to enable building the functional tests.
- `OPTIMIZE=1` to enable optimizations.
- `libraries=libutil programs=` to only build a specific library.
This will fail in the linking phase if the other libraries haven't been built, but is useful for checking types.
- `libraries= programs=nix` to only build a specific program.
This will not work in general, because the programs need the libraries.
## Platforms
Nix can be built for various platforms, as specified in [`flake.nix`]:
@@ -165,38 +175,27 @@ Add more [system types](#system-type) to `crossSystems` in `flake.nix` to bootst
It is useful to perform multiple cross and native builds on the same source tree,
for example to ensure that better support for one platform doesn't break the build for another.
Meson thankfully makes this very easy by confining all build products to the build directory --- one simple shares the source directory between multiple build directories, each of which contains the build for Nix to a different platform.
In order to facilitate this, Nix has some support for being built out of tree – that is, placing build artefacts in a different directory than the source code:
Nixpkgs's `mesonConfigurePhase` always chooses `build` in the current directory as the name and location of the build.
This makes having multiple build directories slightly more inconvenient.
The good news is that Meson/Ninja seem to cope well with relocating the build directory after it is created.
Here's how to do that
1. Configure as usual
1. Create a directory for the build, e.g.
```bash
mesonConfigurePhase
mkdir build
```
2. Rename the build directory
2. Run the configure script from that directory, e.g.
```bash
cd .. # since `mesonConfigurePhase` cd'd inside
mv build build-linux # or whatever name we want
cd build-linux
cd build
../configure <configure flags>
```
3. Build as usual
3. Run make from the source directory, but with the build directory specified, e.g.
```bash
ninjaBuildPhase
make builddir=build <make flags>
```
> **N.B.**
> [`nixpkgs#335818`](https://github.com/NixOS/nixpkgs/issues/335818) tracks giving `mesonConfigurePhase` proper support for custom build directories.
> When it is fixed, we can simplify these instructions and then remove this notice.
## System type
Nix uses a string with the following format to identify the *system type* or *platform* it runs on:
@@ -258,8 +257,11 @@ You can use any of the other supported environments in place of `nix-ccacheStden
The `clangd` LSP server is installed by default on the `clang`-based `devShell`s.
See [supported compilation environments](#compilation-environments) and instructions how to set up a shell [with flakes](#nix-with-flakes) or in [classic Nix](#classic-nix).
To use the LSP with your editor, you will want a `compile_commands.json` file telling `clangd` how we are compiling the code.
Meson's configure always produces this inside the build directory.
To use the LSP with your editor, you first need to [set up `clangd`](https://clangd.llvm.org/installation#project-setup) by running:
```console
make compile_commands.json
```
Configure your editor to use the `clangd` from the `.#native-clangStdenvPackages` shell. You can do that either by running it inside the development shell, or by using [nix-direnv](https://github.com/nix-community/nix-direnv) and [the appropriate editor plugin](https://github.com/direnv/direnv/wiki#editor-integration).
@@ -274,7 +276,7 @@ Configure your editor to use the `clangd` from the `.#native-clangStdenvPackages
You may run the formatters as a one-off using:
```console
./maintainers/format.sh
make format
```
If you'd like to run the formatters before every commit, install the hooks:
+2 -2
View File
@@ -13,13 +13,13 @@ Incremental refactorings of the documentation build setup to make it faster or e
Build the manual from scratch:
```console
nix-build -E '(import ./.).packages.${builtins.currentSystem}.nix.doc'
nix-build $(nix-instantiate)'!doc'
```
or
```console
nix build .#nix^doc
nix build .#^doc
```
and open `./result-doc/share/doc/nix/manual/index.html`.
+40 -28
View File
@@ -135,52 +135,60 @@ Functional tests are run during `installCheck` in the `nix` package build, as we
### Running the whole test suite
The whole test suite (functional and unit tests) can be run with:
The whole test suite can be run with:
```shell-session
$ mesonCheckPhase
$ make install && make installcheck
ran test tests/functional/foo.sh... [PASS]
ran test tests/functional/bar.sh... [PASS]
...
```
### Grouping tests
Sometimes it is useful to group related tests so they can be easily run together without running the entire test suite.
Each test group is in a subdirectory of `tests`.
For example, `tests/functional/ca/meson.build` defines a `ca` test group for content-addressed derivation outputs.
For example, `tests/functional/ca/local.mk` defines a `ca` test group for content-addressed derivation outputs.
That test group can be run like this:
```shell-session
$ meson test --suite ca
ninja: Entering directory `/home/jcericson/src/nix/master/build'
ninja: no work to do.
[1-20/20] 🌑 nix-functional-tests:ca / ca/why-depends 1/20 nix-functional-tests:ca / ca/nix-run OK 0.16s
[2-20/20] 🌒 nix-functional-tests:ca / ca/why-depends 2/20 nix-functional-tests:ca / ca/import-derivation OK 0.17s
$ make ca.test-group -j50
ran test tests/functional/ca/nix-run.sh... [PASS]
ran test tests/functional/ca/import-derivation.sh... [PASS]
...
```
The test group is defined in Make like this:
```makefile
$(test-group-name)-tests := \
$(d)/test0.sh \
$(d)/test1.sh \
...
install-tests-groups += $(test-group-name)
```
### Running individual tests
Individual tests can be run with `meson`:
Individual tests can be run with `make`:
```shell-session
$ meson test ${testName}
ninja: Entering directory `/home/jcericson/src/nix/master/build'
ninja: no work to do.
1/1 nix-functional-tests:main / ${testName} OK 0.41s
Ok: 1
Expected Fail: 0
Fail: 0
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to /home/jcericson/src/nix/master/build/meson-logs/testlog.txt
$ make tests/functional/${testName}.sh.test
ran test tests/functional/${testName}.sh... [PASS]
```
or without `meson`, showing the output:
or without `make`:
```shell-session
$ TEST_NAME=${testName} NIX_REMOTE='' PS4='+(${BASH_SOURCE[0]-$0}:$LINENO) tests/functional/${testName}.sh
$ ./mk/run-test.sh tests/functional/${testName}.sh
ran test tests/functional/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/functional/${testName}.sh
+(${testName}.sh:1) foo
output from foo
+(${testName}.sh:2) bar
@@ -246,7 +254,7 @@ It is frequently useful to regenerate the expected output.
To do that, rerun the failed test(s) with `_NIX_TEST_ACCEPT=1`.
For example:
```bash
_NIX_TEST_ACCEPT=1 meson test lang
_NIX_TEST_ACCEPT=1 make tests/functional/lang.sh.test
```
This convention is shared with the [characterisation unit tests](#characterisation-testing-unit) too.
@@ -268,12 +276,14 @@ To ensure that characterisation testing doesn't make it harder to intentionally
We run the functional tests not just in the build, but also in VM tests.
This helps us ensure that Nix works correctly on NixOS, and environments that have similar characteristics that are hard to reproduce in a build environment.
These can be run with:
The recommended way to run these tests during development is:
```shell
nix build .#hydraJobs.tests.functional_user
nix build .#hydraJobs.tests.functional_user.quickBuild
```
The `quickBuild` attribute configures the test to use a `nix` package that's built without integration tests, so that you can iterate on the tests without performing recompilations due to the changed sources for `installCheck`.
Generally, this build is sufficient, but in nightly or CI we also test the attributes `functional_root` and `functional_trusted`, in which the test suite is run with different levels of authorization.
## Integration tests
@@ -284,6 +294,8 @@ Because these tests are expensive and require more than what the standard github
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`.
If you are testing a build of `nix` that you haven't compiled yet, you may iterate faster by appending the `quickBuild` attribute: `nix build .#hydraJobs.tests.{testName}.quickBuild`.
## Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
@@ -296,7 +308,7 @@ Creating a Cachix cache for your installer tests and adding its authorisation to
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
- The `installer_test` job (which runs on `ubuntu-24.04` and `macos-14`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup
+7 -12
View File
@@ -119,7 +119,7 @@
A store object consists of a [file system object], [references][reference] to other store objects, and other metadata.
It can be referred to by a [store path].
See [Store Object](@docroot@/store/store-object.md) for details.
See [Store Object](@docroot@/store/index.md#store-object) for details.
[store object]: #gloss-store-object
@@ -182,18 +182,13 @@
- [Nix expression]{#gloss-nix-expression}
A syntactically valid use of the [Nix language].
1. Commonly, a high-level description of software packages and compositions
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions specify [derivations][derivation],
which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
> **Example**
>
> The contents of a `.nix` file form a Nix expression.
Nix expressions specify [derivations][derivation], which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
> **Example**
>
> Building and deploying software using Nix entails writing Nix expressions as a high-level description of packages and compositions thereof.
2. A syntactically valid use of the [Nix language]. For example, the contents of a `.nix` file form an expression.
- [reference]{#gloss-reference}
+13 -8
View File
@@ -1,26 +1,31 @@
# Building Nix from Source
Nix is built with [Meson](https://mesonbuild.com/).
It is broken up into multiple Meson packages, which are optionally combined in a single project using Meson's [subprojects](https://mesonbuild.com/Subprojects.html) feature.
After cloning Nix's Git repository, issue the following commands:
There are no mandatory extra steps to the building process:
generic Meson installation instructions like [this](https://mesonbuild.com/Quick-guide.html#using-meson-as-a-distro-packager) should work.
```console
$ autoreconf -vfi
$ ./configure options...
$ make
$ make install
```
The installation path can be specified by passing the `-Dprefix=prefix`
Nix requires GNU Make so you may need to invoke `gmake` instead.
The installation path can be specified by passing the `--prefix=prefix`
to `configure`. The default installation directory is `/usr/local`. You
can change this to any location you like. You must have write permission
to the *prefix* path.
Nix keeps its *store* (the place where packages are stored) in
`/nix/store` by default. This can be changed using
`-Dstore-dir=path`.
`--with-store-dir=path`.
> **Warning**
>
>
> It is best *not* to change the Nix store from its default, since doing
> so makes it impossible to use pre-built binaries from the standard
> Nixpkgs channels — that is, all packages will need to be built from
> source.
Nix keeps state (such as its database and log files) in `/nix/var` by
default. This can be changed using `-Dlocalstatedir=path`.
default. This can be changed using `--localstatedir=path`.
@@ -4,8 +4,11 @@
>
> *lookup-path* = `<` *identifier* [ `/` *identifier* ]... `>`
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/types.md#type-path) if the identifier matches a search path entry in [`builtins.nixPath`](@docroot@/language/builtins.md#builtins-nixPath).
The algorithm for lookup path resolution is described in the documentation on [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile).
A lookup path is an identifier with an optional path suffix that resolves to a [path value](@docroot@/language/types.md#type-path) if the identifier matches a search path entry.
The value of a lookup path is determined by [`builtins.nixPath`](@docroot@/language/builtins.md#builtins-nixPath).
See [`builtins.findFile`](@docroot@/language/builtins.md#builtins-findFile) for details on lookup path resolution.
> **Example**
>
+2 -2
View File
@@ -113,7 +113,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
### Optional
- [`args`]{#attr-args} ([List](@docroot@/language/types.md#list) of [String](@docroot@/language/types.md#type-string))
- [`args`]{#attr-args} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
Default: `[ ]`
@@ -132,7 +132,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
> };
> ```
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/types.md#list) of [String](@docroot@/language/types.md#type-string))
- [`outputs`]{#attr-outputs} ([List](@docroot@/language/types.md#type-list) of [String](@docroot@/language/types.md#type-string))
Default: `[ "out" ]`
+2 -3
View File
@@ -16,14 +16,13 @@ An *identifier* is an [ASCII](https://en.wikipedia.org/wiki/ASCII) character seq
# Names
A *name* can be written as an [identifier](#identifier) or a [string literal](./string-literals.md).
A name can be an [identifier](#identifier) or a [string literal](./syntax.md#string-literal).
> **Syntax**
>
> *name* → *identifier* | *string*
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting-attributes).
Two names are the same if they represent the same sequence of characters, regardless of whether they are written as identifiers or strings.
Names are used in [attribute sets](./syntax.md#attrs-literal), [`let` bindings](./syntax.md#let-expressions), and [`inherit`](./syntax.md#inheriting attributes).
# Keywords
+4 -8
View File
@@ -33,7 +33,7 @@
[path]: ./types.md#type-path
[number]: ./types.md#type-float
[list]: ./types.md#type-list
[attribute set]: ./types.md#attribute-set
[attribute set]: ./types.md#type-attrs
<!-- TODO(@rhendric, #10970): ^ rationalize number -> int/float -->
@@ -75,7 +75,7 @@ The result is a [Boolean] value.
See also: [`builtins.hasAttr`](@docroot@/language/builtins.md#builtins-hasAttr)
[Boolean]: ./types.md#type-boolean
[Boolean]: ./types.md#type-bool
[Has attribute]: #has-attribute
@@ -83,12 +83,8 @@ After evaluating *attrset* and *attrpath*, the computational complexity is O(log
## Arithmetic
Numbers will retain their type unless mixed with other numeric types:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number returns a floating point number.
Evaluation of the following numeric operations throws an evaluation error:
- Division by zero
- Integer overflow, that is, any operation yielding a result outside of the representable range of [Nix language integers](./syntax.md#number-literal)
Numbers are type-compatible:
Pure integer operations will always return integers, whereas any operation involving at least one floating point number return a floating point number.
See also [Comparison] and [Equality].
+9 -23
View File
@@ -1,28 +1,14 @@
# Scoping rules
A *scope* in the Nix language is a dictionary keyed by [name](./identifiers.md#names), mapping each name to an expression and a *definition type*.
The definition type is either *explicit* or *implicit*.
Each entry in this dictionary is a *definition*.
Nix is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope), but with multiple scopes and shadowing rules.
Explicit definitions are created by the following expressions:
- [let-expressions](syntax.md#let-expressions)
- [recursive attribute set literals](syntax.md#recursive-sets) (`rec`)
- [function literals](syntax.md#functions)
* primary scope: explicitly-bound variables
* [`let`](./syntax.md#let-expressions)
* [`inherit`](./syntax.md#inheriting-attributes)
* [function](./syntax.md#functions) arguments
Implicit definitions are only created by [with-expressions](./syntax.md#with-expressions).
* secondary scope: implicitly-bound variables
* [`with`](./syntax.md#with-expressions)
Every expression is *enclosed* by a scope.
The outermost expression is enclosed by the [built-in, global scope](./builtins.md), which contains only explicit definitions.
The expressions listed above *extend* their enclosing scope by adding new definitions, or replacing existing ones with the same name.
An explicit definition can replace a definition of any type; an implicit definition can only replace another implicit definition.
Each of the above expressions defines which of its subexpressions are enclosed by the extended scope.
In all other cases, the same scope that encloses an expression is the enclosing scope for its subexpressions.
The Nix language is [statically scoped](https://en.wikipedia.org/wiki/Scope_(computer_science)#Lexical_scope);
the value of a variable is determined only by the variable's enclosing scope, and not by the dynamic context in which the variable is evaluated.
> **Note**
>
> Expressions entered into the [Nix REPL](@docroot@/command-ref/new-cli/nix3-repl.md) are enclosed by a scope that can be extended by command line arguments or previous REPL commands.
> These ways of extending scope are not, strictly speaking, part of the Nix language.
Primary scope takes precedence over secondary scope.
See [`with`](./syntax.md#with-expressions) for a detailed example.
@@ -8,10 +8,6 @@ Such a construct is called *interpolated string*, and the expression inside is a
[path]: ./types.md#type-path
[attribute set]: ./types.md#attribute-set
> **Syntax**
>
> *interpolation_element* → `${` *expression* `}`
## Examples
### String
+170 -9
View File
@@ -6,7 +6,175 @@ This section covers syntax and semantics of the Nix language.
### String {#string-literal}
See [String literals](string-literals.md).
*Strings* can be written in three ways.
The most common way is to enclose the string between double quotes, e.g., `"foo bar"`.
Strings can span multiple lines.
The results of other expressions can be included into a string by enclosing them in `${ }`, a feature known as [string interpolation].
[string interpolation]: ./string-interpolation.md
The following must be escaped to represent them within a string, by prefixing with a backslash (`\`):
- Double quote (`"`)
> **Example**
>
> ```nix
> "\""
> ```
>
> "\""
- Backslash (`\`)
> **Example**
>
> ```nix
> "\\"
> ```
>
> "\\"
- Dollar sign followed by an opening curly bracket (`${`) – "dollar-curly"
> **Example**
>
> ```nix
> "\${"
> ```
>
> "\${"
The newline, carriage return, and tab characters can be written as `\n`, `\r` and `\t`, respectively.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> "$${"
> ```
>
> "$\${"
String values are output on the terminal with Nix-specific escaping.
Strings written to files will contain the characters encoded by the escaping.
The second way to write string literals is as an *indented string*, which is enclosed between pairs of *double single-quotes* (`''`), like so:
```nix
''
This is the first line.
This is the second line.
This is the third line.
''
```
This kind of string literal intelligently strips indentation from
the start of each line. To be precise, it strips from each line a
number of spaces equal to the minimal indentation of the string as a
whole (disregarding the indentation of empty lines). For instance,
the first and second line are indented two spaces, while the third
line is indented four spaces. Thus, two spaces are stripped from
each line, so the resulting string is
```nix
"This is the first line.\nThis is the second line.\n This is the third line.\n"
```
> **Note**
>
> Whitespace and newline following the opening `''` is ignored if there is no non-whitespace text on the initial line.
> **Warning**
>
> Prefixed tab characters are not stripped.
>
> > **Example**
> >
> > The following indented string is prefixed with tabs:
> >
> > ''
> > all:
> > @echo hello
> > ''
> >
> > "\tall:\n\t\t@echo hello\n"
Indented strings support [string interpolation].
The following must be escaped to represent them in an indented string:
- `$` is escaped by prefixing it with two single quotes (`''`)
> **Example**
>
> ```nix
> ''
> ''$
> ''
> ```
>
> "$\n"
- `''` is escaped by prefixing it with one single quote (`'`)
> **Example**
>
> ```nix
> ''
> '''
> ''
> ```
>
> "''\n"
These special characters are escaped as follows:
- Linefeed (`\n`): `''\n`
- Carriage return (`\r`): `''\r`
- Tab (`\t`): `''\t`
`''\` escapes any other character.
A "double-dollar-curly" (`$${`) can be written literally.
> **Example**
>
> ```nix
> ''
> $${
> ''
> ```
>
> "$\${\n"
Indented strings are primarily useful in that they allow multi-line
string literals to follow the indentation of the enclosing Nix
expression, and that less escaping is typically necessary for
strings representing languages such as shell scripts and
configuration files because `''` is much less common than `"`.
Example:
```nix
stdenv.mkDerivation {
...
postInstall =
''
mkdir $out/bin $out/etc
cp foo $out/bin
echo "Hello World" > $out/etc/foo.conf
${if enableBar then "cp bar $out/bin" else ""}
'';
...
}
```
Finally, as a convenience, *URIs* as defined in appendix B of
[RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
is*, without quotes. For instance, the string
`"http://example.org/foo.tar.bz2"` can also be written as
`http://example.org/foo.tar.bz2`.
### Number {#number-literal}
@@ -15,13 +183,6 @@ See [String literals](string-literals.md).
Numbers, which can be *integers* (like `123`) or *floating point*
(like `123.43` or `.27e13`).
Integers in the Nix language are 64-bit [two's complement] signed integers, with a range of -9223372036854775808 to 9223372036854775807, inclusive.
[two's complement]: https://en.wikipedia.org/wiki/Two%27s_complement
Note that negative numeric literals are actually parsed as unary negation of positive numeric literals.
This means that the minimum integer `-9223372036854775808` cannot be written as-is as a literal, since the positive number `9223372036854775808` is one past the maximum range.
See [arithmetic] and [comparison] operators for semantics.
[arithmetic]: ./operators.md#arithmetic
@@ -92,7 +253,7 @@ Attribute sets are written enclosed in curly brackets (`{ }`).
Attribute names and attribute values are separated by an equal sign (`=`).
Each value can be an arbitrary expression, terminated by a semicolon (`;`)
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](string-literals.md)).
An attribute name is a string without context, and is denoted by a [name] (an [identifier](./identifiers.md#identifiers) or [string literal](#string-literal)).
[name]: ./identifiers.md#names
+1 -1
View File
@@ -45,7 +45,7 @@ The function [`builtins.isBool`](builtins.md#builtins-isBool) can be used to det
A _string_ in the Nix language is an immutable, finite-length sequence of bytes, along with a [string context](string-context.md).
Nix does not assume or support working natively with character encodings.
String values without string context can be expressed as [string literals](string-literals.md).
String values without string context can be expressed as [string literals](syntax.md#string-literal).
The function [`builtins.isString`](builtins.md#builtins-isString) can be used to determine if a value is a string.
### Path {#type-path}
-10
View File
@@ -1,10 +0,0 @@
# Variables
A *variable* is an [identifier](identifiers.md) used as an expression.
> **Syntax**
>
> *expression* → *identifier*
A variable must have the same name as a definition in the [scope](./scope.md) that encloses it.
The value of a variable is the value of the corresponding expression in the enclosing scope.
+4
View File
@@ -75,3 +75,7 @@
(experimental) can be found by any program that follows the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
- A new command `nix store add` has been added. It replaces `nix store add-file` and `nix store add-path` which are now deprecated.
- A new option [`always-allow-substitutes`](@docroot@/command-ref/conf-file.md#conf-always-allow-substitutes) has been added.
When set to `true`, Nix will always try to substitute a derivation, even if it has the [`allowSubstitutes`]{#adv-attr-allowSubstitutes} attribute set to `false`.
+1 -1
View File
@@ -96,7 +96,7 @@
Nix can now warn when evaluation of a Nix expression causes a large
path to be copied to the Nix store. The threshold for this warning can
be configured using [the `warn-large-path-threshold`
setting](@docroot@/command-ref/conf-file.md#warn-large-path-threshold),
setting](@docroot@/command-ref/conf-file.md#conf-warn-large-path-threshold),
e.g. `--warn-large-path-threshold 100M` will warn about paths larger
than 100 MiB.
+21
View File
@@ -261,6 +261,12 @@
Author: [**Eelco Dolstra (@edolstra)**](https://github.com/edolstra)
- Improve handling of tarballs that don't consist of a single top-level directory [#11195](https://github.com/NixOS/nix/pull/11195)
In previous Nix releases, the tarball fetcher (used by `builtins.fetchTarball`) erroneously merged top-level directories into a single directory, and silently discarded top-level files that are not directories. This is no longer the case.
Author: [**Eelco Dolstra (@edolstra)**](https://github.com/edolstra)
- Setting to warn about large paths [#10778](https://github.com/NixOS/nix/pull/10778)
Nix can now warn when evaluation of a Nix expression causes a large
@@ -268,6 +274,21 @@
be configured using the `warn-large-path-threshold` setting,
e.g. `--warn-large-path-threshold 100M`.
- Wrap filesystem exceptions more correctly [#11378](https://github.com/NixOS/nix/pull/11378)
With the switch to `std::filesystem` in different places, Nix started to throw `std::filesystem::filesystem_error` in many places instead of its own exceptions.
This led to no longer generating error traces, for example when listing a non-existing directory.
This version catches these types of exception correctly and wraps them into Nix's own exeception type.
Author: [**@Mic92**](https://github.com/Mic92)
- `<nix/fetchurl.nix>` uses TLS verification [#11585](https://github.com/NixOS/nix/pull/11585)
Previously `<nix/fetchurl.nix>` did not do TLS verification. This was because the Nix sandbox in the past did not have access to TLS certificates, and Nix checks the hash of the fetched file anyway. However, this can expose authentication data from `netrc` and URLs to man-in-the-middle attackers. In addition, Nix now in some cases (such as when using impure derivations) does *not* check the hash. Therefore we have now enabled TLS verification. This means that downloads by `<nix/fetchurl.nix>` will now fail if you're fetching from a HTTPS server that does not have a valid certificate.
`<nix/fetchurl.nix>` is also known as the builtin derivation builder `builtin:fetchurl`. It's not to be confused with the evaluation-time function `builtins.fetchurl`, which was not affected by this issue.
# Contributors
-2
View File
@@ -141,8 +141,6 @@ more than 2800 commits from 195 contributors since release 2.3.
the evaluation cache. This is made possible by the hermetic
evaluation model of flakes.
Intermediate results are not cached.
* The new `--offline` flag disables substituters and causes all
locally cached tarballs and repositories to be considered
up-to-date.
+31 -13
View File
@@ -11,10 +11,15 @@ rec {
concatStrings = concatStringsSep "";
attrsToList = a:
map (name: { inherit name; value = a.${name}; }) (builtins.attrNames a);
attrsToList =
a:
map (name: {
inherit name;
value = a.${name};
}) (builtins.attrNames a);
replaceStringsRec = from: to: string:
replaceStringsRec =
from: to: string:
# recursively replace occurrences of `from` with `to` within `string`
# example:
# replaceStringRec "--" "-" "hello-----world"
@@ -22,16 +27,18 @@ rec {
let
replaced = replaceStrings [ from ] [ to ] string;
in
if replaced == string then string else replaceStringsRec from to replaced;
if replaced == string then string else replaceStringsRec from to replaced;
toLower = replaceStrings upperChars lowerChars;
squash = replaceStringsRec "\n\n\n" "\n\n";
trim = string:
trim =
string:
# trim trailing spaces and squash non-leading spaces
let
trimLine = line:
trimLine =
line:
let
# separate leading spaces from the rest
parts = split "(^ *)" line;
@@ -39,19 +46,30 @@ rec {
rest = elemAt parts 2;
# drop trailing spaces
body = head (split " *$" rest);
in spaces + replaceStringsRec " " " " body;
in concatStringsSep "\n" (map trimLine (splitLines string));
in
spaces + replaceStringsRec " " " " body;
in
concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [];
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [ ];
nameValuePair = name: value: { inherit name value; };
filterAttrs = pred: set:
listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set));
filterAttrs =
pred: set:
listToAttrs (
concatMap (
name:
let
v = set.${name};
in
if pred name v then [ (nameValuePair name v) ] else [ ]
) (attrNames set)
);
optionalString = cond: string: if cond then string else "";
indent = prefix: s:
concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
indent =
prefix: s: concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
}
+200 -181
View File
@@ -1,70 +1,72 @@
{ pkgs ? import <nixpkgs> { }
, lib ? pkgs.lib
, name ? "nix"
, tag ? "latest"
, bundleNixpkgs ? true
, channelName ? "nixpkgs"
, channelURL ? "https://nixos.org/channels/nixpkgs-unstable"
, extraPkgs ? []
, maxLayers ? 100
, nixConf ? {}
, flake-registry ? null
{
pkgs ? import <nixpkgs> { },
lib ? pkgs.lib,
name ? "nix",
tag ? "latest",
bundleNixpkgs ? true,
channelName ? "nixpkgs",
channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
extraPkgs ? [ ],
maxLayers ? 100,
nixConf ? { },
flake-registry ? null,
}:
let
defaultPkgs = with pkgs; [
nix
bashInteractive
coreutils-full
gnutar
gzip
gnugrep
which
curl
less
wget
man
cacert.out
findutils
iana-etc
git
openssh
] ++ extraPkgs;
defaultPkgs =
with pkgs;
[
nix
bashInteractive
coreutils-full
gnutar
gzip
gnugrep
which
curl
less
wget
man
cacert.out
findutils
iana-etc
git
openssh
]
++ extraPkgs;
users = {
users =
{
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
root = {
uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
} // lib.listToAttrs (
map
(
n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}
)
(lib.lists.range 1 32)
);
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups = {
root.gid = 0;
@@ -74,24 +76,20 @@ let
userToPasswd = (
k:
{ uid
, gid ? 65534
, home ? "/var/empty"
, description ? ""
, shell ? "/bin/false"
, groups ? [ ]
}: "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
);
passwdContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToPasswd users))
{
uid,
gid ? 65534,
home ? "/var/empty",
description ? "",
shell ? "/bin/false",
groups ? [ ],
}:
"${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
);
passwdContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToPasswd users)));
userToShadow = k: { ... }: "${k}:!:1::::::";
shadowContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToShadow users))
);
shadowContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToShadow users)));
# Map groups to members
# {
@@ -101,42 +99,35 @@ let
let
# Create a flat list of user/group mappings
mappings = (
builtins.foldl'
(
acc: user:
let
groups = users.${user}.groups or [ ];
in
acc ++ map
(group: {
inherit user group;
})
groups
)
[ ]
(lib.attrNames users)
builtins.foldl' (
acc: user:
let
groups = users.${user}.groups or [ ];
in
acc
++ map (group: {
inherit user group;
}) groups
) [ ] (lib.attrNames users)
);
in
(
builtins.foldl'
(
acc: v: acc // {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
}
)
{ }
mappings)
(builtins.foldl' (
acc: v:
acc
// {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
}
) { } mappings)
);
groupToGroup = k: { gid }:
groupToGroup =
k:
{ gid }:
let
members = groupMemberMap.${k} or [ ];
in
"${k}:x:${toString gid}:${lib.concatStringsSep "," members}";
groupContents = (
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs groupToGroup groups))
);
groupContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs groupToGroup groups)));
defaultNixConf = {
sandbox = "false";
@@ -144,11 +135,17 @@ let
trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
};
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n";
nixConfContents =
(lib.concatStringsSep "\n" (
lib.mapAttrsFlatten (
n: v:
let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}"
) (defaultNixConf // nixConf)
))
+ "\n";
baseSystem =
let
@@ -167,21 +164,29 @@ let
manifest = pkgs.buildPackages.runCommand "manifest.nix" { } ''
cat > $out <<EOF
[
${lib.concatStringsSep "\n" (builtins.map (drv: let
outputs = drv.outputsToInstall or [ "out" ];
in ''
{
${lib.concatStringsSep "\n" (builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs)}
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
'') defaultPkgs)}
${lib.concatStringsSep "\n" (
builtins.map (
drv:
let
outputs = drv.outputsToInstall or [ "out" ];
in
''
{
${lib.concatStringsSep "\n" (
builtins.map (output: ''
${output} = { outPath = "${lib.getOutput output drv}"; };
'') outputs
)}
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
''
) defaultPkgs
)}
]
EOF
'';
@@ -190,16 +195,22 @@ let
cp -a ${rootEnv}/* $out/
ln -s ${manifest} $out/manifest.nix
'';
flake-registry-path = if (flake-registry == null) then
null
else if (builtins.readFileType (toString flake-registry)) == "directory" then
"${flake-registry}/flake-registry.json"
else
flake-registry;
flake-registry-path =
if (flake-registry == null) then
null
else if (builtins.readFileType (toString flake-registry)) == "directory" then
"${flake-registry}/flake-registry.json"
else
flake-registry;
in
pkgs.runCommand "base-system"
{
inherit passwdContents groupContents shadowContents nixConfContents;
inherit
passwdContents
groupContents
shadowContents
nixConfContents
;
passAsFile = [
"passwdContents"
"groupContents"
@@ -208,62 +219,66 @@ let
];
allowSubstitutes = false;
preferLocalBuild = true;
} (''
env
set -x
mkdir -p $out/etc
}
(
''
env
set -x
mkdir -p $out/etc
mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd
cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group
cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group
cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow
cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow
mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/nix/var/nix/gcroots
mkdir -p $out/nix/var/nix/gcroots
mkdir $out/tmp
mkdir $out/tmp
mkdir -p $out/var/tmp
mkdir -p $out/var/tmp
mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out/root
mkdir -p $out/nix/var/nix/profiles/per-user/root
mkdir -p $out/root
mkdir -p $out/nix/var/nix/profiles/per-user/root
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s $out/nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
ln -s $out/nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
ln -s /nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
mkdir -p $out/root/.nix-defexpr
ln -s $out/nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out/root/.nix-channels
mkdir -p $out/root/.nix-defexpr
ln -s /nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out/root/.nix-channels
mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
'' + (lib.optionalString (flake-registry-path != null) ''
nixCacheDir="/root/.cache/nix"
mkdir -p $out$nixCacheDir
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
mkdir -p $out/nix/var/nix/gcroots/auto
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
''));
''
+ (lib.optionalString (flake-registry-path != null) ''
nixCacheDir="/root/.cache/nix"
mkdir -p $out$nixCacheDir
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
mkdir -p $out/nix/var/nix/gcroots/auto
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
'')
);
in
pkgs.dockerTools.buildLayeredImageWithNixDb {
@@ -285,15 +300,19 @@ pkgs.dockerTools.buildLayeredImageWithNixDb {
Cmd = [ "/root/.nix-profile/bin/bash" ];
Env = [
"USER=root"
"PATH=${lib.concatStringsSep ":" [
"/root/.nix-profile/bin"
"/nix/var/nix/profiles/default/bin"
"/nix/var/nix/profiles/default/sbin"
]}"
"MANPATH=${lib.concatStringsSep ":" [
"/root/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]}"
"PATH=${
lib.concatStringsSep ":" [
"/root/.nix-profile/bin"
"/nix/var/nix/profiles/default/bin"
"/nix/var/nix/profiles/default/sbin"
]
}"
"MANPATH=${
lib.concatStringsSep ":" [
"/root/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]
}"
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
Generated
+4 -4
View File
@@ -80,16 +80,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1726674819,
"narHash": "sha256-qyfnYJsZq8cBBat6oGjHnu5MaWbNw47Ph31fX0iWPIU=",
"lastModified": 1723688146,
"narHash": "sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz+NG82pbdg=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "3a458f7c763ca62c6bf454b8d828bd86b7250671",
"rev": "c3d4ac725177c030b1e289015989da2ad9d56af0",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "release-24.05",
"ref": "nixos-24.05",
"repo": "nixpkgs",
"type": "github"
}
+348 -277
View File
@@ -1,11 +1,17 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-24.05";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
inputs.libgit2 = { url = "github:libgit2/libgit2/v1.8.1"; flake = false; };
inputs.flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
inputs.libgit2 = {
url = "github:libgit2/libgit2/v1.8.1";
flake = false;
};
# dev tooling
inputs.flake-parts.url = "github:hercules-ci/flake-parts";
@@ -18,18 +24,39 @@
inputs.git-hooks-nix.inputs.flake-compat.follows = "";
inputs.git-hooks-nix.inputs.gitignore.follows = "";
outputs = inputs@{ self, nixpkgs, nixpkgs-regression, libgit2, ... }:
outputs =
inputs@{
self,
nixpkgs,
nixpkgs-regression,
libgit2,
...
}:
let
inherit (nixpkgs) lib;
officialRelease = false;
officialRelease = true;
version = lib.fileContents ./.version + versionSuffix;
versionSuffix =
if officialRelease then
""
else
"pre${
builtins.substring 0 8 (self.lastModifiedDate or self.lastModified or "19700101")
}_${self.shortRev or "dirty"}";
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ];
linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ];
darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
systems = linuxSystems ++ darwinSystems;
crossSystems = [
@@ -58,62 +85,76 @@
(Provided that the names are unique.)
See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs
*/
*/
flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs;
forAllSystems = lib.genAttrs systems;
forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs = f:
lib.listToAttrs
(map
(stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
})
stdenvs);
forAllStdenvs =
f:
lib.listToAttrs (
map (stdenvName: {
name = "${stdenvName}Packages";
value = f stdenvName;
}) stdenvs
);
# We don't apply flake-parts to the whole flake so that non-development attributes
# load without fetching any development inputs.
devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } {
imports = [ ./maintainers/flake-module.nix ];
systems = lib.subtractLists crossSystems systems;
perSystem = { system, ... }: {
_module.args.pkgs = nixpkgsFor.${system}.native;
};
perSystem =
{ system, ... }:
{
_module.args.pkgs = nixpkgsFor.${system}.native;
};
};
# Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems
(system: let
make-pkgs = crossSystem: stdenv: import nixpkgs {
localSystem = {
inherit system;
nixpkgsFor = forAllSystems (
system:
let
make-pkgs =
crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem =
if crossSystem == null then
null
else
{
config = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
};
crossSystem = if crossSystem == null then null else {
config = crossSystem;
} // lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
};
stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages;
in {
in
{
inherit stdenvs native;
static = native.pkgsStatic;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv");
});
}
);
binaryTarball = nix: pkgs: pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
binaryTarball =
nix: pkgs:
pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
overlayFor = getStdenv: final: prev:
overlayFor =
getStdenv: final: prev:
let
stdenv = getStdenv final;
in
@@ -124,37 +165,41 @@
# without "polluting" the top level "`pkgs`" attrset.
# This also has the benefit of providing us with a distinct set of packages
# we can iterate over.
nixComponents = lib.makeScope final.nixDependencies.newScope (import ./packaging/components.nix {
inherit (final) lib;
inherit officialRelease;
src = self;
});
nixComponents = lib.makeScope final.nixDependencies.newScope (import ./packaging/components.nix);
# The dependencies are in their own scope, so that they don't have to be
# in Nixpkgs top level `pkgs` or `nixComponents`.
nixDependencies = lib.makeScope final.newScope (import ./packaging/dependencies.nix {
inherit inputs stdenv;
pkgs = final;
});
nixDependencies = lib.makeScope final.newScope (
import ./packaging/dependencies.nix {
inherit inputs stdenv versionSuffix;
pkgs = final;
}
);
nix = final.nixComponents.nix;
nix-serve =
prev.nix-serve.override {
# undo potential version pinning
nix = final.nix;
};
nix_noTests = final.nix.override {
doInstallCheck = false;
doCheck = false;
};
# See https://github.com/NixOS/nixpkgs/pull/214409
# Remove when fixed in this flake's nixpkgs
pre-commit =
if prev.stdenv.hostPlatform.system == "i686-linux"
then (prev.pre-commit.override (o: { dotnet-sdk = ""; })).overridePythonAttrs (o: { doCheck = false; })
else prev.pre-commit;
if prev.stdenv.hostPlatform.system == "i686-linux" then
(prev.pre-commit.override (o: {
dotnet-sdk = "";
})).overridePythonAttrs
(o: {
doCheck = false;
})
else
prev.pre-commit;
};
in {
in
{
# A Nixpkgs overlay that overrides the 'nix' and
# 'nix-perl-bindings' packages.
overlays.default = overlayFor (p: p.stdenv);
@@ -169,238 +214,264 @@
linux64BitSystems
nixpkgsFor
self
officialRelease
;
};
checks = forAllSystems (system: {
binaryTarball = self.hydraJobs.binaryTarball.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
let pkgs = nixpkgsFor.${system}.native;
in pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
'';
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
} // (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
checks = forAllSystems (
system:
{
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
installTests = self.hydraJobs.installTests.${system};
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
rl-next =
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
'';
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
}
// (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
}
// (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system};
}
/*
# Add "passthru" tests
// flatMapAttrs ({
"" = nixpkgsFor.${system}.native;
} // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
# TODO: disabled to speed up GHA CI.
#"static-" = nixpkgsFor.${system}.static;
})
(nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents
(pkgName: pkg:
flatMapAttrs pkg.tests or {}
(testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test;
})
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system};
}
# Add "passthru" tests
//
flatMapAttrs
(
{
"" = nixpkgsFor.${system}.native;
}
// lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
"static-" = nixpkgsFor.${system}.static;
}
)
// lib.optionalAttrs (nixpkgs.stdenv.hostPlatform == nixpkgs.stdenv.buildPlatform) {
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests;
}
)
*/
// devFlake.checks.${system} or {}
(
nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents (
pkgName: pkg:
flatMapAttrs pkg.tests or { } (
testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test;
}
)
)
)
// devFlake.checks.${system} or { }
);
packages = forAllSystems (system:
{ # Here we put attributes that map 1:1 into packages.<system>, ie
packages = forAllSystems (
system:
{
# Here we put attributes that map 1:1 into packages.<system>, ie
# for which we don't apply the full build matrix such as cross or static.
inherit (nixpkgsFor.${system}.native)
changelog-d;
default = self.packages.${system}.nix-ng;
changelog-d
;
default = self.packages.${system}.nix;
binaryTarball = self.hydraJobs.binaryTarball.${system};
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
nix-internal-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-internal-api-docs;
nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs;
}
# We need to flatten recursive attribute sets of derivations to pass `flake check`.
// flatMapAttrs
{ # Components we'll iterate over in the upcoming lambda
"nix" = { };
"nix-util" = { };
"nix-util-c" = { };
"nix-util-test-support" = { };
"nix-util-tests" = { };
"nix-store" = { };
"nix-store-c" = { };
"nix-store-test-support" = { };
"nix-store-tests" = { };
"nix-fetchers" = { };
"nix-fetchers-tests" = { };
"nix-expr" = { };
"nix-expr-c" = { };
"nix-expr-test-support" = { };
"nix-expr-tests" = { };
"nix-flake" = { };
"nix-flake-tests" = { };
"nix-main" = { };
"nix-main-c" = { };
"nix-cmd" = { };
"nix-cli" = { };
"nix-functional-tests" = { supportsCross = false; };
"nix-perl-bindings" = { supportsCross = false; };
"nix-ng" = { };
}
(pkgName: { supportsCross ? true }: {
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
}
// lib.optionalAttrs supportsCross (flatMapAttrs (lib.genAttrs crossSystems (_: { })) (crossSystem: {}: {
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
}))
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (stdenvName: {}: {
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
})
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix { inherit pkgs; tag = pkgs.nix.version; };
in
pkgs.runCommand
"docker-image-tarball-${pkgs.nix.version}"
{ meta.description = "Docker image with Nix for ${system}"; }
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
});
devShells = let
makeShell = pkgs: stdenv: (pkgs.nix.override { inherit stdenv; forDevShell = true; }).overrideAttrs (attrs:
let
buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag = prefix: flag:
assert builtins.isString flag;
let
rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in {
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ./.version;
name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc";
shellHook = ''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
# We use this shell with the local checkout, not unpackPhase.
src = null;
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG =
"${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml" modular.pre-commit.settings.rawConfig}";
};
mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags))
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags)
;
nativeBuildInputs = attrs.nativeBuildInputs or []
++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-functional-tests.baseNativeBuildInputs
++ lib.optional
(!buildCanExecuteHost
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages))
pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install"
modular.pre-commit.settings.installationScript)
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (lib.hiPrio pkgs.buildPackages.clang-tools);
buildInputs = attrs.buildInputs or []
++ [
pkgs.gtest
pkgs.rapidcheck
]
++ lib.optional havePerl pkgs.perl
;
});
in
forAllSystems (system:
let
makeShells = prefix: pkgs:
lib.mapAttrs'
(k: v: lib.nameValuePair "${prefix}-${k}" v)
(forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName}));
in
(makeShells "native" nixpkgsFor.${system}.native) //
(lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin)
(makeShells "static" nixpkgsFor.${system}.static) //
(forAllCrossSystems (crossSystem: let pkgs = nixpkgsFor.${system}.cross.${crossSystem}; in makeShell pkgs pkgs.stdenv))) //
//
flatMapAttrs
{
default = self.devShells.${system}.native-stdenvPackages;
# Components we'll iterate over in the upcoming lambda
"nix" = { };
# Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build
# system, we should reenable these.
#"nix-util" = { };
#"nix-store" = { };
#"nix-fetchers" = { };
}
(
pkgName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
}
// flatMapAttrs (lib.genAttrs crossSystems (_: { })) (
crossSystem:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
}
)
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (
stdenvName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" =
nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
}
)
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage =
let
pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix {
inherit pkgs;
tag = version;
};
in
pkgs.runCommand "docker-image-tarball-${version}"
{ meta.description = "Docker image with Nix for ${system}"; }
''
mkdir -p $out/nix-support
image=$out/image.tar.gz
ln -s ${image} $image
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
}
);
devShells =
let
makeShell =
pkgs: stdenv:
(pkgs.nix.override {
inherit stdenv;
forDevShell = true;
}).overrideAttrs
(
attrs:
let
modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag =
prefix: flag:
assert builtins.isString flag;
let
rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in
{
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ./.version;
name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc";
shellHook = ''
PATH=$prefix/bin:$PATH
unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH
# Make bash completion work.
XDG_DATA_DIRS+=:$out/share
'';
# We use this shell with the local checkout, not unpackPhase.
src = null;
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
modular.pre-commit.settings.rawConfig
}";
};
mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (
map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)
)
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags);
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ lib.optional (
!stdenv.buildPlatform.canExecute stdenv.hostPlatform
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
) pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (
stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform
) pkgs.buildPackages.clang-tools;
buildInputs =
attrs.buildInputs or [ ]
++ [
pkgs.gtest
pkgs.rapidcheck
]
++ lib.optional havePerl pkgs.perl;
}
);
in
forAllSystems (
system:
let
makeShells =
prefix: pkgs:
lib.mapAttrs' (k: v: lib.nameValuePair "${prefix}-${k}" v) (
forAllStdenvs (stdenvName: makeShell pkgs pkgs.${stdenvName})
);
in
(makeShells "native" nixpkgsFor.${system}.native)
// (
lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) (
makeShells "static" nixpkgsFor.${system}.static
)
// (forAllCrossSystems (
crossSystem:
let
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
in
makeShell pkgs pkgs.stdenv
))
)
// {
default = self.devShells.${system}.native-stdenvPackages;
}
);
};
};
}
File diff suppressed because it is too large. Load diff
-11
View File
@@ -1,11 +0,0 @@
#!/usr/bin/env bash
if ! type -p pre-commit &>/dev/null; then
echo "format.sh: pre-commit not found. Please use \`nix develop\`.";
exit 1;
fi;
if test -z "$_NIX_PRE_COMMIT_HOOKS_CONFIG"; then
echo "format.sh: _NIX_PRE_COMMIT_HOOKS_CONFIG not set. Please use \`nix develop\`.";
exit 1;
fi;
pre-commit run --config "$_NIX_PRE_COMMIT_HOOKS_CONFIG" --all-files
+9 -2
View File
@@ -3,6 +3,13 @@
print-top-help += echo ' format: Format source code'
# This uses the cached .pre-commit-hooks.yaml file
fmt_script := $(d)/format.sh
format:
@$(fmt_script)
@if ! type -p pre-commit &>/dev/null; then \
echo "make format: pre-commit not found. Please use \`nix develop\`."; \
exit 1; \
fi; \
if test -z "$$_NIX_PRE_COMMIT_HOOKS_CONFIG"; then \
echo "make format: _NIX_PRE_COMMIT_HOOKS_CONFIG not set. Please use \`nix develop\`."; \
exit 1; \
fi; \
pre-commit run --config $$_NIX_PRE_COMMIT_HOOKS_CONFIG --all-files
+2
View File
@@ -2,6 +2,8 @@
# vim: set filetype=bash:
#!nix shell .#changelog-d --command bash
set -euo pipefail
# --- CONFIGURATION ---
# This does double duty for
+1 -4
View File
@@ -3,9 +3,7 @@
project('nix-dev-shell', 'cpp',
version : files('.version'),
default_options : [
'localstatedir=/nix/var',
]
subproject_dir : 'src',
)
# Internal Libraries
@@ -44,4 +42,3 @@ subproject('nix-fetchers-tests')
subproject('nix-expr-test-support')
subproject('nix-expr-tests')
subproject('nix-flake-tests')
subproject('nix-functional-tests')
-8
View File
@@ -1,8 +0,0 @@
configure_file(
input : 'completion.sh',
output : 'nix',
install : true,
install_dir : get_option('datadir') / 'bash-completion' / 'completions',
install_mode : 'rw-r--r--',
copy : true,
)
-8
View File
@@ -1,8 +0,0 @@
configure_file(
input : 'completion.fish',
output : 'nix.fish',
install : true,
install_dir : get_option('datadir') / 'fish' / 'vendor_completions.d',
install_mode : 'rw-r--r--',
copy : true,
)
@@ -2,6 +2,11 @@
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>EnvironmentVariables</key>
<dict>
<key>OBJC_DISABLE_INITIALIZE_FORK_SAFETY</key>
<string>YES</string>
</dict>
<key>Label</key>
<string>org.nixos.nix-daemon</string>
<key>KeepAlive</key>
-5
View File
@@ -1,5 +0,0 @@
subdir('bash')
subdir('fish')
subdir('zsh')
subdir('systemd')
-25
View File
@@ -1,25 +0,0 @@
foreach config : [ 'nix-daemon.socket', 'nix-daemon.service' ]
configure_file(
input : config + '.in',
output : config,
install : true,
install_dir : get_option('prefix') / 'lib/systemd/system',
install_mode : 'rw-r--r--',
configuration : {
'storedir' : store_dir,
'localstatedir' : localstatedir,
'bindir' : bindir,
},
)
endforeach
configure_file(
input : 'nix-daemon.conf.in',
output : 'nix-daemon.conf',
install : true,
install_dir : get_option('prefix') / 'lib/tmpfiles.d',
install_mode : 'rw-r--r--',
configuration : {
'localstatedir' : localstatedir,
},
)
-1
View File
@@ -11,7 +11,6 @@ ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
KillMode=process
LimitNOFILE=1048576
TasksMax=1048576
Delegate=yes
[Install]
WantedBy=multi-user.target
-10
View File
@@ -1,10 +0,0 @@
foreach script : [ [ 'completion.zsh', '_nix' ], [ 'run-help-nix' ] ]
configure_file(
input : script[0],
output : script.get(1, script[0]),
install : true,
install_dir : get_option('datadir') / 'zsh/site-functions',
install_mode : 'rw-r--r--',
copy : true,
)
endforeach
+3 -1
View File
@@ -86,7 +86,9 @@ define build-library
else
ifndef HOST_DARWIN
ifndef HOST_WINDOWS
$(1)_LDFLAGS += -Wl,-z,defs
ifndef HOST_OPENBSD
$(1)_LDFLAGS += -Wl,-z,defs
endif
endif
endif
endif
+4
View File
@@ -21,6 +21,10 @@ ifdef HOST_OS
HOST_NETBSD = 1
HOST_UNIX = 1
endif
ifeq ($(patsubst openbsd%,,$(HOST_KERNEL)),)
HOST_OPENBSD = 1
HOST_UNIX = 1
endif
ifeq ($(HOST_KERNEL), linux)
HOST_LINUX = 1
HOST_UNIX = 1
+365 -304
View File
@@ -1,119 +1,123 @@
{ lib
, stdenv
, releaseTools
, autoconf-archive
, autoreconfHook
, aws-sdk-cpp
, boehmgc
, nlohmann_json
, bison
, boost
, brotli
, bzip2
, curl
, editline
, readline
, flex
, git
, gtest
, jq
, libarchive
, libcpuid
, libgit2
, libseccomp
, libsodium
, man
, lowdown
, mdbook
, mdbook-linkcheck
, mercurial
, nix-perl-bindings
, openssh
, openssl
, pkg-config
, rapidcheck
, sqlite
, toml11
, unixtools
, xz
{
lib,
stdenv,
releaseTools,
autoconf-archive,
autoreconfHook,
aws-sdk-cpp,
boehmgc,
nlohmann_json,
bison,
boost,
brotli,
bzip2,
curl,
editline,
readline,
flex,
git,
gtest,
jq,
libarchive,
libcpuid,
libgit2,
libseccomp,
libsodium,
man,
darwin,
lowdown,
mdbook,
mdbook-linkcheck,
mercurial,
openssh,
openssl,
pkg-config,
rapidcheck,
sqlite,
toml11,
unixtools,
xz,
, busybox-sandbox-shell ? null
busybox-sandbox-shell ? null,
# Configuration Options
#:
# This probably seems like too many degrees of freedom, but it
# faithfully reflects how the underlying configure + make build system
# work. The top-level flake.nix will choose useful combinations of these
# options to CI.
# Configuration Options
#:
# This probably seems like too many degrees of freedom, but it
# faithfully reflects how the underlying configure + make build system
# work. The top-level flake.nix will choose useful combinations of these
# options to CI.
, pname ? "nix"
pname ? "nix",
, version
, versionSuffix
versionSuffix ? "",
# Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix
, doBuild ? true
# Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix
doBuild ? true,
# Run the unit tests as part of the build. See `installUnitTests` for an
# alternative to this.
, doCheck ? __forDefaults.canRunInstalled
# Run the unit tests as part of the build. See `installUnitTests` for an
# alternative to this.
doCheck ? __forDefaults.canRunInstalled,
# Run the functional tests as part of the build.
, doInstallCheck ? test-client != null || __forDefaults.canRunInstalled
# Run the functional tests as part of the build.
doInstallCheck ? test-client != null || __forDefaults.canRunInstalled,
# Check test coverage of Nix. Probably want to use with at least
# one of `doCHeck` or `doInstallCheck` enabled.
, withCoverageChecks ? false
# Check test coverage of Nix. Probably want to use with at least
# one of `doCHeck` or `doInstallCheck` enabled.
withCoverageChecks ? false,
# Whether to build the regular manual
, enableManual ? __forDefaults.canRunInstalled
# Whether to build the regular manual
enableManual ? __forDefaults.canRunInstalled,
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows
# Whether to use garbage collection for the Nix language evaluator.
#
# If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.)
#
# Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking.
#
# Disabled on OpenBSD because of missing `_data_start` symbol while linking
enableGC ? !stdenv.hostPlatform.isWindows && !stdenv.hostPlatform.isOpenBSD,
# Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows
# Whether to enable Markdown rendering in the Nix binary.
enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline"
# Which interactive line editor library to use for Nix's repl.
#
# Currently supported choices are:
#
# - editline (default)
# - readline
readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
# Whether to install unit tests. This is useful when cross compiling
# since we cannot run them natively during the build, but can do so
# later.
, installUnitTests ? doBuild && !__forDefaults.canExecuteHost
# Whether to install unit tests. This is useful when cross compiling
# since we cannot run them natively during the build, but can do so
# later.
installUnitTests ? doBuild && !__forDefaults.canExecuteHost,
# For running the functional tests against a pre-built Nix. Probably
# want to use in conjunction with `doBuild = false;`.
, test-daemon ? null
, test-client ? null
# For running the functional tests against a pre-built Nix. Probably
# want to use in conjunction with `doBuild = false;`.
test-daemon ? null,
test-client ? null,
# Avoid setting things that would interfere with a functioning devShell
, forDevShell ? false
# Avoid setting things that would interfere with a functioning devShell
forDevShell ? false,
# Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults.
, __forDefaults ? {
# Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults.
__forDefaults ? {
canExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
canRunInstalled = doBuild && __forDefaults.canExecuteHost;
}
},
}:
let
inherit (lib) fileset;
version = lib.fileContents ./.version + versionSuffix;
# selected attributes with defaults, will be used to define some
# things which should instead be gotten via `finalAttrs` in order to
# work with overriding.
@@ -122,247 +126,304 @@ let
};
mkDerivation =
if withCoverageChecks
then
if withCoverageChecks then
# TODO support `finalAttrs` args function in
# `releaseTools.coverageAnalysis`.
argsFun:
releaseTools.coverageAnalysis (let args = argsFun args; in args)
else stdenv.mkDerivation;
releaseTools.coverageAnalysis (
let
args = argsFun args;
in
args
)
else
stdenv.mkDerivation;
in
mkDerivation (finalAttrs: let
mkDerivation (
finalAttrs:
let
inherit (finalAttrs)
doCheck
doInstallCheck
;
inherit (finalAttrs)
doCheck
doInstallCheck
;
doBuild = !finalAttrs.dontBuild;
doBuild = !finalAttrs.dontBuild;
# Either running the unit tests during the build, or installing them
# to be run later, requiresthe unit tests to be built.
buildUnitTests = doCheck || installUnitTests;
# Either running the unit tests during the build, or installing them
# to be run later, requiresthe unit tests to be built.
buildUnitTests = doCheck || installUnitTests;
in {
inherit pname version;
in
{
inherit pname version;
src =
let
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
in
src =
let
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
in
fileset.toSource {
root = ./.;
fileset = fileset.intersection baseFiles (fileset.unions ([
# For configure
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
# This could be put behind a conditional
./maintainers/local.mk
# For make, regardless of what we are building
./local.mk
./Makefile
./Makefile.config.in
./mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
] ++ lib.optionals doBuild [
./doc
./misc
./precompiled-headers.h
(fileset.difference ./src ./src/perl)
./COPYING
./scripts/local.mk
] ++ lib.optionals enableManual [
./doc/manual
] ++ lib.optionals buildUnitTests [
./tests/unit
] ++ lib.optionals doInstallCheck [
./tests/functional
]));
fileset = fileset.intersection baseFiles (
fileset.unions (
[
# For configure
./.version
./configure.ac
./m4
# TODO: do we really need README.md? It doesn't seem used in the build.
./README.md
# This could be put behind a conditional
./maintainers/local.mk
# For make, regardless of what we are building
./local.mk
./Makefile
./Makefile.config.in
./mk
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
]
++ lib.optionals doBuild [
./doc
./misc
./precompiled-headers.h
(fileset.difference ./src ./src/perl)
./COPYING
./scripts/local.mk
]
++ lib.optionals enableManual [
./doc/manual
]
++ lib.optionals buildUnitTests [
./tests/unit
]
++ lib.optionals doInstallCheck [
./tests/functional
]
)
);
};
VERSION_SUFFIX = versionSuffix;
VERSION_SUFFIX = versionSuffix;
outputs = [ "out" ]
++ lib.optional doBuild "dev"
# If we are doing just build or just docs, the one thing will use
# "out". We only need additional outputs if we are doing both.
++ lib.optional (doBuild && enableManual) "doc"
++ lib.optional installUnitTests "check"
++ lib.optional doCheck "testresults"
;
outputs =
[ "out" ]
++ lib.optional doBuild "dev"
# If we are doing just build or just docs, the one thing will use
# "out". We only need additional outputs if we are doing both.
++ lib.optional (doBuild && enableManual) "doc"
++ lib.optional installUnitTests "check"
++ lib.optional doCheck "testresults";
nativeBuildInputs = [
autoconf-archive
autoreconfHook
pkg-config
] ++ lib.optionals doBuild [
bison
flex
] ++ lib.optionals enableManual [
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
] ++ lib.optionals doInstallCheck [
git
mercurial
openssh
man # for testing `nix-* --help`
] ++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
] ++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump
;
nativeBuildInputs =
[
autoconf-archive
autoreconfHook
pkg-config
]
++ lib.optionals doBuild [
bison
flex
]
++ lib.optionals enableManual [
(lib.getBin lowdown)
mdbook
mdbook-linkcheck
]
++ lib.optionals doInstallCheck [
git
mercurial
openssh
]
++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
]
++ lib.optionals enableManual [
man
]
++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump;
buildInputs = lib.optionals doBuild (
[
brotli
bzip2
curl
libarchive
libgit2
libsodium
openssl
sqlite
toml11
xz
({ inherit readline editline; }.${readlineFlavor})
] ++ lib.optionals enableMarkdown [
lowdown
] ++ lib.optionals buildUnitTests [
gtest
rapidcheck
] ++ lib.optional stdenv.isLinux libseccomp
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin))
aws-sdk-cpp
);
buildInputs = lib.optionals doBuild (
[
brotli
bzip2
curl
libarchive
libgit2
libsodium
openssl
sqlite
toml11
xz
({ inherit readline editline; }.${readlineFlavor})
]
++ lib.optionals enableMarkdown [
lowdown
]
++ lib.optionals buildUnitTests [
gtest
rapidcheck
]
++ lib.optional stdenv.isLinux libseccomp
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
# There have been issues building these dependencies
++ lib.optional (
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)
) aws-sdk-cpp
);
propagatedBuildInputs = lib.optionals doBuild ([
boost
nlohmann_json
] ++ lib.optional enableGC boehmgc
);
propagatedBuildInputs = lib.optionals doBuild (
[
boost
nlohmann_json
]
++ lib.optional enableGC boehmgc
);
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck;
configureFlags = [
(lib.enableFeature doBuild "build")
(lib.enableFeature buildUnitTests "unit-tests")
(lib.enableFeature doInstallCheck "functional-tests")
(lib.enableFeature enableManual "doc-gen")
(lib.enableFeature enableGC "gc")
(lib.enableFeature enableMarkdown "markdown")
(lib.enableFeature installUnitTests "install-unit-tests")
(lib.withFeatureAs true "readline-flavor" readlineFlavor)
] ++ lib.optionals (!forDevShell) [
"--sysconfdir=/etc"
] ++ lib.optionals installUnitTests [
"--with-check-bin-dir=${builtins.placeholder "check"}/bin"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib"
] ++ lib.optionals (doBuild) [
"--with-boost=${boost}/lib"
] ++ lib.optionals (doBuild && stdenv.isLinux) [
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox"
] ++ lib.optional (doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
"LDFLAGS=-fuse-ld=gold"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell"
;
configureFlags =
[
(lib.enableFeature doBuild "build")
(lib.enableFeature buildUnitTests "unit-tests")
(lib.enableFeature doInstallCheck "functional-tests")
(lib.enableFeature enableManual "doc-gen")
(lib.enableFeature enableGC "gc")
(lib.enableFeature enableMarkdown "markdown")
(lib.enableFeature installUnitTests "install-unit-tests")
(lib.withFeatureAs true "readline-flavor" readlineFlavor)
]
++ lib.optionals (!forDevShell) [
"--sysconfdir=/etc"
]
++ lib.optionals installUnitTests [
"--with-check-bin-dir=${builtins.placeholder "check"}/bin"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib"
]
++ lib.optionals (doBuild) [
"--with-boost=${boost}/lib"
]
++ lib.optionals (doBuild && stdenv.isLinux) [
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox"
]
++ lib.optional (
doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")
) "LDFLAGS=-fuse-ld=gold"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell";
enableParallelBuilding = true;
enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
preCheck = ''
mkdir $testresults
'';
installTargets = lib.optional doBuild "install";
installFlags = "sysconfdir=$(out)/etc";
# In this case we are probably just running tests, and so there isn't
# anything to install, we just make an empty directory to signify tests
# succeeded.
installPhase = if finalAttrs.installTargets != [] then null else ''
mkdir -p $out
'';
postInstall = lib.optionalString doBuild (
lib.optionalString stdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''
) + lib.optionalString enableManual ''
mkdir -p ''${!outputDoc}/nix-support
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
# So the check output gets links for DLLs in the out output.
preFixup = lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs) ''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
doInstallCheck = attrs.doInstallCheck;
installCheckFlags = "sysconfdir=$(out)/etc";
# Work around buggy detection in stdenv.
installCheckTarget = "installcheck";
# Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase = if (!doBuild && doInstallCheck) then ''
runHook preInstallCheck
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
'' else null;
# Needed for tests if we are not doing a build, but testing existing
# built Nix.
preInstallCheck =
lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel
preCheck = ''
mkdir $testresults
'';
separateDebugInfo = !stdenv.hostPlatform.isStatic;
installTargets = lib.optional doBuild "install";
# TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564
strictDeps = !withCoverageChecks;
installFlags = "sysconfdir=$(out)/etc";
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
# In this case we are probably just running tests, and so there isn't
# anything to install, we just make an empty directory to signify tests
# succeeded.
installPhase =
if finalAttrs.installTargets != [ ] then
null
else
''
mkdir -p $out
'';
passthru = lib.optionalAttrs (stdenv.buildPlatform.canExecute stdenv.hostPlatform) {
perl-bindings = nix-perl-bindings;
};
postInstall =
lib.optionalString doBuild (
lib.optionalString stdenv.hostPlatform.isStatic ''
mkdir -p $out/nix-support
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
''
)
+ lib.optionalString enableManual ''
mkdir -p ''${!outputDoc}/nix-support
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix";
broken = !(lib.all (a: a) [
# We cannot run or install unit tests if we don't build them or
# Nix proper (which they depend on).
(installUnitTests -> doBuild)
(doCheck -> doBuild)
# The build process for the manual currently requires extracting
# data from the Nix executable we are trying to document.
(enableManual -> doBuild)
]);
};
# So the check output gets links for DLLs in the out output.
preFixup =
lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs)
''
ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
} // lib.optionalAttrs withCoverageChecks {
lcovFilter = [ "*/boost/*" "*-tab.*" ];
doInstallCheck = attrs.doInstallCheck;
hardeningDisable = ["fortify"];
installCheckFlags = "sysconfdir=$(out)/etc";
# Work around buggy detection in stdenv.
installCheckTarget = "installcheck";
NIX_CFLAGS_COMPILE = "-DCOVERAGE=1";
# Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase =
if (!doBuild && doInstallCheck) then
''
runHook preInstallCheck
mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
''
else
null;
dontInstall = false;
} // lib.optionalAttrs (test-daemon != null) {
NIX_DAEMON_PACKAGE = test-daemon;
} // lib.optionalAttrs (test-client != null) {
NIX_CLIENT_PACKAGE = test-client;
})
# Needed for tests if we are not doing a build, but testing existing
# built Nix.
preInstallCheck =
lib.optionalString (!doBuild) ''
mkdir -p src/nix-channel
''
# See https://github.com/NixOS/nix/issues/2523
# Occurs often in tests since https://github.com/NixOS/nix/pull/9900
+ lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
separateDebugInfo = !stdenv.hostPlatform.isStatic;
# TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564
strictDeps = !withCoverageChecks;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix";
broken =
!(lib.all (a: a) [
# We cannot run or install unit tests if we don't build them or
# Nix proper (which they depend on).
(installUnitTests -> doBuild)
(doCheck -> doBuild)
# The build process for the manual currently requires extracting
# data from the Nix executable we are trying to document.
(enableManual -> doBuild)
]);
};
}
// lib.optionalAttrs withCoverageChecks {
lcovFilter = [
"*/boost/*"
"*-tab.*"
];
hardeningDisable = [ "fortify" ];
NIX_CFLAGS_COMPILE = "-DCOVERAGE=1";
dontInstall = false;
}
// lib.optionalAttrs (test-daemon != null) {
NIX_DAEMON_PACKAGE = test-daemon;
}
// lib.optionalAttrs (test-client != null) {
NIX_CLIENT_PACKAGE = test-client;
}
)
+5 -32
View File
@@ -1,34 +1,11 @@
{
lib,
src,
officialRelease,
}:
scope:
let
inherit (scope) callPackage;
baseVersion = lib.fileContents ../.version;
versionSuffix = lib.optionalString (!officialRelease) "pre";
fineVersionSuffix = lib.optionalString
(!officialRelease)
"pre${builtins.substring 0 8 (src.lastModifiedDate or src.lastModified or "19700101")}_${src.shortRev or "dirty"}";
fineVersion = baseVersion + fineVersionSuffix;
in
# This becomes the pkgs.nixComponents attribute set
{
version = baseVersion + versionSuffix;
inherit versionSuffix;
nix = callPackage ../package.nix {
version = fineVersion;
versionSuffix = fineVersionSuffix;
};
nix = callPackage ../package.nix { };
nix-util = callPackage ../src/libutil/package.nix { };
nix-util-c = callPackage ../src/libutil-c/package.nix { };
@@ -56,15 +33,11 @@ in
nix-cmd = callPackage ../src/libcmd/package.nix { };
nix-cli = callPackage ../src/nix/package.nix { version = fineVersion; };
# Will replace `nix` once the old build system is gone.
nix-ng = callPackage ../src/nix/package.nix { };
nix-functional-tests = callPackage ../src/nix-functional-tests/package.nix { version = fineVersion; };
nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { version = fineVersion; };
nix-external-api-docs = callPackage ../src/external-api-docs/package.nix { version = fineVersion; };
nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { };
nix-external-api-docs = callPackage ../src/external-api-docs/package.nix { };
nix-perl-bindings = callPackage ../src/perl/package.nix { };
# Will replace `nix` once the old build system is gone.
nix-ng = callPackage ../packaging/everything.nix { };
}
+83 -92
View File
@@ -8,6 +8,7 @@
pkgs,
stdenv,
versionSuffix,
}:
let
@@ -19,9 +20,7 @@ let
root = ../.;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64
then darwinStdenv
else prevStdenv;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64 then darwinStdenv else prevStdenv;
# Fix the following error with the default x86_64-darwin SDK:
#
@@ -38,7 +37,8 @@ let
# Indirection for Nixpkgs to override when package.nix files are vendored
filesetToSource = lib.fileset.toSource;
localSourceLayer = finalAttrs: prevAttrs:
localSourceLayer =
finalAttrs: prevAttrs:
let
workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
@@ -47,8 +47,13 @@ let
prevAttrs.workDir;
workDirSubpath = lib.path.removePrefix root workDirPath;
sources = assert prevAttrs.fileset._type == "fileset"; prevAttrs.fileset;
src = lib.fileset.toSource { fileset = sources; inherit root; };
sources =
assert prevAttrs.fileset._type == "fileset";
prevAttrs.fileset;
src = lib.fileset.toSource {
fileset = sources;
inherit root;
};
in
{
@@ -62,28 +67,35 @@ let
# Work around weird `--as-needed` linker behavior with BSD, see
# https://github.com/mesonbuild/meson/issues/3593
bsdNoLinkAsNeeded = finalAttrs: prevAttrs:
bsdNoLinkAsNeeded =
finalAttrs: prevAttrs:
lib.optionalAttrs stdenv.hostPlatform.isBSD {
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [];
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
};
miscGoodPractice = finalAttrs: prevAttrs:
{
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
};
miscGoodPractice = finalAttrs: prevAttrs: {
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
};
in
scope: {
inherit stdenv;
inherit stdenv versionSuffix;
version = lib.fileContents ../.version + versionSuffix;
aws-sdk-cpp = (pkgs.aws-sdk-cpp.override {
apis = [ "s3" "transfer" ];
customMemoryManagement = false;
}).overrideAttrs {
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
aws-sdk-cpp =
(pkgs.aws-sdk-cpp.override {
apis = [
"s3"
"transfer"
];
customMemoryManagement = false;
}).overrideAttrs
{
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
libseccomp = pkgs.libseccomp.overrideAttrs (_: rec {
version = "2.5.5";
@@ -98,71 +110,50 @@ scope: {
};
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost = (pkgs.boost.override {
extraB2Args = [
"--with-container"
"--with-context"
"--with-coroutine"
];
}).overrideAttrs (old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
boost =
(pkgs.boost.override {
extraB2Args = [
"--with-container"
"--with-context"
"--with-coroutine"
];
}).overrideAttrs
(old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 = pkgs.libgit2.overrideAttrs (attrs: {
src = inputs.libgit2;
version = inputs.libgit2.lastModifiedDate;
cmakeFlags = attrs.cmakeFlags or []
++ [ "-DUSE_SSH=exec" ];
nativeBuildInputs = attrs.nativeBuildInputs or []
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch = attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
cmakeFlags = attrs.cmakeFlags or [ ] ++ [ "-DUSE_SSH=exec" ];
});
busybox-sandbox-shell =
pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
patches = attrs.patches or []
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
});
# TODO change in Nixpkgs, Windows works fine. First commit of
# https://github.com/NixOS/nixpkgs/pull/322977 backported will fix.
@@ -172,14 +163,14 @@ scope: {
inherit resolvePath filesetToSource;
mkMesonDerivation = f: let
exts = [
miscGoodPractice
bsdNoLinkAsNeeded
localSourceLayer
];
in stdenv.mkDerivation
(lib.extends
(lib.foldr lib.composeExtensions (_: _: {}) exts)
f);
mkMesonDerivation =
f:
let
exts = [
miscGoodPractice
bsdNoLinkAsNeeded
localSourceLayer
];
in
stdenv.mkDerivation (lib.extends (lib.foldr lib.composeExtensions (_: _: { }) exts) f);
}
-129
View File
@@ -1,129 +0,0 @@
{
lib,
stdenv,
buildEnv,
nix-util,
nix-util-c,
nix-util-test-support,
nix-util-tests,
nix-store,
nix-store-c,
nix-store-test-support,
nix-store-tests,
nix-fetchers,
nix-fetchers-tests,
nix-expr,
nix-expr-c,
nix-expr-test-support,
nix-expr-tests,
nix-flake,
nix-flake-tests,
nix-main,
nix-main-c,
nix-cmd,
nix-cli,
nix-functional-tests,
nix-internal-api-docs,
nix-external-api-docs,
nix-perl-bindings,
}:
(buildEnv rec {
name = "nix-${nix-cli.version}";
paths = [
nix-util
nix-util-c
nix-util-test-support
nix-util-tests
nix-store
nix-store-c
nix-store-test-support
nix-store-tests
nix-fetchers
nix-fetchers-tests
nix-expr
nix-expr-c
nix-expr-test-support
nix-expr-tests
nix-flake
nix-flake-tests
nix-main
nix-main-c
nix-cmd
nix-cli
nix-internal-api-docs
nix-external-api-docs
] ++ lib.optionals (stdenv.buildPlatform.canExecute stdenv.hostPlatform) [
nix-perl-bindings
];
}).overrideAttrs (finalAttrs: prevAttrs: {
doCheck = true;
doInstallCheck = true;
checkInputs = [
# Actually run the unit tests too
nix-util-tests.tests.run
nix-store-tests.tests.run
nix-expr-tests.tests.run
nix-flake-tests.tests.run
];
installCheckInputs = [
nix-functional-tests
];
passthru = prevAttrs.passthru // {
/**
These are the libraries that are part of the Nix project. They are used
by the Nix CLI and other tools.
If you need to use these libraries in your project, we recommend to use
the `-c` C API libraries exclusively, if possible.
We also recommend that you build the complete package to ensure that the unit tests pass.
You could do this in CI, or by passing it in an unused environment variable. e.g in a `mkDerivation` call:
```nix
buildInputs = [ nix.libs.nix-util-c nix.libs.nix-store-c ];
# Make sure the nix libs we use are ok
unusedInputsForTests = [ nix ];
disallowedReferences = nix.all;
```
*/
libs = {
inherit
nix-util
nix-util-c
nix-store
nix-store-c
nix-fetchers
nix-expr
nix-expr-c
nix-flake
nix-main
nix-main-c
;
};
}
// lib.optionalAttrs (stdenv.buildPlatform.canExecute stdenv.hostPlatform) {
perl-bindings = nix-perl-bindings;
};
})
+97 -86
View File
@@ -1,40 +1,36 @@
{ inputs
, binaryTarball
, forAllCrossSystems
, forAllSystems
, lib
, linux64BitSystems
, nixpkgsFor
, self
, officialRelease
{
inputs,
binaryTarball,
forAllCrossSystems,
forAllSystems,
lib,
linux64BitSystems,
nixpkgsFor,
self,
}:
let
inherit (inputs) nixpkgs nixpkgs-regression;
installScriptFor = tarballs:
installScriptFor =
tarballs:
nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix {
inherit tarballs;
};
testNixVersions = pkgs: client: daemon:
pkgs.nixComponents.callPackage ../package.nix {
testNixVersions =
pkgs: client: daemon:
pkgs.callPackage ../package.nix {
pname =
"nix-tests"
+ lib.optionalString
(lib.versionAtLeast daemon.version "2.4pre20211005" &&
lib.versionAtLeast client.version "2.4pre20211005")
"-${client.version}-against-${daemon.version}";
+ lib.optionalString (
lib.versionAtLeast daemon.version "2.4pre20211005"
&& lib.versionAtLeast client.version "2.4pre20211005"
) "-${client.version}-against-${daemon.version}";
test-client = client;
test-daemon = daemon;
doBuild = false;
# This could be more accurate, but a shorter version will match the
# fine version with rev. This functionality is already covered in
# the normal test, so it's fine.
version = pkgs.nixComponents.version;
versionSuffix = pkgs.nixComponents.versionSuffix;
};
# Technically we could just return `pkgs.nixComponents`, but for Hydra it's
@@ -61,36 +57,40 @@ let
"nix-main"
"nix-main-c"
"nix-cmd"
"nix-cli"
"nix-functional-tests"
"nix-ng"
];
in
{
# Binary package for various platforms.
build = forAllPackages (pkgName:
forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName}));
build = forAllPackages (
pkgName: forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})
);
shellInputs = forAllSystems (system: self.devShells.${system}.default.inputDerivation);
buildStatic = forAllPackages (pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName}));
buildCross = forAllPackages (pkgName:
# Hack to avoid non-evaling package
(if pkgName == "nix-functional-tests" then lib.flip builtins.removeAttrs ["x86_64-w64-mingw32"] else lib.id)
(forAllCrossSystems (crossSystem:
lib.genAttrs [ "x86_64-linux" ] (system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}))));
buildNoGc = forAllSystems (system:
self.packages.${system}.nix.override { enableGC = false; }
buildStatic = forAllPackages (
pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})
);
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-cli);
buildCross = forAllPackages (
pkgName:
forAllCrossSystems (
crossSystem:
lib.genAttrs [ "x86_64-linux" ] (
system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}
)
)
);
buildNoGc = forAllSystems (system: self.packages.${system}.nix.override { enableGC = false; });
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nix_noTests);
# Toggles some settings for better coverage. Windows needs these
# library combinations, and Debian build Nix with GNU readline too.
buildReadlineNoMarkdown = forAllSystems (system:
buildReadlineNoMarkdown = forAllSystems (
system:
self.packages.${system}.nix.override {
enableMarkdown = false;
readlineFlavor = "readline";
@@ -103,13 +103,18 @@ in
# Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of
# the installation script.
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native);
binaryTarball = forAllSystems (
system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native
);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (system:
forAllCrossSystems (crossSystem:
binaryTarball
nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}));
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (
system:
forAllCrossSystems (
crossSystem:
binaryTarball nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}
)
);
# The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary
@@ -127,15 +132,13 @@ in
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
installerScriptForGHA = installScriptFor [
# Native
self.hydraJobs.binaryTarball."x86_64-linux"
self.hydraJobs.binaryTarball."aarch64-darwin"
# Cross
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
installerScriptForGHA = forAllSystems (
system:
nixpkgsFor.${system}.native.callPackage ../scripts/installer.nix {
tarballs = [ self.hydraJobs.binaryTarball.${system} ];
}
);
# docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
@@ -153,16 +156,24 @@ in
external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs;
# System tests.
tests = import ../tests/nixos { inherit lib nixpkgs nixpkgsFor self; } // {
tests =
import ../tests/nixos {
inherit
lib
nixpkgs
nixpkgsFor
self
;
}
// {
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in
runCommand "eval-nixos" { buildInputs = [ nix ]; }
''
# Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs.
evalNixpkgs =
let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in
runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
(
@@ -173,36 +184,36 @@ in
mkdir $out
'';
nixpkgsLibTests =
forAllSystems (system:
import (nixpkgs + "/lib/tests/test-with-nix.nix")
{
lib = nixpkgsFor.${system}.native.lib;
nix = self.packages.${system}.nix;
pkgs = nixpkgsFor.${system}.native;
}
nixpkgsLibTests = forAllSystems (
system:
import (nixpkgs + "/lib/tests/test-with-nix.nix") {
lib = nixpkgsFor.${system}.native.lib;
nix = self.packages.${system}.nix;
pkgs = nixpkgsFor.${system}.native;
}
);
};
};
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native;
nixpkgs = nixpkgs-regression;
};
installTests = forAllSystems (system:
let pkgs = nixpkgsFor.${system}.native; in
pkgs.runCommand "install-tests"
{
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentLatest =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux"
then testNixVersions pkgs pkgs.nix pkgs.nixVersions.latest
else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out");
installTests = forAllSystems (
system:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.runCommand "install-tests" {
againstSelf = testNixVersions pkgs pkgs.nix pkgs.pkgs.nix;
againstCurrentLatest =
# FIXME: temporarily disable this on macOS because of #3605.
if system == "x86_64-linux" then testNixVersions pkgs pkgs.nix pkgs.nixVersions.latest else null;
# Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nix pkgs.nixStable;
} "touch $out"
);
installerTests = import ../tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball;
@@ -1,282 +0,0 @@
commit 9bacade4a3ef4b6b26e2c02f549eef0e9eb9eaa2
Author: Robert Hensing <robert@roberthensing.nl>
Date: Sun Aug 18 20:20:36 2024 +0200
Add unoptimized git_mempack_write_thin_pack
diff --git a/include/git2/sys/mempack.h b/include/git2/sys/mempack.h
index 17da590a3..3688bdd50 100644
--- a/include/git2/sys/mempack.h
+++ b/include/git2/sys/mempack.h
@@ -44,6 +44,29 @@ GIT_BEGIN_DECL
*/
GIT_EXTERN(int) git_mempack_new(git_odb_backend **out);
+/**
+ * Write a thin packfile with the objects in the memory store.
+ *
+ * A thin packfile is a packfile that does not contain its transitive closure of
+ * references. This is useful for efficiently distributing additions to a
+ * repository over the network, but also finds use in the efficient bulk
+ * addition of objects to a repository, locally.
+ *
+ * This operation performs the (shallow) insert operations into the
+ * `git_packbuilder`, but does not write the packfile to disk;
+ * see `git_packbuilder_write_buf`.
+ *
+ * It also does not reset the memory store; see `git_mempack_reset`.
+ *
+ * @note This function may or may not write trees and blobs that are not
+ * referenced by commits. Currently everything is written, but this
+ * behavior may change in the future as the packer is optimized.
+ *
+ * @param backend The mempack backend
+ * @param pb The packbuilder to use to write the packfile
+ */
+GIT_EXTERN(int) git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb);
+
/**
* Dump all the queued in-memory writes to a packfile.
*
diff --git a/src/libgit2/odb_mempack.c b/src/libgit2/odb_mempack.c
index 6f27f45f8..0b61e2b66 100644
--- a/src/libgit2/odb_mempack.c
+++ b/src/libgit2/odb_mempack.c
@@ -132,6 +132,35 @@ cleanup:
return err;
}
+int git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb)
+{
+ struct memory_packer_db *db = (struct memory_packer_db *)backend;
+ const git_oid *oid;
+ size_t iter = 0;
+ int err = -1;
+
+ /* TODO: Implement the recency heuristics.
+ For this it probably makes sense to only write what's referenced
+ through commits, an option I've carved out for you in the docs.
+ wrt heuristics: ask your favorite LLM to translate https://git-scm.com/docs/pack-heuristics/en
+ to actual normal reference documentation. */
+ while (true) {
+ err = git_oidmap_iterate(NULL, db->objects, &iter, &oid);
+ if (err == GIT_ITEROVER) {
+ err = 0;
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ err = git_packbuilder_insert(pb, oid, NULL);
+ if (err != 0)
+ return err;
+ }
+
+ return 0;
+}
+
int git_mempack_dump(
git_buf *pack,
git_repository *repo,
diff --git a/tests/libgit2/mempack/thinpack.c b/tests/libgit2/mempack/thinpack.c
new file mode 100644
index 000000000..604a4dda2
--- /dev/null
+++ b/tests/libgit2/mempack/thinpack.c
@@ -0,0 +1,196 @@
+#include "clar_libgit2.h"
+#include "git2/indexer.h"
+#include "git2/odb_backend.h"
+#include "git2/tree.h"
+#include "git2/types.h"
+#include "git2/sys/mempack.h"
+#include "git2/sys/odb_backend.h"
+#include "util.h"
+
+static git_repository *_repo;
+static git_odb_backend * _mempack_backend;
+
+void test_mempack_thinpack__initialize(void)
+{
+ git_odb *odb;
+
+ _repo = cl_git_sandbox_init_new("mempack_thinpack_repo");
+
+ cl_git_pass(git_mempack_new(&_mempack_backend));
+ cl_git_pass(git_repository_odb(&odb, _repo));
+ cl_git_pass(git_odb_add_backend(odb, _mempack_backend, 999));
+ git_odb_free(odb);
+}
+
+void _mempack_thinpack__cleanup(void)
+{
+ cl_git_sandbox_cleanup();
+}
+
+/*
+ Generating a packfile for an unchanged repo works and produces an empty packfile.
+ Even if we allow this scenario to be detected, it shouldn't misbehave if the
+ application is unaware of it.
+*/
+void test_mempack_thinpack__empty(void)
+{
+ git_packbuilder *pb;
+ int version;
+ int n;
+ git_buf buf = GIT_BUF_INIT;
+
+ git_packbuilder_new(&pb, _repo);
+
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_assert_in_range(12, buf.size, 1024 /* empty packfile is >0 bytes, but certainly not that big */);
+ cl_assert(buf.ptr[0] == 'P');
+ cl_assert(buf.ptr[1] == 'A');
+ cl_assert(buf.ptr[2] == 'C');
+ cl_assert(buf.ptr[3] == 'K');
+ version = (buf.ptr[4] << 24) | (buf.ptr[5] << 16) | (buf.ptr[6] << 8) | buf.ptr[7];
+ /* Subject to change. https://git-scm.com/docs/pack-format: Git currently accepts version number 2 or 3 but generates version 2 only.*/
+ cl_assert_equal_i(2, version);
+ n = (buf.ptr[8] << 24) | (buf.ptr[9] << 16) | (buf.ptr[10] << 8) | buf.ptr[11];
+ cl_assert_equal_i(0, n);
+ git_buf_dispose(&buf);
+
+ git_packbuilder_free(pb);
+}
+
+#define LIT_LEN(x) x, sizeof(x) - 1
+
+/*
+ Check that git_mempack_write_thin_pack produces a thin packfile.
+*/
+void test_mempack_thinpack__thin(void)
+{
+ /* Outline:
+ - Create tree 1
+ - Flush to packfile A
+ - Create tree 2
+ - Flush to packfile B
+
+ Tree 2 has a new blob and a reference to a blob from tree 1.
+
+ Expectation:
+ - Packfile B is thin and does not contain the objects from packfile A
+ */
+
+
+ git_oid oid_blob_1;
+ git_oid oid_blob_2;
+ git_oid oid_blob_3;
+ git_oid oid_tree_1;
+ git_oid oid_tree_2;
+ git_treebuilder *tb;
+
+ git_packbuilder *pb;
+ git_buf buf = GIT_BUF_INIT;
+ git_indexer *indexer;
+ git_indexer_progress stats;
+ char pack_dir_path[1024];
+
+ char sbuf[1024];
+ const char * repo_path;
+ const char * pack_name_1;
+ const char * pack_name_2;
+ git_str pack_path_1 = GIT_STR_INIT;
+ git_str pack_path_2 = GIT_STR_INIT;
+ git_odb_backend * pack_odb_backend_1;
+ git_odb_backend * pack_odb_backend_2;
+
+
+ cl_assert_in_range(0, snprintf(pack_dir_path, sizeof(pack_dir_path), "%s/objects/pack", git_repository_path(_repo)), sizeof(pack_dir_path));
+
+ /* Create tree 1 */
+
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_2, _repo, LIT_LEN("thinpack blob 2")));
+
+
+ cl_git_pass(git_treebuilder_new(&tb, _repo, NULL));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob2", &oid_blob_2, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_1, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_1 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_1);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+
+ /* Create tree 2 */
+
+ cl_git_pass(git_treebuilder_clear(tb));
+ /* blob 1 won't be used, but we add it anyway to test that just "declaring" an object doesn't
+ necessarily cause its inclusion in the next thin packfile. It must only be included if new. */
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_3, _repo, LIT_LEN("thinpack blob 3")));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob3", &oid_blob_3, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_2, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_2 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_2);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+ git_treebuilder_free(tb);
+
+ /* Assertions */
+
+ assert(pack_name_1);
+ assert(pack_name_2);
+
+ repo_path = git_repository_path(_repo);
+
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_1);
+ git_str_joinpath(&pack_path_1, repo_path, sbuf);
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_2);
+ git_str_joinpath(&pack_path_2, repo_path, sbuf);
+
+ /* If they're the same, something definitely went wrong. */
+ cl_assert(strcmp(pack_name_1, pack_name_2) != 0);
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_1, pack_path_1.ptr));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_1));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_2));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_3));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_1));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_2));
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_2, pack_path_2.ptr));
+ /* blob 1 is already in the packfile 1, so packfile 2 must not include it, in order to be _thin_. */
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_1));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_2));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_3));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_1));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_2));
+
+ pack_odb_backend_1->free(pack_odb_backend_1);
+ pack_odb_backend_2->free(pack_odb_backend_2);
+ free((void *)pack_name_1);
+ free((void *)pack_name_2);
+ git_str_dispose(&pack_path_1);
+ git_str_dispose(&pack_path_2);
+
+}
@@ -1,930 +0,0 @@
commit e9823c5da4fa977c46bcb97167fbdd0d70adb5ff
Author: Robert Hensing <robert@roberthensing.nl>
Date: Mon Aug 26 20:07:04 2024 +0200
Make packbuilder interruptible using progress callback
Forward errors from packbuilder->progress_cb
This allows the callback to terminate long-running operations when
the application is interrupted.
diff --git a/include/git2/pack.h b/include/git2/pack.h
index 0f6bd2ab9..bee72a6c0 100644
--- a/include/git2/pack.h
+++ b/include/git2/pack.h
@@ -247,6 +247,9 @@ typedef int GIT_CALLBACK(git_packbuilder_progress)(
* @param progress_cb Function to call with progress information during
* pack building. Be aware that this is called inline with pack building
* operations, so performance may be affected.
+ * When progress_cb returns an error, the pack building process will be
+ * aborted and the error will be returned from the invoked function.
+ * `pb` must then be freed.
* @param progress_cb_payload Payload for progress callback.
* @return 0 or an error code
*/
diff --git a/src/libgit2/pack-objects.c b/src/libgit2/pack-objects.c
index b2d80cba9..7c331c2d5 100644
--- a/src/libgit2/pack-objects.c
+++ b/src/libgit2/pack-objects.c
@@ -932,6 +932,9 @@ static int report_delta_progress(
{
int ret;
+ if (pb->failure)
+ return pb->failure;
+
if (pb->progress_cb) {
uint64_t current_time = git_time_monotonic();
uint64_t elapsed = current_time - pb->last_progress_report_time;
@@ -943,8 +946,10 @@ static int report_delta_progress(
GIT_PACKBUILDER_DELTAFICATION,
count, pb->nr_objects, pb->progress_cb_payload);
- if (ret)
+ if (ret) {
+ pb->failure = ret;
return git_error_set_after_callback(ret);
+ }
}
}
@@ -976,7 +981,10 @@ static int find_deltas(git_packbuilder *pb, git_pobject **list,
}
pb->nr_deltified += 1;
- report_delta_progress(pb, pb->nr_deltified, false);
+ if ((error = report_delta_progress(pb, pb->nr_deltified, false)) < 0) {
+ GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
+ goto on_error;
+ }
po = *list++;
(*list_size)--;
@@ -1124,6 +1132,10 @@ struct thread_params {
size_t depth;
size_t working;
size_t data_ready;
+
+ /* A pb->progress_cb can stop the packing process by returning an error.
+ When that happens, all threads observe the error and stop voluntarily. */
+ bool stopped;
};
static void *threaded_find_deltas(void *arg)
@@ -1133,7 +1145,12 @@ static void *threaded_find_deltas(void *arg)
while (me->remaining) {
if (find_deltas(me->pb, me->list, &me->remaining,
me->window, me->depth) < 0) {
- ; /* TODO */
+ me->stopped = true;
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
+ me->working = false;
+ git_cond_signal(&me->pb->progress_cond);
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_unlock(me->pb) == 0, NULL);
+ return NULL;
}
GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
@@ -1175,8 +1192,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
pb->nr_threads = git__online_cpus();
if (pb->nr_threads <= 1) {
- find_deltas(pb, list, &list_size, window, depth);
- return 0;
+ return find_deltas(pb, list, &list_size, window, depth);
}
p = git__mallocarray(pb->nr_threads, sizeof(*p));
@@ -1195,6 +1211,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
p[i].depth = depth;
p[i].working = 1;
p[i].data_ready = 0;
+ p[i].stopped = 0;
/* try to split chunks on "path" boundaries */
while (sub_size && sub_size < list_size &&
@@ -1262,7 +1279,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
(!victim || victim->remaining < p[i].remaining))
victim = &p[i];
- if (victim) {
+ if (victim && !target->stopped) {
sub_size = victim->remaining / 2;
list = victim->list + victim->list_size - sub_size;
while (sub_size && list[0]->hash &&
@@ -1286,7 +1303,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
target->list_size = sub_size;
target->remaining = sub_size;
- target->working = 1;
+ target->working = 1; /* even when target->stopped, so that we don't process this thread again */
GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
if (git_mutex_lock(&target->mutex)) {
@@ -1299,7 +1316,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
git_cond_signal(&target->cond);
git_mutex_unlock(&target->mutex);
- if (!sub_size) {
+ if (target->stopped || !sub_size) {
git_thread_join(&target->thread, NULL);
git_cond_free(&target->cond);
git_mutex_free(&target->mutex);
@@ -1308,7 +1325,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
git__free(p);
- return 0;
+ return pb->failure;
}
#else
@@ -1319,6 +1336,7 @@ int git_packbuilder__prepare(git_packbuilder *pb)
{
git_pobject **delta_list;
size_t i, n = 0;
+ int error;
if (pb->nr_objects == 0 || pb->done)
return 0; /* nothing to do */
@@ -1327,8 +1345,10 @@ int git_packbuilder__prepare(git_packbuilder *pb)
* Although we do not report progress during deltafication, we
* at least report that we are in the deltafication stage
*/
- if (pb->progress_cb)
- pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload);
+ if (pb->progress_cb) {
+ if ((error = pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload)) < 0)
+ return git_error_set_after_callback(error);
+ }
delta_list = git__mallocarray(pb->nr_objects, sizeof(*delta_list));
GIT_ERROR_CHECK_ALLOC(delta_list);
@@ -1345,31 +1365,33 @@ int git_packbuilder__prepare(git_packbuilder *pb)
if (n > 1) {
git__tsort((void **)delta_list, n, type_size_sort);
- if (ll_find_deltas(pb, delta_list, n,
+ if ((error = ll_find_deltas(pb, delta_list, n,
GIT_PACK_WINDOW + 1,
- GIT_PACK_DEPTH) < 0) {
+ GIT_PACK_DEPTH)) < 0) {
git__free(delta_list);
- return -1;
+ return error;
}
}
- report_delta_progress(pb, pb->nr_objects, true);
+ error = report_delta_progress(pb, pb->nr_objects, true);
pb->done = true;
git__free(delta_list);
- return 0;
+ return error;
}
-#define PREPARE_PACK if (git_packbuilder__prepare(pb) < 0) { return -1; }
+#define PREPARE_PACK error = git_packbuilder__prepare(pb); if (error < 0) { return error; }
int git_packbuilder_foreach(git_packbuilder *pb, int (*cb)(void *buf, size_t size, void *payload), void *payload)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, cb, payload);
}
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, &write_pack_buf, buf);
diff --git a/src/libgit2/pack-objects.h b/src/libgit2/pack-objects.h
index bbc8b9430..380a28ebe 100644
--- a/src/libgit2/pack-objects.h
+++ b/src/libgit2/pack-objects.h
@@ -100,6 +100,10 @@ struct git_packbuilder {
uint64_t last_progress_report_time;
bool done;
+
+ /* A non-zero error code in failure causes all threads to shut themselves
+ down. Some functions will return this error code. */
+ volatile int failure;
};
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb);
diff --git a/tests/libgit2/pack/cancel.c b/tests/libgit2/pack/cancel.c
new file mode 100644
index 000000000..a0aa9716a
--- /dev/null
+++ b/tests/libgit2/pack/cancel.c
@@ -0,0 +1,240 @@
+#include "clar_libgit2.h"
+#include "futils.h"
+#include "pack.h"
+#include "hash.h"
+#include "iterator.h"
+#include "vector.h"
+#include "posix.h"
+#include "hash.h"
+#include "pack-objects.h"
+
+static git_repository *_repo;
+static git_revwalk *_revwalker;
+static git_packbuilder *_packbuilder;
+static git_indexer *_indexer;
+static git_vector _commits;
+static int _commits_is_initialized;
+static git_indexer_progress _stats;
+
+extern bool git_disable_pack_keep_file_checks;
+
+static void pack_packbuilder_init(const char *sandbox) {
+ _repo = cl_git_sandbox_init(sandbox);
+ /* cl_git_pass(p_chdir(sandbox)); */
+ cl_git_pass(git_revwalk_new(&_revwalker, _repo));
+ cl_git_pass(git_packbuilder_new(&_packbuilder, _repo));
+ cl_git_pass(git_vector_init(&_commits, 0, NULL));
+ _commits_is_initialized = 1;
+ memset(&_stats, 0, sizeof(_stats));
+ p_fsync__cnt = 0;
+}
+
+void test_pack_cancel__initialize(void)
+{
+ pack_packbuilder_init("small.git");
+}
+
+void test_pack_cancel__cleanup(void)
+{
+ git_oid *o;
+ unsigned int i;
+
+ cl_git_pass(git_libgit2_opts(GIT_OPT_ENABLE_FSYNC_GITDIR, 0));
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, false));
+
+ if (_commits_is_initialized) {
+ _commits_is_initialized = 0;
+ git_vector_foreach(&_commits, i, o) {
+ git__free(o);
+ }
+ git_vector_free(&_commits);
+ }
+
+ git_packbuilder_free(_packbuilder);
+ _packbuilder = NULL;
+
+ git_revwalk_free(_revwalker);
+ _revwalker = NULL;
+
+ git_indexer_free(_indexer);
+ _indexer = NULL;
+
+ /* cl_git_pass(p_chdir("..")); */
+ cl_git_sandbox_cleanup();
+ _repo = NULL;
+}
+
+static int seed_packbuilder(void)
+{
+ int error;
+ git_oid oid, *o;
+ unsigned int i;
+
+ git_revwalk_sorting(_revwalker, GIT_SORT_TIME);
+ cl_git_pass(git_revwalk_push_ref(_revwalker, "HEAD"));
+
+ while (git_revwalk_next(&oid, _revwalker) == 0) {
+ o = git__malloc(sizeof(git_oid));
+ cl_assert(o != NULL);
+ git_oid_cpy(o, &oid);
+ cl_git_pass(git_vector_insert(&_commits, o));
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ if((error = git_packbuilder_insert(_packbuilder, o, NULL)) < 0)
+ return error;
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ git_object *obj;
+ cl_git_pass(git_object_lookup(&obj, _repo, o, GIT_OBJECT_COMMIT));
+ error = git_packbuilder_insert_tree(_packbuilder,
+ git_commit_tree_id((git_commit *)obj));
+ git_object_free(obj);
+ if (error < 0)
+ return error;
+ }
+
+ return 0;
+}
+
+static int fail_stage;
+
+static int packbuilder_cancel_after_n_calls_cb(int stage, uint32_t current, uint32_t total, void *payload)
+{
+
+ /* Force the callback to run again on the next opportunity regardless
+ of how fast we're running. */
+ _packbuilder->last_progress_report_time = 0;
+
+ if (stage == fail_stage) {
+ int *calls = (int *)payload;
+ int n = *calls;
+ /* Always decrement, including past zero. This way the error is only
+ triggered once, making sure it is picked up immediately. */
+ --*calls;
+ if (n == 0)
+ return GIT_EUSER;
+ }
+
+ return 0;
+}
+
+static void test_cancel(int n)
+{
+
+ int calls_remaining = n;
+ int err;
+ git_buf buf = GIT_BUF_INIT;
+
+ /* Switch to a small repository, so that `packbuilder_cancel_after_n_calls_cb`
+ can hack the time to call the callback on every opportunity. */
+
+ cl_git_pass(git_packbuilder_set_callbacks(_packbuilder, &packbuilder_cancel_after_n_calls_cb, &calls_remaining));
+ err = seed_packbuilder();
+ if (!err)
+ err = git_packbuilder_write_buf(&buf, _packbuilder);
+
+ cl_assert_equal_i(GIT_EUSER, err);
+}
+void test_pack_cancel__cancel_after_add_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_add_1(void)
+{
+ cl_skip();
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_delta_1(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+#else
+ cl_skip();
+#endif
+}
+
+void test_pack_cancel__cancel_after_delta_1_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+#else
+ cl_skip();
+#endif
+}
+
+static int foreach_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *) payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return 0;
+}
+
+void test_pack_cancel__foreach(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_pass(git_packbuilder_foreach(_packbuilder, foreach_cb, idx));
+ cl_git_pass(git_indexer_commit(idx, &_stats));
+ git_indexer_free(idx);
+}
+
+static int foreach_cancel_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *)payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return (_stats.total_objects > 2) ? -1111 : 0;
+}
+
+void test_pack_cancel__foreach_with_cancel(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_fail_with(
+ git_packbuilder_foreach(_packbuilder, foreach_cancel_cb, idx), -1111);
+ git_indexer_free(idx);
+}
+
+void test_pack_cancel__keep_file_check(void)
+{
+ assert(!git_disable_pack_keep_file_checks);
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, true));
+ assert(git_disable_pack_keep_file_checks);
+}
diff --git a/tests/resources/small.git/HEAD b/tests/resources/small.git/HEAD
new file mode 100644
index 0000000000000000000000000000000000000000..cb089cd89a7d7686d284d8761201649346b5aa1c
GIT binary patch
literal 23
ecmXR)O|w!cN=+-)&qz&7Db~+TEG|hc;sO9;xClW2
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/config b/tests/resources/small.git/config
new file mode 100644
index 0000000000000000000000000000000000000000..07d359d07cf1ed0c0074fdad71ffff5942f0adfa
GIT binary patch
literal 66
zcmaz}&M!)h<>D+#Eyyp<EXgmbOv^9IO)M!(Eh^5;&r`5fFyP`$%gjm5%}+@M@=A(I
MQ@J>k5{uv*03B5png9R*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/description b/tests/resources/small.git/description
new file mode 100644
index 0000000000000000000000000000000000000000..498b267a8c7812490d6479839c5577eaaec79d62
GIT binary patch
literal 73
zcmWH|%S+5nO;IRHEyyp<EXgmbv{pz>$t+PQ$;d2LNXyJgRZve!Elw`VEGWs$&r??@
Q$yWgB0LrH#Y0~2Y0PnOK(EtDd
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/applypatch-msg.sample b/tests/resources/small.git/hooks/applypatch-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..dcbf8167fa503f96ff6a39c68409007eadc9b1f3
GIT binary patch
literal 535
zcmY+AX;Q;542A#a6e8^~FyI8r&I~hf2QJ{GO6(?HuvEG*+#R{4EI%zhfA8r{j%sh$
zHE~E-UtQd8{bq4@*S%jq3@bmxwQDXGv#o!N`o3AHMw3xD)hy0#>&E&zzl%vRffo<B
z)-H|+CWHZ~O*S%cfYx9;02_ohIA<Bg(1SxF-6OCb&_lBkf{t<AM9r;%E(Hf#h{|a@
z9>mqo=v6>_2NRa#TwDdYvTVQyueO*15Nlo%=#DXgC0bhF3vTa`LQGaO9;jeD$OP?~
za$G4Q{z+Q_{5V?5h;a-noM$P{<>Q~j4o7u%#P6^o^16{y*jU=-K8GYD_dUtdj4FSx
zSC0C!DvAnv%S!4d<Yg@O<;m`;oSw)=Fz+hrL<mY{rBr8j4pi^88FX3}jKrYUP)>gk
XB^)11aoGMJPCqWs%IS0YSv(eBT&%T6
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/commit-msg.sample b/tests/resources/small.git/hooks/commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..f3780f92349638ebe32f6baf24c7c3027675d7c9
GIT binary patch
literal 953
zcmaJ<TW`}a6n<`g#Ya}rZCaYOzy?UGG&Tf#VG`?}7@eHtB=MTqneFUS%75oLS;atz
zm&iUo=ey->y@-{3h^^Cx;#d0zEA@DDc$nY4ez&|=%jTg@_HU*ub=!!y$xW09TSjlj
z(`I@QCsM`!9&80$I98wsQ8yK#)Orb<8re8FjkKh630D$QUDwi~(gkX=RunYm$rDjk
zlp%RUSnzA#6yjdG5?T?2DcYKp+v_lts0ljn&bh3J0bD5@N@1UKZ190O6ZeWr-BuZ^
zWRebCX%(%=Xoj#(xYk1Cjtr!=tyBesf@m6}8zY6Ijbz9i9ziI_jG9Mv<Cz(ymp*>R
zDH*e>^ga9IR?2wrSrAVm;eButj4<aWB@zzNl|1Wp@4;}1O?MUF>Y>7(E2?b~jsu>&
zRKCJ7bp#19sqYh627wD%D9R$8=Ml$TNlumDypl~$jBu*G>5fIR^FB0h0Ex&TGZNr>
zL5hs1_K>taRb!|ThN9ns7^@4MXKP+6aGI_UK)T-M#rcP$;kN(Vcf#P)+5GzWa{l@J
z>-E{`$1iiNVYxq27}<DnwLRXQUG0o_hw&da-s5T#H=`Y9D_8=eTZ?cpWatp#a1vs@
z2BjrO)z@aTuI#g#`)oJcnhM7oYLT@~CHX@CNXv4>j;uo%;)r3kJI2xCFF~Ux;$Q%)
wjbk6JlDCM`jU&P+UVOvg`|iYl<7~9k>HHB4I;pdlQ=I-^$DrHaN$@lH1?P!0U;qFB
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/fsmonitor-watchman.sample b/tests/resources/small.git/hooks/fsmonitor-watchman.sample
new file mode 100755
index 0000000000000000000000000000000000000000..41184ebc318c159f51cd1ebe2290559805df89d8
GIT binary patch
literal 4777
zcmbtYYi}F368$Xwipg4lq(BeHMvzvH-4;n7DGJBPqq#tw3aed8+IU5-m)yvL>;Cqh
z8FFRGj$`9CA8ao<GaSz2oMCnz4Rv-gw9b@j_$0GQT1?XYi|;S??Y`Z6`Z;}C6#27N
z8ShS>J?j^$%==FV``-=rhLcPW`McSytRm~mEO7_&_cAVZrf1fFy*ha@8oe%*-aBYE
zcjzZg>LOkgxuUr-XJnHyD;zmPnRaSc#!k_P*d_BttRdc+J6G7za5#+<HG#rlmbrN~
z8DwU-3}VABEwM=0VLP@^Dy6ERR5_J6cmg|GEh*M1EliqCGwe^ZT-iZ$2Yc`4!I#WZ
z5nGGhn7*jeW=2ydsmfAmm#=8AD<<;TI+#z{Q)kW;yE!%GB6f~7EtEMLdM47Qaz*8=
zIObA(VVj-VG{Ax|66d*hi`+bRG>^Y1nkc2Oowk`ya47uUR3Feu?B<phm31&PQB<lt
zb{W(W4wf#Bab%|Q_tKPS?3^o=5)Z8^Vh(#slNI}pO(f^|{U0GZhLnycSaNd&h?CaC
z0XklU6^<ky6rES9T=na$M8P<_)aKMAMo+UDewAu4wF{#&6diFshiudixAoh|&0<BJ
zR>(w;S{(VYzxh}q-=#zP@uxSx{wbyPUMFU;K(06)$o{07&3yI?q{GqMcQ1c_^M<0<
zF4acAV)Il-V(rCTC1(;bsZ*}bl8dmejAk~yb`B}!^0;g^(o9kGUfZfDOvyp@x4OQt
zSgWh6T|3eq;9MFs8-#z+FDM1h(IjRUP|``PxupgJ7CUHOH90gbgl^2~97`?_X{P))
zB*$r1cDlF-%azKND}?Gv`2K8-9v5e`gQoft=j?T<&a13c^!wY_$D`5z-X1g?ty&6-
zQN50{8?bUk9AI->^W@~~nkOghHIC2YN+<JiT_ob7ttND1oh`HH28Y+OV~HedG&uB`
zy}rA*r_xT#bR`Q7-*)3t*(!Hf?jKzyxk=8hdi3L^d<p<uU9q_<4k&xEr4@YWv_vsW
zp(#32bYtA5u|s#1+}h`0kwpz4kUD&+>AXkLQG_2-{Pq3%{`3KUMeG$iIn%%^6*NYb
zn|_BdV#C)n4565Vcc<EWC-nglECZGy!j9I4&;hUCzr(?6TftH=0^@!mI^E@y5HZw8
ztH&kaSNyg=O6riqR^MPOX6oD__Jz@K;*icS)?m$!p{S$n;*EwlO<L!d7;utu(W9D!
zaF!B~xV^2i?wH0s?Lw%Q)(`aPkajs1XojlPv@Y-d5#YFg#KG+!r7A(dJLnkiJMs`p
zV|_=d!upN{TsxH1?sZNdzxeHsmtzTV`1{pykJ_~+^*>X;uT8&z3vSi!HXGbUj2B!R
zdz~&#<?<tHJql=W&((YpzS06y-Z6Cn^H!*9qH?pOrD~(GV=JJ~z{tpLnGK|xp&C1`
zsbn7O86xjF<~G*S1l*;;Bh%6><Up=oKy99?62P^?M&22U6QFRBXLb&u%=Ur<74wRy
zMRG!f{AvZ>fk#L-&k$fLwo$4?>12g@AXOKFekuo#6EHB%gmpD?1eyh%N8s{2wGoTu
z*@6cEZ^ZW!FAF_|JL`NkV7k}0ow|-2jHwbgH0;c@Dq*o?@&c*HnGdyx6^su8Qk%2{
z*ye(dxO*6-&>qn1+zw}tc6;=sOX{4WB=VqjTS^))y1jlX2Q;=e!qMmFA5lC$#;BxC
z=Y%tRpWxb+_uQAvAw7Q{HGV#R$xb&udLCzZ+HN?kTyB};1EJ8UlQ5!>5eGW@)RX0n
zkjj>EF!3=0Gl^8dzv$B^NMGRxJoqN4A`xq-@wCbrx*u2NmIJ1<fUDQ=*^)h6`vzco
z3F+ro$F!F6pc<B;<;isobIgbVGKUBByoQ4#CO({S7g?<Dh0^!7uJ3gxS=6F;+^gQc
zeKi4`4`Fm3p|BU2v{M|-u!#QGCXiM=k=%np0<ZOPQqEjP_nneyOdgEuG9IH&YYPtp
zKB_dvcYCcyhyT#<uhUEL$o~!TUz;cb&|`uSM{Dkv%&W2lcpYL&kv)tUvVv?&>xZ%H
zh;{|4T3(!E9sY#Ni(wUJYs1MmIc9bl)(4Nl3_wD_BWB>i<1S(LX7m*{Q7PU$muMS*
zM!%0EZx-Vw=Zey;erC?SNxF;pY@^A%-krqzfLV2meBp1vWdyArFYn`DD19T)Hw(?n
z)}{NP(Lk(o*?gl#B@pP7^*r|=;PIDT4|F#{2Hzh-AL0Rv$6uT;<CP7qxbYms@WU7}
z%}TsHJ!V_56ZFF{BfI=8jTBxMEATG376pS6a;u1@c?{~sL<N52`U6fuLkz4P@Mb^b
z2`Q48y&!C0&A+WRCJAdmo3u2#?eI=si9Vm47$|`m**x<wKkM=QR&g?C63@P5X@xP8
zi5QG2b-Fdz%S0%1{kKvL%^RTdpK|SU^VJ7JCmKdwp1`;HRoGM7ef^k_s_}2An=cql
za|{IaxQWpdq<ae&x3BOJq+M5QNIk#C{Nv@u>n|WzE4=slK?on@(fZeGhRgQCu56qB
z{+n81Az96qnQjMY*-*r-KV*7;Z#4Q<xfjbcBx_6JAN-HP@bq+eI%HhAB9&vLyOap{
bw<Ywj(b#kdcCk7dCBY;|DBNpPjIa1F6*dXN
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/post-update.sample b/tests/resources/small.git/hooks/post-update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..9f95e41a39cf974958277c8881ac6cce405ebb20
GIT binary patch
literal 246
zcmXZVO?HDY3<Ti4Poaiwjq^yGw9DKf7mz^&Ly=V5q$H=W^Rt|p_r9s#9Ea7VERo!9
zyT9>uJRJJV$M^KdldiMhj?ImK6~FvwJ*L5a){QoM=L5TYHkGO1$UrO3`a>{?Opw|b
zG(#59NQ#jFL9v~vgOVkM@^^(^A}onOE))yWEwhIlk&{ZyseZ^O0b=w8&O=BK{k<5B
k^Q-B@eG}LeHrquz%(SVEp_N)VhYZikCW__82JXfD17`J9Qvd(}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-applypatch.sample b/tests/resources/small.git/hooks/pre-applypatch.sample
new file mode 100755
index 0000000000000000000000000000000000000000..625837e25f91421b8809a097f4a3103dd387ef31
GIT binary patch
literal 481
zcmY+ATTa6;5Jms9iouO45IBJXEg&Jm9@v1LPHMM_ZR|;#6tQ<EeSrA%_2`_rGr1_8
z?aM?yVtIc%-@9SGSk&8x=grP-Lf`7!^=$7xgL=|ysZ}!av6zL~ywui}<2##V6L@!k
zy=p^)V7%Wzs-g`9<Y9}^)&uN}BCrXR_T3@Z2$gSJON2`X=mAs+%@7n-2I}ZrP|TFA
zvJJGDl3HPLP<@!Q!}zXQvey#qEE#a#$vs97i4=A0stF@YQ)k_ZajaoS^dVYBc&37_
zVI(L=X<V335r9~7T<;|HfKF+yM}}LB9d96V)Si;sj(;9Rh$#P>h$71hSXq*MxP;V&
zj0cY7SCL=x4`a46sF)C>94Gk%=3q$W2s;j6iHtB2$R0%gix4oK@&T~=ALd_o*CKxt
I-`Pv{1Bpzc>;M1&
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-commit.sample b/tests/resources/small.git/hooks/pre-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..10b39b2e26981b8f87ea424e735ef87359066dbb
GIT binary patch
literal 1706
zcmZuxU2ohr5PY_N#pZ0-F<{-<Zkx1j&cMNSQ3FN`GzR*R1O+9oPV`BnOj7q@1pV!u
zrF0Hl^i33(yR)-1d-!H%&2|=|^E~_R{N1zNJ-&Zmt-t?iwXv&i+ZN}Km(TX8Q$H4u
zd7(m`|1iDmF5k@xV`p;C4zojASmLc}yN0QDZbhN=ri&CEt=XGuN1IwjGJ#a#`t-kG
zDqY)}7+Ft|;YKwLYbtg$S(-TBO=x3cP1cd}%f4kB!<6Wu-dCwz-)KDMEuM^_Hh*UC
zC`1)|)T<(U6b`+yOH!6p*Ll}@qastwA*dyjsgOf5C=?LTprfORG6O{5L%@S0wyHpj
zu|_A-=NWnYYR5m7kvm6|&T~GzoJ_OKR3sgFUgw?ifho^NQhvK#{6g0=&Fh)%n}#m0
zk1sNmwb_AMDq};OOGw5|;OyX#?yQMMH6yAk(x$3tjFjHE?c$E2XC_xXav8tnIeIG?
zYMI|~MLEVGkuT*>v&v-X^RA+u>k}E$4d&uD7=g_fA8+pNNV=4s0|iD3p<=DTXClTS
zXV23tJ;ECmN@M0j@zUAKEYW@3bv!SeYZ8ZH`YQNTApFVNc;F|9r5p4TqGs=>8E?6y
zi|gY{iM#PG1nL?UE9YCnWTk72kgZPG*Usqw!~Qd3c?~@w2?%eg@~)+VlSs6N5Yf2^
zz;ow<fjf3n`imj7u5lnzt||q9QFM(H@<3EJCK|l5P!$yDfn~U-(5Vu7s+GqxNKyeI
zP=-Oa(KH&gK`NhUa`cLj3B8%qL};DR7dk!`A^h&3-hFB6p($5Ufy^tGir)3et}qK4
zpkPKYWzC+?=&gw-L6S)S=<lfq)%uLUAa%~81Jf9hk)w~U!DItnoSy`m^}#38M}K-o
z!PqisQkV!&z4E*V01ro~qlVK^0BI`pLk6D+)f~*y!hCvwHq8zY9BGh<2s$@b^A<8G
zRaqk}&qZyyv&|0QDFPA%h4TgF&vdlc|JUq*=>F#K#r^&KMq1A`oqVGFpD&-!Pv|Rc
zO3KSqA@h9nSc%bm`0)Amk6*J}@14J*1-219l%%7D!Pl}UK>|lVi0Dfgu2jN3WC!uL
z0ej??b2iSehVgdnWHmZV4kUo*QL#aiIp}U=9x)IXk}JJ7VQ;CI9Rtn5e0VcjbY<bp
zv{}WlG6L;H!EzFKdY>cVt+`x5D+svCGD<sXw4|)E|JX43I1_3P(sI4{wj87bPSrNG
w!SIr>;Z5hm*<gY+X;)Ryx4=nzaab9m`bwE*^s(%u*E3HbUuOR@+&s_z1=MCi2LJ#7
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-merge-commit.sample b/tests/resources/small.git/hooks/pre-merge-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..d8e8ad4bcf8337f23c78a98537ecab4831301202
GIT binary patch
literal 473
zcmaJ-O;f@!5WV+TJd4B0whSt$H%Dh2t`2u6q1!glCNbGU;n%yxdNsG~zR#WA6xIwy
zWEZHoU#u?nykD=Y<HPgeWDkDm^kTof*l(|%^gh!nHrZpo^vhMDjV;E1GD~K7wV*+D
zz9lry9T0cHcm_KhDVXYvQ==FrLTT4u=bEr{U1yl7%thf%wJnv<XQZ`ZbQEezaWdS%
zI;c?h9a)Y!ux<WK8rQd_aA^?61hv_Pf<t7*z1USuL40FxYz<|hybsO?qnN}aMpcuf
z6phFw1%Xx=wUk(m>E$jSEQZ%SQ(}oLgslTvrKK@9Qf#b!hajVFnp9@oIix;NcI9Wk
xjnh0ya!AWet{I7YpD;y6HXyzI*lfSvH=o6*7mJZPkuaYpm>vzZ`wyGEBtOQPo|pgt
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-push.sample b/tests/resources/small.git/hooks/pre-push.sample
new file mode 100755
index 0000000000000000000000000000000000000000..02cbd80c287f959fe33975bb66c56293e3f5b396
GIT binary patch
literal 1431
zcmaJ>U60!~5PUX&#a1@z9B{IIZkjLT0t5kq9#8~D(I5{+8&J~9;#ndUk~-ZT`r|uG
z$#K$$J{TsK<m~Lsu9iP+t-0TZ=sa(K+C6);54X>s*LP1}9!GoZ@4I4myMMG_di|of
z%?llx{O8TS-#^<H#%^V=)RNv>;(OioEmPy%kwWQBA1OMzV{hsQ8XFzS1k!~YQoLa5
zhtP1fA$q6VmMbbAC_9)4I628k*O5J$NR19uHe4QYDK<==I~SQk)Nu%xQ~<Hy8U>KH
z53w=!ke(FGb_PpnZfd*+hnXDTn;2*`u^~;?+5C~cn?bRka7NR%06%e6O91{MAgN6J
zmlO8{Biw4&wr&&(z4p3eln`E}XR9m9bNYZ7Ibrg(4yZIXrfgD7N*AFD7L3YSM#j}%
zo__rOS5fr;@8UM<6cl+cv_$YB$PQ&9dv($eM*))g!_cu!QcSh-mqE9i#QDZT)=o#`
z?8!RtE?w6p?GkGZ-6yt_p~5~4ecu|Sf^)6096%h*q-eNiEA1;Xwg)p~Q&iGSG7-IQ
z9aII&`ps$WOojFA`*bjG<mBv1n0hcYZWN0~(X01-hx(ExqWqaXgSk*@-GMp|K_3!5
z9|O21N3%~izh(4fbp9wzd+!b&7cVwSP5H00)m5ej-(s=Pl#(90UOhn@OA9u+D{i@r
za4*CP0I#<d-)-#xq5q-iY5nIef2s5OuQjcA>kFk|E@sHHuD}W^d`7YJ3YE^zrQnqR
zGoq?;YGKe)93o|_=^f%3U1KYZGPOXRRxK7w`UUbMMa3<86OmVH!EKP$8RCrn9mWX+
zC?9yF!fRVLmud3hF<}x;;sR}f(*r}6Gap3fR6zLHR~kbMgD{98N`L+r&?3p~*0+FX
zcAL%j=(SO}xTJUTvA`&Lf`2mv4koPG9&|<CA~EHbWHMoFPwT(&U=7t0`RoFZPO9Kq
zwwe$i=T|AXY#hD$aZlNMH`wZ%gwilGJ(zeYpOn*F3cy0XKXio^Sj#WXx=PWV`WGaA
B&~*R+
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-rebase.sample b/tests/resources/small.git/hooks/pre-rebase.sample
new file mode 100755
index 0000000000000000000000000000000000000000..193ee3f127387f469894675f0d621b5bb9c275da
GIT binary patch
literal 5006
zcmb7IX>;2+68$XxiXKL@ma;l5d2^5Ba_rPh_DHI-u1#&_upttZXp;no03$20|NFiM
zK#D#xQ>!Z3JkX8T-LDVm!B5j7y_{;JDmmTTef+K1oIiPzeEr+Ai*<2PUgnG4^ZB>p
z_fkAvoR1emuf~ri^K$-px=4#D-v<wZ2Xv&$O_eTJh6d4)=DWL(NBs9G{k<+yMMw0T
z$VH*-+LM)}u&m^`l8~1nt(3Z;R8v(KbY5#i3z+~8h0D}Xvq&3J8BMWDizPNpaeb~9
zBN9bSkthfXzskapf%Zt{*e#}{QaNiaAVZ4{$;;I6<vKNhO@%7P-(;l-x=pPoExHC!
zB(hA#cDdD?s4P=!)=-K{<kHAWKetl-8I8wwO<ihJNs-$dEvr;&S_@6E=mNSJ5;mg#
zyb)MbqKH<one{qrV;ZQ6WL}yLtyi*ekNLf|uC6M!)Cmq7*l?g0d6`MlE49|}>Y9w&
z`bCv#<YfTKtb`!}CyNYd;|(C?vRVQmWOfR9X?FZ#=f$No)^#4>2zVn=YnJyeNey(Y
zRh`9vtLw~A+5zsjp|W0Nsa|29Rm!B>OoG5a+vi;ari8O>KkU!KAWg_fa3btK2x*_@
z0bEc7J;Ubghm}n9bOi(Sv_B66nQ7U)J7f0fO}<cB8i8vG{r39s_>8Wuf*uorcIgEG
zOHc|-V6+HlRhOP}?Cn?@5iwSl43abmBA^2lyL$+cpabCGVES+v^j^FO_}?FIp<qP?
z#_?*YMHIkyZxJxS;h@A)WDJ0bN&+F-#_lFjAf^g_Pb#5YXqYe|dZUpa^zI)VOBXQQ
zAMhT>%En%Ll?Z*7*}TwrZyg5OSZ9rY-`aU~Mc-jjv{Ll)FLMgtB4ujktfQ`Xhqrka
zT=P!A;9w^;Z?PqpLwOLu=cj3L>TdUKw2;DMu)`oVkj}<z_EjO_2uWYuvKG==%Zu?h
zJiMVR^c30RbpW}<+z;jjoNC}YI4f6QC7d-08*4mCB1>#bcDx4tYg=j%D`+i{W~fVM
zVmZ>W9VMyin9c-0KzI_;iZ-g|OyzuG`Yq%(%dvl;ifnVr0;jWE&S`z|rQu=!yHBBO
zx`OJ;oOQ(KKM<$(bC38o>pD0%|HA(E0TRw7qj$fJ_pRN+7Nm>dS<q{AcOz#-;d7_2
zL$z(_nhH{vOzT(}>C(gLg{(`t+5Z=?o+}wXU4tHy+&%F&aRhFebeEhR2R5|<c6J);
zEY(q5F5<n*XP0|=PtPBn$B)V~d$Os-?&8UlaVe_|jdkzoWNsTP-_uyq4$R6o<h`&@
z{loXa{^#TF=NJBYu9qB?hs}x=It_O}ZjX(_wy9@X(lmkRpNi0{8T{O_b_j*JC^_SM
zz3G?1$KCNWF-|`Jbx2cQ-y5LW?Z2eikngTZmsx5C(@({8<l)Ue+gIp##Mosfa~iZN
zZ|NLN9uE6Xaqpwk+@D+f?$tg2JRCY`pwZwbR9OvEn*zYm`ffKIzl4{r{ZgjfpbuX)
z_r0=0y3)T-j$gljPyEJO*6Y<pj7G72aLoqaTpc=#u)*xJcVUm0;k(n;r)^DQNa6Oj
zWvlHEGg~lz`Q_8`yQ9<BZ;ylE1Z}bD<8}<uB9Y5lRB=;JUD0h?_)4H&EhJjvwzJx?
zr<o_vk&75j_5^d$8Z$_Oc1=R-I_DlNZ2@~81oV*J6%o3RuiCz}^VEW>$#Ycbp^w@t
zTl%=f1t=w+WpJzF<|CE@?SCNAz)%9?w33lQ8vrHJqPfH9@}qs*QXOG71W=ylx;wOB
zcx!Bj^)Yy6WX$a^vBkBJ5Cob<oubBW+a#9bX{0bkMTX_2sIrs`HMk@$q{dK5h2-g}
z({`~#gm#G?c#>qlaDx_B0c<3b+8)f84LCrt;e;qxc+7>VbwVK{skNv!wvBiTa^9Iu
zkwP;VK)jH$WJ{`MRwAA9fal!y0dtV;FWg8PTkWU>CwnqD>1ZX2B@;$DlX%C5MI+}{
z9xQVnffR*~v2KAUj*hCd<gRCjR7~6Q(vF%VT7j97iv3j4Z0p%mU`7t061~l7b$!#t
z3*Pveii}aQ?QD9aiX>gul~`bk#mk`o>zk9)<2Uc8?hUZAEvd!`9em)~$Z)zev>w^8
zyAgCP_$&Y)7HSQ84`xG}OeTavaEswwF|8Xpi5iZzZa@hCiv(J-%bfFC&)HLlO+Rhw
zG6g?9eL5&A!SuJnQ6}LxG%tU+@vZ`i+!+Rz6iYvsTdhnPo7lW{m-}{hya@viX4)XZ
zngaw+j;gloB#|UwI@8sOmQpc`h+bicQJnQIB5eifIMQNgD2+oai33m!34~xU|0Azj
zhu$8z+T5^;Pxx@d{N)pzOJLSa^e;aDf$W%N5XcOf!mGC9l9j$Ev2h6N+6ZQC+CJzl
zaM7?S!SrFLS2DASjj(h6y1WN3N?|bmqmyzm!&nLoE|`rKBOc_yDF$a#FsUn!IQf(t
zdC&Us(kQz*7mv<H12p@UD8XaLXdK{>H^j*^MC@>wTDb}g%~sx*ng#>{@lR=XG-Z5_
z#<9*Oh0joMzt;nS)ObAp)347`D=}r-;nV!TbIq&xrGRGsF6fZg+!VkfUei@_&l-M&
zPqQ+Dw)RV}+)I8RuqAxa`Pv8e&!_gXS=e2-un>=Ktn}-;%lLZxaVn?Q>yZCb2R3Wk
z77zr%;Rq&h|2ncqyKYmFI0148JVY7Q$V5p=dWj<MQ<8r+@QLq+UROk&%quICq^O2C
zp(17882jXI)_GaqzAY4AjoB_nh8k*r1mJ>+Qqpu%i|xp2<qF`Tw6&3`h654ceBjZ7
z4>C=WaOb2Wudn^h0EcD%$p9YVU1fnoRV9`(cy(vv6K>FXS!2jY>1GnU--7)4usH&K
zao*&P^@9~YmUe|ZdLW@C>H;!*<TIU}-!Tw#3oqZA*99}b3&uHiGO<{I6!pMnAiQdS
P!fA@Yk4s_DjDP<F98V`a
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-receive.sample b/tests/resources/small.git/hooks/pre-receive.sample
new file mode 100755
index 0000000000000000000000000000000000000000..32b597f8c32399017c3d7d51134d1f6490ad9575
GIT binary patch
literal 601
zcmZ`#+iu%141Kn~f>Vt3>Nw4M*;=?j(TBD#O@XCv0|MEhA;z}kTFRv@`tPHhp=&Yh
zg%Zhg4i7o_k{a5i&f5;tZ==%}^Sn4aD_6%qs<o-wO_Prn;}`SPs_*$C$(7T|$#C3`
zPt%-C8gelZ1GqAP8`ZQmg0{8-S9H{R@D>_XAuJt&EumdH4Yu`UjT<s+s_~uXh}qA8
zg|_HG)%7Pdc&$7*uR0HF@)~vmFjqyD?XZwCbLen^h5t)sm9<90Oa!@Y%8!~rF8G?W
zkzmCF8kMtuuelMHIAlqqnm?72LeGM1J4`w(kX9&%LQn}ForlDLjBoCyvxmo@x3kH^
z^loxLyPiDWPo-cBMnsg2M6}kuPGHEGBqVkC{D&9Kt%xFAsTw4QC1$_=fwBl=3dI+e
zaSxI}JS}=Z(Ec80eF`!Zq3mqapXI|UN!a)t;@4hcu%Eq2xcoW}%!><-+XHTuHss+b
YOmM2;hq8Egm*4=7_P9T{21QBYH*F=mfB*mh
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/prepare-commit-msg.sample b/tests/resources/small.git/hooks/prepare-commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..b1970da1b6d3f42f00069fd17c325de72cda812e
GIT binary patch
literal 1702
zcmb_cTW{Mo6n>t6#i?x6xmZ$SFLf{QfG*3r0L?Pg?px55l8$UTGO3bO;spKi{V3XX
z))weX0X>M9bNMcZ-6yG%>(n}JI2|25dr<ew@wmMG{l(3lx~bQz>}WZBP@ih?JX^+@
zu#5O48P>yRX(m<b*PU*sORp92TCD1dX`%HE+1$w5k<(Ngu7zQ83#MGJR?<<W=d@yL
z#heqwo{FmCg0g#x<~R+PBD#}q(MBn;V$x;%UrJPP3*l%XtlvTWChI2SfJ$9e`YvSj
zRSOQ?NUgSMLI`3vL48YBHzwzVXoe7v0ef|0YHgV$N@?N(-R)rPqRDrK$n(%+OF$`P
zWdjC5N~`#RjV9}aYwQ4_yF5O-$h2`>fDIhYP)doc1&TADZa@ZGpusJ$6G+e$ZMcmC
zoOosDQPS}l{H?YPsq(4;0SGkATa9eeqAaDcj<jNAU+LTSmM1jo(ti~T0B7acJnnTX
zTarYy;Husdxal0!S<ba8=uu&a*SNYtr7|d7$g-q3_JHER2*oBsVW~i~XXdMx%LW~0
zT*960LF<qZ6K&FZ;vGmtyywBU3^Q>q8n2wALbFwU@2i@FAaRV!=uw-nwx1gKn2SvY
z>Ff>;2sg!+Hxfkwv1lsiii=p6WenF=5)6LZc<a$zDCD$GRuwvG4Fr+B#h?#9gTbio
zk#MdxC@WY%zSGN#i}Ts_#q`bf-{)`7CcWeB*7WlIyHjioJJWw&A5VItPUq3^9!r}S
zbykelFV-VFvcr>QaZ=aS_}+-4Y&?!@HWh|<^gJ21!|T@+%On#w6azxPHV}XsRbe*w
zR_TZ2XEsQa1lPK~biYqg@0-RW@5J1@=<87cFzEUABdCoFH2CZo?}l(Z*!OFqUxo>K
z_d`l#4d9|H6;VPT{X?^{VJ>oL|D7K{BJwwqB>`YcPoGk+9hbvHnoQ{EM|kPgD_`wk
zKm4#2xu;-y`RAm!=L_BnLvJ8$AZm8@?)v<%vwvsw8AF2x6!mTT;c72A_~U9nIq0ST
zv)N0!I!^1p=g8-RQfx5)E_Mb_4I2vtQpI30XZ&t<!9D6nI|;V7YR3)l6=S~QhuwLQ
g$e&^kTY-M99*<-Iw@(78*M5W!4}U}|8YyMx3->-9h5!Hn
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/push-to-checkout.sample b/tests/resources/small.git/hooks/push-to-checkout.sample
new file mode 100755
index 0000000000000000000000000000000000000000..a80611e18896f212c390d845e49a3f6d5693b41d
GIT binary patch
literal 2840
zcmai0U31$u5PXh)#YOS7cE^-rw@uolNhe9&aUS|HtvhX>G$45tVUYj>fRdF?|9kfU
zNR~aG=E)WbEbeyq7JTw}ZuHIE2kUtL<<n;$&G!2F^Je|kx2ug=4L5!H^!ogx`7o$&
z%Il(3zAe6<oe$^F=A|}s`8}CDp*M#3M)gC-)LOeDUpYMl3YNy9R)I-T)pE7sy09aj
zJ7%&5PnSB-F#2{jc><WLR{I2izuK%VHc+{hRfXe<^_q)8RjcE(6WX+F2)iAtDtI{x
ztAHVBq)eSp_E^xcV^i_5KLIHA$g{zEjh?rsacu+(Eyvve2~Kmw%;n3g(kWB56j~Js
z<yE5tYUsAR&PY0wgRvM8x!zgLX8SI!eVY&}YZ|>AoeCNptd-NM1aZLhESzC;I`+Ns
zfmNNjdAp^W8#Q*}l>CT7RB9F5(BbI8ly2l~+E};JW|>&d1)=epZ-8vm8ppkbEVn#R
zt30a5A-c(YQR8eM5%;|UAnO>rt!&@x@G@yp+92%w-}%(5P_+P&Wf_zb$f-Qrl5(7z
z2ah(bkE;!DK(&aAMuQ%1TS>ai?wSXCOCSj=_}8x4IbCx^$}9q)<W{Y<9o<WqZo^oV
z3bR;Qa%VT-M~kU@2n{=+=)C!MD`12;@<F*EtPfV3K#g^1%&ggH?4{m<R$WEKcUI4%
zl6{ik6Bo46pmNjdXg5@?hn;SjG$}rr3Gy#-BGgVD$8oD)OcK(oqca)L_kk)UBZ_&6
z*ourb#Yc8l3J+uSda_Y$GY--5Zp3QK9w^?P%E0xb57?fY+Q#+KU4)+R>whwv)SBt|
zg#MX4;;Oau`m=MI9(^&zPbueY@~>3*ixX%mvR5m_1&nAg@ZKvY1E$O}&EtLiG;mhV
z1xhMIm~fGjmf_#{62f`y;09?I7M1W2tWQvz<}i9lR>OpQyUJi45_&*pQus&EkwY<>
zI|ZAx=*3i9a-)g)hXkvO7>UJ5MNgL(Z+-wpXVcgbSgpmFmbf1~DPA(OVGI&FNLeIE
zNH!_aiH$vsif$_j7=T2{cS(!DOI`~bn@)vSd-0d7xL=DF;UNP|tW}4i<qWTSNCe|y
zg9kV&L9g;FhC@tvsVu#V-brqShHy2qZpA!gg{ZTY>h>DvHtu9tY_pbJ6x(6E*hxgC
zzNDao%qlr-IE%YGbS4hF!n!on7#W3$bX-_hbZAaws^nHu#)Dx=WzdbJ>AKzAy@T$x
zSWE^x9+|TEHVEPyaPYa0DOChp?AeHSBBDbZNokQpAY{lE!7geZI=jV)G^2@<iI(N4
zJLJjy@Y<VI$yq;3bVpJICW3D?YDMDl4Oe5pDf{d`i1_3Qx%4uX`$dOz<9e}jm2B-u
z8-?%!7XuiNF2O&MAdl-iw{drG+$F^zT2Z7WRV#c6;IRZEf>l)&91Zb1+`T+oq9wWF
zRV~kGTGce0O~p^6mj{kT5kL(pv>r;Lvd7VDX*P>A^Th`$3cWO<svk?_?FeP@458*2
zA1PqUOdd%VP5&4~ocLK16{1GLll64c=mU81RMFstpnMoLuI7hNIE4N)U%h*#<Fz{C
z9#>0<l7lRrls|W(8=W1O80P~6+U7<4BqC}N4-4GR3w#{O7YmH?JxwJfru2d?e){$5
z@5R+`7Z;1)FW;OkE-(HPisCS;5F4O^Q>L81p4Ysdo3ZP1(SrR-peEdTo;-@bkB((G
zPHYQXUL!@Q$e(OQ;R9r%@Afz+50I7>*^^c&&|E*r-jN)LH=pM4AqMwWxSv|nqjddE
Z4{_hwv8!W(<d3>T<BC%y-6O5i(|^OS%`gA}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/sendemail-validate.sample b/tests/resources/small.git/hooks/sendemail-validate.sample
new file mode 100755
index 0000000000000000000000000000000000000000..effa6f54f2a5c1f0cebe258bf4805904be4b4852
GIT binary patch
literal 2365
zcmb_dU2oe)5PUX&#g-Nv2{5JDX_MA~3%Iq?8g*kpRgpYZIFU=~BI=I5J6e{T{`bz^
zk+$48h#x9Ika!=nv$M0y{clD}-j1x(hDWbnzP?l2k8j?TH{brS+Nf21yPm)Nczma>
zYw`X3V>TCdnSD1ru8&`j=2DIPbCT@SnIgUw>$+lEYP}+x8(BMYnr=iT3*ndq)xzaV
z>I+qjv}vC#8_9M+b1p#uNS0M0)q<p>8!3p_LRQ0MA3M`!2foxzRUjbFY@}O~(ki=S
zqscnq8cU*dY)D$$cqE}n)V0yIk>CNKHCrndOtSP*HbOb;nbwAHSb;R+gs^?^Dve%)
zoW}t(*D}$>O3ab0TS^-;J|u&sb-PkZzo#kn*#xYt(;<xzKW(YtQZ$u23?yV#kyh1~
z@+Idh;EG5jXx8@%<;Y_W8SA=|T;MPQ)TB!!<QcbU)YR4)79eeeg4|vp-8jm%Dl3^I
zRS7+i3>FGuwzSb^g&RDiGcOz9TB;Hu`nJh)$W=C=XCSm2AY=$w3G3P-V#Oo+N*;#2
z4ijJ-pBZ=;T(RTgp_HYrD!uW-dTMfkuqY5jwOy)~gM;#=P^i{!l7`pXTS^s(&^{RU
zydaw}OpS#^D1cXM8?FW+fh`t7D(g;yr6|}fdaNtZBx3hlK~IpkTu3!Qq%R+zAo#<L
zU&m+XIFB0>t}Bs8^3$vHD+-TGT@`F>H1Cc#WAVW;&$S6%fE2d6@kLS0g&ihIM{}0z
z8#XhD>b>3{(BH|Px7}&lJ4%y1v<t$W+!F|W@<gaU4;MqSHKR(wdML<XnCv;zaPrSi
zxVCvek26-bf#Q!H+uAgy_{e_1UZCq>(CihZJx@8MPoGdl*BJGD;usf*iS7%;{Joe;
zNFuBa>*~o&qETDPo~u&~$FxE1xb^x&(CbE`Y3GfsibL2rl+L;>P6j&Y3U>K$mkp*6
zd`Q{<^+^&;GskGjwD-%!boR&i-TC<Uvy02w+l$Nb?B}aL-%ZDpluqd=K_@}z*fyuV
zzAs1Hf?3v$k!X7GTc8Q=r`WJ_Uu=>A9UOR|@=GYb5x#<f&WSMH%mCJU<#=7=qFdL6
zG?Wz&6z&J<@I(B>+dhd7fkaVIR^pol`Mv+rUbmZ43dVL6^S7g3{NsPiG$iy$5EDB%
z6KIgnb$H(n&t3e4E6d4V7w^B?JS}JkG)PM6+X3Co`SQs($O*AA+MG~{S7RJ=cy-l&
z>~%3y`tjfx2>uOu<lDGWewcb=oL@}B@B6FCZ?oxSJWldrm%UfOduahk%C0H>tB_^s
ziwG=e=ch|FQ0IkN91US7rhdQkXhwwt$gU0WEVDjo=IPb+?6PC=s8}J*ua(Ms))`UL
fi$|vMHn?H<rred|1&u#kOoJ`%vx)-*g-ZSfgGvdP
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/update.sample b/tests/resources/small.git/hooks/update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..c3430f6fe27e572a1cebdb2f2bec70add0642a60
GIT binary patch
literal 3707
zcmb_fU2oeq6n(aS#jRq*br9K3y0z;^QgkUc^kKlTrB6js&@yE)mPu8l<0$L?`wmIT
zrsXba))@gJi@e|G+<SfSXe`CeSQ}OG@sr8ZTUlQ{dzM}Q@O-hBi}GeUom`#X%FiYH
zX?m4Rna-0RN2lfK)A3ZuvHcz$L<jUn62D=~vfz{}wIH2VqBLX_O$(JSXeF7H$}q!c
zWY}C&R;eX%X?P{%d;|>_tSE3ettp-hLlsZCxaLX8(nU;bVRB;Ce6@s#eu2|WvLz>-
zvy(&>Gyfp@+BtKnpqWkKi^+v{4jn_pNw_zeuxE<mRXKx8G3;9pl+45&4~hHW!G@wo
za7?X(0B}HbX*ExkDmas*xzV)FxygC8AL?2Z1x-0QJqS@qn8sD7r{bm30@<%eL_gOw
z;~85O$Xw2AS}Qp)5ViRUe3|ir8;&&I<B7Y6^!kkNyYXF4EY(b8_5DsTYn_&?wkdEz
z0y$tADo<&}nGs5kg2-J=0KlEGPb(%<An(pXY{K`qIZCuwiT|2{8JD&5o_~`o6<;dD
zi@J#zCE4={8j%<uy|iutvHu1QKo5Tno-BAF2FwD%%O#UDDum=w!;!PNe-cOFNX4)5
zd>TifiGO|)w}OANj2n2D^K=o3j6P6uOL70#cbA{uzWXDlk1wr9GV1X(2W{RuTvjXV
zCmd<W?kH^?PXjkbF`XZtwu1B+%4@ZvHIwGpJ*8@8`MWAhq^B|Hj1lzj3R8bVuMpNb
zz4Gzk!3T3bY;WEGIww&kq9BYW6EP*q$K|EB-@TH(Fjtz*`HMTO?i+3E;5tdSah&xZ
z+t!x4K7)dpy5wiJhlJz~8qF|r8a&-SV7^I3C@_q=P`yt@_x_F-;PQR)fzP<zNN>8u
zH%V`94=q3)Dk)PHNrnFC(T1)Om6f{Usj;u1R->&XoCYVK2V3ZlgZuF?N}1+33<P7e
z<0yVF?Qoa{l#7q(3&jv=Ab)gpM8A7`p%3InNzSxy)ZER2U0C#9zKpnLY0I?>OER*x
z*9Z=L=zI8CN>A_^jYjt0F$psO$sL=38q5q|SG)qCN6{^>RFh5E&l5GZ$pEahnF&d+
z5c>64t}uJPkf~_!VUj#&N%nC-gUMj%=@B=!V>&}xtj2%@-mOm#rQUSJ3(ccmc+fza
znZ#uxF>N?QN5UrIEd!5RgHEf<eGg}P45aAs(Xs6u!XW9r1I*E6XJ^1movX@xYLuPz
z|7xBN4z@b}#x>W#;(nKYF+D<*rdshJ$X-z2OZ2X;)nn@KSVdVhaA?}@3;6gZxb4<W
z`9sa`0lR_azMX|=t_(FvG@%w2);9P}SG0u&K1(*oX3};~=<<!N*F$UTSxD{V&6mgL
ztw9Ntc2eOF@c!OJytNC4T^#)Ien7-`dI{6s#co~0f^E3J<0Ty)l3xq2u@Y8DXTK>v
zozoWSr{{+!h}zGpumG3H`=AvWpm^9kW;J$Jp^Xl*?8ckr`fqN%c|Z;VC0|cM4vSrk
zH_O8Yvh85nvJp^;``wo8=z0f`FWg?`>gO#y1hjX1{}rTlg9rwIKia8eyGexA3GnuR
z`Rg~XZoW;0pA)vI8=p5!+6sIn#C^FCvR>ffv39h6SCNi9v);%WD;WZ`of_MgwyRWy
z-yY%n*Y>X8<Sf+RyB|Sr2YG@1w~%U$o`(5EDkJ|3$u=eMZA&_wxEsy<Xxh2k^tP?4
RGx&ZHQs^8zuIpu!=pQhfp8Eg*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/info/exclude b/tests/resources/small.git/info/exclude
new file mode 100644
index 0000000000000000000000000000000000000000..a5196d1be8fb59edf8062bef36d3a602e0812139
GIT binary patch
literal 240
zcmXYqK?=e!6h!x)VxWtv*mf_t5?px$aS_{}Hj?C*<cHXeXE&AZhT*-L3ZoI&*l1%Z
zqG?zr3TvQGZ__}H4(u*%p*rI=cU!%ya5ugfGATh66$IJHgu1Gs0-<N;$V+SsdE)?u
zIq;i$f#WE4f$_MWicZjMEob9LWKMR#iwZq54~QgST^6=i%u0lUkJu-_J**QBMq}ZG
Wth_)NDbl|`oQr&HAFQ5h`0jqAIZ*BZ
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e b/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e
new file mode 100644
index 0000000000000000000000000000000000000000..7d80b8d78e0dc55669d831a6638f48ec9fed0982
GIT binary patch
literal 50
zcmV-20L}k+0V^p=O;s>9W-v4`Ff%bx$Vkn}$!Ay}rnY6F$m-Kg*KD_+;Lx#g4|^&N
I02NaX#p`nv=Kufz
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b b/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b
new file mode 100644
index 0000000000000000000000000000000000000000..822bc151862ec3763cf2d3fa2372b93bbd3a4b65
GIT binary patch
literal 30
mcmb<m^geacKghr&@q@?NlP9kSYMj?U<r(;diNWtH+YSKNt_|)0
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4 b/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4
new file mode 100644
index 0000000000000000000000000000000000000000..5adfa88cc6b00bb19f2031b8ab61f6d07f9ccdb8
GIT binary patch
literal 130
zcmV-|0Db>>0i}&W3IZ_@1U=^!a~EV1casc=c+{&un1qQN*i9hD|0|m(2n|iwp*q%W
z%N;b$hu%cM`$TMo*~EnC1BFP&Pfj~;jZVKXQ96s_PhV<-XAROi+@-v8dBLUa`!;GB
k^i<X>XlEv8$>R)1G>9th&t3j;s7J{?^9n<zzF|~BaA?ar-~a#s
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/refs/heads/master b/tests/resources/small.git/refs/heads/master
new file mode 100644
index 0000000000000000000000000000000000000000..4eb36d22298f060fd324155ab854d9d6486fc498
GIT binary patch
literal 41
ucmV~$!4Uu;2m`Rc)5uXl$AMP&AHjriQg~T$i(A>|7U^`%mXoWC24Q^m!3%@{
literal 0
HcmV?d00001
+2 -2
View File
@@ -2,7 +2,7 @@
((NEW_NIX_FIRST_BUILD_UID=351))
id_unavailable(){
id_available(){
dscl . list /Users UniqueID | grep -E '\b'"$1"'\b' >/dev/null
}
@@ -15,7 +15,7 @@ change_nixbld_names_and_ids(){
while read -r name uid; do
echo " Checking $name (uid: $uid)"
# iterate for a clean ID
while id_unavailable "$next_id"; do
while id_available "$next_id"; do
((next_id++))
if ((next_id >= 400)); then
echo "We've hit UID 400 without placing all of your users :("
+11 -7
View File
@@ -1,14 +1,18 @@
{ runCommand
, system
, buildPackages
, cacert
, nix
{
runCommand,
system,
buildPackages,
cacert,
nix,
}:
let
installerClosureInfo = buildPackages.closureInfo {
rootPaths = [ nix cacert ];
rootPaths = [
nix
cacert
];
};
inherit (nix) version;
@@ -65,7 +69,7 @@ runCommand "nix-binary-tarball-${version}" env ''
fn=$out/$dir.tar.xz
mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \
tar cfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \
--absolute-names \
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
system=$(nix eval --raw --impure --expr builtins.currentSystem)
nix eval --json ".#checks.$system" --apply builtins.attrNames | \
jq -r '.[]' | \
xargs -P0 -I '{}' sh -c "nix build -L .#checks.$system.{} || { echo 'FAILED: \033[0;31mnix build -L .#checks.$system.{}\\033[0m'; kill 0; }"
+1 -1
View File
@@ -463,7 +463,7 @@ EOF
EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF
:a
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,suid,owners
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,nosuid,noatime,owners
.
:x
EOF
+3
View File
@@ -96,6 +96,9 @@ poly_configure_nix_daemon_service() {
if [ -e /run/systemd/system ]; then
task "Setting up the nix-daemon systemd service"
_sudo "to create parent of the nix-daemon tmpfiles config" \
mkdir -p "$(dirname "$TMPFILES_DEST")"
_sudo "to create the nix-daemon tmpfiles config" \
ln -sfn "/nix/var/nix/profiles/default$TMPFILES_SRC" "$TMPFILES_DEST"
+37 -31
View File
@@ -1,36 +1,42 @@
{ lib
, runCommand
, nix
, tarballs
{
lib,
runCommand,
nix,
tarballs,
}:
runCommand "installer-script" {
buildInputs = [ nix ];
} ''
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
runCommand "installer-script"
{
buildInputs = [ nix ];
}
''
mkdir -p $out/nix-support
substitute ${./install.in} $out/install \
${lib.concatMapStrings
(tarball: let
inherit (tarball.stdenv.hostPlatform) system;
in '' \
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
)
tarballs
} --replace '@nixVersion@' ${nix.version}
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
substitute ${./install.in} $out/install \
${
lib.concatMapStrings (
tarball:
let
inherit (tarball.stdenv.hostPlatform) system;
in
''
\
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
) tarballs
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
-29
View File
@@ -1,29 +0,0 @@
# configures `scripts/nix-profile.sh.in` (and copies the original to the build directory).
# this is only needed for tests, but running it unconditionally does not hurt enough to care.
configure_file(
input : 'nix-profile.sh.in',
output : 'nix-profile.sh',
configuration : {
'localstatedir': localstatedir,
}
)
# https://github.com/mesonbuild/meson/issues/860
configure_file(
input : 'nix-profile.sh.in',
output : 'nix-profile.sh.in',
copy : true,
)
foreach rc : [ '.sh', '.fish', '-daemon.sh', '-daemon.fish' ]
configure_file(
input : 'nix-profile' + rc + '.in',
output : 'nix' + rc,
install : true,
install_dir : get_option('profile-dir'),
install_mode : 'rw-r--r--',
configuration : {
'localstatedir': localstatedir,
},
)
endforeach
+2 -2
View File
@@ -1,7 +1,7 @@
# Only execute this file once per shell.
# This file is tested by tests/installer/default.nix.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1
export __ETC_PROFILE_NIX_SOURCED=1
NIX_LINK=$HOME/.nix-profile
if [ -n "${XDG_STATE_HOME-}" ]; then
@@ -52,7 +52,7 @@ elif [ -e /etc/pki/tls/certs/ca-bundle.crt ]; then # Fedora, CentOS
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
check_nix_profiles() {
if [ -n "${ZSH_VERSION:-}" ]; then
if [ -n "$ZSH_VERSION" ]; then
# Zsh by default doesn't split words in unquoted parameter expansion.
# Set local_options for these options to be reverted at the end of the function
# and shwordsplit to force splitting words in $NIX_PROFILES below.
+21 -25
View File
@@ -1,35 +1,31 @@
# This file is tested by tests/installer/default.nix.
if [ -n "$HOME" ] && [ -n "$USER" ]; then
if [ -n "${HOME-}" ] && [ -n "${USER-}" ]; then
# Set up the per-user profile.
if [ -n "$NIX_STATE_HOME" ]; then
NIX_LINK="$NIX_STATE_HOME/profile"
NIX_LINK="$HOME/.nix-profile"
if [ -n "${XDG_STATE_HOME-}" ]; then
NIX_LINK_NEW="$XDG_STATE_HOME/nix/profile"
else
NIX_LINK="$HOME/.nix-profile"
if [ -n "${XDG_STATE_HOME-}" ]; then
NIX_LINK_NEW="$XDG_STATE_HOME/nix/profile"
else
NIX_LINK_NEW="$HOME/.local/state/nix/profile"
fi
if [ -e "$NIX_LINK_NEW" ]; then
if [ -t 2 ] && [ -e "$NIX_LINK" ]; then
warning="\033[1;35mwarning:\033[0m"
printf "$warning Both %s and legacy %s exist; using the former.\n" "$NIX_LINK_NEW" "$NIX_LINK" 1>&2
if [ "$(realpath "$NIX_LINK")" = "$(realpath "$NIX_LINK_NEW")" ]; then
printf " Since the profiles match, you can safely delete either of them.\n" 1>&2
else
# This should be an exceptionally rare occasion: the only way to get it would be to
# 1. Update to newer Nix;
# 2. Remove .nix-profile;
# 3. Set the $NIX_LINK_NEW to something other than the default user profile;
# 4. Roll back to older Nix.
# If someone did all that, they can probably figure out how to migrate the profile.
printf "$warning Profiles do not match. You should manually migrate from %s to %s.\n" "$NIX_LINK" "$NIX_LINK_NEW" 1>&2
fi
NIX_LINK_NEW="$HOME/.local/state/nix/profile"
fi
if [ -e "$NIX_LINK_NEW" ]; then
if [ -t 2 ] && [ -e "$NIX_LINK" ]; then
warning="\033[1;35mwarning:\033[0m"
printf "$warning Both %s and legacy %s exist; using the former.\n" "$NIX_LINK_NEW" "$NIX_LINK" 1>&2
if [ "$(realpath "$NIX_LINK")" = "$(realpath "$NIX_LINK_NEW")" ]; then
printf " Since the profiles match, you can safely delete either of them.\n" 1>&2
else
# This should be an exceptionally rare occasion: the only way to get it would be to
# 1. Update to newer Nix;
# 2. Remove .nix-profile;
# 3. Set the $NIX_LINK_NEW to something other than the default user profile;
# 4. Roll back to older Nix.
# If someone did all that, they can probably figure out how to migrate the profile.
printf "$warning Profiles do not match. You should manually migrate from %s to %s.\n" "$NIX_LINK" "$NIX_LINK_NEW" 1>&2
fi
NIX_LINK="$NIX_LINK_NEW"
fi
NIX_LINK="$NIX_LINK_NEW"
fi
# Set up environment.
+7 -6
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bash
set -e
set -euo pipefail
script=$(nix-build -A outputs.hydraJobs.installerScriptForGHA --no-out-link)
installerHash=$(echo "$script" | cut -b12-43 -)
nix build -L ".#installerScriptForGHA" ".#binaryTarball"
installerURL=https://$CACHIX_NAME.cachix.org/serve/$installerHash/install
echo "::set-output name=installerURL::$installerURL"
mkdir -p out
cp ./result/install "out/install"
name="$(basename "$(realpath ./result-1)")"
# everything before the first dash
cp -r ./result-1 "out/${name%%-*}"
+3
View File
@@ -2,6 +2,9 @@
set -eo pipefail
# stock path to avoid unexpected command versions
PATH="$(/usr/bin/getconf PATH)"
((NEW_NIX_FIRST_BUILD_UID=351))
((TEMP_NIX_FIRST_BUILD_UID=31000))
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
if [[ ! -d out ]]; then
echo "run prepare-installer-for-github-actions first"
exit 1
fi
cd out
PORT=${PORT:-8126}
nohup python -m http.server "$PORT" >/dev/null 2>&1 &
pid=$!
while ! curl -s "http://localhost:$PORT"; do
sleep 1
if ! kill -0 $pid; then
echo "Failed to start http server"
exit 1
fi
done
echo 'To install nix, run the following command:'
echo "sh <(curl http://localhost:$PORT/install) --tarball-url-prefix http://localhost:$PORT"
+12 -12
View File
@@ -1,13 +1,14 @@
{ lib
, mkMesonDerivation
{
lib,
mkMesonDerivation,
, meson
, ninja
, doxygen
meson,
ninja,
doxygen,
# Configuration Options
# Configuration Options
, version
version,
}:
let
@@ -42,11 +43,10 @@ mkMesonDerivation (finalAttrs: {
doxygen
];
preConfigure =
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
preConfigure = ''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
postInstall = ''
mkdir -p ''${!outputDoc}/nix-support
+25 -23
View File
@@ -1,13 +1,14 @@
{ lib
, mkMesonDerivation
{
lib,
mkMesonDerivation,
, meson
, ninja
, doxygen
meson,
ninja,
doxygen,
# Configuration Options
# Configuration Options
, version
version,
}:
let
@@ -19,17 +20,19 @@ mkMesonDerivation (finalAttrs: {
inherit version;
workDir = ./.;
fileset = let
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
in fileset.unions [
./.version
../../.version
./meson.build
./doxygen.cfg.in
# Source is not compiled, but still must be available for Doxygen
# to gather comments.
(cpp ../.)
];
fileset =
let
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
in
fileset.unions [
./.version
../../.version
./meson.build
./doxygen.cfg.in
# Source is not compiled, but still must be available for Doxygen
# to gather comments.
(cpp ../.)
];
nativeBuildInputs = [
meson
@@ -37,11 +40,10 @@ mkMesonDerivation (finalAttrs: {
doxygen
];
preConfigure =
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
preConfigure = ''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
postInstall = ''
mkdir -p ''${!outputDoc}/nix-support
+2 -4
View File
@@ -18,8 +18,6 @@
namespace nix {
namespace fs { using namespace std::filesystem; }
fetchers::Settings fetchSettings;
static GlobalConfig::Register rFetchSettings(&fetchSettings);
@@ -121,8 +119,8 @@ MixEvalArgs::MixEvalArgs()
.category = category,
.labels = {"original-ref", "resolved-ref"},
.handler = {[&](std::string _from, std::string _to) {
auto from = parseFlakeRef(fetchSettings, _from, fs::current_path().string());
auto to = parseFlakeRef(fetchSettings, _to, fs::current_path().string());
auto from = parseFlakeRef(fetchSettings, _from, absPath("."));
auto to = parseFlakeRef(fetchSettings, _to, absPath("."));
fetchers::Attrs extraAttrs;
if (to.subdir != "") extraAttrs["dir"] = to.subdir;
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
-10
View File
@@ -32,16 +32,6 @@ InstallableDerivedPath InstallableDerivedPath::parse(
// store path.
[&](const ExtendedOutputsSpec::Default &) -> DerivedPath {
auto storePath = store->followLinksToStorePath(prefix);
// Remove this prior to stabilizing the new CLI.
if (storePath.isDerivation()) {
auto oldDerivedPath = DerivedPath::Built {
.drvPath = makeConstantStorePathRef(storePath),
.outputs = OutputsSpec::All { },
};
warn(
"The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '%s'",
oldDerivedPath.to_string(*store));
};
return DerivedPath::Opaque {
.path = std::move(storePath),
};
+2 -2
View File
@@ -104,12 +104,12 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
auto drvPath = attr->forceDerivation();
std::optional<NixInt::Inner> priority;
std::optional<NixInt> priority;
if (attr->maybeGetAttr(state->sOutputSpecified)) {
} else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aPriority = aMeta->maybeGetAttr("priority"))
priority = aPriority->getInt().value;
priority = aPriority->getInt();
}
return {{
+1 -1
View File
@@ -40,7 +40,7 @@ struct ExtraPathInfoValue : ExtraPathInfo
/**
* An optional priority for use with "build envs". See Package
*/
std::optional<NixInt::Inner> priority;
std::optional<NixInt> priority;
/**
* The attribute path associated with this value. The idea is
Loaded 100 of 699 files, more files were not shown because too many files have changed in this diff. Show more