Compare commits

...
Author SHA1 Message Date
mergify[bot] 38de6fbe38 Merge pull request #12841 from NixOS/mergify/bp/2.25-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:49:36 +00:00
Thomas Miedema 0f181346e4 nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:47:29 +00:00
mergify[bot] d41420c184 Merge pull request #12753 from NixOS/mergify/bp/2.25-maintenance/pr-12105
local-derivation-goal: improve "illegal reference" error (backport #12105)
2025-03-25 16:23:23 +00:00
Sergei Trofimovich 89028b3388 tests/functional/check-refs.nix: format newly added test
(cherry picked from commit 4d72e0f73b)
2025-03-25 15:48:25 +00:00
Sergei Trofimovich 257b660483 tests/functional/check-refs.sh: guard test12 against too old nix daemon
Otherwise without the change the test fails on nix-2.26 as:

    error: derivation contains an illegal reference specifier 'dev'

Note: the error message does not match intended change.
(cherry picked from commit 1e7c7244cf)
2025-03-25 15:48:25 +00:00
Robert Hensing 23cd3720c0 test illegal reference specifier error message
(cherry picked from commit f4def47c89)
2025-03-25 15:48:24 +00:00
Robert Hensing 4c2259b5fe checkRefs: use concatMapStringsSep
(cherry picked from commit 2b4d461c14)
2025-03-25 15:48:24 +00:00
Robert Hensing 0fb8157642 nix-util: Use small_vector in concatMapStringsSep
(cherry picked from commit 32898dc46a)
2025-03-25 15:48:24 +00:00
Robert Hensing 2710d238da nix-util: Add concatMapStrings
(cherry picked from commit f3dbaa3f54)
2025-03-25 15:48:23 +00:00
Sergei Trofimovich dcca4fd4bb local-derivation-goal: improve "illegal reference" error
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.

Today's `nixpkgs` example:

Before the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation contains an illegal reference specifier 'man'

After the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])

(cherry picked from commit bbdc3197a9)
2025-03-25 15:48:23 +00:00
mergify[bot] 1d162593be Merge pull request #12746 from NixOS/mergify/bp/2.25-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-24 16:41:10 +00:00
Sergei Zimmerman 0823a0c6e1 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-24 16:05:32 +00:00
Sergei Zimmerman b7c8d5246e libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-24 16:05:31 +00:00
Sergei Zimmerman 7eaf8ec028 libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-24 16:05:31 +00:00
Sergei Zimmerman fd86343b89 {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-24 16:05:31 +00:00
tomberek 7e9be2b924 Merge pull request #12744 from NixOS/mergify/bp/2.25-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-24 11:21:55 -04:00
Jörg Thalheim 76098180d2 MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)

# Conflicts:
#	src/libutil/unix/monitor-fd.hh
2025-03-24 15:48:34 +01:00
John Ericson e1c559b27a MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-24 15:48:34 +01:00
Jade Lovelace ef12785644 daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-24 15:48:34 +01:00
Félix Baylac Jacqué cf7d72c733 MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-24 15:48:34 +01:00
John Ericson 336d07397d MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-24 13:20:56 +00:00
John Ericson 7be9829896 MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-24 13:20:56 +00:00
Jörg Thalheim a00b385d2a MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-24 13:20:55 +00:00
John Ericson 9883341ea8 monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-24 13:20:55 +00:00
mergify[bot] 1130be437b Merge pull request #12684 from NixOS/mergify/bp/2.25-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 20:59:01 +00:00
Pierre-Etienne Meunier 7fab0741d8 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:21:38 +00:00
mergify[bot] 43286afc8c Merge pull request #12279 from NixOS/mergify/bp/2.25-maintenance/pr-12258
sequoia-nixbld-user-migration: nail down PATH (backport #12258)
2025-03-12 16:22:54 +00:00
Travis A. Everett d87e46cfeb sequoia-nixbld-user-migration: nail down PATH
Fixes a user report of trouble with toybox grep and avoids
potential of same basic issue with other utils.

(cherry picked from commit 6a874c2865)
2025-03-12 16:13:35 +00:00
mergify[bot] 40f33b5e17 Merge pull request #12536 from NixOS/mergify/bp/2.25-maintenance/pr-12391
Only try to chmod /nix/var/nix/profiles/per-user when necessary (backport #12391)
2025-02-20 16:53:37 +00:00
Sandro Jäckel 807d186289 Only try to chmod /nix/var/nix/profiles/per-user when necessary
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit dcbf4dcc09)
2025-02-20 16:11:27 +00:00
mergify[bot] d99bcd62eb Merge pull request #12529 from NixOS/mergify/bp/2.25-maintenance/pr-12439
Set FD_CLOEXEC on sockets created by curl (backport #12439)
2025-02-19 20:21:40 +00:00
MaxHearnden a5fa357fdf Set FD_CLOEXEC on sockets created by curl
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

(cherry picked from commit 12d2527276)
2025-02-19 19:47:45 +00:00
mergify[bot] 9e7663fe38 Merge pull request #12522 from NixOS/mergify/bp/2.25-maintenance/pr-12518
packaging: Restore libgit2 USE_SSH=exec (backport #12518)
2025-02-19 09:18:52 +00:00
Robert Hensing e2558b8c26 packaging: Restore libgit2 USE_SSH=exec
... when nixpkgs is nixos-unstable or the overlay is used.

(cherry picked from commit 5488e29d2f)
2025-02-19 10:10:45 +01:00
mergify[bot] 9442b9bd7b Merge pull request #12516 from NixOS/mergify/bp/2.25-maintenance/pr-12484
Support libgit2 1.9.0 (backport #12484)
2025-02-18 23:32:16 +00:00
Robert Hensing c03bde0133 Support libgit2 1.9.0
For when the overlay is used with nixos-unstable.
1.9.0 has our patches.

(cherry picked from commit b0bbb1252a)
2025-02-19 00:26:46 +01:00
mergify[bot] d652513e45 Merge pull request #12452 from NixOS/mergify/bp/2.25-maintenance/pr-12448
copyPathToStore(): Preserve symlinks (backport #12448)
2025-02-12 20:00:54 +00:00
Eelco Dolstra 4aba8ad022 Delete unused test 2025-02-12 20:05:02 +01:00
Eelco Dolstra d856328ab5 Fix build 2025-02-12 20:04:48 +01:00
Eelco Dolstra 9e83b1aa19 Merge pull request #12456 from NixOS/mergify/bp/2.25-maintenance/pr-12450
packaging/components.nix: Don't import a symlink (backport #12450)
2025-02-12 12:48:48 +01:00
Eelco Dolstra a0fb43e229 Don't import a symlink
This is a workaround to avoid differing evaluation results between Nix
2.19 and >= 2.20 (#12449).

(cherry picked from commit 2e20a5f822)
2025-02-12 11:41:45 +00:00
Eelco Dolstra 7bfd71e10d copyPathToStore(): Preserve symlinks
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

(cherry picked from commit 26b87e78b5)
2025-02-11 21:51:38 +00:00
mergify[bot] 37393f0c69 Merge pull request #12366 from NixOS/mergify/bp/2.25-maintenance/pr-12362
refactor: Extract EvalState::realiseString (backport #12362)
2025-01-27 16:19:54 +00:00
Robert Hensing 7d3021f59f refactor: Extract EvalState::realiseString
(cherry picked from commit 7465fbe926)
2025-01-27 15:51:13 +00:00
mergify[bot] 95253b1f7c Merge pull request #12358 from NixOS/mergify/bp/2.25-maintenance/pr-12356
Improve "illegal path references in fixed output derivation" error (backport #12356)
2025-01-26 20:12:55 +00:00
Ben Millwood 3fd1316438 Improve "illegal path references in fixed output derivation" error
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.

(cherry picked from commit 4e5d1b281e)
2025-01-26 19:38:50 +00:00
Robert Hensing 79a2993e23 Merge pull request #12353 from NixOS/nixfmt-on-2.25
Apply nixfmt (backport 2.25)
2025-01-24 22:41:36 +01:00
Robert Hensing c69579114b test: Fix shifted source positions after formatting 2025-01-24 22:09:21 +01:00
Robert Hensing 24c78e06cf Format .nix files
This does not include any automation for the release branch, but
is based on the configuration of https://github.com/NixOS/nix/pull/12349

    pre-commit run -a nixfmt-rfc-style
2025-01-24 20:37:38 +01:00
mergify[bot] 566b33b0bd Merge pull request #12345 from NixOS/mergify/bp/2.25-maintenance/pr-12336
libstore: Fix progress bars (backport #12336)
2025-01-24 11:47:01 +00:00
Philipp Otterbein 6224d87564 libstore: fix progress bars
(cherry picked from commit be97dc1efc)
2025-01-24 11:10:36 +00:00
Eelco Dolstra 636c9cfdf0 Merge pull request #12333 from NixOS/mergify/bp/2.25-maintenance/pr-12331
GitRepo::fetch(): Ignore $GIT_DIR (backport #12331)
2025-01-22 22:04:19 +01:00
Eelco Dolstra e64fcf62f9 GitRepo::fetch(): Ignore $GIT_DIR
Fixes #12325.

(cherry picked from commit 41983dba8f)
2025-01-22 20:30:25 +00:00
Eelco Dolstra 08d4e67e84 GitRepo::fetch(): Cleanup
(cherry picked from commit bd10b859f7)
2025-01-22 20:30:25 +00:00
Eelco Dolstra 5d2f26e3f6 Bump version 2025-01-22 10:29:21 +01:00
mergify[bot] 132c992ce0 Merge pull request #12319 from NixOS/mergify/bp/2.25-maintenance/pr-12315
tests: Wait for network *online* targets and/or multi-user targets (more) (backport #12315)
2025-01-20 18:28:05 +00:00
Robert Hensing 0284eef493 tests: Wait for network *online* targets
More prevalent than I thought in 15073e86a

See also https://github.com/NixOS/nix/actions/runs/12872412321/job/35887830320?pr=12310
which is a failed github-flakes test without "Network is Online"

(cherry picked from commit 6f0bdd9ae7)

# Conflicts:
#	tests/nixos/nix-docker.nix
2025-01-20 18:16:43 +00:00
Eelco Dolstra df5246dd8e Merge pull request #12310 from NixOS/mergify/bp/2.25-maintenance/pr-12302
tests: Wait for network *online* and multi-user targets (backport #12302)
2025-01-20 18:19:24 +01:00
Robert Hensing 8b205302f3 tests: Wait for network *online* and multi-user targets
This should help prevent some test stalls.

By default, multi-user.target does not imply that the network is fully up.

(cherry picked from commit 15073e86a8)
2025-01-20 16:40:04 +00:00
Eelco Dolstra 8e87c19125 Merge pull request #12308 from NixOS/mergify/bp/2.25-maintenance/pr-12294
processGraph(): Don't throw ThreadPoolShutDown if there is an exception (backport #12294)
2025-01-20 17:19:12 +01:00
Eelco Dolstra c38234bffb Merge pull request #12300 from NixOS/mergify/bp/2.25-maintenance/pr-12296
addMultipleToStore(): Move pathsToCopy (backport #12296)
2025-01-20 16:26:35 +01:00
Eelco Dolstra 4d0e2b5ade processGraph(): Don't throw ThreadPoolShutDown if there is an exception
Fixes

  $ nix copy --derivation --to /tmp/nix /nix/store/...
  error: cannot enqueue a work item while the thread pool is shutting down

The ThreadPoolShutDown exception was hiding the reason for the thread
pool shut down, e.g.

  error: cannot add path '/nix/store/03sl46khd8gmjpsad7223m32ma965vy9-fix-static.patch' because it lacks a signature by a trusted key

(cherry picked from commit a8c69cc907)
2025-01-20 15:24:55 +00:00
Eelco Dolstra aabf6e6c65 Add comment
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit 2669e4ac4f)
2025-01-20 14:18:26 +00:00
Eelco Dolstra 7dd6b980aa addMultipleToStore(): Move pathsToCopy
This allows RemoteStore::addMultipleToStore() to free the Source
objects early (and in particular the associated sinkToSource()
buffers). This should fix #7359. For example, memory consumption of

  nix copy --derivation --to ssh-ng://localhost?remote-store=/tmp/nix --derivation --no-check-sigs \
    /nix/store/4p9xmfgnvclqpii8pxqcwcvl9bxqy2xf-nixos-system-...drv

went from 353 MB to 74 MB.

(cherry picked from commit cc838e8181)
2025-01-20 14:18:25 +00:00
Eelco Dolstra daf101d487 Merge pull request #12280 from DeterminateSystems/fix-2.25-build
Don't absolutize 'git+file:' in parseURLFlakeRef()
2025-01-17 13:45:37 +01:00
Eelco Dolstra 938f0f4fd9 Don't absolutize 'git+file:' in parseURLFlakeRef()
https://github.com/NixOS/nix/issues/12273#issuecomment-2596069519
2025-01-16 19:17:08 +01:00
Eelco Dolstra 40bb5ec675 Merge pull request #12269 from NixOS/mergify/bp/2.25-maintenance/pr-12254
Fix relative 'path:' flakerefs in the CLI (backport #12254)
2025-01-15 21:54:43 +01:00
mergify[bot] 98506a5ef1 Merge pull request #12266 from NixOS/mergify/bp/2.25-maintenance/pr-12107
fetchers/git: make relative path absolute for local repo (backport #12107)
2025-01-15 20:46:25 +00:00
Eelco Dolstra 639f526405 Use isAbsolute()
(cherry picked from commit ff9d886f3c)
2025-01-15 20:20:46 +00:00
Eelco Dolstra 0b57e7daf5 Fix relative 'path:' flakerefs in the CLI
And handle relative 'git+file:' flakerefs while we're at it (these
crashed with an assertion failure).

Fixes #12248.

(cherry picked from commit ff8e2fe84e)
2025-01-15 20:20:45 +00:00
Robert Hensing dcfd80610b Clarify cd call in tests/functional/flakes/flakes.sh
(cherry picked from commit d9a50c0af2)
2025-01-15 19:56:17 +00:00
Bryan Lai 4fea745fe0 tests/flake-in-submodule: git+file:./* input
(cherry picked from commit 37ac18d1d9)
2025-01-15 19:56:17 +00:00
Bryan Lai 335e0d9508 tests/flakes: check git+file:./${submodule} protocol
Relative, local git repo used to work (for submodules), but it
fails after 3e0129ce3b.

This commit adds a test to prevent such failure in the future.

(cherry picked from commit 9d088fa502)
2025-01-15 19:56:16 +00:00
Bryan Lai 6963fd4027 fetchers/git: make path absolute for local repo
(cherry picked from commit 96bd9bad2f)
2025-01-15 19:56:16 +00:00
mergify[bot] b8264c4c54 Merge pull request #12260 from NixOS/mergify/bp/2.25-maintenance/pr-11922
gc: resume GC after a pathinuse error (backport #11922)
2025-01-15 16:31:35 +00:00
Picnoir 30ef0a1f4c gc: resume GC after a pathinuse error
First the motivation: I recently faced a bug that I assume is coming
from the topoSortPaths function where the GC was trying to delete a
path having some alive referrers. I resolved this by manually deleting
the faulty path referrers using nix-store --query --referrers. I sadly
did not manage to reproduce this bug.

This bug alone is not a big deal. However, this bug is
triggering a cascading failure: invalidatePathChecked is throwing a
PathInUse exception. This exception is not catched and fails the whole GC
run. From there, the machine (a builder machine) was unable to GC its
Nix store, which led to an almost full disk with no way to
automatically delete the dead Nix paths.

Instead, I think we should log the error for the specific store path
we're trying to delete, specifying we can't delete this path because
it still has referrers. Once we're done with logging that, the GC run
should continue to delete the dead store paths it can delete.

(cherry picked from commit ced8d311a5)
2025-01-15 15:59:26 +00:00
Eelco Dolstra 50a7056b02 Bump version 2025-01-14 14:40:30 +01:00
mergify[bot] 4c35e6ec6e Merge pull request #12247 from NixOS/mergify/bp/2.25-maintenance/pr-12245
scripts/install-multi-user: fix typo (backport #12245)
2025-01-13 20:15:49 +00:00
Siddarth Kumar 3f96cfd99c scripts/install-multi-user: fix typo
(cherry picked from commit fd053fdcad)
2025-01-13 20:06:13 +00:00
Eelco Dolstra 5b32a021a9 Bump version 2025-01-13 16:55:24 +01:00
Jörg Thalheim cadd9fbe88 Merge pull request #12243 from NixOS/mergify/bp/2.25-maintenance/pr-12235
libutil: thread-pool: ensure threads finished on error (backport #12235)
2025-01-13 09:42:12 +01:00
Dominique Martinet 0ccd1fa0a4 libutil: thread-pool: ensure threads finished on error
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.

Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.

It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.

By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.

[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]

Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
(cherry picked from commit afac093b34)
2025-01-13 08:03:12 +00:00
mergify[bot] 8615df48a1 Merge pull request #12232 from NixOS/mergify/bp/2.25-maintenance/pr-12166
nix upgrade-nix: Give a better error message if the profile is using 'nix profile' (backport #12166)
2025-01-11 21:00:48 +00:00
Eelco Dolstra 6b03f1219e nix upgrade-nix: Give a better error message if the profile is using 'nix profile'
(cherry picked from commit fccfdbea57)
2025-01-11 20:07:00 +00:00
mergify[bot] 08cd0acf65 Merge pull request #12210 from NixOS/mergify/bp/2.25-maintenance/pr-11294
parser-state: fix attribute merging (backport #11294)
2025-01-10 20:39:33 +00:00
mergify[bot] 40e35dc567 Merge pull request #12189 from NixOS/mergify/bp/2.25-maintenance/pr-11853
fix(libmain/common-args): do not exceed maximum allowed verbosity (backport #11853)
2025-01-10 18:55:16 +00:00
mergify[bot] 1521a82077 Merge pull request #12184 from NixOS/mergify/bp/2.25-maintenance/pr-12051
Skip tests on systems with restricted usernamespaces (backport #12051)
2025-01-10 18:55:14 +00:00
mergify[bot] 009de1f7ac Merge pull request #12179 from NixOS/mergify/bp/2.25-maintenance/pr-12103
fix: ignore symlinks in fsync-store-paths (backport #12103)
2025-01-10 18:55:11 +00:00
mergify[bot] 7a7a3d20d1 Merge pull request #12177 from NixOS/mergify/bp/2.25-maintenance/pr-12114
fix documentation of substring (backport #12114)
2025-01-10 18:55:08 +00:00
mergify[bot] 70ab8bbb28 Merge pull request #12175 from NixOS/mergify/bp/2.25-maintenance/pr-12115
libcmd: update to support lowdown-1.4 API (backport #12115)
2025-01-10 18:55:05 +00:00
Eelco Dolstra c902a299a8 Merge pull request #12164 from NixOS/mergify/bp/2.25-maintenance/pr-12157
parsePathFlakeRefWithFragment(): Handle 'path?query' without a fragment (backport #12157)
2025-01-10 18:11:30 +01:00
Eelco Dolstra dee91873bc Merge pull request #12186 from NixOS/mergify/bp/2.25-maintenance/pr-12016
Disable suid and atime on the /nix mount point on Darwin (backport #12016)
2025-01-10 15:19:26 +01:00
Eelco Dolstra 31d6afb476 Merge pull request #12181 from NixOS/mergify/bp/2.25-maintenance/pr-12091
libstore: fixup unformatted uri when S3 getObject fails (backport #12091)
2025-01-10 15:17:50 +01:00
Eelco Dolstra ef1e704707 Attempt to make the FlakeRef test succeed on macOS
(cherry picked from commit 3ad0f45e79)
2025-01-10 13:37:40 +01:00
Eelco Dolstra 21fe544ed9 Remove unused variable
(cherry picked from commit 1a38e62a09)
2025-01-10 13:37:36 +01:00
Eelco Dolstra e1178b8a39 parsePathFlakeRefWithFragment(): Add unit tests
(cherry picked from commit 5f7b535b81)
2025-01-10 13:37:32 +01:00
Eelco Dolstra 31fa189123 parsePathFlakeRefWithFragment(): Handle query params in the non-git case
Backported from lazy-trees.

(cherry picked from commit 83ff523865)
2025-01-10 13:37:28 +01:00
Eelco Dolstra faecb6e306 parsePathFlakeRefWithFragment(): Handle 'path?query' without a fragment
Commands like `nix flake metadata '.?submodules=1'` ignored the query
part of the URL, while `nix build '.?submodules=1#foo'` did work
correctly because of the presence of the fragment part.

(cherry picked from commit 28caa35a97)
2025-01-10 13:37:23 +01:00
Eelco Dolstra 200e3be41a ParsedURL: Remove base field
(cherry picked from commit 4077aa43a8)
2025-01-10 13:37:05 +01:00
Eelco Dolstra 8856b5f2ca ParsedURL: Remove url field
This prevents a 'url' field that is out of sync with the other
fields. You can use to_string() to get the full URL.

(cherry picked from commit f705ce7f9a)
2025-01-10 13:37:00 +01:00
Eelco Dolstra aa5246bfe3 Drop std::make_pair
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
(cherry picked from commit ebb19cc1cd)
2025-01-10 13:36:43 +01:00
Eelco Dolstra 8b4a89f4bb Clean up flakeref parsing
This factors out some commonality in calling fromURL() and handling
the "dir" parameter into a fromParsedURL() helper function.

(cherry picked from commit 850281908c)
2025-01-10 13:36:37 +01:00
Eelco Dolstra ab4159ff99 Merge pull request #12174 from NixOS/mergify/bp/2.25-maintenance/pr-12127
toJSON: re-throw serialization exception (backport #12127)
2025-01-10 13:32:53 +01:00
mergify[bot] b9f89c846d Merge pull request #12171 from NixOS/mergify/bp/2.25-maintenance/pr-12155
Document `--max-freed` for `nix-collect-garbage` (backport #12155)
2025-01-10 11:00:31 +00:00
Ryan Hendrickson 5575a2f1c3 parser-state: fix attribute merging
(cherry picked from commit 8034589d7e)
2025-01-10 10:40:07 +00:00
Sergei Zimmerman 2cb2fbbca7 fix(libmain/common-args): do not exceed maximum allowed verbosity
This patch gets rid of UB when verbosity exceeds the maximum logging value of `lvlVomit = 7` and
reaches invalid values (e.g. 8). This is actually triggered in functional tests.
There are too many occurrences to list, but here's one from the UBSAN log:

../src/libstore/gc.cc:610:5: runtime error: load of value 8, which is not a valid value for type 'Verbosity'

(cherry picked from commit b9f8c4af40)
2025-01-10 10:04:40 +00:00
Graham Christensen 6e9281be6d Disable suid and atime on the /nix mount point on Darwin
The Determinate Nix Installer has set nosuid and noatime in https://github.com/DeterminateSystems/nix-installer/pull/1338, and figured this perf and security improvement is worthy of upstreaming.

The /nix volume shouldn't have setuid binaries anyway, and filesystems seem to generally be noatime on macOS.
Further, the garbage collector doesn't use atime.

(cherry picked from commit 4137ead7a1)
2025-01-10 09:56:36 +00:00
Jörg Thalheim 4e990b0bcb functional-tests: skip tests if the kernel restricts unprivileged user namespaces
Update tests/functional/common/functions.sh

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
(cherry picked from commit da7f7ba810)
2025-01-10 09:52:32 +00:00
Cole Helbling 8f5ea8b8ca libstore: fixup unformatted uri when S3 getObject fails
(cherry picked from commit b978fa8450)
2025-01-10 09:48:23 +00:00
Cole Helbling f8999cd4ee tests/nixos/s3-binary-cache-store: disable default substituter so it runs faster
Since networking is disabled in these VMs, trying to talk to the default
cache.nixos.org slows the test down (since it can't resolve it).

(cherry picked from commit f0c1262d23)
2025-01-10 09:48:23 +00:00
Cole Helbling 6aaf623058 tests/nixos/s3-binary-cache-store: test that "object does not exist" error message is properly formatted
(cherry picked from commit 535724fd79)
2025-01-10 09:48:22 +00:00
Yaroslav Bolyukin 24a9c6fad9 fix: ignore symlinks in fsync-store-paths
Fixes: https://github.com/NixOS/nix/issues/12099
(cherry picked from commit 4a91e627a7)
2025-01-10 09:45:22 +00:00
Philipp Otterbein d54c283821 fix documentation of substring
(cherry picked from commit 1e2cace5f1)
2025-01-10 09:44:18 +00:00
Sergei Trofimovich bead70acc2 libcmd: update to support lowdown-1.4 API
Upstream change
https://github.com/kristapsdz/lowdown/commit/bab1d75079ac3866fc14e1fa4825ae64a94a7c0e
moved a few fields from `lowdown_opts` toa  new `lowdown_opts_term`
struct. As a result the build started failing as:

    nix-cmd> [2/17] Compiling C++ object libnixcmd.so.p/markdown.cc.o
    nix-cmd> FAILED: libnixcmd.so.p/markdown.cc.o
    nix-cmd> g++ -Ilibnixcmd.so.p -I. -I.. -I/nix/store/b0bnrk5lacxbpgxgnc28r8q3wcazrgxj-nix-util-2.26.0pre-dev/include/nix -I/nix/store/cxnynq9ykyj4xxv6wf6dw7r0aw5x6n9k-libarchive-3.7.7-dev/include -I/nix/store/bfgjwkcb8snkizx578rzdahi75m8zyh4-nlohmann_json-3.11.3/include -I/nix/store/3sx8bq3sip6j2nv1m5xx4gbdp33v7iy6-nix-store-2.26.0pre-dev/include/nix -I/nix/store/sih2dgqzvsbv7p510lkfmas7s7wbsl4j-nix-fetchers-2.26.0pre-dev/include/nix -I/nix/store/68p8s20fsiiakj7nys7grbaixfnhsdzs-nix-expr-2.26.0pre-dev/include/nix -I/nix/store/gw7wknhzhfzzj9zww2kyi5xrzgf1ndki-boehm-gc-8.2.8-dev/include -I/nix/store/3jwb9j4vnsk5saq3wfyyp9il3mhs41l9-nix-flake-2.26.0pre-dev/include/nix -I/nix/store/8nwjvmq7m48v8g646jrxkikv6x47bc3m-nix-main-2.26.0pre-dev/include/nix -I/nix/store/rb0hzsw5wc1a7daizhpj824mbxlvijrq-lowdown-1.4.0-dev/include -I/nix/store/m388ywpk53fsp8r98brfd7nf1f5sskv0-editline-1.17.1-dev/include -fdiagnostics-color=always -D_GLIBCXX_ASSERTIONS=1 -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++2a -include config-util.hh -include config-store.hh -include config-expr.hh -include config-main.hh -include config-cmd.hh -Wdeprecated-copy -Werror=suggest-override -Werror=switch -Werror=switch-enum -Werror=unused-result -Wignored-qualifiers -Wimplicit-fallthrough -Wno-deprecated-declarations -O3 -fPIC -pthread -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -std=c++2a -MD -MQ libnixcmd.so.p/markdown.cc.o -MF libnixcmd.so.p/markdown.cc.o.d -o libnixcmd.so.p/markdown.cc.o -c ../markdown.cc
    nix-cmd> ../markdown.cc: In function 'std::string nix::doRenderMarkdownToTerminal(std::string_view)':
    nix-cmd> ../markdown.cc:28:5: error: 'lowdown_opts' has no non-static data member named 'cols'
    nix-cmd>    28 |     };
    nix-cmd>       |     ^

The change adds version-based conditional to support both pre-1.4 and
1.4 forms of the initialization.

Closes: https://github.com/NixOS/nix/issues/12113
(cherry picked from commit edbfe863ce)
2025-01-10 09:43:18 +00:00
Philipp Otterbein 4bf020de50 toJSON: re-throw serialization exception
(cherry picked from commit 4a2310a3a0)
2025-01-10 09:42:14 +00:00
Domagoj Mišković 6b5b538d98 Document --max-freed for nix-collect-garbage (#12155)
* Update nix-collect-garbage.md

Referencing issue at: https://github.com/NixOS/nix/issues/12132

Copied the description of `--max-freed` option from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44

(cherry picked from commit 5230d3ecc4)
2025-01-10 09:37:10 +00:00
mergify[bot] 85a4f62400 Merge pull request #12151 from NixOS/mergify/bp/2.25-maintenance/pr-12143
mergify: fix installer test name (backport #12143)
2025-01-07 19:42:57 +00:00
Jörg Thalheim f9fc7fea5c mergify: fix installer test name
(cherry picked from commit 438a20427f)
2025-01-07 19:18:11 +00:00
Jörg Thalheim e67cf5f306 Merge pull request #12145 from NixOS/mergify/bp/2.25-maintenance/pr-12046
CLI symlink fixes (backport #12046)
2025-01-07 07:54:59 +01:00
Robert Hensing 336fdaa119 test: Avoid regressing accidental use of weakly_canonical instead of makeParentCanonical
I'd messed up a rebase in my previous iteration, causing `weakly_canonical` to reappear,
but not trigger a test failure.

These two functions behave similarly when the argument is a path that points to a broken
symlink. `weakly_canonical` would not resolve it because the target doesn't exist, and
`makeParentCanonical` would not resolve it, because it never resolves the final path
element.
This new test case now also tests a valid symlink, "differentiating" the two.

(cherry picked from commit 4c74d679b6)
2025-01-07 06:03:58 +00:00
Jörg Thalheim b0784dd69b makeParentCanonical: test case where parent is empty
(cherry picked from commit 5a5a86949a)
2025-01-07 06:03:58 +00:00
Robert Hensing 7c4e2fdfd7 test: Add hydraJobs.tests.functional_symlinked-home
(cherry picked from commit 628c11d237)
2025-01-07 06:03:57 +00:00
Robert Hensing 3fbcd8d7ba fix: Resolve CLI parent symlinks before adding to store
Fixes https://github.com/NixOS/nix/issues/11941

(cherry picked from commit ddbbf53767)
2025-01-07 06:03:57 +00:00
Robert Hensing fdd61cf529 refactor: Don't re-construct SourcePath unnecessarily
(cherry picked from commit c0b64f3377)
2025-01-07 06:03:57 +00:00
Robert Hensing 6d7ef8f29d doc: Document nix-store --add-fixed symlink behavior
Tested with

    nix run nix/2.3-maintenance#nix-store -- --add some_symlink
    nix run nix/2.3-maintenance#nix-store -- --add-fixed sha256 --recursive some_symlink

(cherry picked from commit 91e91f62fa)
2025-01-07 06:03:56 +00:00
Robert Hensing 255379daf1 fix: Handle symlinks and FIFOs in nix hash where possible
Fixes https://github.com/NixOS/nix/issues/11756
Fixes https://github.com/NixOS/nix/issues/11681

(cherry picked from commit 36563c69a4)
2025-01-07 06:03:56 +00:00
Robert Hensing 0213f22650 Add makeParentCanonical()
(cherry picked from commit 69853c067c)
2025-01-07 06:03:55 +00:00
Jörg Thalheim 8abff3cf05 Merge pull request #12075 from NixOS/mergify/bp/2.25-maintenance/pr-12059
installer: make sure we can always test the installer in ci and locally (backport #12059)
2025-01-07 00:32:01 +01:00
Robert Hensing d910cab254 .github/ci: Use fixed names
This lets us update "runs-on" without creating a mismatch with the
required checks that are configured for the repo in the github ui.
2025-01-07 00:05:23 +01:00
NAHO a3522b4696 ci: lock macOS runner to macos-14 2025-01-07 00:05:23 +01:00
NAHO 29a8ad18ff ci: update Ubuntu runner to ubuntu-24.04
Link: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:05:23 +01:00
NAHO 2d63629ed0 ci: lock Ubuntu runner to ubuntu-22.04
Lock the Ubuntu runner to ubuntu-22.04 to avoid accidental updates [1]
and increase reproducibility.

[1]: https://github.com/actions/runner-images/issues/10636
2025-01-07 00:05:23 +01:00
Jörg Thalheim 92b14e072f mergify: updated required tests
(cherry picked from commit df3eb4f3f6)
2025-01-07 00:05:23 +01:00
Jörg Thalheim 9a2cf3468c installer: make sure we can always test the installer in ci and locally
Just now there is a dependency on cachix, which means we cannot test
the installer in CI if forks do not have the necessary secrets set up.
We replace this with a simple http server that serves the installer and
can be both used in CI and locally.

(cherry picked from commit 6162e1220e)
2025-01-07 00:05:18 +01:00
Eelco Dolstra c396e24c3a Merge pull request #12081 from NixOS/mergify/bp/2.25-maintenance/pr-12080
Fix `flake.lock`/`flake.nix` mix-up (backport #12080)
2024-12-16 23:26:35 +01:00
Jason Yundt f1a4544886 Fix flake.lock/flake.nix mix-up
src/nix/flake.md describes the format of flake.lock files. Before this
change, it said that the original field was “The original input
specification from `flake.lock`[…]” The original input specification is
in flake.nix, not flake.lock.

(cherry picked from commit 51463d2280)
2024-12-16 22:03:50 +00:00
mergify[bot] bb08cc7cc2 Merge pull request #12065 from NixOS/mergify/bp/2.25-maintenance/pr-12048
Fix `nix upgrade-nix` profile search (backport #12048)
2024-12-15 21:24:09 +00:00
Jörg Thalheim 51166bb388 upgrade-nix: don't double quote path on error
the format error already adds quotes.

(cherry picked from commit ccaa4c259a)
2024-12-15 20:52:06 +00:00
Greg Curtis e467269a44 Fix nix upgrade-nix profile search
Commit cfe66dbec updated `nix upgrade-nix` to use
`ExecutablePath::load().find`, which broke the logic for finding the
profile associated with the nix executable. The error looks something
like:

```
$ sudo -i nix upgrade-nix --debug
found Nix in '"/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin/nix"'
found profile '/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin'
error: directory '"/nix/store/46p1z0w9ad605kky62dr53z4h24k2a5r-nix-2.25.2/bin/nix"' does not appear to be part of a Nix profile
```

This seems to happen for two reasons:

1. The original PATH search resulted in a directory, but `find` returns
   the path to the executable. Fixed by getting the path's parent.
2. The profile symlink cannot be found because
   `ExecutablePath::load().find` canonicalizes the executable path. I
   updated find to normalize the path instead, which seems more in line
   with how other programs resolve paths. I'm not sure if this affects
   other callers though.

I manually tested this on macOS and Linux, and it seemed to fix
upgrading from 2.25.2 to 2.25.3.

(cherry picked from commit 4f831e2be5)
2024-12-15 20:52:05 +00:00
mergify[bot] fd087ceec6 Merge pull request #12055 from NixOS/mergify/bp/2.25-maintenance/pr-12050
speed up ci by splitting off eval and build + fix ci with restricted namespaces (backport #12050)
2024-12-14 15:51:12 +00:00
Jörg Thalheim 75b657cf7c ci: disable apparmor restrictions
For our tests we need to map the root user for some tests.
However ubuntu no longer allows this by default:
https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces

(cherry picked from commit 9f2b9256b6)
2024-12-14 15:24:51 +00:00
Jörg Thalheim cdd3e3015d binary-tarball: create tarball without listing files
This is producing quite a bit of log output in CI.

(cherry picked from commit 31b3a34e5b)
2024-12-14 15:24:51 +00:00
Jörg Thalheim 79a52afe82 speed up ci by splitting off eval and build
- This speeds up macOS builds from 30 minutes to 11 minutes (3x faster).
- Also improve error reporting e.g. printing out what actually failed to build.
- As a result we also no longer need swap.

(cherry picked from commit 691b1ea237)
2024-12-14 15:24:50 +00:00
Jörg Thalheim fe3c94d5c2 Merge pull request #12035 from NixOS/mergify/bp/2.25-maintenance/pr-11921
Make @nix json structured build log parsing warn instead of fail (backport #11921)
2024-12-10 05:37:59 +01:00
Robert Hensing 9f2df5899c tests: Make unusual logging test conditional on fixed daemon version
(cherry picked from commit c783cd22ac)
2024-12-10 00:16:09 +01:00
Robert Hensing 2eb816226e Remove redundant warning: prefix from structured build log warning
(cherry picked from commit f3c722cab2)
2024-12-09 21:40:41 +00:00
Robert Hensing a867747c05 test: Move unusual-logging to run only in logging test case
(cherry picked from commit 1421420e86)
2024-12-09 21:40:37 +00:00
Robert Hensing c709ca6e36 Push log source description out of libutil and report build hook @nix warning correctly
(cherry picked from commit 03d4bfd852)
2024-12-09 21:40:32 +00:00
Robert Hensing b1058ee0d2 tests/functional/dependencies.nix: Check that we tolerate syntax and type errors
(cherry picked from commit 1485937b89)
2024-12-09 21:40:27 +00:00
Linus Heckemann 7c98167eab libutil: handle json builder log messages with unexpected format
Before this change, expressions like:

with import <nixpkgs> {};
runCommand "foo" {} ''
  echo '@nix {}' >&$NIX_LOG_FD
''

would result in Lix crashing, because accessing nonexistent fields of
a JSON object throws an exception.

Rather than handling each field individually, we just catch JSON
exceptions wholesale. Since these log messages are an unusual
circumstance, log a warning when this happens.

Fixes #544.

Change-Id: Idc2d8acf6e37046b3ec212f42e29269163dca893
(cherry picked from commit e55cd3beea710db727fd966f265a1b715b7285f3)
(cherry picked from commit ee03fd478e)
2024-12-09 21:40:23 +00:00
Robert Hensing c797848b88 tests/functional/dependencies.nix: Refactor, replace arcane let
(cherry picked from commit e82ff51726)
2024-12-09 21:40:19 +00:00
Eelco Dolstra 8a81c2428a Merge pull request #12018 from NixOS/mergify/bp/2.25-maintenance/pr-12013
nix hash convert: Support SRI hashes that lack trailing '=' characters (backport #12013)
2024-12-09 13:14:29 +01:00
Eelco Dolstra a5ee9a3e99 Resolve conflicts 2024-12-06 12:13:51 +01:00
Eelco Dolstra 0a2545b23a nix hash: Don't print 'nix hash' deprecation message
Fixes #11997.

(cherry picked from commit 408c2faf93)

# Conflicts:
#	src/nix/hash.cc
2024-12-05 22:30:33 +00:00
Eelco Dolstra 97f0c68389 nix hash convert: Don't fail on uppercase base-16 hashes
(cherry picked from commit 33b645cedf)
2024-12-05 22:30:32 +00:00
Eelco Dolstra 26741bcfda nix hash convert: Support SRI hashes that lack trailing '=' characters
Fixes #11996.

(cherry picked from commit 52f1cd0595)
2024-12-05 22:30:32 +00:00
Eelco Dolstra 218cd6c16c Merge pull request #11981 from h0nIg/patch-2
docker: Fix command "nix profile install", Don't require --impure (fix for 2.25)
2024-11-29 12:53:47 +01:00
h0nIg 61950c3375 docker: Fix command "nix profile install", Don't require --impure 2024-11-27 13:35:18 +01:00
Jörg Thalheim 282bfbdacb Merge pull request #11973 from NixOS/mergify/bp/2.25-maintenance/pr-11959
more readable errors if symlinks cannot be created (backport #11959)
2024-11-27 08:04:06 +01:00
Jörg Thalheim 14432ea4a2 Merge pull request #11962 from NixOS/mergify/bp/2.25-maintenance/pr-11915
Bump fetcher cache version (backport #11915)
2024-11-27 07:23:52 +01:00
Jörg Thalheim 8d51c90872 more readable errors if symlinks cannot be created
Before:

filesystem error: cannot create symlink: Permission denied [/nix/store/1s2p3a4rs172336hj2l8n20nz74hf71j-nix-eval-jobs-2.24.1.drv] [/1s2p3a4rs172336hj2l8n20nz74hf71j-nix-eval-jobs-2.24.1.drv.tmp-2772352-1316231068]

Now:

creating symlink '/wfxz2q489c811n08cdqj7ywxm3n4z6m5-nix-eval-jobs-2.24.1.drv.tmp-2971297-324653080' -> '/nix/store/wfxz2q489c811n08cdqj7ywxm3n4z6m5-nix-eval-jobs-2.24.1.drv': Permission denied

(cherry picked from commit 5b8728d393)

# Conflicts:
#	src/nix/flake.cc
2024-11-27 07:21:10 +01:00
Eelco Dolstra 118d50992c Merge pull request #11968 from NixOS/mergify/bp/2.25-maintenance/pr-11878
Remove warning that interpretation of .drv has changed (backport #11878)
2024-11-26 14:51:50 +01:00
Eelco Dolstra 9f9b2ed48c Merge pull request #11966 from NixOS/mergify/bp/2.25-maintenance/pr-11876
Filter OSC 8 correctly (backport #11876)
2024-11-26 14:10:16 +01:00
Eelco Dolstra eb50683e81 Merge pull request #11965 from NixOS/mergify/bp/2.25-maintenance/pr-11884
nix path-info: Don't write to std::cout directly (backport #11884)
2024-11-26 13:58:42 +01:00
Eelco Dolstra c381199e97 Merge pull request #11964 from NixOS/mergify/bp/2.25-maintenance/pr-11901
tests/functional/flakes/develop.sh: Don't hang (backport #11901)
2024-11-26 13:35:14 +01:00
Eelco Dolstra 072ebeee79 Merge pull request #11963 from NixOS/mergify/bp/2.25-maintenance/pr-11902
Fix issue #11892 (backport #11902)
2024-11-26 13:16:35 +01:00
Robert Hensing 3501c737d3 Remove warning that interpretation of .drv has changed
This was first tagged as 2.15.0, 1½ years ago; plenty of time for
everyone to catch up.

By now, the warning is causing more confusion than that it is helpful,
because passing a `.drv` or `drvPath` has legitimate use cases.

(cherry picked from commit f534a7a524)
2024-11-26 11:27:41 +00:00
Jack Wilsdon 12eff94815 Filter OSC 8 correctly
This allows Nix to use lowdown 1.2.0 which outputs OSC-8 links.

(cherry picked from commit 1301f8434d)
2024-11-26 11:20:35 +00:00
Eelco Dolstra 89f4ac3d2c nix path-info: Don't write to std::cout directly
This interferes with the progress bar, resulting in output like

  evaluating derivation 'git+file:///home/eelco/Dev/nix-master#packages.x86_64-linux.default'/nix/store/zz8v96j5md952x0mxfix12xqnvq5qv5x-nix-2.26.0pre20241114_a95f6ea.drv

(cherry picked from commit 33a0fa882f)
2024-11-26 11:20:35 +00:00
Eelco Dolstra 1dbe60e073 tests/functional/flakes/develop.sh: Don't hang
The bash shell started by `nix develop` waited forever for stdin
input.

Fixes #11827.

(cherry picked from commit 428af8c66f)
2024-11-26 11:19:33 +00:00
Robert Hensing 4e2824ce93 Bump fetcher cache version
We're getting more reports in https://github.com/NixOS/nix/issues/10985

It appears that something hasn't gone right process-wise.
I find this mistake not to be worth investigating, but rather something
to pay attention to going forward.

Let's nip this in the bud.

Closes https://github.com/NixOS/nix/issues/10985

(cherry picked from commit e948c8e033)
2024-11-26 11:19:29 +00:00
Robert Hensing 3bc3e55736 Fix issue 11892
It seems that I copied the expression for baseDir thoughtlessly and
did not come back to it.

- `baseDir` was only used in the `fromArgs` branch.
- `fromArgs` is true when `packages` is true.

(cherry picked from commit c4b95dbdd1)
2024-11-26 11:19:29 +00:00
Eelco Dolstra d97ebe519a Bump version 2024-11-14 11:37:15 +01:00
Eelco Dolstra f87f87120a Bump version 2024-11-13 21:04:28 +01:00
Eelco Dolstra 5ae53d4730 nix-everything: Pass through a version attribute
The existence of this attribute is assumed by the Determinate
Installer packaging and maybe others.

(cherry picked from commit 06769eb2bf)
2024-11-13 14:49:51 +01:00
Eelco Dolstra 7bc52df684 Merge pull request #11861 from DeterminateSystems/nix-2.25-default
[2.25] Make the default package point to the old build
2024-11-12 17:12:12 +01:00
Eelco Dolstra 3e883b9eaf Make the default package point to the old build
There are some differences that are causing issues, like the fact that
nix-ng doesn't have a version attribute.
2024-11-12 16:26:40 +01:00
Eelco Dolstra 6b96c66302 Merge pull request #11856 from nix-windows/fix-windows-env-var-arg-order
[Backport 2.25] Fix argument order in the Windows implementation of `getEnvOs`
2024-11-12 11:04:59 +01:00
Eelco Dolstra 9213bf55ce Bump version 2024-11-11 17:08:24 +01:00
Eelco Dolstra 5c5a737885 Mark stable release 2024-11-11 15:14:57 +01:00
428 changed files with 9654 additions and 6335 deletions

No files matched your search

+85 -102
View File
@@ -7,14 +7,28 @@ on:
permissions: read-all permissions: read-all
jobs: jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v30
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
tests: tests:
needs: [check_secrets]
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ubuntu-latest, macos-latest] include:
runs-on: ${{ matrix.os }} - scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60 timeout-minutes: 60
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@@ -23,103 +37,54 @@ jobs:
- uses: cachix/install-nix-action@v30 - uses: cachix/install-nix-action@v30
with: with:
# The sandbox would otherwise be disabled by default on Darwin # The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true" extra_nix_config: |
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV sandbox = true
- uses: cachix/cachix-action@v15 max-jobs = 1
if: needs.check_secrets.outputs.cachix == 'true' - uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
if: matrix.os == 'linux'
- run: scripts/build-checks
- run: scripts/prepare-installer-for-github-actions
- name: Upload installer tarball
uses: actions/upload-artifact@v4
with: with:
name: '${{ env.CACHIX_NAME }}' name: installer-${{matrix.os}}
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}' path: out/*
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- if: matrix.os == 'ubuntu-latest'
run: |
free -h
swapon --show
swap=$(swapon --show --noheadings | head -n 1 | awk '{print $1}')
echo "Found swap: $swap"
sudo swapoff $swap
# resize it (fallocate)
sudo fallocate -l 10G $swap
sudo mkswap $swap
sudo swapon $swap
free -h
(
while sleep 60; do
free -h
done
) &
- run: nix --experimental-features 'nix-command flakes' flake check -L
- run: nix --experimental-features 'nix-command flakes' flake show --all-systems --json
# Steps to test CI automation in your own fork.
# Cachix:
# 1. Sign-up for https://www.cachix.org/
# 2. Create a cache for $githubuser-nix-install-tests
# 3. Create a cachix auth token and save it in https://github.com/$githubuser/nix/settings/secrets/actions in "Repository secrets" as CACHIX_AUTH_TOKEN
# Dockerhub:
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Cachix and Docker secrets present for installer tests
runs-on: ubuntu-latest
outputs:
cachix: ${{ steps.secret.outputs.cachix }}
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_CACHIX_SECRETS: ${{ secrets.CACHIX_SIGNING_KEY }}${{ secrets.CACHIX_AUTH_TOKEN }}
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=cachix::${{ env._CACHIX_SECRETS != '' }}"
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
installer:
needs: [tests, check_secrets]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
runs-on: ubuntu-latest
outputs:
installerURL: ${{ steps.prepare-installer.outputs.installerURL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@v30
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v15
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
cachixArgs: '-v'
- id: prepare-installer
run: scripts/prepare-installer-for-github-actions
installer_test: installer_test:
needs: [installer, check_secrets] needs: [tests]
if: github.event_name == 'push' && needs.check_secrets.outputs.cachix == 'true'
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ubuntu-latest, macos-latest] include:
runs-on: ${{ matrix.os }} - scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
- scenario: on macos
runs-on: macos-14
os: darwin
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV - name: Download installer tarball
uses: actions/download-artifact@v4
with:
name: installer-${{matrix.os}}
path: out
- name: Serving installer
id: serving_installer
run: ./scripts/serve-installer-for-github-actions
- uses: cachix/install-nix-action@v30 - uses: cachix/install-nix-action@v30
with: with:
install_url: '${{needs.installer.outputs.installerURL}}' install_url: 'http://localhost:8126/install'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve" install_options: "--tarball-url-prefix http://localhost:8126/"
- run: sudo apt install fish zsh - run: sudo apt install fish zsh
if: matrix.os == 'ubuntu-latest' if: matrix.os == 'linux'
- run: brew install fish - run: brew install fish
if: matrix.os == 'macos-latest' if: matrix.os == 'darwin'
- run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval" - run: exec bash -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval" - run: exec sh -c "nix-instantiate -E 'builtins.currentTime' --eval"
- run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval" - run: exec zsh -c "nix-instantiate -E 'builtins.currentTime' --eval"
@@ -127,32 +92,50 @@ jobs:
- run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs" - run: exec bash -c "nix-channel --add https://releases.nixos.org/nixos/unstable/nixos-23.05pre466020.60c1d71f2ba nixpkgs"
- run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello" - run: exec bash -c "nix-channel --update && nix-env -iA nixpkgs.hello && hello"
# Steps to test CI automation in your own fork.
# 1. Sign-up for https://hub.docker.com/
# 2. Store your dockerhub username as DOCKERHUB_USERNAME in "Repository secrets" of your fork repository settings (https://github.com/$githubuser/nix/settings/secrets/actions)
# 3. Create an access token in https://hub.docker.com/settings/security and store it as DOCKERHUB_TOKEN in "Repository secrets" of your fork
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
docker_push_image: docker_push_image:
needs: [check_secrets, tests] needs: [tests, vm_tests, check_secrets]
permissions: permissions:
contents: read contents: read
packages: write packages: write
if: >- if: >-
needs.check_secrets.outputs.docker == 'true' &&
github.event_name == 'push' && github.event_name == 'push' &&
github.ref_name == 'master' && github.ref_name == 'master'
needs.check_secrets.outputs.cachix == 'true' && runs-on: ubuntu-24.04
needs.check_secrets.outputs.docker == 'true'
runs-on: ubuntu-latest
steps: steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
fetch-depth: 0 fetch-depth: 0
- uses: cachix/install-nix-action@v30 - uses: cachix/install-nix-action@v30
with: with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV - uses: DeterminateSystems/magic-nix-cache-action@main
- run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV - run: echo NIX_VERSION="$(nix --experimental-features 'nix-command flakes' eval .\#nix.version | tr -d \")" >> $GITHUB_ENV
- uses: cachix/cachix-action@v15
if: needs.check_secrets.outputs.cachix == 'true'
with:
name: '${{ env.CACHIX_NAME }}'
signingKey: '${{ secrets.CACHIX_SIGNING_KEY }}'
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L - run: nix --experimental-features 'nix-command flakes' build .#dockerImage -L
- run: docker load -i ./result/image.tar.gz - run: docker load -i ./result/image.tar.gz
- run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION - run: docker tag nix:$NIX_VERSION ${{ secrets.DOCKERHUB_USERNAME }}/nix:$NIX_VERSION
@@ -189,7 +172,7 @@ jobs:
docker push $IMAGE_ID:master docker push $IMAGE_ID:master
vm_tests: vm_tests:
runs-on: ubuntu-22.04 runs-on: ubuntu-24.04
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main - uses: DeterminateSystems/nix-installer-action@main
@@ -198,7 +181,7 @@ jobs:
flake_regressions: flake_regressions:
needs: vm_tests needs: vm_tests
runs-on: ubuntu-22.04 runs-on: ubuntu-24.04
steps: steps:
- name: Checkout nix - name: Checkout nix
uses: actions/checkout@v4 uses: actions/checkout@v4
+1 -1
View File
@@ -15,7 +15,7 @@ permissions:
jobs: jobs:
labels: labels:
runs-on: ubuntu-latest runs-on: ubuntu-24.04
if: github.repository_owner == 'NixOS' if: github.repository_owner == 'NixOS'
steps: steps:
- uses: actions/labeler@v5 - uses: actions/labeler@v5
+4 -2
View File
@@ -5,8 +5,10 @@ queue_rules:
- check-success=installer - check-success=installer
- check-success=installer_test (macos-latest) - check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest) - check-success=installer_test (ubuntu-latest)
- check-success=tests (macos-latest) - check-success=tests on macos
- check-success=tests (ubuntu-latest) - check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests - check-success=vm_tests
merge_method: rebase merge_method: rebase
batch_size: 5 batch_size: 5
+1 -1
View File
@@ -1 +1 @@
2.25.0 2.25.6
+9 -10
View File
@@ -1,10 +1,9 @@
(import (import (
( let
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in lock = builtins.fromJSON (builtins.readFile ./flake.lock);
fetchTarball { in
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; fetchTarball {
sha256 = lock.nodes.flake-compat.locked.narHash; url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
} sha256 = lock.nodes.flake-compat.locked.narHash;
) }
{ src = ./.; } ) { src = ./.; }).defaultNix
).defaultNix
+9 -1
View File
@@ -5,7 +5,15 @@ in
builtinsInfo: builtinsInfo:
let let
showBuiltin = name: { doc, type ? null, args ? [ ], experimental-feature ? null, impure-only ? false }: showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
let let
type' = optionalString (type != null) " (${type})"; type' = optionalString (type != null) " (${type})";
+90 -62
View File
@@ -32,7 +32,13 @@ let
commandInfo = fromJSON commandDump; commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }: showCommand =
{
command,
details,
filename,
toplevel,
}:
let let
result = '' result = ''
@@ -56,26 +62,27 @@ let
${maybeOptions} ${maybeOptions}
''; '';
showSynopsis = command: args: showSynopsis =
command: args:
let let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "..."; showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args); arguments = concatStringsSep " " (map showArgument args);
in '' in
''
`${command}` [*option*...] ${arguments} `${command}` [*option*...] ${arguments}
''; '';
maybeSubcommands = optionalString (details ? commands && details.commands != {}) maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
'' where *subcommand* is one of the following:
where *subcommand* is one of the following:
${subcommands} ${subcommands}
''; '';
subcommands = if length categories > 1 subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
then listCategories
else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands))); categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories); listCategories = concatStrings (map showCategory categories);
@@ -99,38 +106,39 @@ let
${allStores} ${allStores}
''; '';
index = replaceStrings index =
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ] replaceStrings
[ storesOverview "#local-store" "#local-daemon-store" ] [ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
details.doc; [ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview = storesOverview =
let let
showEntry = store: showEntry = store: "- [${store.name}](#${store.slug})";
"- [${store.name}](#${store.slug})";
in in
concatStringsSep "\n" (map showEntry storesList) + "\n"; concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages); allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs storePages = listToAttrs (
(map (s: { name = s.filename; value = s.page; }) storesList); map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storesList = showStoreDocs { storesList = showStoreDocs {
storeInfo = commandInfo.stores; storeInfo = commandInfo.stores;
inherit inlineHTML; inherit inlineHTML;
}; };
hasInfix = infix: content: hasInfix =
infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content); builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in in
optionalString (details ? doc) ( optionalString (details ? doc) (
# An alternate implementation with builtins.match stack overflowed on some systems. # An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc if hasInfix "@store-types@" details.doc then help-stores else details.doc
then help-stores
else details.doc
); );
maybeOptions = maybeOptions =
let let
allVisibleOptions = filterAttrs allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
in in
optionalString (allVisibleOptions != { }) '' optionalString (allVisibleOptions != { }) ''
# Options # Options
@@ -142,55 +150,73 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags. > See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
''; '';
showOptions = inlineHTML: allOptions: showOptions =
inlineHTML: allOptions:
let let
showCategory = cat: opts: '' showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"} ${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))} ${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
''; '';
showOption = name: option: showOption =
name: option:
let let
result = trim '' result = trim ''
- ${item} - ${item}
${option.description} ${option.description}
''; '';
item = if inlineHTML item =
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}'' if inlineHTML then
else "`--${name}` ${shortName} ${labels}"; ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
shortName = optionalString else
(option ? shortName) "`--${name}` ${shortName} ${labels}";
("/ `-${option.shortName}`"); shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
(option ? labels) in
(concatStringsSep " " (map (s: "*${s}*") option.labels)); result;
in result; categories =
categories = mapAttrs mapAttrs
# Convert each group from a list of key-value pairs back to an attrset # Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs) (_: listToAttrs)
(groupBy (groupBy (cmd: cmd.value.category) (attrsToList allOptions));
(cmd: cmd.value.category) in
(attrsToList allOptions)); concatStrings (attrValues (mapAttrs showCategory categories));
in concatStrings (attrValues (mapAttrs showCategory categories)); in
in squash result; squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name; appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }: processCommand =
{
command,
details,
filename,
toplevel,
}:
let let
cmd = { cmd = {
inherit command; inherit command;
name = filename + ".md"; name = filename + ".md";
value = showCommand { inherit command details filename toplevel; }; value = showCommand {
inherit
command
details
filename
toplevel
;
};
}; };
subcommand = subCmd: processCommand { subcommand =
command = command + " " + subCmd; subCmd:
details = details.commands.${subCmd}; processCommand {
filename = appendName filename subCmd; command = command + " " + subCmd;
inherit toplevel; details = details.commands.${subCmd};
}; filename = appendName filename subCmd;
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {}); inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand { manpages = processCommand {
command = "nix"; command = "nix";
@@ -199,9 +225,11 @@ let
toplevel = commandInfo.args; toplevel = commandInfo.args;
}; };
tableOfContents = let tableOfContents =
showEntry = page: let
" - [${page.command}](command-ref/new-cli/${page.name})"; showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n"; in
concatStringsSep "\n" (map showEntry manpages) + "\n";
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; } in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+74 -42
View File
@@ -1,67 +1,99 @@
let let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs; inherit (builtins)
inherit (import <nix/utils.nix>) concatStrings indent optionalString squash; attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
in in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown` # `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo: {
prefix,
inlineHTML ? true,
}:
settingsInfo:
let let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }: showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
let let
result = squash '' result = squash ''
- ${item} - ${item}
${indent " " body} ${indent " " body}
''; '';
item = if inlineHTML item =
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>'' if inlineHTML then
else "`${setting}`"; ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
# separate body to cleanly handle indentation # separate body to cleanly handle indentation
body = '' body = ''
${experimentalFeatureNote} ${experimentalFeatureNote}
${description} ${description}
**Default:** ${showDefault documentDefault defaultValue} **Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases} ${showAliases aliases}
''; '';
experimentalFeatureNote = optionalString (experimentalFeature != null) '' experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning** > **Warning**
> >
> This setting is part of an > This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md). > [experimental feature](@docroot@/development/experimental-features.md).
> >
> To change this setting, make sure the > To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature}) > [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled. > is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md): > For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
> >
> ``` > ```
> extra-experimental-features = ${experimentalFeature} > extra-experimental-features = ${experimentalFeature}
> ${setting} = ... > ${setting} = ...
> ``` > ```
''; '';
showDefault = documentDefault: defaultValue: showDefault =
documentDefault: defaultValue:
if documentDefault then if documentDefault then
# a StringMap value type is specified as a string, but # a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in # this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here. # JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
then "*empty*" "*empty*"
else if isBool defaultValue then else if isBool defaultValue then
if defaultValue then "`true`" else "`false`" if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`" else
else "*machine-specific*"; "`${toString defaultValue}`"
else
"*machine-specific*";
showAliases = aliases: showAliases =
optionalString (aliases != []) aliases:
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}"; optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result; in
result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo)) in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+37 -14
View File
@@ -1,6 +1,20 @@
let let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings; inherit (builtins)
inherit (import <nix/utils.nix>) optionalString filterAttrs trim squash toLower unique indent; attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
showSettings = import <nix/generate-settings.nix>; showSettings = import <nix/generate-settings.nix>;
in in
@@ -14,7 +28,13 @@ in
let let
showStore = { name, slug }: { settings, doc, experimentalFeature }: showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
let let
result = squash '' result = squash ''
# ${name} # ${name}
@@ -25,7 +45,10 @@ let
## Settings ## Settings
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings} ${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
''; '';
experimentalFeatureNote = optionalString (experimentalFeature != null) '' experimentalFeatureNote = optionalString (experimentalFeature != null) ''
@@ -43,15 +66,15 @@ let
> extra-experimental-features = ${experimentalFeature} > extra-experimental-features = ${experimentalFeature}
> ``` > ```
''; '';
in result; in
result;
storesList = map storesList = map (name: rec {
(name: rec { inherit name;
inherit name; slug = replaceStrings [ " " ] [ "-" ] (toLower name);
slug = replaceStrings [ " " ] [ "-" ] (toLower name); filename = "${slug}.md";
filename = "${slug}.md"; page = showStore { inherit name slug; } storeInfo.${name};
page = showStore { inherit name slug; } storeInfo.${name}; }) (attrNames storeInfo);
})
(attrNames storeInfo);
in storesList in
storesList
+18 -10
View File
@@ -1,5 +1,11 @@
let let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings; inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
showSettings = import <nix/generate-settings.nix>; showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>; showStoreDocs = import <nix/generate-store-info.nix>;
in in
@@ -14,26 +20,28 @@ let
index = index =
let let
showEntry = store: showEntry = store: "- [${store.name}](./${store.filename})";
"- [${store.name}](./${store.filename})";
in in
concatStringsSep "\n" (map showEntry storesList); concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings "index.md" =
[ "@store-types@" ] [ index ] replaceStrings [ "@store-types@" ] [ index ]
(readFile ./source/store/types/index.md.in); (readFile ./source/store/types/index.md.in);
tableOfContents = tableOfContents =
let let
showEntry = store: showEntry = store: " - [${store.name}](store/types/${store.filename})";
" - [${store.name}](store/types/${store.filename})";
in in
concatStringsSep "\n" (map showEntry storesList) + "\n"; concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents; "SUMMARY.md" = tableOfContents;
storePages = listToAttrs storePages = listToAttrs (
(map (s: { name = s.filename; value = s.page; }) storesList); map (s: {
name = s.filename;
value = s.page;
}) storesList
);
in in
storePages // { inherit "index.md" "SUMMARY.md"; } storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -2,8 +2,8 @@ with builtins;
with import <nix/utils.nix>; with import <nix/utils.nix>;
let let
showExperimentalFeature = name: doc: showExperimentalFeature = name: doc: ''
'' - [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name}) '';
''; in
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps))) xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+2 -1
View File
@@ -2,7 +2,8 @@ with builtins;
with import <nix/utils.nix>; with import <nix/utils.nix>;
let let
showExperimentalFeature = name: doc: showExperimentalFeature =
name: doc:
squash '' squash ''
## [`${name}`]{#xp-feature-${name}} ## [`${name}`]{#xp-feature-${name}}
+32 -28
View File
@@ -1,19 +1,20 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, meson meson,
, ninja ninja,
, lowdown lowdown,
, mdbook mdbook,
, mdbook-linkcheck mdbook-linkcheck,
, jq jq,
, python3 python3,
, rsync rsync,
, nix-cli nix-cli,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -25,18 +26,22 @@ mkMesonDerivation (finalAttrs: {
inherit version; inherit version;
workDir = ./.; workDir = ./.;
fileset = fileset.difference fileset =
(fileset.unions [ fileset.difference
../../.version (fileset.unions [
# Too many different types of files to filter for now ../../.version
../../doc/manual # Too many different types of files to filter for now
./. ../../doc/manual
]) ./.
# Do a blacklist instead ])
../../doc/manual/package.nix; # Do a blacklist instead
../../doc/manual/package.nix;
# TODO the man pages should probably be separate # TODO the man pages should probably be separate
outputs = [ "out" "man" ]; outputs = [
"out"
"man"
];
# Hack for sake of the dev shell # Hack for sake of the dev shell
passthru.externalNativeBuildInputs = [ passthru.externalNativeBuildInputs = [
@@ -54,11 +59,10 @@ mkMesonDerivation (finalAttrs: {
nix-cli nix-cli
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''$out/nix-support mkdir -p ''$out/nix-support
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
@@ -62,6 +62,15 @@ These options are for deleting old [profiles] prior to deleting unreachable [sto
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile. This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument. See the documentation of that command for additional information about the *period* argument.
- <span id="opt-max-freed">[`--max-freed`](#opt-max-freed)</span> *bytes*
<!-- duplication from https://github.com/NixOS/nix/blob/442a2623e48357ff72c77bb11cf2cf06d94d2f90/doc/manual/source/command-ref/nix-store/gc.md?plain=1#L39-L44 -->
Keep deleting paths until at least *bytes* bytes have been deleted,
then stop. The argument *bytes* can be followed by the
multiplicative suffix `K`, `M`, `G` or `T`, denoting KiB, MiB, GiB
or TiB units.
{{#include ./opt-common.md}} {{#include ./opt-common.md}}
{{#include ./env-common.md}} {{#include ./env-common.md}}
@@ -21,6 +21,9 @@ This operation has the following options:
Use recursive instead of flat hashing mode, used when adding Use recursive instead of flat hashing mode, used when adding
directories to the store. directories to the store.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}} {{#include ./opt-common.md}}
{{#include ../opt-common.md}} {{#include ../opt-common.md}}
@@ -11,6 +11,9 @@
The operation `--add` adds the specified paths to the Nix store. It The operation `--add` adds the specified paths to the Nix store. It
prints the resulting paths in the Nix store on standard output. prints the resulting paths in the Nix store on standard output.
*paths* that refer to symlinks are not dereferenced, but added to the store
as symlinks with the same target.
{{#include ./opt-common.md}} {{#include ./opt-common.md}}
{{#include ../opt-common.md}} {{#include ../opt-common.md}}
+1 -1
View File
@@ -297,7 +297,7 @@ Creating a Cachix cache for your installer tests and adding its authorisation to
- `armv7l-linux` - `armv7l-linux`
- `x86_64-darwin` - `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command. - The `installer_test` job (which runs on `ubuntu-24.04` and `macos-14`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup ### One-time setup
+31 -13
View File
@@ -11,10 +11,15 @@ rec {
concatStrings = concatStringsSep ""; concatStrings = concatStringsSep "";
attrsToList = a: attrsToList =
map (name: { inherit name; value = a.${name}; }) (builtins.attrNames a); a:
map (name: {
inherit name;
value = a.${name};
}) (builtins.attrNames a);
replaceStringsRec = from: to: string: replaceStringsRec =
from: to: string:
# recursively replace occurrences of `from` with `to` within `string` # recursively replace occurrences of `from` with `to` within `string`
# example: # example:
# replaceStringRec "--" "-" "hello-----world" # replaceStringRec "--" "-" "hello-----world"
@@ -22,16 +27,18 @@ rec {
let let
replaced = replaceStrings [ from ] [ to ] string; replaced = replaceStrings [ from ] [ to ] string;
in in
if replaced == string then string else replaceStringsRec from to replaced; if replaced == string then string else replaceStringsRec from to replaced;
toLower = replaceStrings upperChars lowerChars; toLower = replaceStrings upperChars lowerChars;
squash = replaceStringsRec "\n\n\n" "\n\n"; squash = replaceStringsRec "\n\n\n" "\n\n";
trim = string: trim =
string:
# trim trailing spaces and squash non-leading spaces # trim trailing spaces and squash non-leading spaces
let let
trimLine = line: trimLine =
line:
let let
# separate leading spaces from the rest # separate leading spaces from the rest
parts = split "(^ *)" line; parts = split "(^ *)" line;
@@ -39,19 +46,30 @@ rec {
rest = elemAt parts 2; rest = elemAt parts 2;
# drop trailing spaces # drop trailing spaces
body = head (split " *$" rest); body = head (split " *$" rest);
in spaces + replaceStringsRec " " " " body; in
in concatStringsSep "\n" (map trimLine (splitLines string)); spaces + replaceStringsRec " " " " body;
in
concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2) # FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) []; unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [ ];
nameValuePair = name: value: { inherit name value; }; nameValuePair = name: value: { inherit name value; };
filterAttrs = pred: set: filterAttrs =
listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set)); pred: set:
listToAttrs (
concatMap (
name:
let
v = set.${name};
in
if pred name v then [ (nameValuePair name v) ] else [ ]
) (attrNames set)
);
optionalString = cond: string: if cond then string else ""; optionalString = cond: string: if cond then string else "";
indent = prefix: s: indent =
concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s)); prefix: s: concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
} }
+200 -181
View File
@@ -1,70 +1,72 @@
{ pkgs ? import <nixpkgs> { } {
, lib ? pkgs.lib pkgs ? import <nixpkgs> { },
, name ? "nix" lib ? pkgs.lib,
, tag ? "latest" name ? "nix",
, bundleNixpkgs ? true tag ? "latest",
, channelName ? "nixpkgs" bundleNixpkgs ? true,
, channelURL ? "https://nixos.org/channels/nixpkgs-unstable" channelName ? "nixpkgs",
, extraPkgs ? [] channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
, maxLayers ? 100 extraPkgs ? [ ],
, nixConf ? {} maxLayers ? 100,
, flake-registry ? null nixConf ? { },
flake-registry ? null,
}: }:
let let
defaultPkgs = with pkgs; [ defaultPkgs =
nix with pkgs;
bashInteractive [
coreutils-full nix
gnutar bashInteractive
gzip coreutils-full
gnugrep gnutar
which gzip
curl gnugrep
less which
wget curl
man less
cacert.out wget
findutils man
iana-etc cacert.out
git findutils
openssh iana-etc
] ++ extraPkgs; git
openssh
]
++ extraPkgs;
users = { users =
{
root = { root = {
uid = 0; uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash"; shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root"; home = "/root";
gid = 0; gid = 0;
groups = [ "root" ]; groups = [ "root" ];
description = "System administrator"; description = "System administrator";
}; };
nobody = { nobody = {
uid = 65534; uid = 65534;
shell = "${pkgs.shadow}/bin/nologin"; shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty"; home = "/var/empty";
gid = 65534; gid = 65534;
groups = [ "nobody" ]; groups = [ "nobody" ];
description = "Unprivileged account (don't use!)"; description = "Unprivileged account (don't use!)";
}; };
} // lib.listToAttrs ( }
map // lib.listToAttrs (
( map (n: {
n: { name = "nixbld${toString n}";
name = "nixbld${toString n}"; value = {
value = { uid = 30000 + n;
uid = 30000 + n; gid = 30000;
gid = 30000; groups = [ "nixbld" ];
groups = [ "nixbld" ]; description = "Nix build user ${toString n}";
description = "Nix build user ${toString n}"; };
}; }) (lib.lists.range 1 32)
} );
)
(lib.lists.range 1 32)
);
groups = { groups = {
root.gid = 0; root.gid = 0;
@@ -74,24 +76,20 @@ let
userToPasswd = ( userToPasswd = (
k: k:
{ uid {
, gid ? 65534 uid,
, home ? "/var/empty" gid ? 65534,
, description ? "" home ? "/var/empty",
, shell ? "/bin/false" description ? "",
, groups ? [ ] shell ? "/bin/false",
}: "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}" groups ? [ ],
); }:
passwdContents = ( "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToPasswd users))
); );
passwdContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToPasswd users)));
userToShadow = k: { ... }: "${k}:!:1::::::"; userToShadow = k: { ... }: "${k}:!:1::::::";
shadowContents = ( shadowContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToShadow users)));
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToShadow users))
);
# Map groups to members # Map groups to members
# { # {
@@ -101,42 +99,35 @@ let
let let
# Create a flat list of user/group mappings # Create a flat list of user/group mappings
mappings = ( mappings = (
builtins.foldl' builtins.foldl' (
( acc: user:
acc: user: let
let groups = users.${user}.groups or [ ];
groups = users.${user}.groups or [ ]; in
in acc
acc ++ map ++ map (group: {
(group: { inherit user group;
inherit user group; }) groups
}) ) [ ] (lib.attrNames users)
groups
)
[ ]
(lib.attrNames users)
); );
in in
( (builtins.foldl' (
builtins.foldl' acc: v:
( acc
acc: v: acc // { // {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ]; ${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
} }
) ) { } mappings)
{ }
mappings)
); );
groupToGroup = k: { gid }: groupToGroup =
k:
{ gid }:
let let
members = groupMemberMap.${k} or [ ]; members = groupMemberMap.${k} or [ ];
in in
"${k}:x:${toString gid}:${lib.concatStringsSep "," members}"; "${k}:x:${toString gid}:${lib.concatStringsSep "," members}";
groupContents = ( groupContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs groupToGroup groups)));
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs groupToGroup groups))
);
defaultNixConf = { defaultNixConf = {
sandbox = "false"; sandbox = "false";
@@ -144,11 +135,17 @@ let
trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ]; trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
}; };
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v: nixConfContents =
let (lib.concatStringsSep "\n" (
vStr = if builtins.isList v then lib.concatStringsSep " " v else v; lib.mapAttrsFlatten (
in n: v:
"${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n"; let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}"
) (defaultNixConf // nixConf)
))
+ "\n";
baseSystem = baseSystem =
let let
@@ -167,21 +164,29 @@ let
manifest = pkgs.buildPackages.runCommand "manifest.nix" { } '' manifest = pkgs.buildPackages.runCommand "manifest.nix" { } ''
cat > $out <<EOF cat > $out <<EOF
[ [
${lib.concatStringsSep "\n" (builtins.map (drv: let ${lib.concatStringsSep "\n" (
outputs = drv.outputsToInstall or [ "out" ]; builtins.map (
in '' drv:
{ let
${lib.concatStringsSep "\n" (builtins.map (output: '' outputs = drv.outputsToInstall or [ "out" ];
${output} = { outPath = "${lib.getOutput output drv}"; }; in
'') outputs)} ''
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ]; {
name = "${drv.name}"; ${lib.concatStringsSep "\n" (
outPath = "${drv}"; builtins.map (output: ''
system = "${drv.system}"; ${output} = { outPath = "${lib.getOutput output drv}"; };
type = "derivation"; '') outputs
meta = { }; )}
} outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
'') defaultPkgs)} name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
''
) defaultPkgs
)}
] ]
EOF EOF
''; '';
@@ -190,16 +195,22 @@ let
cp -a ${rootEnv}/* $out/ cp -a ${rootEnv}/* $out/
ln -s ${manifest} $out/manifest.nix ln -s ${manifest} $out/manifest.nix
''; '';
flake-registry-path = if (flake-registry == null) then flake-registry-path =
null if (flake-registry == null) then
else if (builtins.readFileType (toString flake-registry)) == "directory" then null
"${flake-registry}/flake-registry.json" else if (builtins.readFileType (toString flake-registry)) == "directory" then
else "${flake-registry}/flake-registry.json"
flake-registry; else
flake-registry;
in in
pkgs.runCommand "base-system" pkgs.runCommand "base-system"
{ {
inherit passwdContents groupContents shadowContents nixConfContents; inherit
passwdContents
groupContents
shadowContents
nixConfContents
;
passAsFile = [ passAsFile = [
"passwdContents" "passwdContents"
"groupContents" "groupContents"
@@ -208,62 +219,66 @@ let
]; ];
allowSubstitutes = false; allowSubstitutes = false;
preferLocalBuild = true; preferLocalBuild = true;
} ('' }
env (
set -x ''
mkdir -p $out/etc env
set -x
mkdir -p $out/etc
mkdir -p $out/etc/ssl/certs mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
cat $passwdContentsPath > $out/etc/passwd cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd echo "" >> $out/etc/passwd
cat $groupContentsPath > $out/etc/group cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group echo "" >> $out/etc/group
cat $shadowContentsPath > $out/etc/shadow cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow echo "" >> $out/etc/shadow
mkdir -p $out/usr mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/ ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/nix/var/nix/gcroots mkdir -p $out/nix/var/nix/gcroots
mkdir $out/tmp mkdir $out/tmp
mkdir -p $out/var/tmp mkdir -p $out/var/tmp
mkdir -p $out/etc/nix mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out/root mkdir -p $out/root
mkdir -p $out/nix/var/nix/profiles/per-user/root mkdir -p $out/nix/var/nix/profiles/per-user/root
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s $out/nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out/root/.nix-profile ln -s /nix/var/nix/profiles/default $out/root/.nix-profile
ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link ln -s ${channel} $out/nix/var/nix/profiles/per-user/root/channels-1-link
ln -s $out/nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels ln -s /nix/var/nix/profiles/per-user/root/channels-1-link $out/nix/var/nix/profiles/per-user/root/channels
mkdir -p $out/root/.nix-defexpr mkdir -p $out/root/.nix-defexpr
ln -s $out/nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels ln -s /nix/var/nix/profiles/per-user/root/channels $out/root/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out/root/.nix-channels echo "${channelURL} ${channelName}" > $out/root/.nix-channels
mkdir -p $out/bin $out/usr/bin mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
'' + (lib.optionalString (flake-registry-path != null) '' ''
nixCacheDir="/root/.cache/nix" + (lib.optionalString (flake-registry-path != null) ''
mkdir -p $out$nixCacheDir nixCacheDir="/root/.cache/nix"
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json" mkdir -p $out$nixCacheDir
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
mkdir -p $out/nix/var/nix/gcroots/auto ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath)) mkdir -p $out/nix/var/nix/gcroots/auto
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
'')); ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
'')
);
in in
pkgs.dockerTools.buildLayeredImageWithNixDb { pkgs.dockerTools.buildLayeredImageWithNixDb {
@@ -285,15 +300,19 @@ pkgs.dockerTools.buildLayeredImageWithNixDb {
Cmd = [ "/root/.nix-profile/bin/bash" ]; Cmd = [ "/root/.nix-profile/bin/bash" ];
Env = [ Env = [
"USER=root" "USER=root"
"PATH=${lib.concatStringsSep ":" [ "PATH=${
"/root/.nix-profile/bin" lib.concatStringsSep ":" [
"/nix/var/nix/profiles/default/bin" "/root/.nix-profile/bin"
"/nix/var/nix/profiles/default/sbin" "/nix/var/nix/profiles/default/bin"
]}" "/nix/var/nix/profiles/default/sbin"
"MANPATH=${lib.concatStringsSep ":" [ ]
"/root/.nix-profile/share/man" }"
"/nix/var/nix/profiles/default/share/man" "MANPATH=${
]}" lib.concatStringsSep ":" [
"/root/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]
}"
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
+270 -174
View File
@@ -4,8 +4,14 @@
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05"; inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2"; inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446"; inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; }; inputs.flake-compat = {
inputs.libgit2 = { url = "github:libgit2/libgit2/v1.8.1"; flake = false; }; url = "github:edolstra/flake-compat";
flake = false;
};
inputs.libgit2 = {
url = "github:libgit2/libgit2/v1.8.1";
flake = false;
};
# dev tooling # dev tooling
inputs.flake-parts.url = "github:hercules-ci/flake-parts"; inputs.flake-parts.url = "github:hercules-ci/flake-parts";
@@ -18,18 +24,30 @@
inputs.git-hooks-nix.inputs.flake-compat.follows = ""; inputs.git-hooks-nix.inputs.flake-compat.follows = "";
inputs.git-hooks-nix.inputs.gitignore.follows = ""; inputs.git-hooks-nix.inputs.gitignore.follows = "";
outputs = inputs@{ self, nixpkgs, nixpkgs-regression, libgit2, ... }: outputs =
inputs@{
self,
nixpkgs,
nixpkgs-regression,
libgit2,
...
}:
let let
inherit (nixpkgs) lib; inherit (nixpkgs) lib;
officialRelease = false; officialRelease = true;
linux32BitSystems = [ "i686-linux" ]; linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ]; linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linuxSystems = linux32BitSystems ++ linux64BitSystems; linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ]; darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
systems = linuxSystems ++ darwinSystems; systems = linuxSystems ++ darwinSystems;
crossSystems = [ crossSystems = [
@@ -58,62 +76,76 @@
(Provided that the names are unique.) (Provided that the names are unique.)
See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs
*/ */
flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs; flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs;
forAllSystems = lib.genAttrs systems; forAllSystems = lib.genAttrs systems;
forAllCrossSystems = lib.genAttrs crossSystems; forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs = f: forAllStdenvs =
lib.listToAttrs f:
(map lib.listToAttrs (
(stdenvName: { map (stdenvName: {
name = "${stdenvName}Packages"; name = "${stdenvName}Packages";
value = f stdenvName; value = f stdenvName;
}) }) stdenvs
stdenvs); );
# We don't apply flake-parts to the whole flake so that non-development attributes # We don't apply flake-parts to the whole flake so that non-development attributes
# load without fetching any development inputs. # load without fetching any development inputs.
devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } { devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } {
imports = [ ./maintainers/flake-module.nix ]; imports = [ ./maintainers/flake-module.nix ];
systems = lib.subtractLists crossSystems systems; systems = lib.subtractLists crossSystems systems;
perSystem = { system, ... }: { perSystem =
_module.args.pkgs = nixpkgsFor.${system}.native; { system, ... }:
}; {
_module.args.pkgs = nixpkgsFor.${system}.native;
};
}; };
# Memoize nixpkgs for different platforms for efficiency. # Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems nixpkgsFor = forAllSystems (
(system: let system:
make-pkgs = crossSystem: stdenv: import nixpkgs { let
localSystem = { make-pkgs =
inherit system; crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem =
if crossSystem == null then
null
else
{
config = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
}; };
crossSystem = if crossSystem == null then null else {
config = crossSystem;
} // lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
};
stdenvs = forAllStdenvs (make-pkgs null); stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages; native = stdenvs.stdenvPackages;
in { in
{
inherit stdenvs native; inherit stdenvs native;
static = native.pkgsStatic; static = native.pkgsStatic;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv"); cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv");
}); }
);
binaryTarball = nix: pkgs: pkgs.callPackage ./scripts/binary-tarball.nix { binaryTarball =
inherit nix; nix: pkgs:
}; pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
overlayFor = getStdenv: final: prev: overlayFor =
getStdenv: final: prev:
let let
stdenv = getStdenv final; stdenv = getStdenv final;
in in
@@ -124,31 +156,42 @@
# without "polluting" the top level "`pkgs`" attrset. # without "polluting" the top level "`pkgs`" attrset.
# This also has the benefit of providing us with a distinct set of packages # This also has the benefit of providing us with a distinct set of packages
# we can iterate over. # we can iterate over.
nixComponents = lib.makeScope final.nixDependencies.newScope (import ./packaging/components.nix { nixComponents = lib.makeScope final.nixDependencies.newScope (
inherit (final) lib; import ./packaging/components.nix {
inherit officialRelease; inherit (final) lib;
src = self; inherit officialRelease;
}); src = self;
}
);
# The dependencies are in their own scope, so that they don't have to be # The dependencies are in their own scope, so that they don't have to be
# in Nixpkgs top level `pkgs` or `nixComponents`. # in Nixpkgs top level `pkgs` or `nixComponents`.
nixDependencies = lib.makeScope final.newScope (import ./packaging/dependencies.nix { nixDependencies = lib.makeScope final.newScope (
inherit inputs stdenv; import ./packaging/dependencies.nix {
pkgs = final; inherit inputs stdenv;
}); pkgs = final;
}
);
nix = final.nixComponents.nix; nix = final.nixComponents.nix;
# See https://github.com/NixOS/nixpkgs/pull/214409 # See https://github.com/NixOS/nixpkgs/pull/214409
# Remove when fixed in this flake's nixpkgs # Remove when fixed in this flake's nixpkgs
pre-commit = pre-commit =
if prev.stdenv.hostPlatform.system == "i686-linux" if prev.stdenv.hostPlatform.system == "i686-linux" then
then (prev.pre-commit.override (o: { dotnet-sdk = ""; })).overridePythonAttrs (o: { doCheck = false; }) (prev.pre-commit.override (o: {
else prev.pre-commit; dotnet-sdk = "";
})).overridePythonAttrs
(o: {
doCheck = false;
})
else
prev.pre-commit;
}; };
in { in
{
# A Nixpkgs overlay that overrides the 'nix' and # A Nixpkgs overlay that overrides the 'nix' and
# 'nix-perl-bindings' packages. # 'nix-perl-bindings' packages.
overlays.default = overlayFor (p: p.stdenv); overlays.default = overlayFor (p: p.stdenv);
@@ -167,151 +210,204 @@
; ;
}; };
checks = forAllSystems (system: { checks = forAllSystems (
binaryTarball = self.hydraJobs.binaryTarball.${system}; system:
installTests = self.hydraJobs.installTests.${system}; {
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system}; installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
rl-next = installTests = self.hydraJobs.installTests.${system};
let pkgs = nixpkgsFor.${system}.native; nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
in pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } '' rl-next =
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out let
''; pkgs = nixpkgsFor.${system}.native;
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { }; in
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) { pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
dockerImage = self.hydraJobs.dockerImage.${system}; LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
} // (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) { '';
# Some perl dependencies are broken on i686-linux. repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
# Since the support is only best-effort there, disable the perl }
# bindings // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
}
// (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
# Temporarily disabled because GitHub Actions OOM issues. Once # Temporarily disabled because GitHub Actions OOM issues. Once
# the old build system is gone and we are back to one build # the old build system is gone and we are back to one build
# system, we should reenable this. # system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system}; #perlBindings = self.hydraJobs.perlBindings.${system};
} }
/* /*
# Add "passthru" tests # Add "passthru" tests
// flatMapAttrs ({ // flatMapAttrs ({
"" = nixpkgsFor.${system}.native; "" = nixpkgsFor.${system}.native;
} // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) { } // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on: # TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448 # https://github.com/NixOS/nixpkgs/issues/320448
# TODO: disabled to speed up GHA CI. # TODO: disabled to speed up GHA CI.
#"static-" = nixpkgsFor.${system}.static; #"static-" = nixpkgsFor.${system}.static;
}) })
(nixpkgsPrefix: nixpkgs: (nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents flatMapAttrs nixpkgs.nixComponents
(pkgName: pkg: (pkgName: pkg:
flatMapAttrs pkg.tests or {} flatMapAttrs pkg.tests or {}
(testName: test: { (testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test; "${nixpkgsPrefix}${pkgName}-${testName}" = test;
}) })
)
// lib.optionalAttrs (nixpkgs.stdenv.hostPlatform == nixpkgs.stdenv.buildPlatform) {
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests;
}
) )
// lib.optionalAttrs (nixpkgs.stdenv.hostPlatform == nixpkgs.stdenv.buildPlatform) { */
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests; // devFlake.checks.${system} or { }
}
)
*/
// devFlake.checks.${system} or {}
); );
packages = forAllSystems (system: packages = forAllSystems (
{ # Here we put attributes that map 1:1 into packages.<system>, ie system:
{
# Here we put attributes that map 1:1 into packages.<system>, ie
# for which we don't apply the full build matrix such as cross or static. # for which we don't apply the full build matrix such as cross or static.
inherit (nixpkgsFor.${system}.native) inherit (nixpkgsFor.${system}.native)
changelog-d; changelog-d
default = self.packages.${system}.nix-ng; ;
default = self.packages.${system}.nix;
binaryTarball = self.hydraJobs.binaryTarball.${system};
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
nix-manual = nixpkgsFor.${system}.native.nixComponents.nix-manual; nix-manual = nixpkgsFor.${system}.native.nixComponents.nix-manual;
nix-internal-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-internal-api-docs; nix-internal-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-internal-api-docs;
nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs; nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs;
} }
# We need to flatten recursive attribute sets of derivations to pass `flake check`. # We need to flatten recursive attribute sets of derivations to pass `flake check`.
// flatMapAttrs //
{ # Components we'll iterate over in the upcoming lambda flatMapAttrs
"nix" = { }; {
"nix-util" = { }; # Components we'll iterate over in the upcoming lambda
"nix-util-c" = { }; "nix" = { };
"nix-util-test-support" = { }; "nix-util" = { };
"nix-util-tests" = { }; "nix-util-c" = { };
"nix-util-test-support" = { };
"nix-util-tests" = { };
"nix-store" = { }; "nix-store" = { };
"nix-store-c" = { }; "nix-store-c" = { };
"nix-store-test-support" = { }; "nix-store-test-support" = { };
"nix-store-tests" = { }; "nix-store-tests" = { };
"nix-fetchers" = { }; "nix-fetchers" = { };
"nix-fetchers-tests" = { }; "nix-fetchers-tests" = { };
"nix-expr" = { }; "nix-expr" = { };
"nix-expr-c" = { }; "nix-expr-c" = { };
"nix-expr-test-support" = { }; "nix-expr-test-support" = { };
"nix-expr-tests" = { }; "nix-expr-tests" = { };
"nix-flake" = { }; "nix-flake" = { };
"nix-flake-tests" = { }; "nix-flake-tests" = { };
"nix-main" = { }; "nix-main" = { };
"nix-main-c" = { }; "nix-main-c" = { };
"nix-cmd" = { }; "nix-cmd" = { };
"nix-cli" = { }; "nix-cli" = { };
"nix-functional-tests" = { supportsCross = false; }; "nix-functional-tests" = {
supportsCross = false;
};
"nix-perl-bindings" = { supportsCross = false; }; "nix-perl-bindings" = {
"nix-ng" = { }; supportsCross = false;
} };
(pkgName: { supportsCross ? true }: { "nix-ng" = { };
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
} }
// lib.optionalAttrs supportsCross (flatMapAttrs (lib.genAttrs crossSystems (_: { })) (crossSystem: {}: { (
# These attributes go right into `packages.<system>`. pkgName:
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}; {
})) supportsCross ? true,
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (stdenvName: {}: { }:
# These attributes go right into `packages.<system>`. {
"${pkgName}-${stdenvName}" = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName}; # These attributes go right into `packages.<system>`.
}) "${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
) "${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
}
// lib.optionalAttrs supportsCross (
flatMapAttrs (lib.genAttrs crossSystems (_: { })) (
crossSystem:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
}
)
)
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (
stdenvName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" =
nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
}
)
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) { // lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage = dockerImage =
let let
pkgs = nixpkgsFor.${system}.native; pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix { inherit pkgs; tag = pkgs.nix.version; }; image = import ./docker.nix {
in inherit pkgs;
pkgs.runCommand tag = pkgs.nix.version;
"docker-image-tarball-${pkgs.nix.version}" };
{ meta.description = "Docker image with Nix for ${system}"; } in
'' pkgs.runCommand "docker-image-tarball-${pkgs.nix.version}"
mkdir -p $out/nix-support { meta.description = "Docker image with Nix for ${system}"; }
image=$out/image.tar.gz ''
ln -s ${image} $image mkdir -p $out/nix-support
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products image=$out/image.tar.gz
''; ln -s ${image} $image
}); echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
}
);
devShells = let devShells =
makeShell = import ./packaging/dev-shell.nix { inherit lib devFlake; }; let
prefixAttrs = prefix: lib.concatMapAttrs (k: v: { "${prefix}-${k}" = v; }); makeShell = import ./packaging/dev-shell.nix { inherit lib devFlake; };
in prefixAttrs = prefix: lib.concatMapAttrs (k: v: { "${prefix}-${k}" = v; });
forAllSystems (system: in
prefixAttrs "native" (forAllStdenvs (stdenvName: makeShell { forAllSystems (
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages"; system:
})) // prefixAttrs "native" (
lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) ( forAllStdenvs (
prefixAttrs "static" (forAllStdenvs (stdenvName: makeShell { stdenvName:
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsStatic; makeShell {
})) // pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages";
prefixAttrs "cross" (forAllCrossSystems (crossSystem: makeShell { }
pkgs = nixpkgsFor.${system}.cross.${crossSystem}; )
})) )
) // // lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) (
{ prefixAttrs "static" (
forAllStdenvs (
stdenvName:
makeShell {
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsStatic;
}
)
)
// prefixAttrs "cross" (
forAllCrossSystems (
crossSystem:
makeShell {
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
}
)
)
)
// {
default = self.devShells.${system}.native-stdenvPackages; default = self.devShells.${system}.native-stdenvPackages;
} }
); );
}; };
} }
File diff suppressed because it is too large. Load diff
+346 -293
View File
@@ -1,116 +1,117 @@
{ lib {
, stdenv lib,
, releaseTools stdenv,
, autoconf-archive releaseTools,
, autoreconfHook autoconf-archive,
, aws-sdk-cpp autoreconfHook,
, boehmgc aws-sdk-cpp,
, nlohmann_json boehmgc,
, bison nlohmann_json,
, boost bison,
, brotli boost,
, bzip2 brotli,
, curl bzip2,
, editline curl,
, readline editline,
, flex readline,
, git flex,
, gtest git,
, jq gtest,
, libarchive jq,
, libcpuid libarchive,
, libgit2 libcpuid,
, libseccomp libgit2,
, libsodium libseccomp,
, man libsodium,
, darwin man,
, lowdown darwin,
, mdbook lowdown,
, mdbook-linkcheck mdbook,
, mercurial mdbook-linkcheck,
, openssh mercurial,
, openssl openssh,
, pkg-config openssl,
, rapidcheck pkg-config,
, sqlite rapidcheck,
, toml11 sqlite,
, unixtools toml11,
, xz unixtools,
xz,
, busybox-sandbox-shell ? null busybox-sandbox-shell ? null,
# Configuration Options # Configuration Options
#: #:
# This probably seems like too many degrees of freedom, but it # This probably seems like too many degrees of freedom, but it
# faithfully reflects how the underlying configure + make build system # faithfully reflects how the underlying configure + make build system
# work. The top-level flake.nix will choose useful combinations of these # work. The top-level flake.nix will choose useful combinations of these
# options to CI. # options to CI.
, pname ? "nix" pname ? "nix",
, version version,
, versionSuffix versionSuffix,
# Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix # Whether to build Nix. Useful to skip for tasks like testing existing pre-built versions of Nix
, doBuild ? true doBuild ? true,
# Run the unit tests as part of the build. See `installUnitTests` for an # Run the unit tests as part of the build. See `installUnitTests` for an
# alternative to this. # alternative to this.
, doCheck ? __forDefaults.canRunInstalled doCheck ? __forDefaults.canRunInstalled,
# Run the functional tests as part of the build. # Run the functional tests as part of the build.
, doInstallCheck ? test-client != null || __forDefaults.canRunInstalled doInstallCheck ? test-client != null || __forDefaults.canRunInstalled,
# Check test coverage of Nix. Probably want to use with at least # Check test coverage of Nix. Probably want to use with at least
# one of `doCHeck` or `doInstallCheck` enabled. # one of `doCHeck` or `doInstallCheck` enabled.
, withCoverageChecks ? false withCoverageChecks ? false,
# Whether to build the regular manual # Whether to build the regular manual
, enableManual ? __forDefaults.canRunInstalled enableManual ? __forDefaults.canRunInstalled,
# Whether to use garbage collection for the Nix language evaluator. # Whether to use garbage collection for the Nix language evaluator.
# #
# If it is disabled, we just leak memory, but this is not as bad as it # If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived # sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is # processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.) # only leaked *within* the process.)
# #
# Temporarily disabled on Windows because the `GC_throw_bad_alloc` # Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking. # symbol is missing during linking.
# #
# Disabled on OpenBSD because of missing `_data_start` symbol while linking # Disabled on OpenBSD because of missing `_data_start` symbol while linking
, enableGC ? !stdenv.hostPlatform.isWindows && !stdenv.hostPlatform.isOpenBSD enableGC ? !stdenv.hostPlatform.isWindows && !stdenv.hostPlatform.isOpenBSD,
# Whether to enable Markdown rendering in the Nix binary. # Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Which interactive line editor library to use for Nix's repl. # Which interactive line editor library to use for Nix's repl.
# #
# Currently supported choices are: # Currently supported choices are:
# #
# - editline (default) # - editline (default)
# - readline # - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline" readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
# Whether to install unit tests. This is useful when cross compiling # Whether to install unit tests. This is useful when cross compiling
# since we cannot run them natively during the build, but can do so # since we cannot run them natively during the build, but can do so
# later. # later.
, installUnitTests ? doBuild && !__forDefaults.canExecuteHost installUnitTests ? doBuild && !__forDefaults.canExecuteHost,
# For running the functional tests against a pre-built Nix. Probably # For running the functional tests against a pre-built Nix. Probably
# want to use in conjunction with `doBuild = false;`. # want to use in conjunction with `doBuild = false;`.
, test-daemon ? null test-daemon ? null,
, test-client ? null test-client ? null,
# Avoid setting things that would interfere with a functioning devShell # Avoid setting things that would interfere with a functioning devShell
, forDevShell ? false forDevShell ? false,
# Not a real argument, just the only way to approximate let-binding some # Not a real argument, just the only way to approximate let-binding some
# stuff for argument defaults. # stuff for argument defaults.
, __forDefaults ? { __forDefaults ? {
canExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform; canExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
canRunInstalled = doBuild && __forDefaults.canExecuteHost; canRunInstalled = doBuild && __forDefaults.canExecuteHost;
} },
}: }:
let let
@@ -124,243 +125,295 @@ let
}; };
mkDerivation = mkDerivation =
if withCoverageChecks if withCoverageChecks then
then
# TODO support `finalAttrs` args function in # TODO support `finalAttrs` args function in
# `releaseTools.coverageAnalysis`. # `releaseTools.coverageAnalysis`.
argsFun: argsFun:
releaseTools.coverageAnalysis (let args = argsFun args; in args) releaseTools.coverageAnalysis (
else stdenv.mkDerivation; let
args = argsFun args;
in
args
)
else
stdenv.mkDerivation;
in in
mkDerivation (finalAttrs: let mkDerivation (
finalAttrs:
let
inherit (finalAttrs) inherit (finalAttrs)
doCheck doCheck
doInstallCheck doInstallCheck
; ;
doBuild = !finalAttrs.dontBuild; doBuild = !finalAttrs.dontBuild;
# Either running the unit tests during the build, or installing them # Either running the unit tests during the build, or installing them
# to be run later, requiresthe unit tests to be built. # to be run later, requiresthe unit tests to be built.
buildUnitTests = doCheck || installUnitTests; buildUnitTests = doCheck || installUnitTests;
in { in
inherit pname version; {
inherit pname version;
src = src =
let let
baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.; baseFiles = fileset.fileFilter (f: f.name != ".gitignore") ./.;
in in
fileset.toSource { fileset.toSource {
root = ./.; root = ./.;
fileset = fileset.intersection baseFiles (fileset.unions ([ fileset = fileset.intersection baseFiles (
# For configure fileset.unions (
./.version [
./configure.ac # For configure
./m4 ./.version
# TODO: do we really need README.md? It doesn't seem used in the build. ./configure.ac
./README.md ./m4
# This could be put behind a conditional # TODO: do we really need README.md? It doesn't seem used in the build.
./maintainers/local.mk ./README.md
# For make, regardless of what we are building # This could be put behind a conditional
./local.mk ./maintainers/local.mk
./Makefile # For make, regardless of what we are building
./Makefile.config.in ./local.mk
./mk ./Makefile
(fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts) ./Makefile.config.in
] ++ lib.optionals doBuild [ ./mk
./doc (fileset.fileFilter (f: lib.strings.hasPrefix "nix-profile" f.name) ./scripts)
./misc ]
./precompiled-headers.h ++ lib.optionals doBuild [
(fileset.difference ./src ./src/perl) ./doc
./COPYING ./misc
./scripts/local.mk ./precompiled-headers.h
] ++ lib.optionals enableManual [ (fileset.difference ./src ./src/perl)
./doc/manual ./COPYING
] ++ lib.optionals doInstallCheck [ ./scripts/local.mk
./tests/functional ]
])); ++ lib.optionals enableManual [
./doc/manual
]
++ lib.optionals doInstallCheck [
./tests/functional
]
)
);
}; };
VERSION_SUFFIX = versionSuffix; VERSION_SUFFIX = versionSuffix;
outputs = [ "out" ] outputs =
++ lib.optional doBuild "dev" [ "out" ]
# If we are doing just build or just docs, the one thing will use ++ lib.optional doBuild "dev"
# "out". We only need additional outputs if we are doing both. # If we are doing just build or just docs, the one thing will use
++ lib.optional (doBuild && enableManual) "doc" # "out". We only need additional outputs if we are doing both.
++ lib.optional installUnitTests "check" ++ lib.optional (doBuild && enableManual) "doc"
++ lib.optional doCheck "testresults" ++ lib.optional installUnitTests "check"
; ++ lib.optional doCheck "testresults";
nativeBuildInputs = [ nativeBuildInputs =
autoconf-archive [
autoreconfHook autoconf-archive
pkg-config autoreconfHook
] ++ lib.optionals doBuild [ pkg-config
bison ]
flex ++ lib.optionals doBuild [
] ++ lib.optionals enableManual [ bison
(lib.getBin lowdown) flex
mdbook ]
mdbook-linkcheck ++ lib.optionals enableManual [
] ++ lib.optionals doInstallCheck [ (lib.getBin lowdown)
git mdbook
mercurial mdbook-linkcheck
openssh ]
] ++ lib.optionals (doInstallCheck || enableManual) [ ++ lib.optionals doInstallCheck [
jq # Also for custom mdBook preprocessor. git
] ++ lib.optionals enableManual [ mercurial
man openssh
] ++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump ]
; ++ lib.optionals (doInstallCheck || enableManual) [
jq # Also for custom mdBook preprocessor.
]
++ lib.optionals enableManual [
man
]
++ lib.optional stdenv.hostPlatform.isStatic unixtools.hexdump;
buildInputs = lib.optionals doBuild ( buildInputs = lib.optionals doBuild (
[ [
brotli brotli
bzip2 bzip2
curl curl
libarchive libarchive
libgit2 libgit2
libsodium libsodium
openssl openssl
sqlite sqlite
toml11 toml11
xz xz
({ inherit readline editline; }.${readlineFlavor}) ({ inherit readline editline; }.${readlineFlavor})
] ++ lib.optionals enableMarkdown [ ]
lowdown ++ lib.optionals enableMarkdown [
] ++ lib.optionals buildUnitTests [ lowdown
gtest ]
rapidcheck ++ lib.optionals buildUnitTests [
] ++ lib.optional stdenv.isLinux libseccomp gtest
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox rapidcheck
++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid ]
# There have been issues building these dependencies ++ lib.optional stdenv.isLinux libseccomp
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)) ++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
aws-sdk-cpp ++ lib.optional stdenv.hostPlatform.isx86_64 libcpuid
); # There have been issues building these dependencies
++ lib.optional (
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)
) aws-sdk-cpp
);
propagatedBuildInputs = lib.optionals doBuild ([ propagatedBuildInputs = lib.optionals doBuild (
boost [
nlohmann_json boost
] ++ lib.optional enableGC boehmgc nlohmann_json
); ]
++ lib.optional enableGC boehmgc
);
dontBuild = !attrs.doBuild; dontBuild = !attrs.doBuild;
doCheck = attrs.doCheck; doCheck = attrs.doCheck;
configureFlags = [ configureFlags =
(lib.enableFeature doBuild "build") [
(lib.enableFeature buildUnitTests "unit-tests") (lib.enableFeature doBuild "build")
(lib.enableFeature doInstallCheck "functional-tests") (lib.enableFeature buildUnitTests "unit-tests")
(lib.enableFeature enableManual "doc-gen") (lib.enableFeature doInstallCheck "functional-tests")
(lib.enableFeature enableGC "gc") (lib.enableFeature enableManual "doc-gen")
(lib.enableFeature enableMarkdown "markdown") (lib.enableFeature enableGC "gc")
(lib.enableFeature installUnitTests "install-unit-tests") (lib.enableFeature enableMarkdown "markdown")
(lib.withFeatureAs true "readline-flavor" readlineFlavor) (lib.enableFeature installUnitTests "install-unit-tests")
] ++ lib.optionals (!forDevShell) [ (lib.withFeatureAs true "readline-flavor" readlineFlavor)
"--sysconfdir=/etc" ]
] ++ lib.optionals installUnitTests [ ++ lib.optionals (!forDevShell) [
"--with-check-bin-dir=${builtins.placeholder "check"}/bin" "--sysconfdir=/etc"
"--with-check-lib-dir=${builtins.placeholder "check"}/lib" ]
] ++ lib.optionals (doBuild) [ ++ lib.optionals installUnitTests [
"--with-boost=${boost}/lib" "--with-check-bin-dir=${builtins.placeholder "check"}/bin"
] ++ lib.optionals (doBuild && stdenv.isLinux) [ "--with-check-lib-dir=${builtins.placeholder "check"}/lib"
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox" ]
] ++ lib.optional (doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) ++ lib.optionals (doBuild) [
"LDFLAGS=-fuse-ld=gold" "--with-boost=${boost}/lib"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell" ]
; ++ lib.optionals (doBuild && stdenv.isLinux) [
"--with-sandbox-shell=${busybox-sandbox-shell}/bin/busybox"
]
++ lib.optional (
doBuild && stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")
) "LDFLAGS=-fuse-ld=gold"
++ lib.optional (doBuild && stdenv.hostPlatform.isStatic) "--enable-embedded-sandbox-shell";
enableParallelBuilding = true; enableParallelBuilding = true;
makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1"; makeFlags = "profiledir=$(out)/etc/profile.d PRECOMPILE_HEADERS=1";
preCheck = '' preCheck = ''
mkdir $testresults mkdir $testresults
''; '';
installTargets = lib.optional doBuild "install"; installTargets = lib.optional doBuild "install";
installFlags = "sysconfdir=$(out)/etc"; installFlags = "sysconfdir=$(out)/etc";
# In this case we are probably just running tests, and so there isn't # In this case we are probably just running tests, and so there isn't
# anything to install, we just make an empty directory to signify tests # anything to install, we just make an empty directory to signify tests
# succeeded. # succeeded.
installPhase = if finalAttrs.installTargets != [] then null else '' installPhase =
mkdir -p $out if finalAttrs.installTargets != [ ] then
''; null
else
''
mkdir -p $out
'';
postInstall = lib.optionalString doBuild ( postInstall =
lib.optionalString stdenv.hostPlatform.isStatic '' lib.optionalString doBuild (
mkdir -p $out/nix-support lib.optionalString stdenv.hostPlatform.isStatic ''
echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products mkdir -p $out/nix-support
'' echo "file binary-dist $out/bin/nix" >> $out/nix-support/hydra-build-products
) + lib.optionalString enableManual '' ''
mkdir -p ''${!outputDoc}/nix-support )
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products + lib.optionalString enableManual ''
''; mkdir -p ''${!outputDoc}/nix-support
echo "doc manual ''${!outputDoc}/share/doc/nix/manual" >> ''${!outputDoc}/nix-support/hydra-build-products
'';
# So the check output gets links for DLLs in the out output. # So the check output gets links for DLLs in the out output.
preFixup = lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs) '' preFixup =
ln -s "$check/lib/"*.dll "$check/bin" lib.optionalString (stdenv.hostPlatform.isWindows && builtins.elem "check" finalAttrs.outputs)
ln -s "$out/bin/"*.dll "$check/bin" ''
''; ln -s "$check/lib/"*.dll "$check/bin"
ln -s "$out/bin/"*.dll "$check/bin"
'';
doInstallCheck = attrs.doInstallCheck; doInstallCheck = attrs.doInstallCheck;
installCheckFlags = "sysconfdir=$(out)/etc"; installCheckFlags = "sysconfdir=$(out)/etc";
# Work around buggy detection in stdenv. # Work around buggy detection in stdenv.
installCheckTarget = "installcheck"; installCheckTarget = "installcheck";
# Work around weird bug where it doesn't think there is a Makefile. # Work around weird bug where it doesn't think there is a Makefile.
installCheckPhase = if (!doBuild && doInstallCheck) then '' installCheckPhase =
runHook preInstallCheck if (!doBuild && doInstallCheck) then
mkdir -p src/nix-channel ''
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES runHook preInstallCheck
'' else null; mkdir -p src/nix-channel
make installcheck -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES
''
else
null;
# Needed for tests if we are not doing a build, but testing existing # Needed for tests if we are not doing a build, but testing existing
# built Nix. # built Nix.
preInstallCheck = preInstallCheck = lib.optionalString (!doBuild) ''
lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel mkdir -p src/nix-channel
''; '';
separateDebugInfo = !stdenv.hostPlatform.isStatic; separateDebugInfo = !stdenv.hostPlatform.isStatic;
# TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564 # TODO Always true after https://github.com/NixOS/nixpkgs/issues/318564
strictDeps = !withCoverageChecks; strictDeps = !withCoverageChecks;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie"; hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix"; mainProgram = "nix";
broken = !(lib.all (a: a) [ broken =
# We cannot run or install unit tests if we don't build them or !(lib.all (a: a) [
# Nix proper (which they depend on). # We cannot run or install unit tests if we don't build them or
(installUnitTests -> doBuild) # Nix proper (which they depend on).
(doCheck -> doBuild) (installUnitTests -> doBuild)
# The build process for the manual currently requires extracting (doCheck -> doBuild)
# data from the Nix executable we are trying to document. # The build process for the manual currently requires extracting
(enableManual -> doBuild) # data from the Nix executable we are trying to document.
]); (enableManual -> doBuild)
}; ]);
};
} // lib.optionalAttrs withCoverageChecks { }
lcovFilter = [ "*/boost/*" "*-tab.*" ]; // lib.optionalAttrs withCoverageChecks {
lcovFilter = [
"*/boost/*"
"*-tab.*"
];
hardeningDisable = ["fortify"]; hardeningDisable = [ "fortify" ];
NIX_CFLAGS_COMPILE = "-DCOVERAGE=1"; NIX_CFLAGS_COMPILE = "-DCOVERAGE=1";
dontInstall = false; dontInstall = false;
} // lib.optionalAttrs (test-daemon != null) { }
NIX_DAEMON_PACKAGE = test-daemon; // lib.optionalAttrs (test-daemon != null) {
} // lib.optionalAttrs (test-client != null) { NIX_DAEMON_PACKAGE = test-daemon;
NIX_CLIENT_PACKAGE = test-client; }
}) // lib.optionalAttrs (test-client != null) {
NIX_CLIENT_PACKAGE = test-client;
}
)
+8 -4
View File
@@ -13,9 +13,11 @@ let
versionSuffix = lib.optionalString (!officialRelease) "pre"; versionSuffix = lib.optionalString (!officialRelease) "pre";
fineVersionSuffix = lib.optionalString fineVersionSuffix =
(!officialRelease) lib.optionalString (!officialRelease)
"pre${builtins.substring 0 8 (src.lastModifiedDate or src.lastModified or "19700101")}_${src.shortRev or "dirty"}"; "pre${
builtins.substring 0 8 (src.lastModifiedDate or src.lastModified or "19700101")
}_${src.shortRev or "dirty"}";
fineVersion = baseVersion + fineVersionSuffix; fineVersion = baseVersion + fineVersionSuffix;
in in
@@ -58,7 +60,9 @@ in
nix-cli = callPackage ../src/nix/package.nix { version = fineVersion; }; nix-cli = callPackage ../src/nix/package.nix { version = fineVersion; };
nix-functional-tests = callPackage ../src/nix-functional-tests/package.nix { version = fineVersion; }; nix-functional-tests = callPackage ../tests/functional/package.nix {
version = fineVersion;
};
nix-manual = callPackage ../doc/manual/package.nix { version = fineVersion; }; nix-manual = callPackage ../doc/manual/package.nix { version = fineVersion; };
nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { version = fineVersion; }; nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { version = fineVersion; };
+145 -129
View File
@@ -19,9 +19,7 @@ let
root = ../.; root = ../.;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64 stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64 then darwinStdenv else prevStdenv;
then darwinStdenv
else prevStdenv;
# Fix the following error with the default x86_64-darwin SDK: # Fix the following error with the default x86_64-darwin SDK:
# #
@@ -38,11 +36,14 @@ let
# Indirection for Nixpkgs to override when package.nix files are vendored # Indirection for Nixpkgs to override when package.nix files are vendored
filesetToSource = lib.fileset.toSource; filesetToSource = lib.fileset.toSource;
/** Given a set of layers, create a mkDerivation-like function */ /**
mkPackageBuilder = exts: userFn: Given a set of layers, create a mkDerivation-like function
stdenv.mkDerivation (lib.extends (lib.composeManyExtensions exts) userFn); */
mkPackageBuilder =
exts: userFn: stdenv.mkDerivation (lib.extends (lib.composeManyExtensions exts) userFn);
localSourceLayer = finalAttrs: prevAttrs: localSourceLayer =
finalAttrs: prevAttrs:
let let
workDirPath = workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has # Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
@@ -51,8 +52,13 @@ let
prevAttrs.workDir; prevAttrs.workDir;
workDirSubpath = lib.path.removePrefix root workDirPath; workDirSubpath = lib.path.removePrefix root workDirPath;
sources = assert prevAttrs.fileset._type == "fileset"; prevAttrs.fileset; sources =
src = lib.fileset.toSource { fileset = sources; inherit root; }; assert prevAttrs.fileset._type == "fileset";
prevAttrs.fileset;
src = lib.fileset.toSource {
fileset = sources;
inherit root;
};
in in
{ {
@@ -64,52 +70,54 @@ let
workDir = null; workDir = null;
}; };
mesonLayer = finalAttrs: prevAttrs: mesonLayer = finalAttrs: prevAttrs: {
{ nativeBuildInputs = [
nativeBuildInputs = [ pkgs.buildPackages.meson
pkgs.buildPackages.meson pkgs.buildPackages.ninja
pkgs.buildPackages.ninja ] ++ prevAttrs.nativeBuildInputs or [ ];
] ++ prevAttrs.nativeBuildInputs or []; };
};
mesonBuildLayer = finalAttrs: prevAttrs: mesonBuildLayer = finalAttrs: prevAttrs: {
{ nativeBuildInputs = prevAttrs.nativeBuildInputs or [ ] ++ [
nativeBuildInputs = prevAttrs.nativeBuildInputs or [] ++ [ pkgs.buildPackages.pkg-config
pkgs.buildPackages.pkg-config ];
]; separateDebugInfo = !stdenv.hostPlatform.isStatic;
separateDebugInfo = !stdenv.hostPlatform.isStatic; hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie"; };
};
mesonLibraryLayer = finalAttrs: prevAttrs: mesonLibraryLayer = finalAttrs: prevAttrs: {
{ outputs = prevAttrs.outputs or [ "out" ] ++ [ "dev" ];
outputs = prevAttrs.outputs or [ "out" ] ++ [ "dev" ]; };
};
# Work around weird `--as-needed` linker behavior with BSD, see # Work around weird `--as-needed` linker behavior with BSD, see
# https://github.com/mesonbuild/meson/issues/3593 # https://github.com/mesonbuild/meson/issues/3593
bsdNoLinkAsNeeded = finalAttrs: prevAttrs: bsdNoLinkAsNeeded =
finalAttrs: prevAttrs:
lib.optionalAttrs stdenv.hostPlatform.isBSD { lib.optionalAttrs stdenv.hostPlatform.isBSD {
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or []; mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
}; };
miscGoodPractice = finalAttrs: prevAttrs: miscGoodPractice = finalAttrs: prevAttrs: {
{ strictDeps = prevAttrs.strictDeps or true;
strictDeps = prevAttrs.strictDeps or true; enableParallelBuilding = true;
enableParallelBuilding = true; };
};
in in
scope: { scope: {
inherit stdenv; inherit stdenv;
aws-sdk-cpp = (pkgs.aws-sdk-cpp.override { aws-sdk-cpp =
apis = [ "s3" "transfer" ]; (pkgs.aws-sdk-cpp.override {
customMemoryManagement = false; apis = [
}).overrideAttrs { "s3"
# only a stripped down version is built, which takes a lot less resources "transfer"
# to build, so we don't need a "big-parallel" machine. ];
requiredSystemFeatures = [ ]; customMemoryManagement = false;
}; }).overrideAttrs
{
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
libseccomp = pkgs.libseccomp.overrideAttrs (_: rec { libseccomp = pkgs.libseccomp.overrideAttrs (_: rec {
version = "2.5.5"; version = "2.5.5";
@@ -124,71 +132,44 @@ scope: {
}; };
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed. # TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost = (pkgs.boost.override { boost =
extraB2Args = [ (pkgs.boost.override {
"--with-container" extraB2Args = [
"--with-context" "--with-container"
"--with-coroutine" "--with-context"
]; "--with-coroutine"
}).overrideAttrs (old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 = pkgs.libgit2.overrideAttrs (attrs: {
src = inputs.libgit2;
version = inputs.libgit2.lastModifiedDate;
cmakeFlags = attrs.cmakeFlags or []
++ [ "-DUSE_SSH=exec" ];
nativeBuildInputs = attrs.nativeBuildInputs or []
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
]; ];
# Only `git apply` can handle git binary patches }).overrideAttrs
prePatch = attrs.prePatch or "" (old: {
+ lib.optionalString (!stdenv.hostPlatform.isWindows) '' # Need to remove `--with-*` to use `--with-libraries=...`
patch() { buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
git apply installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
} });
busybox-sandbox-shell =
pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
''; '';
patches = attrs.patches or [] });
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
useMusl = true;
enableStatic = true;
enableMinimal = true;
extraConfig = ''
CONFIG_FEATURE_FANCY_ECHO y
CONFIG_FEATURE_SH_MATH y
CONFIG_FEATURE_SH_MATH_64 y
CONFIG_ASH y
CONFIG_ASH_OPTIMIZE_FOR_SIZE y
CONFIG_ASH_ALIAS y
CONFIG_ASH_BASH_COMPAT y
CONFIG_ASH_CMDCMD y
CONFIG_ASH_ECHO y
CONFIG_ASH_GETOPTS y
CONFIG_ASH_INTERNAL_GLOB y
CONFIG_ASH_JOB_CONTROL y
CONFIG_ASH_PRINTF y
CONFIG_ASH_TEST y
'';
});
# TODO change in Nixpkgs, Windows works fine. First commit of # TODO change in Nixpkgs, Windows works fine. First commit of
# https://github.com/NixOS/nixpkgs/pull/322977 backported will fix. # https://github.com/NixOS/nixpkgs/pull/322977 backported will fix.
@@ -198,27 +179,62 @@ scope: {
inherit resolvePath filesetToSource; inherit resolvePath filesetToSource;
mkMesonDerivation = mkMesonDerivation = mkPackageBuilder [
mkPackageBuilder [ miscGoodPractice
miscGoodPractice localSourceLayer
localSourceLayer mesonLayer
mesonLayer ];
]; mkMesonExecutable = mkPackageBuilder [
mkMesonExecutable = miscGoodPractice
mkPackageBuilder [ bsdNoLinkAsNeeded
miscGoodPractice localSourceLayer
bsdNoLinkAsNeeded mesonLayer
localSourceLayer mesonBuildLayer
mesonLayer ];
mesonBuildLayer mkMesonLibrary = mkPackageBuilder [
]; miscGoodPractice
mkMesonLibrary = bsdNoLinkAsNeeded
mkPackageBuilder [ localSourceLayer
miscGoodPractice mesonLayer
bsdNoLinkAsNeeded mesonBuildLayer
localSourceLayer mesonLibraryLayer
mesonLayer ];
mesonBuildLayer
mesonLibraryLayer libgit2 = pkgs.libgit2.overrideAttrs (
]; attrs:
{
cmakeFlags = attrs.cmakeFlags or [ ] ++ [ "-DUSE_SSH=exec" ];
}
# libgit2: Nixpkgs 24.11 has < 1.9.0, which needs our patches
// lib.optionalAttrs (!lib.versionAtLeast pkgs.libgit2.version "1.9.0") {
src = inputs.libgit2;
version = inputs.libgit2.lastModifiedDate;
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch =
attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
'';
patches =
attrs.patches or [ ]
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
}
);
} }
+90 -82
View File
@@ -2,95 +2,103 @@
{ pkgs }: { pkgs }:
(pkgs.nix.override { forDevShell = true; }).overrideAttrs (attrs: (pkgs.nix.override { forDevShell = true; }).overrideAttrs (
attrs:
let let
stdenv = pkgs.nixDependencies.stdenv; stdenv = pkgs.nixDependencies.stdenv;
buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform; buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
modular = devFlake.getSystem stdenv.buildPlatform.system; modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag = prefix: flag: transformFlag =
assert builtins.isString flag; prefix: flag:
let assert builtins.isString flag;
rest = builtins.substring 2 (builtins.stringLength flag) flag; let
in rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}"; "-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix; havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags; ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in { in
pname = "shell-for-" + attrs.pname; {
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop # Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ../.version; version = lib.fileContents ../.version;
name = attrs.pname; name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc"; installFlags = "sysconfdir=$(out)/etc";
shellHook = '' shellHook = ''
PATH=$prefix/bin:$PATH PATH=$prefix/bin:$PATH
unset PYTHONPATH unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH export MANPATH=$out/share/man:$MANPATH
# Make bash completion work. # Make bash completion work.
XDG_DATA_DIRS+=:$out/share XDG_DATA_DIRS+=:$out/share
''; '';
# We use this shell with the local checkout, not unpackPhase. # We use this shell with the local checkout, not unpackPhase.
src = null; src = null;
env = { env = {
# Needed for Meson to find Boost. # Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131. # https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include"; BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib"; BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit # For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = _NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
"${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml" modular.pre-commit.settings.rawConfig}"; modular.pre-commit.settings.rawConfig
}; }";
};
mesonFlags = mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags) map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags) ++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags) ++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)) ++ lib.optionals havePerl (
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags) map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags) )
; ++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags);
nativeBuildInputs = attrs.nativeBuildInputs or [] nativeBuildInputs =
++ pkgs.nixComponents.nix-util.nativeBuildInputs attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents.nix-store.nativeBuildInputs ++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs ++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-expr.nativeBuildInputs ++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs ++ pkgs.nixComponents.nix-expr.nativeBuildInputs
++ lib.optionals buildCanExecuteHost pkgs.nixComponents.nix-manual.externalNativeBuildInputs ++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs ++ lib.optionals buildCanExecuteHost pkgs.nixComponents.nix-manual.externalNativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs ++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-functional-tests.externalNativeBuildInputs ++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ lib.optional ++ pkgs.nixComponents.nix-functional-tests.externalNativeBuildInputs
(!buildCanExecuteHost ++ lib.optional (
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479 !buildCanExecuteHost
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin) # Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages && !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)) && stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
pkgs.buildPackages.mesonEmulatorHook && lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
++ [ ) pkgs.buildPackages.mesonEmulatorHook
pkgs.buildPackages.cmake ++ [
pkgs.buildPackages.shellcheck pkgs.buildPackages.cmake
pkgs.buildPackages.changelog-d pkgs.buildPackages.shellcheck
modular.pre-commit.settings.package pkgs.buildPackages.changelog-d
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.package
modular.pre-commit.settings.installationScript) (pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
] ]
# TODO: Remove the darwin check once # TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available # https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear ++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (lib.hiPrio pkgs.buildPackages.clang-tools); ++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
);
buildInputs = attrs.buildInputs or [] buildInputs =
++ pkgs.nixComponents.nix-util.buildInputs attrs.buildInputs or [ ]
++ pkgs.nixComponents.nix-store.buildInputs ++ pkgs.nixComponents.nix-util.buildInputs
++ pkgs.nixComponents.nix-fetchers.buildInputs ++ pkgs.nixComponents.nix-store.buildInputs
++ pkgs.nixComponents.nix-expr.buildInputs ++ pkgs.nixComponents.nix-fetchers.buildInputs
++ pkgs.nixComponents.nix-store-tests.externalBuildInputs ++ pkgs.nixComponents.nix-expr.buildInputs
++ lib.optional havePerl pkgs.perl ++ pkgs.nixComponents.nix-store-tests.externalBuildInputs
; ++ lib.optional havePerl pkgs.perl;
}) }
)
+78 -71
View File
@@ -42,89 +42,96 @@
(buildEnv { (buildEnv {
name = "nix-${nix-cli.version}"; name = "nix-${nix-cli.version}";
paths = [ paths =
nix-util [
nix-util-c nix-util
nix-util-test-support nix-util-c
nix-util-tests nix-util-test-support
nix-util-tests
nix-store nix-store
nix-store-c nix-store-c
nix-store-test-support nix-store-test-support
nix-store-tests nix-store-tests
nix-fetchers nix-fetchers
nix-fetchers-tests nix-fetchers-tests
nix-expr nix-expr
nix-expr-c nix-expr-c
nix-expr-test-support nix-expr-test-support
nix-expr-tests nix-expr-tests
nix-flake nix-flake
nix-flake-tests nix-flake-tests
nix-main nix-main
nix-main-c nix-main-c
nix-cmd nix-cmd
nix-cli nix-cli
nix-manual nix-manual
nix-internal-api-docs nix-internal-api-docs
nix-external-api-docs nix-external-api-docs
] ++ lib.optionals (stdenv.buildPlatform.canExecute stdenv.hostPlatform) [ ]
nix-perl-bindings ++ lib.optionals (stdenv.buildPlatform.canExecute stdenv.hostPlatform) [
]; nix-perl-bindings
];
meta.mainProgram = "nix"; meta.mainProgram = "nix";
}).overrideAttrs (finalAttrs: prevAttrs: { }).overrideAttrs
doCheck = true; (
doInstallCheck = true; finalAttrs: prevAttrs: {
doCheck = true;
doInstallCheck = true;
checkInputs = [ checkInputs = [
# Actually run the unit tests too # Actually run the unit tests too
nix-util-tests.tests.run nix-util-tests.tests.run
nix-store-tests.tests.run nix-store-tests.tests.run
nix-expr-tests.tests.run nix-expr-tests.tests.run
nix-flake-tests.tests.run nix-flake-tests.tests.run
]; ];
installCheckInputs = [ installCheckInputs = [
nix-functional-tests nix-functional-tests
]; ];
passthru = prevAttrs.passthru // { passthru = prevAttrs.passthru // {
/** inherit (nix-cli) version;
These are the libraries that are part of the Nix project. They are used
by the Nix CLI and other tools.
If you need to use these libraries in your project, we recommend to use /**
the `-c` C API libraries exclusively, if possible. These are the libraries that are part of the Nix project. They are used
by the Nix CLI and other tools.
We also recommend that you build the complete package to ensure that the unit tests pass. If you need to use these libraries in your project, we recommend to use
You could do this in CI, or by passing it in an unused environment variable. e.g in a `mkDerivation` call: the `-c` C API libraries exclusively, if possible.
```nix We also recommend that you build the complete package to ensure that the unit tests pass.
buildInputs = [ nix.libs.nix-util-c nix.libs.nix-store-c ]; You could do this in CI, or by passing it in an unused environment variable. e.g in a `mkDerivation` call:
# Make sure the nix libs we use are ok
unusedInputsForTests = [ nix ]; ```nix
disallowedReferences = nix.all; buildInputs = [ nix.libs.nix-util-c nix.libs.nix-store-c ];
``` # Make sure the nix libs we use are ok
*/ unusedInputsForTests = [ nix ];
libs = { disallowedReferences = nix.all;
inherit ```
nix-util */
nix-util-c libs = {
nix-store inherit
nix-store-c nix-util
nix-fetchers nix-util-c
nix-expr nix-store
nix-expr-c nix-store-c
nix-flake nix-fetchers
nix-main nix-expr
nix-main-c nix-expr-c
; nix-flake
}; nix-main
}; nix-main-c
}) ;
};
};
}
)
+130 -90
View File
@@ -1,29 +1,32 @@
{ inputs {
, binaryTarball inputs,
, forAllCrossSystems binaryTarball,
, forAllSystems forAllCrossSystems,
, lib forAllSystems,
, linux64BitSystems lib,
, nixpkgsFor linux64BitSystems,
, self nixpkgsFor,
, officialRelease self,
officialRelease,
}: }:
let let
inherit (inputs) nixpkgs nixpkgs-regression; inherit (inputs) nixpkgs nixpkgs-regression;
installScriptFor = tarballs: installScriptFor =
tarballs:
nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix { nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix {
inherit tarballs; inherit tarballs;
}; };
testNixVersions = pkgs: daemon: testNixVersions =
pkgs: daemon:
pkgs.nixComponents.nix-functional-tests.override { pkgs.nixComponents.nix-functional-tests.override {
pname = pname =
"nix-tests" "nix-tests"
+ lib.optionalString + lib.optionalString (
(lib.versionAtLeast daemon.version "2.4pre20211005" && lib.versionAtLeast daemon.version "2.4pre20211005"
lib.versionAtLeast pkgs.nix.version "2.4pre20211005") && lib.versionAtLeast pkgs.nix.version "2.4pre20211005"
"-${pkgs.nix.version}-against-${daemon.version}"; ) "-${pkgs.nix.version}-against-${daemon.version}";
test-daemon = daemon; test-daemon = daemon;
}; };
@@ -59,44 +62,70 @@ let
in in
{ {
# Binary package for various platforms. # Binary package for various platforms.
build = forAllPackages (pkgName: build = forAllPackages (
forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})); pkgName: forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})
);
shellInputs = removeAttrs shellInputs = removeAttrs (forAllSystems (
(forAllSystems (system: self.devShells.${system}.default.inputDerivation)) system: self.devShells.${system}.default.inputDerivation
[ "i686-linux" ]; )) [ "i686-linux" ];
buildStatic = forAllPackages (pkgName: buildStatic = forAllPackages (
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})); pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})
);
buildCross = forAllPackages (pkgName: buildCross = forAllPackages (
pkgName:
# Hack to avoid non-evaling package # Hack to avoid non-evaling package
(if pkgName == "nix-functional-tests" then lib.flip builtins.removeAttrs ["x86_64-w64-mingw32"] else lib.id) (
(forAllCrossSystems (crossSystem: if pkgName == "nix-functional-tests" then
lib.genAttrs [ "x86_64-linux" ] (system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName})))); lib.flip builtins.removeAttrs [ "x86_64-w64-mingw32" ]
else
lib.id
)
(
forAllCrossSystems (
crossSystem:
lib.genAttrs [ "x86_64-linux" ] (
system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}
)
)
)
);
buildNoGc = let buildNoGc =
components = forAllSystems (system: let
nixpkgsFor.${system}.native.nixComponents.overrideScope (self: super: { components = forAllSystems (
nix-expr = super.nix-expr.override { enableGC = false; }; system:
}) nixpkgsFor.${system}.native.nixComponents.overrideScope (
); self: super: {
in forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName})); nix-expr = super.nix-expr.override { enableGC = false; };
}
)
);
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-cli); buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-cli);
# Toggles some settings for better coverage. Windows needs these # Toggles some settings for better coverage. Windows needs these
# library combinations, and Debian build Nix with GNU readline too. # library combinations, and Debian build Nix with GNU readline too.
buildReadlineNoMarkdown = let buildReadlineNoMarkdown =
components = forAllSystems (system: let
nixpkgsFor.${system}.native.nixComponents.overrideScope (self: super: { components = forAllSystems (
nix-cmd = super.nix-cmd.override { system:
enableMarkdown = false; nixpkgsFor.${system}.native.nixComponents.overrideScope (
readlineFlavor = "readline"; self: super: {
}; nix-cmd = super.nix-cmd.override {
}) enableMarkdown = false;
); readlineFlavor = "readline";
in forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName})); };
}
)
);
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
# Perl bindings for various platforms. # Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-perl-bindings); perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-perl-bindings);
@@ -104,13 +133,18 @@ in
# Binary tarball for various platforms, containing a Nix store # Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of # with the closure of 'nix' package, and the second half of
# the installation script. # the installation script.
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native); binaryTarball = forAllSystems (
system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native
);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (system: binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (
forAllCrossSystems (crossSystem: system:
binaryTarball forAllCrossSystems (
nixpkgsFor.${system}.cross.${crossSystem}.nix crossSystem:
nixpkgsFor.${system}.cross.${crossSystem})); binaryTarball nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}
)
);
# The first half of the installation script. This is uploaded # The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary # to https://nixos.org/nix/install. It downloads the binary
@@ -128,15 +162,13 @@ in
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf" self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf"
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu" self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
]; ];
installerScriptForGHA = installScriptFor [
# Native installerScriptForGHA = forAllSystems (
self.hydraJobs.binaryTarball."x86_64-linux" system:
self.hydraJobs.binaryTarball."aarch64-darwin" nixpkgsFor.${system}.native.callPackage ../scripts/installer.nix {
# Cross tarballs = [ self.hydraJobs.binaryTarball.${system} ];
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv6l-unknown-linux-gnueabihf" }
self.hydraJobs.binaryTarballCross."x86_64-linux"."armv7l-unknown-linux-gnueabihf" );
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
];
# docker image with Nix inside # docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage); dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
@@ -157,16 +189,24 @@ in
external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs; external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs;
# System tests. # System tests.
tests = import ../tests/nixos { inherit lib nixpkgs nixpkgsFor self; } // { tests =
import ../tests/nixos {
inherit
lib
nixpkgs
nixpkgsFor
self
;
}
// {
# Make sure that nix-env still produces the exact same result # Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs. # on a particular version of Nixpkgs.
evalNixpkgs = evalNixpkgs =
let let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix; inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in in
runCommand "eval-nixos" { buildInputs = [ nix ]; } runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
''
type -p nix-env type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593. # Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
( (
@@ -177,36 +217,36 @@ in
mkdir $out mkdir $out
''; '';
nixpkgsLibTests = nixpkgsLibTests = forAllSystems (
forAllSystems (system: system:
import (nixpkgs + "/lib/tests/test-with-nix.nix") import (nixpkgs + "/lib/tests/test-with-nix.nix") {
{ lib = nixpkgsFor.${system}.native.lib;
lib = nixpkgsFor.${system}.native.lib; nix = self.packages.${system}.nix;
nix = self.packages.${system}.nix; pkgs = nixpkgsFor.${system}.native;
pkgs = nixpkgsFor.${system}.native; }
}
); );
}; };
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" { metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native; pkgs = nixpkgsFor.x86_64-linux.native;
nixpkgs = nixpkgs-regression; nixpkgs = nixpkgs-regression;
}; };
installTests = forAllSystems (system: installTests = forAllSystems (
let pkgs = nixpkgsFor.${system}.native; in system:
pkgs.runCommand "install-tests" let
{ pkgs = nixpkgsFor.${system}.native;
againstSelf = testNixVersions pkgs pkgs.nix; in
againstCurrentLatest = pkgs.runCommand "install-tests" {
# FIXME: temporarily disable this on macOS because of #3605. againstSelf = testNixVersions pkgs pkgs.nix;
if system == "x86_64-linux" againstCurrentLatest =
then testNixVersions pkgs pkgs.nixVersions.latest # FIXME: temporarily disable this on macOS because of #3605.
else null; if system == "x86_64-linux" then testNixVersions pkgs pkgs.nixVersions.latest else null;
# Disabled because the latest stable version doesn't handle # Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work # `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nixStable; # againstLatestStable = testNixVersions pkgs pkgs.nixStable;
} "touch $out"); } "touch $out"
);
installerTests = import ../tests/installer { installerTests = import ../tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball; binaryTarballs = self.hydraJobs.binaryTarball;
+11 -7
View File
@@ -1,14 +1,18 @@
{ runCommand {
, system runCommand,
, buildPackages system,
, cacert buildPackages,
, nix cacert,
nix,
}: }:
let let
installerClosureInfo = buildPackages.closureInfo { installerClosureInfo = buildPackages.closureInfo {
rootPaths = [ nix cacert ]; rootPaths = [
nix
cacert
];
}; };
inherit (nix) version; inherit (nix) version;
@@ -65,7 +69,7 @@ runCommand "nix-binary-tarball-${version}" env ''
fn=$out/$dir.tar.xz fn=$out/$dir.tar.xz
mkdir -p $out/nix-support mkdir -p $out/nix-support
echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products echo "file binary-dist $fn" >> $out/nix-support/hydra-build-products
tar cvfJ $fn \ tar cfJ $fn \
--owner=0 --group=0 --mode=u+rw,uga+r \ --owner=0 --group=0 --mode=u+rw,uga+r \
--mtime='1970-01-01' \ --mtime='1970-01-01' \
--absolute-names \ --absolute-names \
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
system=$(nix eval --raw --impure --expr builtins.currentSystem)
nix eval --json ".#checks.$system" --apply builtins.attrNames | \
jq -r '.[]' | \
xargs -P0 -I '{}' sh -c "nix build -L .#checks.$system.{} || { echo 'FAILED: \033[0;31mnix build -L .#checks.$system.{}\\033[0m'; kill 0; }"
+1 -1
View File
@@ -463,7 +463,7 @@ EOF
EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF
:a :a
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,suid,owners UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,nosuid,noatime,owners
. .
:x :x
EOF EOF
+1 -1
View File
@@ -557,7 +557,7 @@ create_build_user_for_core() {
if [ "$actual_uid" != "$uid" ]; then if [ "$actual_uid" != "$uid" ]; then
failure <<EOF failure <<EOF
It seems the build user $username already exists, but with the UID It seems the build user $username already exists, but with the UID
with the UID '$actual_uid'. This script can't really handle that right '$actual_uid'. This script can't really handle that right
now, so I'm going to give up. now, so I'm going to give up.
If you already created the users and you know they start from If you already created the users and you know they start from
+37 -31
View File
@@ -1,36 +1,42 @@
{ lib {
, runCommand lib,
, nix runCommand,
, tarballs nix,
tarballs,
}: }:
runCommand "installer-script" { runCommand "installer-script"
buildInputs = [ nix ]; {
} '' buildInputs = [ nix ];
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
} }
''
mkdir -p $out/nix-support
substitute ${./install.in} $out/install \ # Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
${lib.concatMapStrings tarballPath() {
(tarball: let # Remove the store prefix
inherit (tarball.stdenv.hostPlatform) system; local path=''${1#${builtins.storeDir}/}
in '' \ # Get the path relative to the derivation root
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \ local rest=''${path#*/}
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \ # Get the derivation hash
'' local drvHash=''${path%%-*}
) echo "$drvHash/$rest"
tarballs }
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products substitute ${./install.in} $out/install \
'' ${
lib.concatMapStrings (
tarball:
let
inherit (tarball.stdenv.hostPlatform) system;
in
''
\
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
) tarballs
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
+1 -1
View File
@@ -1,7 +1,7 @@
# Only execute this file once per shell. # Only execute this file once per shell.
# This file is tested by tests/installer/default.nix. # This file is tested by tests/installer/default.nix.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1 export __ETC_PROFILE_NIX_SOURCED=1
NIX_LINK=$HOME/.nix-profile NIX_LINK=$HOME/.nix-profile
if [ -n "${XDG_STATE_HOME-}" ]; then if [ -n "${XDG_STATE_HOME-}" ]; then
+7 -6
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -e set -euo pipefail
script=$(nix-build -A outputs.hydraJobs.installerScriptForGHA --no-out-link) nix build -L ".#installerScriptForGHA" ".#binaryTarball"
installerHash=$(echo "$script" | cut -b12-43 -)
installerURL=https://$CACHIX_NAME.cachix.org/serve/$installerHash/install mkdir -p out
cp ./result/install "out/install"
echo "::set-output name=installerURL::$installerURL" name="$(basename "$(realpath ./result-1)")"
# everything before the first dash
cp -r ./result-1 "out/${name%%-*}"
+3
View File
@@ -2,6 +2,9 @@
set -eo pipefail set -eo pipefail
# stock path to avoid unexpected command versions
PATH="$(/usr/bin/getconf PATH)"
((NEW_NIX_FIRST_BUILD_UID=351)) ((NEW_NIX_FIRST_BUILD_UID=351))
((TEMP_NIX_FIRST_BUILD_UID=31000)) ((TEMP_NIX_FIRST_BUILD_UID=31000))
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
set -euo pipefail
if [[ ! -d out ]]; then
echo "run prepare-installer-for-github-actions first"
exit 1
fi
cd out
PORT=${PORT:-8126}
nohup python -m http.server "$PORT" >/dev/null 2>&1 &
pid=$!
while ! curl -s "http://localhost:$PORT"; do
sleep 1
if ! kill -0 $pid; then
echo "Failed to start http server"
exit 1
fi
done
echo 'To install nix, run the following command:'
echo "sh <(curl http://localhost:$PORT/install) --tarball-url-prefix http://localhost:$PORT"
+10 -10
View File
@@ -1,11 +1,12 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, doxygen doxygen,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -38,11 +39,10 @@ mkMesonDerivation (finalAttrs: {
doxygen doxygen
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''${!outputDoc}/nix-support mkdir -p ''${!outputDoc}/nix-support
+23 -21
View File
@@ -1,11 +1,12 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, doxygen doxygen,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -17,27 +18,28 @@ mkMesonDerivation (finalAttrs: {
inherit version; inherit version;
workDir = ./.; workDir = ./.;
fileset = let fileset =
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh"); let
in fileset.unions [ cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
./.version in
../../.version fileset.unions [
./meson.build ./.version
./doxygen.cfg.in ../../.version
# Source is not compiled, but still must be available for Doxygen ./meson.build
# to gather comments. ./doxygen.cfg.in
(cpp ../.) # Source is not compiled, but still must be available for Doxygen
]; # to gather comments.
(cpp ../.)
];
nativeBuildInputs = [ nativeBuildInputs = [
doxygen doxygen
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''${!outputDoc}/nix-support mkdir -p ''${!outputDoc}/nix-support
-10
View File
@@ -32,16 +32,6 @@ InstallableDerivedPath InstallableDerivedPath::parse(
// store path. // store path.
[&](const ExtendedOutputsSpec::Default &) -> DerivedPath { [&](const ExtendedOutputsSpec::Default &) -> DerivedPath {
auto storePath = store->followLinksToStorePath(prefix); auto storePath = store->followLinksToStorePath(prefix);
// Remove this prior to stabilizing the new CLI.
if (storePath.isDerivation()) {
auto oldDerivedPath = DerivedPath::Built {
.drvPath = makeConstantStorePathRef(storePath),
.outputs = OutputsSpec::All { },
};
warn(
"The interpretation of store paths arguments ending in `.drv` recently changed. If this command is now failing try again with '%s'",
oldDerivedPath.to_string(*store));
};
return DerivedPath::Opaque { return DerivedPath::Opaque {
.path = std::move(storePath), .path = std::move(storePath),
}; };
+16 -4
View File
@@ -16,13 +16,25 @@ static std::string doRenderMarkdownToTerminal(std::string_view markdown)
{ {
int windowWidth = getWindowSize().second; int windowWidth = getWindowSize().second;
struct lowdown_opts opts #if HAVE_LOWDOWN_1_4
{ struct lowdown_opts_term opts_term {
.type = LOWDOWN_TERM,
.maxdepth = 20,
.cols = (size_t) std::max(windowWidth - 5, 60), .cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0, .hmargin = 0,
.vmargin = 0, .vmargin = 0,
};
#endif
struct lowdown_opts opts
{
.type = LOWDOWN_TERM,
#if HAVE_LOWDOWN_1_4
.term = opts_term,
#endif
.maxdepth = 20,
#if !HAVE_LOWDOWN_1_4
.cols = (size_t) std::max(windowWidth - 5, 60),
.hmargin = 0,
.vmargin = 0,
#endif
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES, .feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = LOWDOWN_TERM_NOLINK, .oflags = LOWDOWN_TERM_NOLINK,
}; };
+2
View File
@@ -38,6 +38,8 @@ deps_public += nlohmann_json
lowdown = dependency('lowdown', version : '>= 0.9.0', required : get_option('markdown')) lowdown = dependency('lowdown', version : '>= 0.9.0', required : get_option('markdown'))
deps_private += lowdown deps_private += lowdown
configdata.set('HAVE_LOWDOWN', lowdown.found().to_int()) configdata.set('HAVE_LOWDOWN', lowdown.found().to_int())
# The API changed slightly around terminal initialization.
configdata.set('HAVE_LOWDOWN_1_4', lowdown.version().version_compare('>= 1.4.0').to_int())
readline_flavor = get_option('readline-flavor') readline_flavor = get_option('readline-flavor')
if readline_flavor == 'editline' if readline_flavor == 'editline'
+31 -27
View File
@@ -1,32 +1,33 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, nix-expr nix-expr,
, nix-flake nix-flake,
, nix-main nix-main,
, editline editline,
, readline readline,
, lowdown lowdown,
, nlohmann_json nlohmann_json,
# Configuration Options # Configuration Options
, version version,
# Whether to enable Markdown rendering in the Nix binary. # Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Which interactive line editor library to use for Nix's repl. # Which interactive line editor library to use for Nix's repl.
# #
# Currently supported choices are: # Currently supported choices are:
# #
# - editline (default) # - editline (default)
# - readline # - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline" readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
}: }:
let let
@@ -76,9 +77,12 @@ mkMesonLibrary (finalAttrs: {
(lib.mesonOption "readline-flavor" readlineFlavor) (lib.mesonOption "readline-flavor" readlineFlavor)
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+4 -7
View File
@@ -138,16 +138,13 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
out << ANSI_RED "error: " << ANSI_NORMAL; out << ANSI_RED "error: " << ANSI_NORMAL;
out << dt.hint.str() << "\n"; out << dt.hint.str() << "\n";
// prefer direct pos, but if noPos then try the expr. auto pos = dt.getPos(positions);
auto pos = dt.pos
? dt.pos
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
if (pos) { if (pos) {
out << *pos; out << pos;
if (auto loc = pos->getCodeLines()) { if (auto loc = pos.getCodeLines()) {
out << "\n"; out << "\n";
printCodeLines(out, "", *pos, *loc); printCodeLines(out, "", pos, *loc);
out << "\n"; out << "\n";
} }
} }
+1 -5
View File
@@ -613,12 +613,8 @@ nix_realised_string * nix_string_realise(nix_c_context * context, EvalState * st
context->last_err_code = NIX_OK; context->last_err_code = NIX_OK;
try { try {
auto & v = check_value_in(value); auto & v = check_value_in(value);
nix::NixStringContext stringContext;
auto rawStr = state->state.coerceToString(nix::noPos, v, stringContext, "while realising a string").toOwned();
nix::StorePathSet storePaths; nix::StorePathSet storePaths;
auto rewrites = state->state.realiseContext(stringContext, &storePaths); auto s = state->state.realiseString(v, &storePaths, isIFD);
auto s = nix::rewriteStrings(rawStr, rewrites);
// Convert to the C API StorePath type and convert to vector for index-based access // Convert to the C API StorePath type and convert to vector for index-based access
std::vector<StorePath> vec; std::vector<StorePath> vec;
+14 -10
View File
@@ -1,13 +1,14 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-store-c nix-store-c,
, nix-expr nix-expr,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -47,9 +48,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+15 -11
View File
@@ -1,15 +1,16 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-store-test-support nix-store-test-support,
, nix-expr nix-expr,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -49,9 +50,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+36 -26
View File
@@ -1,20 +1,21 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-expr nix-expr,
, nix-expr-c nix-expr-c,
, nix-expr-test-support nix-expr-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -56,22 +57,31 @@ mkMesonExecutable (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+51
View File
@@ -177,6 +177,57 @@ namespace nix {
) )
); );
// The following macros ultimately define 48 tests (16 variations on three
// templates). Each template tests an expression that can be written in 2^4
// different ways, by making four choices about whether to write a particular
// attribute path segment as `x.y = ...;` (collapsed) or `x = { y = ...; };`
// (expanded).
//
// The nestedAttrsetMergeXXXX tests check that the expression
// `{ a.b.c = 1; a.b.d = 2; }` has the same value regardless of how it is
// expanded. (That exact expression is exercised in test
// nestedAttrsetMerge0000, because it is fully collapsed. The test
// nestedAttrsetMerge1001 would instead examine
// `{ a = { b.c = 1; }; a.b = { d = 2; }; }`.)
//
// The nestedAttrsetMergeDupXXXX tests check that the expression
// `{ a.b.c = 1; a.b.c = 2; }` throws a duplicate attribute error, again
// regardless of how it is expanded.
//
// The nestedAttrsetMergeLetXXXX tests check that the expression
// `let a.b.c = 1; a.b.d = 2; in a` has the same value regardless of how it is
// expanded.
#define X_EXPAND_IF0(k, v) k "." v
#define X_EXPAND_IF1(k, v) k " = { " v " };"
#define X4(w, x, y, z) \
TEST_F(TrivialExpressionTest, nestedAttrsetMerge##w##x##y##z) { \
auto v = eval("{ a.b = { c = 1; d = 2; }; } == { " \
X_EXPAND_IF##w("a", X_EXPAND_IF##x("b", "c = 1;")) " " \
X_EXPAND_IF##y("a", X_EXPAND_IF##z("b", "d = 2;")) " }"); \
ASSERT_THAT(v, IsTrue()); \
}; \
TEST_F(TrivialExpressionTest, nestedAttrsetMergeDup##w##x##y##z) { \
ASSERT_THROW(eval("{ " \
X_EXPAND_IF##w("a", X_EXPAND_IF##x("b", "c = 1;")) " " \
X_EXPAND_IF##y("a", X_EXPAND_IF##z("b", "c = 2;")) " }"), Error); \
}; \
TEST_F(TrivialExpressionTest, nestedAttrsetMergeLet##w##x##y##z) { \
auto v = eval("{ b = { c = 1; d = 2; }; } == (let " \
X_EXPAND_IF##w("a", X_EXPAND_IF##x("b", "c = 1;")) " " \
X_EXPAND_IF##y("a", X_EXPAND_IF##z("b", "d = 2;")) " in a)"); \
ASSERT_THAT(v, IsTrue()); \
};
#define X3(...) X4(__VA_ARGS__, 0) X4(__VA_ARGS__, 1)
#define X2(...) X3(__VA_ARGS__, 0) X3(__VA_ARGS__, 1)
#define X1(...) X2(__VA_ARGS__, 0) X2(__VA_ARGS__, 1)
X1(0) X1(1)
#undef X_EXPAND_IF0
#undef X_EXPAND_IF1
#undef X1
#undef X2
#undef X3
#undef X4
TEST_F(TrivialExpressionTest, functor) { TEST_F(TrivialExpressionTest, functor) {
auto v = eval("{ __functor = self: arg: self.v + arg; v = 10; } 5"); auto v = eval("{ __functor = self: arg: self.v + arg; v = 10; } 5");
ASSERT_THAT(v, IsIntEq(15)); ASSERT_THAT(v, IsIntEq(15));
+48 -48
View File
@@ -20,70 +20,70 @@ let
# Resolve a input spec into a node name. An input spec is # Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root # either a node name, or a 'follows' path from the root
# node. # node.
resolveInput = inputSpec: resolveInput =
if builtins.isList inputSpec inputSpec: if builtins.isList inputSpec then getInputByPath lockFile.root inputSpec else inputSpec;
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the # Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node. # root node, returning the final node.
getInputByPath = nodeName: path: getInputByPath =
if path == [] nodeName: path:
then nodeName if path == [ ] then
nodeName
else else
getInputByPath getInputByPath
# Since this could be a 'follows' input, call resolveInput. # Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path}) (resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path); (builtins.tail path);
allNodes = allNodes = builtins.mapAttrs (
builtins.mapAttrs key: node:
(key: node: let
let
sourceInfo = sourceInfo =
if overrides ? ${key} if overrides ? ${key} then
then overrides.${key}.sourceInfo
overrides.${key}.sourceInfo else
else # FIXME: remove obsolete node.info.
# FIXME: remove obsolete node.info. # Note: lock file entries are always final.
# Note: lock file entries are always final. fetchTreeFinal (node.info or { } // removeAttrs node.locked [ "dir" ]);
fetchTreeFinal (node.info or {} // removeAttrs node.locked ["dir"]);
subdir = overrides.${key}.dir or node.locked.dir or ""; subdir = overrides.${key}.dir or node.locked.dir or "";
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir); outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
flake = import (outPath + "/flake.nix"); flake = import (outPath + "/flake.nix");
inputs = builtins.mapAttrs inputs = builtins.mapAttrs (inputName: inputSpec: allNodes.${resolveInput inputSpec}) (
(inputName: inputSpec: allNodes.${resolveInput inputSpec}) node.inputs or { }
(node.inputs or {}); );
outputs = flake.outputs (inputs // { self = result; }); outputs = flake.outputs (inputs // { self = result; });
result = result =
outputs outputs
# We add the sourceInfo attribute for its metadata, as they are # We add the sourceInfo attribute for its metadata, as they are
# relevant metadata for the flake. However, the outPath of the # relevant metadata for the flake. However, the outPath of the
# sourceInfo does not necessarily match the outPath of the flake, # sourceInfo does not necessarily match the outPath of the flake,
# as the flake may be in a subdirectory of a source. # as the flake may be in a subdirectory of a source.
# This is shadowed in the next // # This is shadowed in the next //
// sourceInfo // sourceInfo
// { // {
# This shadows the sourceInfo.outPath # This shadows the sourceInfo.outPath
inherit outPath; inherit outPath;
inherit inputs; inherit outputs; inherit sourceInfo; _type = "flake"; inherit inputs;
}; inherit outputs;
inherit sourceInfo;
_type = "flake";
};
in in
if node.flake or true then if node.flake or true then
assert builtins.isFunction flake.outputs; assert builtins.isFunction flake.outputs;
result result
else else
sourceInfo sourceInfo
) ) lockFile.nodes;
lockFile.nodes;
in allNodes.${lockFile.root} in
allNodes.${lockFile.root}
+1 -1
View File
@@ -45,7 +45,7 @@ EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr
// TODO: check compatibility with nested debugger calls. // TODO: check compatibility with nested debugger calls.
// TODO: What side-effects?? // TODO: What side-effects??
error.state.debugTraces.push_front(DebugTrace{ error.state.debugTraces.push_front(DebugTrace{
.pos = error.state.positions[expr.getPos()], .pos = expr.getPos(),
.expr = expr, .expr = expr,
.env = env, .env = env,
.hint = HintFmt("Fake frame for debugging purposes"), .hint = HintFmt("Fake frame for debugging purposes"),
+26 -22
View File
@@ -396,7 +396,7 @@ void EvalState::checkURI(const std::string & uri)
/* If the URI is a path, then check it against allowedPaths as /* If the URI is a path, then check it against allowedPaths as
well. */ well. */
if (hasPrefix(uri, "/")) { if (isAbsolute(uri)) {
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListSourceAccessor>()) if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListSourceAccessor>())
rootFS2->checkAccess(CanonPath(uri)); rootFS2->checkAccess(CanonPath(uri));
return; return;
@@ -734,18 +734,26 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
if (!debugRepl || inDebugger) if (!debugRepl || inDebugger)
return; return;
auto dts = auto dts = [&]() -> std::unique_ptr<DebugTraceStacker> {
error && expr.getPos() if (error && expr.getPos()) {
? std::make_unique<DebugTraceStacker>( auto trace = DebugTrace{
*this, .pos = [&]() -> std::variant<Pos, PosIdx> {
DebugTrace { if (error->info().pos) {
.pos = error->info().pos ? error->info().pos : positions[expr.getPos()], if (auto * pos = error->info().pos.get())
return *pos;
return noPos;
}
return expr.getPos();
}(),
.expr = expr, .expr = expr,
.env = env, .env = env,
.hint = error->info().msg, .hint = error->info().msg,
.isError = true .isError = true};
})
: nullptr; return std::make_unique<DebugTraceStacker>(*this, std::move(trace));
}
return nullptr;
}();
if (error) if (error)
{ {
@@ -790,7 +798,7 @@ static std::unique_ptr<DebugTraceStacker> makeDebugTraceStacker(
EvalState & state, EvalState & state,
Expr & expr, Expr & expr,
Env & env, Env & env,
std::shared_ptr<Pos> && pos, std::variant<Pos, PosIdx> pos,
const Args & ... formatArgs) const Args & ... formatArgs)
{ {
return std::make_unique<DebugTraceStacker>(state, return std::make_unique<DebugTraceStacker>(state,
@@ -1067,7 +1075,7 @@ void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
*this, *this,
*e, *e,
this->baseEnv, this->baseEnv,
e->getPos() ? std::make_shared<Pos>(positions[e->getPos()]) : nullptr, e->getPos(),
"while evaluating the file '%1%':", resolvedPath.to_string()) "while evaluating the file '%1%':", resolvedPath.to_string())
: nullptr; : nullptr;
@@ -1293,9 +1301,7 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env2, env2,
getPos() getPos(),
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while evaluating a '%1%' expression", "while evaluating a '%1%' expression",
"let" "let"
) )
@@ -1364,7 +1370,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env, env,
state.positions[getPos()], getPos(),
"while evaluating the attribute '%1%'", "while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath)) showAttrPath(state, env, attrPath))
: nullptr; : nullptr;
@@ -1565,7 +1571,7 @@ void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value &
try { try {
auto dts = debugRepl auto dts = debugRepl
? makeDebugTraceStacker( ? makeDebugTraceStacker(
*this, *lambda.body, env2, positions[lambda.pos], *this, *lambda.body, env2, lambda.pos,
"while calling %s", "while calling %s",
lambda.name lambda.name
? concatStrings("'", symbols[lambda.name], "'") ? concatStrings("'", symbols[lambda.name], "'")
@@ -1704,9 +1710,7 @@ void ExprCall::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env, env,
getPos() getPos(),
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while calling a function" "while calling a function"
) )
: nullptr; : nullptr;
@@ -2087,7 +2091,7 @@ void EvalState::forceValueDeep(Value & v)
try { try {
// If the value is a thunk, we're evaling. Otherwise no trace necessary. // If the value is a thunk, we're evaling. Otherwise no trace necessary.
auto dts = debugRepl && i.value->isThunk() auto dts = debugRepl && i.value->isThunk()
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, positions[i.pos], ? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, i.pos,
"while evaluating the attribute '%1%'", symbols[i.name]) "while evaluating the attribute '%1%'", symbols[i.name])
: nullptr; : nullptr;
@@ -2365,7 +2369,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
: [&]() { : [&]() {
auto dstPath = fetchToStore( auto dstPath = fetchToStore(
*store, *store,
path.resolveSymlinks(), path.resolveSymlinks(SymlinkResolution::Ancestors),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy, settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(), path.baseName(),
ContentAddressMethod::Raw::NixArchive, ContentAddressMethod::Raw::NixArchive,
+27 -1
View File
@@ -171,11 +171,28 @@ struct RegexCache;
std::shared_ptr<RegexCache> makeRegexCache(); std::shared_ptr<RegexCache> makeRegexCache();
struct DebugTrace { struct DebugTrace {
std::shared_ptr<Pos> pos; /* WARNING: Converting PosIdx -> Pos should be done with extra care. This is
due to the fact that operator[] of PosTable is incredibly expensive. */
std::variant<Pos, PosIdx> pos;
const Expr & expr; const Expr & expr;
const Env & env; const Env & env;
HintFmt hint; HintFmt hint;
bool isError; bool isError;
Pos getPos(const PosTable & table) const
{
return std::visit(
overloaded{
[&](PosIdx idx) {
// Prefer direct pos, but if noPos then try the expr.
if (!idx)
idx = expr.getPos();
return table[idx];
},
[&](Pos pos) { return pos; },
},
pos);
}
}; };
class EvalState : public std::enable_shared_from_this<EvalState> class EvalState : public std::enable_shared_from_this<EvalState>
@@ -805,6 +822,15 @@ public:
*/ */
[[nodiscard]] StringMap realiseContext(const NixStringContext & context, StorePathSet * maybePaths = nullptr, bool isIFD = true); [[nodiscard]] StringMap realiseContext(const NixStringContext & context, StorePathSet * maybePaths = nullptr, bool isIFD = true);
/**
* Realise the given string with context, and return the string with outputs instead of downstream output placeholders.
* @param[in] str the string to realise
* @param[out] paths all referenced store paths will be added to this set
* @return the realised string
* @throw EvalError if the value is not a string, path or derivation (see `coerceToString`)
*/
std::string realiseString(Value & str, StorePathSet * storePathsOutMaybe, bool isIFD = true, const PosIdx pos = noPos);
/* Call the binary path filter predicate used builtins.path etc. */ /* Call the binary path filter predicate used builtins.path etc. */
bool callPathFilter( bool callPathFilter(
Value * filterFun, Value * filterFun,
+62 -30
View File
@@ -1,40 +1,72 @@
{ system ? "" # obsolete {
, url system ? "", # obsolete
, hash ? "" # an SRI hash url,
hash ? "", # an SRI hash
# Legacy hash specification # Legacy hash specification
, md5 ? "", sha1 ? "", sha256 ? "", sha512 ? "" md5 ? "",
, outputHash ? sha1 ? "",
if hash != "" then hash else if sha512 != "" then sha512 else if sha1 != "" then sha1 else if md5 != "" then md5 else sha256 sha256 ? "",
, outputHashAlgo ? sha512 ? "",
if hash != "" then "" else if sha512 != "" then "sha512" else if sha1 != "" then "sha1" else if md5 != "" then "md5" else "sha256" outputHash ?
if hash != "" then
hash
else if sha512 != "" then
sha512
else if sha1 != "" then
sha1
else if md5 != "" then
md5
else
sha256,
outputHashAlgo ?
if hash != "" then
""
else if sha512 != "" then
"sha512"
else if sha1 != "" then
"sha1"
else if md5 != "" then
"md5"
else
"sha256",
, executable ? false executable ? false,
, unpack ? false unpack ? false,
, name ? baseNameOf (toString url) name ? baseNameOf (toString url),
, impure ? false impure ? false,
}: }:
derivation ({ derivation (
builder = "builtin:fetchurl"; {
builder = "builtin:fetchurl";
# New-style output content requirements. # New-style output content requirements.
outputHashMode = if unpack || executable then "recursive" else "flat"; outputHashMode = if unpack || executable then "recursive" else "flat";
inherit name url executable unpack; inherit
name
url
executable
unpack
;
system = "builtin"; system = "builtin";
# No need to double the amount of network traffic # No need to double the amount of network traffic
preferLocalBuild = true; preferLocalBuild = true;
# This attribute does nothing; it's here to avoid changing evaluation results. # This attribute does nothing; it's here to avoid changing evaluation results.
impureEnvVars = [ impureEnvVars = [
"http_proxy" "https_proxy" "ftp_proxy" "all_proxy" "no_proxy" "http_proxy"
]; "https_proxy"
"ftp_proxy"
"all_proxy"
"no_proxy"
];
# To make "nix-prefetch-url" work. # To make "nix-prefetch-url" work.
urls = [ url ]; urls = [ url ];
} // (if impure }
then { __impure = true; } // (if impure then { __impure = true; } else { inherit outputHashAlgo outputHash; })
else { inherit outputHashAlgo outputHash; })) )
+21 -15
View File
@@ -1,21 +1,27 @@
attrs @ { drvPath, outputs, name, ... }: attrs@{
drvPath,
outputs,
name,
...
}:
let let
commonAttrs = (builtins.listToAttrs outputsList) // commonAttrs = (builtins.listToAttrs outputsList) // {
{ all = map (x: x.value) outputsList; all = map (x: x.value) outputsList;
inherit drvPath name; inherit drvPath name;
type = "derivation"; type = "derivation";
}; };
outputToAttrListElement = outputName: outputToAttrListElement = outputName: {
{ name = outputName; name = outputName;
value = commonAttrs // { value = commonAttrs // {
outPath = builtins.getAttr outputName attrs; outPath = builtins.getAttr outputName attrs;
inherit outputName; inherit outputName;
};
}; };
};
outputsList = map outputToAttrListElement outputs; outputsList = map outputToAttrListElement outputs;
in (builtins.head outputsList).value in
(builtins.head outputsList).value
-2
View File
@@ -175,8 +175,6 @@ headers = [config_h] + files(
# internal: 'lexer-helpers.hh', # internal: 'lexer-helpers.hh',
'nixexpr.hh', 'nixexpr.hh',
'parser-state.hh', 'parser-state.hh',
'pos-idx.hh',
'pos-table.hh',
'primops.hh', 'primops.hh',
'print-ambiguous.hh', 'print-ambiguous.hh',
'print-options.hh', 'print-options.hh',
-35
View File
@@ -601,41 +601,6 @@ void ExprLambda::setDocComment(DocComment docComment) {
} }
}; };
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(
std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
/* Symbol table. */ /* Symbol table. */
size_t SymbolTable::totalSize() const size_t SymbolTable::totalSize() const
+39 -37
View File
@@ -1,33 +1,34 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, bison bison,
, flex flex,
, cmake # for resolving toml11 dep cmake, # for resolving toml11 dep
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, boost boost,
, boehmgc boehmgc,
, nlohmann_json nlohmann_json,
, toml11 toml11,
# Configuration Options # Configuration Options
, version version,
# Whether to use garbage collection for the Nix language evaluator. # Whether to use garbage collection for the Nix language evaluator.
# #
# If it is disabled, we just leak memory, but this is not as bad as it # If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived # sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is # processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.) # only leaked *within* the process.)
# #
# Temporarily disabled on Windows because the `GC_throw_bad_alloc` # Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking. # symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows enableGC ? !stdenv.hostPlatform.isWindows,
}: }:
let let
@@ -51,10 +52,7 @@ mkMesonLibrary (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "hh") ./.) (fileset.fileFilter (file: file.hasExt "hh") ./.)
./lexer.l ./lexer.l
./parser.y ./parser.y
(fileset.difference (fileset.difference (fileset.fileFilter (file: file.hasExt "nix") ./.) ./package.nix)
(fileset.fileFilter (file: file.hasExt "nix") ./.)
./package.nix
)
]; ];
nativeBuildInputs = [ nativeBuildInputs = [
@@ -87,14 +85,18 @@ mkMesonLibrary (finalAttrs: {
(lib.mesonEnable "gc" enableGC) (lib.mesonEnable "gc" enableGC)
]; ];
env = { env =
# Needed for Meson to find Boost. {
# https://github.com/NixOS/nixpkgs/issues/86131. # Needed for Meson to find Boost.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include"; # https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib"; BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
} // lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
LDFLAGS = "-fuse-ld=gold"; }
}; // lib.optionalAttrs
(stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+64 -44
View File
@@ -88,6 +88,7 @@ struct ParserState
void dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos); void dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos);
void dupAttr(Symbol attr, const PosIdx pos, const PosIdx prevPos); void dupAttr(Symbol attr, const PosIdx pos, const PosIdx prevPos);
void addAttr(ExprAttrs * attrs, AttrPath && attrPath, const ParserLocation & loc, Expr * e, const ParserLocation & exprLoc); void addAttr(ExprAttrs * attrs, AttrPath && attrPath, const ParserLocation & loc, Expr * e, const ParserLocation & exprLoc);
void addAttr(ExprAttrs * attrs, AttrPath & attrPath, const Symbol & symbol, ExprAttrs::AttrDef && def);
Formals * validateFormals(Formals * formals, PosIdx pos = noPos, Symbol arg = {}); Formals * validateFormals(Formals * formals, PosIdx pos = noPos, Symbol arg = {});
Expr * stripIndentation(const PosIdx pos, Expr * stripIndentation(const PosIdx pos,
std::vector<std::pair<PosIdx, std::variant<Expr *, StringToken>>> && es); std::vector<std::pair<PosIdx, std::variant<Expr *, StringToken>>> && es);
@@ -120,64 +121,29 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, const
// Checking attrPath validity. // Checking attrPath validity.
// =========================== // ===========================
for (i = attrPath.begin(); i + 1 < attrPath.end(); i++) { for (i = attrPath.begin(); i + 1 < attrPath.end(); i++) {
ExprAttrs * nested;
if (i->symbol) { if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol); ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol);
if (j != attrs->attrs.end()) { if (j != attrs->attrs.end()) {
if (j->second.kind != ExprAttrs::AttrDef::Kind::Inherited) { nested = dynamic_cast<ExprAttrs *>(j->second.e);
ExprAttrs * attrs2 = dynamic_cast<ExprAttrs *>(j->second.e); if (!nested) {
if (!attrs2) dupAttr(attrPath, pos, j->second.pos); attrPath.erase(i + 1, attrPath.end());
attrs = attrs2;
} else
dupAttr(attrPath, pos, j->second.pos); dupAttr(attrPath, pos, j->second.pos);
}
} else { } else {
ExprAttrs * nested = new ExprAttrs; nested = new ExprAttrs;
attrs->attrs[i->symbol] = ExprAttrs::AttrDef(nested, pos); attrs->attrs[i->symbol] = ExprAttrs::AttrDef(nested, pos);
attrs = nested;
} }
} else { } else {
ExprAttrs *nested = new ExprAttrs; nested = new ExprAttrs;
attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, nested, pos)); attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, nested, pos));
attrs = nested;
} }
attrs = nested;
} }
// Expr insertion. // Expr insertion.
// ========================== // ==========================
if (i->symbol) { if (i->symbol) {
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(i->symbol); addAttr(attrs, attrPath, i->symbol, ExprAttrs::AttrDef(e, pos));
if (j != attrs->attrs.end()) {
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto ae = dynamic_cast<ExprAttrs *>(e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
auto j2 = jAttrs->attrs.find(ad.first);
if (j2 != jAttrs->attrs.end()) // Attr already defined in iAttrs, error.
dupAttr(ad.first, j2->second.pos, ad.second.pos);
jAttrs->attrs.emplace(ad.first, ad.second);
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
}
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(), ae->dynamicAttrs.begin(), ae->dynamicAttrs.end());
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
ae->inheritFromExprs->begin(), ae->inheritFromExprs->end());
}
} else {
dupAttr(attrPath, pos, j->second.pos);
}
} else {
// This attr path is not defined. Let's create it.
attrs->attrs.emplace(i->symbol, ExprAttrs::AttrDef(e, pos));
e->setName(i->symbol);
}
} else { } else {
attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, e, pos)); attrs->dynamicAttrs.push_back(ExprAttrs::DynamicAttrDef(i->expr, e, pos));
} }
@@ -189,6 +155,60 @@ inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath && attrPath, const
} }
} }
/**
* Precondition: attrPath is used for error messages and should already contain
* symbol as its last element.
*/
inline void ParserState::addAttr(ExprAttrs * attrs, AttrPath & attrPath, const Symbol & symbol, ExprAttrs::AttrDef && def)
{
ExprAttrs::AttrDefs::iterator j = attrs->attrs.find(symbol);
if (j != attrs->attrs.end()) {
// This attr path is already defined. However, if both
// e and the expr pointed by the attr path are two attribute sets,
// we want to merge them.
// Otherwise, throw an error.
auto ae = dynamic_cast<ExprAttrs *>(def.e);
auto jAttrs = dynamic_cast<ExprAttrs *>(j->second.e);
// N.B. In a world in which we are less bound by our past mistakes, we
// would also test that jAttrs and ae are not recursive. The effect of
// not doing so is that any `rec` marker on ae is discarded, and any
// `rec` marker on jAttrs will apply to the attributes in ae.
// See https://github.com/NixOS/nix/issues/9020.
if (jAttrs && ae) {
if (ae->inheritFromExprs && !jAttrs->inheritFromExprs)
jAttrs->inheritFromExprs = std::make_unique<std::vector<Expr *>>();
for (auto & ad : ae->attrs) {
if (ad.second.kind == ExprAttrs::AttrDef::Kind::InheritedFrom) {
auto & sel = dynamic_cast<ExprSelect &>(*ad.second.e);
auto & from = dynamic_cast<ExprInheritFrom &>(*sel.e);
from.displ += jAttrs->inheritFromExprs->size();
}
attrPath.emplace_back(AttrName(ad.first));
addAttr(jAttrs, attrPath, ad.first, std::move(ad.second));
attrPath.pop_back();
}
ae->attrs.clear();
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(),
std::make_move_iterator(ae->dynamicAttrs.begin()),
std::make_move_iterator(ae->dynamicAttrs.end()));
ae->dynamicAttrs.clear();
if (ae->inheritFromExprs) {
jAttrs->inheritFromExprs->insert(jAttrs->inheritFromExprs->end(),
std::make_move_iterator(ae->inheritFromExprs->begin()),
std::make_move_iterator(ae->inheritFromExprs->end()));
ae->inheritFromExprs = nullptr;
}
} else {
dupAttr(attrPath, def.pos, j->second.pos);
}
} else {
// This attr path is not defined. Let's create it.
attrs->attrs.emplace(symbol, def);
def.e->setName(symbol);
}
}
inline Formals * ParserState::validateFormals(Formals * formals, PosIdx pos, Symbol arg) inline Formals * ParserState::validateFormals(Formals * formals, PosIdx pos, Symbol arg)
{ {
std::sort(formals->formals.begin(), formals->formals.end(), std::sort(formals->formals.begin(), formals->formals.end(),
+11 -2
View File
@@ -47,6 +47,15 @@ static inline Value * mkString(EvalState & state, const std::csub_match & match)
return v; return v;
} }
std::string EvalState::realiseString(Value & s, StorePathSet * storePathsOutMaybe, bool isIFD, const PosIdx pos)
{
nix::NixStringContext stringContext;
auto rawStr = coerceToString(pos, s, stringContext, "while realising a string").toOwned();
auto rewrites = realiseContext(stringContext, storePathsOutMaybe, isIFD);
return nix::rewriteStrings(rawStr, rewrites);
}
StringMap EvalState::realiseContext(const NixStringContext & context, StorePathSet * maybePathsOut, bool isIFD) StringMap EvalState::realiseContext(const NixStringContext & context, StorePathSet * maybePathsOut, bool isIFD)
{ {
std::vector<DerivedPath::Built> drvs; std::vector<DerivedPath::Built> drvs;
@@ -4061,7 +4070,7 @@ static RegisterPrimOp primop_toString({
}); });
/* `substring start len str' returns the substring of `str' starting /* `substring start len str' returns the substring of `str' starting
at character position `min(start, stringLength str)' inclusive and at byte position `min(start, stringLength str)' inclusive and
ending at `min(start + len, stringLength str)'. `start' must be ending at `min(start + len, stringLength str)'. `start' must be
non-negative. */ non-negative. */
static void prim_substring(EvalState & state, const PosIdx pos, Value * * args, Value & v) static void prim_substring(EvalState & state, const PosIdx pos, Value * * args, Value & v)
@@ -4100,7 +4109,7 @@ static RegisterPrimOp primop_substring({
.name = "__substring", .name = "__substring",
.args = {"start", "len", "s"}, .args = {"start", "len", "s"},
.doc = R"( .doc = R"(
Return the substring of *s* from character position *start* Return the substring of *s* from byte position *start*
(zero-based) up to but not including *start + len*. If *start* is (zero-based) up to but not including *start + len*. If *start* is
greater than the length of the string, an empty string is returned. greater than the length of the string, an empty string is returned.
If *start + len* lies beyond the end of the string or *len* is `-1`, If *start + len* lies beyond the end of the string or *len* is `-1`,
+19 -12
View File
@@ -26,27 +26,34 @@
Note that `derivation` is very bare-bones, and provides almost no commands during the build. Note that `derivation` is very bare-bones, and provides almost no commands during the build.
Most likely, you'll want to use functions like `stdenv.mkDerivation` in Nixpkgs to set up a basic environment. Most likely, you'll want to use functions like `stdenv.mkDerivation` in Nixpkgs to set up a basic environment.
*/ */
drvAttrs @ { outputs ? [ "out" ], ... }: drvAttrs@{
outputs ? [ "out" ],
...
}:
let let
strict = derivationStrict drvAttrs; strict = derivationStrict drvAttrs;
commonAttrs = drvAttrs // (builtins.listToAttrs outputsList) // commonAttrs =
{ all = map (x: x.value) outputsList; drvAttrs
// (builtins.listToAttrs outputsList)
// {
all = map (x: x.value) outputsList;
inherit drvAttrs; inherit drvAttrs;
}; };
outputToAttrListElement = outputName: outputToAttrListElement = outputName: {
{ name = outputName; name = outputName;
value = commonAttrs // { value = commonAttrs // {
outPath = builtins.getAttr outputName strict; outPath = builtins.getAttr outputName strict;
drvPath = strict.drvPath; drvPath = strict.drvPath;
type = "derivation"; type = "derivation";
inherit outputName; inherit outputName;
};
}; };
};
outputsList = map outputToAttrListElement outputs; outputsList = map outputToAttrListElement outputs;
in (builtins.head outputsList).value in
(builtins.head outputsList).value
+5 -1
View File
@@ -108,7 +108,11 @@ json printValueAsJSON(EvalState & state, bool strict,
void printValueAsJSON(EvalState & state, bool strict, void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, std::ostream & str, NixStringContext & context, bool copyToStore) Value & v, const PosIdx pos, std::ostream & str, NixStringContext & context, bool copyToStore)
{ {
str << printValueAsJSON(state, strict, v, pos, context, copyToStore); try {
str << printValueAsJSON(state, strict, v, pos, context, copyToStore);
} catch (nlohmann::json::exception & e) {
throw JSONSerializationError("JSON serialization error: %s", e.what());
}
} }
json ExternalValueBase::printValueAsJSON(EvalState & state, bool strict, json ExternalValueBase::printValueAsJSON(EvalState & state, bool strict,
+3
View File
@@ -16,4 +16,7 @@ nlohmann::json printValueAsJSON(EvalState & state, bool strict,
void printValueAsJSON(EvalState & state, bool strict, void printValueAsJSON(EvalState & state, bool strict,
Value & v, const PosIdx pos, std::ostream & str, NixStringContext & context, bool copyToStore = true); Value & v, const PosIdx pos, std::ostream & str, NixStringContext & context, bool copyToStore = true);
MakeError(JSONSerializationError, Error);
} }
+35 -25
View File
@@ -1,19 +1,20 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-fetchers nix-fetchers,
, nix-store-test-support nix-store-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -54,22 +55,31 @@ mkMesonExecutable (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+1 -1
View File
@@ -36,7 +36,7 @@ struct CacheImpl : Cache
{ {
auto state(_state.lock()); auto state(_state.lock());
auto dbPath = getCacheDir() + "/fetcher-cache-v2.sqlite"; auto dbPath = getCacheDir() + "/fetcher-cache-v3.sqlite";
createDirs(dirOf(dbPath)); createDirs(dirOf(dbPath));
state->db = SQLite(dbPath); state->db = SQLite(dbPath);
+1 -1
View File
@@ -66,7 +66,7 @@ Input Input::fromURL(
} }
} }
throw Error("input '%s' is unsupported", url.url); throw Error("input '%s' is unsupported", url);
} }
Input Input::fromAttrs(const Settings & settings, Attrs && attrs) Input Input::fromAttrs(const Settings & settings, Attrs && attrs)
+6 -8
View File
@@ -205,7 +205,8 @@ static git_packbuilder_progress PACKBUILDER_PROGRESS_CHECK_INTERRUPT = &packBuil
} // extern "C" } // extern "C"
static void initRepoAtomically(std::filesystem::path &path, bool bare) { static void initRepoAtomically(std::filesystem::path &path, bool bare)
{
if (pathExists(path.string())) return; if (pathExists(path.string())) return;
Path tmpDir = createTempDir(os_string_to_string(PathViewNG { std::filesystem::path(path).parent_path() })); Path tmpDir = createTempDir(os_string_to_string(PathViewNG { std::filesystem::path(path).parent_path() }));
@@ -538,13 +539,10 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
// then use code that was removed in this commit (see blame) // then use code that was removed in this commit (see blame)
auto dir = this->path; auto dir = this->path;
Strings gitArgs; Strings gitArgs{"-C", dir.string(), "--git-dir", ".", "fetch", "--quiet", "--force"};
if (shallow) { if (shallow)
gitArgs = { "-C", dir.string(), "fetch", "--quiet", "--force", "--depth", "1", "--", url, refspec }; append(gitArgs, {"--depth", "1"});
} append(gitArgs, {std::string("--"), url, refspec});
else {
gitArgs = { "-C", dir.string(), "fetch", "--quiet", "--force", "--", url, refspec };
}
runProgram(RunOptions { runProgram(RunOptions {
.program = "git", .program = "git",
+10 -1
View File
@@ -425,7 +425,16 @@ struct GitInputScheme : InputScheme
auto url = parseURL(getStrAttr(input.attrs, "url")); auto url = parseURL(getStrAttr(input.attrs, "url"));
bool isBareRepository = url.scheme == "file" && !pathExists(url.path + "/.git"); bool isBareRepository = url.scheme == "file" && !pathExists(url.path + "/.git");
repoInfo.isLocal = url.scheme == "file" && !forceHttp && !isBareRepository; repoInfo.isLocal = url.scheme == "file" && !forceHttp && !isBareRepository;
repoInfo.url = repoInfo.isLocal ? url.path : url.base; //
// FIXME: here we turn a possibly relative path into an absolute path.
// This allows relative git flake inputs to be resolved against the
// **current working directory** (as in POSIX), which tends to work out
// ok in the context of flakes, but is the wrong behavior,
// as it should resolve against the flake.nix base directory instead.
//
// See: https://discourse.nixos.org/t/57783 and #9708
//
repoInfo.url = repoInfo.isLocal ? std::filesystem::absolute(url.path).string() : url.to_string();
// If this is a local directory and no ref or revision is // If this is a local directory and no ref or revision is
// given, then allow the use of an unclean working tree. // given, then allow the use of an unclean working tree.
+8 -8
View File
@@ -50,7 +50,7 @@ struct GitArchiveInputScheme : InputScheme
else if (std::regex_match(path[2], refRegex)) else if (std::regex_match(path[2], refRegex))
ref = path[2]; ref = path[2];
else else
throw BadURL("in URL '%s', '%s' is not a commit hash or branch/tag name", url.url, path[2]); throw BadURL("in URL '%s', '%s' is not a commit hash or branch/tag name", url, path[2]);
} else if (size > 3) { } else if (size > 3) {
std::string rs; std::string rs;
for (auto i = std::next(path.begin(), 2); i != path.end(); i++) { for (auto i = std::next(path.begin(), 2); i != path.end(); i++) {
@@ -63,34 +63,34 @@ struct GitArchiveInputScheme : InputScheme
if (std::regex_match(rs, refRegex)) { if (std::regex_match(rs, refRegex)) {
ref = rs; ref = rs;
} else { } else {
throw BadURL("in URL '%s', '%s' is not a branch/tag name", url.url, rs); throw BadURL("in URL '%s', '%s' is not a branch/tag name", url, rs);
} }
} else if (size < 2) } else if (size < 2)
throw BadURL("URL '%s' is invalid", url.url); throw BadURL("URL '%s' is invalid", url);
for (auto &[name, value] : url.query) { for (auto &[name, value] : url.query) {
if (name == "rev") { if (name == "rev") {
if (rev) if (rev)
throw BadURL("URL '%s' contains multiple commit hashes", url.url); throw BadURL("URL '%s' contains multiple commit hashes", url);
rev = Hash::parseAny(value, HashAlgorithm::SHA1); rev = Hash::parseAny(value, HashAlgorithm::SHA1);
} }
else if (name == "ref") { else if (name == "ref") {
if (!std::regex_match(value, refRegex)) if (!std::regex_match(value, refRegex))
throw BadURL("URL '%s' contains an invalid branch/tag name", url.url); throw BadURL("URL '%s' contains an invalid branch/tag name", url);
if (ref) if (ref)
throw BadURL("URL '%s' contains multiple branch/tag names", url.url); throw BadURL("URL '%s' contains multiple branch/tag names", url);
ref = value; ref = value;
} }
else if (name == "host") { else if (name == "host") {
if (!std::regex_match(value, hostRegex)) if (!std::regex_match(value, hostRegex))
throw BadURL("URL '%s' contains an invalid instance host", url.url); throw BadURL("URL '%s' contains an invalid instance host", url);
host_url = value; host_url = value;
} }
// FIXME: barf on unsupported attributes // FIXME: barf on unsupported attributes
} }
if (ref && rev) if (ref && rev)
throw BadURL("URL '%s' contains both a commit hash and a branch/tag name %s %s", url.url, *ref, rev->gitRev()); throw BadURL("URL '%s' contains both a commit hash and a branch/tag name %s %s", url, *ref, rev->gitRev());
Input input{settings}; Input input{settings};
input.attrs.insert_or_assign("type", std::string { schemeName() }); input.attrs.insert_or_assign("type", std::string { schemeName() });
+4 -4
View File
@@ -26,16 +26,16 @@ struct IndirectInputScheme : InputScheme
else if (std::regex_match(path[1], refRegex)) else if (std::regex_match(path[1], refRegex))
ref = path[1]; ref = path[1];
else else
throw BadURL("in flake URL '%s', '%s' is not a commit hash or branch/tag name", url.url, path[1]); throw BadURL("in flake URL '%s', '%s' is not a commit hash or branch/tag name", url, path[1]);
} else if (path.size() == 3) { } else if (path.size() == 3) {
if (!std::regex_match(path[1], refRegex)) if (!std::regex_match(path[1], refRegex))
throw BadURL("in flake URL '%s', '%s' is not a branch/tag name", url.url, path[1]); throw BadURL("in flake URL '%s', '%s' is not a branch/tag name", url, path[1]);
ref = path[1]; ref = path[1];
if (!std::regex_match(path[2], revRegex)) if (!std::regex_match(path[2], revRegex))
throw BadURL("in flake URL '%s', '%s' is not a commit hash", url.url, path[2]); throw BadURL("in flake URL '%s', '%s' is not a commit hash", url, path[2]);
rev = Hash::parseAny(path[2], HashAlgorithm::SHA1); rev = Hash::parseAny(path[2], HashAlgorithm::SHA1);
} else } else
throw BadURL("GitHub URL '%s' is invalid", url.url); throw BadURL("GitHub URL '%s' is invalid", url);
std::string id = path[0]; std::string id = path[0];
if (!std::regex_match(id, flakeRegex)) if (!std::regex_match(id, flakeRegex))
+1 -1
View File
@@ -161,7 +161,7 @@ struct MercurialInputScheme : InputScheme
{ {
auto url = parseURL(getStrAttr(input.attrs, "url")); auto url = parseURL(getStrAttr(input.attrs, "url"));
bool isLocal = url.scheme == "file"; bool isLocal = url.scheme == "file";
return {isLocal, isLocal ? url.path : url.base}; return {isLocal, isLocal ? url.path : url.to_string()};
} }
StorePath fetchToStore(ref<Store> store, Input & input) const StorePath fetchToStore(ref<Store> store, Input & input) const
+16 -12
View File
@@ -1,15 +1,16 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nlohmann_json nlohmann_json,
, libgit2 libgit2,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -49,9 +50,12 @@ mkMesonLibrary (finalAttrs: {
echo ${version} > ../../.version echo ${version} > ../../.version
''; '';
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+4 -4
View File
@@ -14,7 +14,7 @@ struct PathInputScheme : InputScheme
if (url.scheme != "path") return {}; if (url.scheme != "path") return {};
if (url.authority && *url.authority != "") if (url.authority && *url.authority != "")
throw Error("path URL '%s' should not have an authority ('%s')", url.url, *url.authority); throw Error("path URL '%s' should not have an authority ('%s')", url, *url.authority);
Input input{settings}; Input input{settings};
input.attrs.insert_or_assign("type", "path"); input.attrs.insert_or_assign("type", "path");
@@ -27,10 +27,10 @@ struct PathInputScheme : InputScheme
if (auto n = string2Int<uint64_t>(value)) if (auto n = string2Int<uint64_t>(value))
input.attrs.insert_or_assign(name, *n); input.attrs.insert_or_assign(name, *n);
else else
throw Error("path URL '%s' has invalid parameter '%s'", url.to_string(), name); throw Error("path URL '%s' has invalid parameter '%s'", url, name);
} }
else else
throw Error("path URL '%s' has unsupported parameter '%s'", url.to_string(), name); throw Error("path URL '%s' has unsupported parameter '%s'", url, name);
return input; return input;
} }
@@ -97,7 +97,7 @@ struct PathInputScheme : InputScheme
std::optional<std::string> isRelative(const Input & input) const std::optional<std::string> isRelative(const Input & input) const
{ {
auto path = getStrAttr(input.attrs, "path"); auto path = getStrAttr(input.attrs, "path");
if (hasPrefix(path, "/")) if (isAbsolute(path))
return std::nullopt; return std::nullopt;
else else
return path; return path;
+1 -1
View File
@@ -156,7 +156,7 @@ static std::shared_ptr<Registry> getGlobalRegistry(const Settings & settings, re
return std::make_shared<Registry>(settings, Registry::Global); // empty registry return std::make_shared<Registry>(settings, Registry::Global); // empty registry
} }
if (!hasPrefix(path, "/")) { if (!isAbsolute(path)) {
auto storePath = downloadFile(store, path, "flake-registry.json").storePath; auto storePath = downloadFile(store, path, "flake-registry.json").storePath;
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>())
store2->addPermRoot(storePath, getCacheDir() + "/flake-registry.json"); store2->addPermRoot(storePath, getCacheDir() + "/flake-registry.json");
+47 -5
View File
@@ -7,18 +7,60 @@ namespace nix {
/* ----------- tests for flake/flakeref.hh --------------------------------------------------*/ /* ----------- tests for flake/flakeref.hh --------------------------------------------------*/
/* ---------------------------------------------------------------------------- TEST(parseFlakeRef, path) {
* to_string experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
* --------------------------------------------------------------------------*/
fetchers::Settings fetchSettings;
{
auto s = "/foo/bar";
auto flakeref = parseFlakeRef(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "path:/foo/bar");
}
{
auto s = "/foo/bar?revCount=123&rev=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
auto flakeref = parseFlakeRef(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "path:/foo/bar?rev=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&revCount=123");
}
{
auto s = "/foo/bar?xyzzy=123";
EXPECT_THROW(
parseFlakeRef(fetchSettings, s),
Error);
}
{
auto s = "/foo/bar#bla";
EXPECT_THROW(
parseFlakeRef(fetchSettings, s),
Error);
}
{
auto s = "/foo/bar#bla";
auto [flakeref, fragment] = parseFlakeRefWithFragment(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "path:/foo/bar");
ASSERT_EQ(fragment, "bla");
}
{
auto s = "/foo/bar?revCount=123#bla";
auto [flakeref, fragment] = parseFlakeRefWithFragment(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "path:/foo/bar?revCount=123");
ASSERT_EQ(fragment, "bla");
}
}
TEST(to_string, doesntReencodeUrl) { TEST(to_string, doesntReencodeUrl) {
fetchers::Settings fetchSettings; fetchers::Settings fetchSettings;
auto s = "http://localhost:8181/test/+3d.tar.gz"; auto s = "http://localhost:8181/test/+3d.tar.gz";
auto flakeref = parseFlakeRef(fetchSettings, s); auto flakeref = parseFlakeRef(fetchSettings, s);
auto parsed = flakeref.to_string(); auto unparsed = flakeref.to_string();
auto expected = "http://localhost:8181/test/%2B3d.tar.gz"; auto expected = "http://localhost:8181/test/%2B3d.tar.gz";
ASSERT_EQ(parsed, expected); ASSERT_EQ(unparsed, expected);
} }
} }
+35 -25
View File
@@ -1,19 +1,20 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-flake nix-flake,
, nix-expr-test-support nix-expr-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -54,22 +55,31 @@ mkMesonExecutable (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+47 -50
View File
@@ -67,6 +67,20 @@ std::optional<FlakeRef> maybeParseFlakeRef(
} }
} }
static std::pair<FlakeRef, std::string> fromParsedURL(
const fetchers::Settings & fetchSettings,
ParsedURL && parsedURL,
bool isFlake)
{
auto dir = getOr(parsedURL.query, "dir", "");
parsedURL.query.erase("dir");
std::string fragment;
std::swap(fragment, parsedURL.fragment);
return {FlakeRef(fetchers::Input::fromURL(fetchSettings, parsedURL, isFlake), dir), fragment};
}
std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment( std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
const fetchers::Settings & fetchSettings, const fetchers::Settings & fetchSettings,
const std::string & url, const std::string & url,
@@ -74,28 +88,21 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
bool allowMissing, bool allowMissing,
bool isFlake) bool isFlake)
{ {
std::string path = url; static std::regex pathFlakeRegex(
std::string fragment = ""; R"(([^?#]*)(\?([^#]*))?(#(.*))?)",
std::map<std::string, std::string> query; std::regex::ECMAScript);
auto pathEnd = url.find_first_of("#?");
auto fragmentStart = pathEnd; std::smatch match;
if (pathEnd != std::string::npos && url[pathEnd] == '?') { auto succeeds = std::regex_match(url, match, pathFlakeRegex);
fragmentStart = url.find("#"); assert(succeeds);
} auto path = match[1].str();
if (pathEnd != std::string::npos) { auto query = decodeQuery(match[3]);
path = url.substr(0, pathEnd); auto fragment = percentDecode(match[5].str());
}
if (fragmentStart != std::string::npos) {
fragment = percentDecode(url.substr(fragmentStart+1));
}
if (pathEnd != std::string::npos && fragmentStart != std::string::npos && url[pathEnd] == '?') {
query = decodeQuery(url.substr(pathEnd+1, fragmentStart-pathEnd-1));
}
if (baseDir) { if (baseDir) {
/* Check if 'url' is a path (either absolute or relative /* Check if 'url' is a path (either absolute or relative
to 'baseDir'). If so, search upward to the root of the to 'baseDir'). If so, search upward to the root of the
repo (i.e. the directory containing .git). */ repo (i.e. the directory containing .git). */
path = absPath(path, baseDir); path = absPath(path, baseDir);
@@ -144,15 +151,12 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
while (flakeRoot != "/") { while (flakeRoot != "/") {
if (pathExists(flakeRoot + "/.git")) { if (pathExists(flakeRoot + "/.git")) {
auto base = std::string("git+file://") + flakeRoot;
auto parsedURL = ParsedURL{ auto parsedURL = ParsedURL{
.url = base, // FIXME
.base = base,
.scheme = "git+file", .scheme = "git+file",
.authority = "", .authority = "",
.path = flakeRoot, .path = flakeRoot,
.query = query, .query = query,
.fragment = fragment,
}; };
if (subdir != "") { if (subdir != "") {
@@ -164,9 +168,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
if (pathExists(flakeRoot + "/.git/shallow")) if (pathExists(flakeRoot + "/.git/shallow"))
parsedURL.query.insert_or_assign("shallow", "1"); parsedURL.query.insert_or_assign("shallow", "1");
return std::make_pair( return fromParsedURL(fetchSettings, std::move(parsedURL), isFlake);
FlakeRef(fetchers::Input::fromURL(fetchSettings, parsedURL), getOr(parsedURL.query, "dir", "")),
fragment);
} }
subdir = std::string(baseNameOf(flakeRoot)) + (subdir.empty() ? "" : "/" + subdir); subdir = std::string(baseNameOf(flakeRoot)) + (subdir.empty() ? "" : "/" + subdir);
@@ -175,21 +177,24 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
} }
} else { } else {
if (!hasPrefix(path, "/")) if (!isAbsolute(path))
throw BadURL("flake reference '%s' is not an absolute path", url); throw BadURL("flake reference '%s' is not an absolute path", url);
path = canonPath(path + "/" + getOr(query, "dir", "")); path = canonPath(path + "/" + getOr(query, "dir", ""));
} }
fetchers::Attrs attrs; return fromParsedURL(fetchSettings, {
attrs.insert_or_assign("type", "path"); .scheme = "path",
attrs.insert_or_assign("path", path); .authority = "",
.path = path,
.query = query,
.fragment = fragment
}, isFlake);
}
return std::make_pair(FlakeRef(fetchers::Input::fromAttrs(fetchSettings, std::move(attrs)), ""), fragment); /**
}; * Check if `url` is a flake ID. This is an abbreviated syntax for
* `flake:<flake-id>?ref=<ref>&rev=<rev>`.
*/
/* Check if 'url' is a flake ID. This is an abbreviated syntax for
'flake:<flake-id>?ref=<ref>&rev=<rev>'. */
static std::optional<std::pair<FlakeRef, std::string>> parseFlakeIdRef( static std::optional<std::pair<FlakeRef, std::string>> parseFlakeIdRef(
const fetchers::Settings & fetchSettings, const fetchers::Settings & fetchSettings,
const std::string & url, const std::string & url,
@@ -205,8 +210,6 @@ static std::optional<std::pair<FlakeRef, std::string>> parseFlakeIdRef(
if (std::regex_match(url, match, flakeRegex)) { if (std::regex_match(url, match, flakeRegex)) {
auto parsedURL = ParsedURL{ auto parsedURL = ParsedURL{
.url = url,
.base = "flake:" + match.str(1),
.scheme = "flake", .scheme = "flake",
.authority = "", .authority = "",
.path = match[1], .path = match[1],
@@ -227,22 +230,16 @@ std::optional<std::pair<FlakeRef, std::string>> parseURLFlakeRef(
bool isFlake bool isFlake
) )
{ {
ParsedURL parsedURL;
try { try {
parsedURL = parseURL(url); auto parsed = parseURL(url);
if (baseDir
&& parsed.scheme == "path"
&& !isAbsolute(parsed.path))
parsed.path = absPath(parsed.path, *baseDir);
return fromParsedURL(fetchSettings, std::move(parsed), isFlake);
} catch (BadURL &) { } catch (BadURL &) {
return std::nullopt; return std::nullopt;
} }
std::string fragment;
std::swap(fragment, parsedURL.fragment);
auto input = fetchers::Input::fromURL(fetchSettings, parsedURL, isFlake);
input.parent = baseDir;
return std::make_pair(
FlakeRef(std::move(input), getOr(parsedURL.query, "dir", "")),
fragment);
} }
std::pair<FlakeRef, std::string> parseFlakeRefWithFragment( std::pair<FlakeRef, std::string> parseFlakeRefWithFragment(
+17 -13
View File
@@ -1,16 +1,17 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, nix-expr nix-expr,
, nlohmann_json nlohmann_json,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -48,9 +49,12 @@ mkMesonLibrary (finalAttrs: {
echo ${version} > ../../.version echo ${version} > ../../.version
''; '';
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+16 -12
View File
@@ -1,15 +1,16 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util-c nix-util-c,
, nix-store nix-store,
, nix-store-c nix-store-c,
, nix-main nix-main,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -51,9 +52,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+3 -1
View File
@@ -17,7 +17,9 @@ MixCommonArgs::MixCommonArgs(const std::string & programName)
.shortName = 'v', .shortName = 'v',
.description = "Increase the logging verbosity level.", .description = "Increase the logging verbosity level.",
.category = loggingCategory, .category = loggingCategory,
.handler = {[]() { verbosity = (Verbosity) (verbosity + 1); }}, .handler = {[]() {
verbosity = (Verbosity) std::min<std::underlying_type_t<Verbosity>>(verbosity + 1, lvlVomit);
}},
}); });
addFlag({ addFlag({
+15 -11
View File
@@ -1,15 +1,16 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, openssl openssl,
, nix-util nix-util,
, nix-store nix-store,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -45,9 +46,12 @@ mkMesonLibrary (finalAttrs: {
echo ${version} > ../../.version echo ${version} > ../../.version
''; '';
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+14 -10
View File
@@ -1,13 +1,14 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util-c nix-util-c,
, nix-store nix-store,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -47,9 +48,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+15 -11
View File
@@ -1,15 +1,16 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util-test-support nix-util-test-support,
, nix-store nix-store,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -49,9 +50,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+48 -37
View File
@@ -1,21 +1,22 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-store nix-store,
, nix-store-c nix-store-c,
, nix-store-test-support nix-store-test-support,
, sqlite sqlite,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, filesetToSource filesetToSource,
}: }:
let let
@@ -62,32 +63,42 @@ mkMesonExecutable (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
passthru = { passthru = {
tests = { tests = {
run = let run =
# Some data is shared with the functional tests: they create it, let
# we consume it. # Some data is shared with the functional tests: they create it,
data = filesetToSource { # we consume it.
root = ../..; data = filesetToSource {
fileset = lib.fileset.unions [ root = ../..;
./data fileset = lib.fileset.unions [
../../tests/functional/derivation ./data
]; ../../tests/functional/derivation
}; ];
in runCommand "${finalAttrs.pname}-run" { };
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; in
} (lib.optionalString stdenv.hostPlatform.isWindows '' runCommand "${finalAttrs.pname}-run"
export HOME="$PWD/home-dir" {
mkdir -p "$HOME" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
'' + '' }
export _NIX_TEST_UNIT_DATA=${data + "/src/libstore-tests/data"} (
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} lib.optionalString stdenv.hostPlatform.isWindows ''
touch $out export HOME="$PWD/home-dir"
''); mkdir -p "$HOME"
''
+ ''
export _NIX_TEST_UNIT_DATA=${data + "/src/libstore-tests/data"}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+4 -4
View File
@@ -1158,7 +1158,7 @@ HookReply DerivationGoal::tryBuildHook()
throw; throw;
} }
}(); }();
if (handleJSONLogMessage(s, worker.act, worker.hook->activities, true)) if (handleJSONLogMessage(s, worker.act, worker.hook->activities, "the build hook", true))
; ;
else if (s.substr(0, 2) == "# ") { else if (s.substr(0, 2) == "# ") {
reply = s.substr(2); reply = s.substr(2);
@@ -1343,9 +1343,9 @@ void DerivationGoal::handleChildOutput(Descriptor fd, std::string_view data)
if (hook && fd == hook->fromHook.readSide.get()) { if (hook && fd == hook->fromHook.readSide.get()) {
for (auto c : data) for (auto c : data)
if (c == '\n') { if (c == '\n') {
auto json = parseJSONMessage(currentHookLine); auto json = parseJSONMessage(currentHookLine, "the derivation builder");
if (json) { if (json) {
auto s = handleJSONLogMessage(*json, worker.act, hook->activities, true); auto s = handleJSONLogMessage(*json, worker.act, hook->activities, "the derivation builder", true);
// ensure that logs from a builder using `ssh-ng://` as protocol // ensure that logs from a builder using `ssh-ng://` as protocol
// are also available to `nix log`. // are also available to `nix log`.
if (s && !isWrittenToLog && logSink) { if (s && !isWrittenToLog && logSink) {
@@ -1387,7 +1387,7 @@ void DerivationGoal::handleEOF(Descriptor fd)
void DerivationGoal::flushLine() void DerivationGoal::flushLine()
{ {
if (handleJSONLogMessage(currentLogLine, *act, builderActivities, false)) if (handleJSONLogMessage(currentLogLine, *act, builderActivities, "the derivation builder", false))
; ;
else { else {
+13 -1
View File
@@ -300,6 +300,14 @@ struct curlFileTransfer : public FileTransfer
return ((TransferItem *) userp)->readCallback(buffer, size, nitems); return ((TransferItem *) userp)->readCallback(buffer, size, nitems);
} }
#if !defined(_WIN32) && LIBCURL_VERSION_NUM >= 0x071000
static int cloexec_callback(void *, curl_socket_t curlfd, curlsocktype purpose) {
unix::closeOnExec(curlfd);
vomit("cloexec set for fd %i", curlfd);
return CURL_SOCKOPT_OK;
}
#endif
void init() void init()
{ {
if (!req) req = curl_easy_init(); if (!req) req = curl_easy_init();
@@ -359,6 +367,10 @@ struct curlFileTransfer : public FileTransfer
curl_easy_setopt(req, CURLOPT_SSL_VERIFYHOST, 0); curl_easy_setopt(req, CURLOPT_SSL_VERIFYHOST, 0);
} }
#if !defined(_WIN32) && LIBCURL_VERSION_NUM >= 0x071000
curl_easy_setopt(req, CURLOPT_SOCKOPTFUNCTION, cloexec_callback);
#endif
curl_easy_setopt(req, CURLOPT_CONNECTTIMEOUT, fileTransferSettings.connectTimeout.get()); curl_easy_setopt(req, CURLOPT_CONNECTTIMEOUT, fileTransferSettings.connectTimeout.get());
curl_easy_setopt(req, CURLOPT_LOW_SPEED_LIMIT, 1L); curl_easy_setopt(req, CURLOPT_LOW_SPEED_LIMIT, 1L);
@@ -767,7 +779,7 @@ struct curlFileTransfer : public FileTransfer
auto s3Res = s3Helper.getObject(bucketName, key); auto s3Res = s3Helper.getObject(bucketName, key);
FileTransferResult res; FileTransferResult res;
if (!s3Res.data) if (!s3Res.data)
throw FileTransferError(NotFound, "S3 object '%s' does not exist", request.uri); throw FileTransferError(NotFound, {}, "S3 object '%s' does not exist", request.uri);
res.data = std::move(*s3Res.data); res.data = std::move(*s3Res.data);
res.urls.push_back(request.uri); res.urls.push_back(request.uri);
callback(std::move(res)); callback(std::move(res));
+10 -4
View File
@@ -4,6 +4,7 @@
#include "finally.hh" #include "finally.hh"
#include "unix-domain-socket.hh" #include "unix-domain-socket.hh"
#include "signals.hh" #include "signals.hh"
#include "posix-fs-canonicalise.hh"
#if !defined(__linux__) #if !defined(__linux__)
// For shelling out to lsof // For shelling out to lsof
@@ -763,13 +764,18 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
} }
} }
} }
for (auto & path : topoSortPaths(visited)) { for (auto & path : topoSortPaths(visited)) {
if (!dead.insert(path).second) continue; if (!dead.insert(path).second) continue;
if (shouldDelete) { if (shouldDelete) {
invalidatePathChecked(path); try {
deleteFromStore(path.to_string()); invalidatePathChecked(path);
referrersCache.erase(path); deleteFromStore(path.to_string());
referrersCache.erase(path);
} catch (PathInUse &e) {
// If we end up here, it's likely a new occurence
// of https://github.com/NixOS/nix/issues/11923
printError("BUG: %s", e.what());
}
} }
} }
}; };
+6 -1
View File
@@ -181,7 +181,12 @@ LocalStore::LocalStore(
for (auto & perUserDir : {profilesDir + "/per-user", gcRootsDir + "/per-user"}) { for (auto & perUserDir : {profilesDir + "/per-user", gcRootsDir + "/per-user"}) {
createDirs(perUserDir); createDirs(perUserDir);
if (!readOnly) { if (!readOnly) {
if (chmod(perUserDir.c_str(), 0755) == -1) auto st = lstat(perUserDir);
// Skip chmod call if the directory already has the correct permissions (0755).
// This is to avoid failing when the executing user lacks permissions to change the directory's permissions
// even if it would be no-op.
if ((st.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO)) != 0755 && chmod(perUserDir.c_str(), 0755) == -1)
throw SysError("could not set permissions on '%s' to 755", perUserDir); throw SysError("could not set permissions on '%s' to 755", perUserDir);
} }
} }
+48 -40
View File
@@ -1,25 +1,26 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, unixtools unixtools,
, darwin darwin,
, nix-util nix-util,
, boost boost,
, curl curl,
, aws-sdk-cpp aws-sdk-cpp,
, libseccomp libseccomp,
, nlohmann_json nlohmann_json,
, sqlite sqlite,
, busybox-sandbox-shell ? null busybox-sandbox-shell ? null,
# Configuration Options # Configuration Options
, version version,
, embeddedSandboxShell ? stdenv.hostPlatform.isStatic embeddedSandboxShell ? stdenv.hostPlatform.isStatic,
}: }:
let let
@@ -48,19 +49,20 @@ mkMesonLibrary (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "sql") ./.) (fileset.fileFilter (file: file.hasExt "sql") ./.)
]; ];
nativeBuildInputs = nativeBuildInputs = lib.optional embeddedSandboxShell unixtools.hexdump;
lib.optional embeddedSandboxShell unixtools.hexdump;
buildInputs = [ buildInputs =
boost [
curl boost
sqlite curl
] ++ lib.optional stdenv.hostPlatform.isLinux libseccomp sqlite
]
++ lib.optional stdenv.hostPlatform.isLinux libseccomp
# There have been issues building these dependencies # There have been issues building these dependencies
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox ++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)) ++ lib.optional (
aws-sdk-cpp stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)
; ) aws-sdk-cpp;
propagatedBuildInputs = [ propagatedBuildInputs = [
nix-util nix-util
@@ -75,21 +77,27 @@ mkMesonLibrary (finalAttrs: {
echo ${version} > ../../.version echo ${version} > ../../.version
''; '';
mesonFlags = [ mesonFlags =
(lib.mesonEnable "seccomp-sandboxing" stdenv.hostPlatform.isLinux) [
(lib.mesonBool "embedded-sandbox-shell" embeddedSandboxShell) (lib.mesonEnable "seccomp-sandboxing" stdenv.hostPlatform.isLinux)
] ++ lib.optionals stdenv.hostPlatform.isLinux [ (lib.mesonBool "embedded-sandbox-shell" embeddedSandboxShell)
(lib.mesonOption "sandbox-shell" "${busybox-sandbox-shell}/bin/busybox") ]
]; ++ lib.optionals stdenv.hostPlatform.isLinux [
(lib.mesonOption "sandbox-shell" "${busybox-sandbox-shell}/bin/busybox")
];
env = { env =
# Needed for Meson to find Boost. {
# https://github.com/NixOS/nixpkgs/issues/86131. # Needed for Meson to find Boost.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include"; # https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib"; BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
} // lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
LDFLAGS = "-fuse-ld=gold"; }
}; // lib.optionalAttrs
(stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+17 -3
View File
@@ -534,14 +534,27 @@ void RemoteStore::addToStore(const ValidPathInfo & info, Source & source,
void RemoteStore::addMultipleToStore( void RemoteStore::addMultipleToStore(
PathsSource & pathsToCopy, PathsSource && pathsToCopy,
Activity & act, Activity & act,
RepairFlag repair, RepairFlag repair,
CheckSigsFlag checkSigs) CheckSigsFlag checkSigs)
{ {
// `addMultipleToStore` is single threaded
size_t bytesExpected = 0;
for (auto & [pathInfo, _] : pathsToCopy) {
bytesExpected += pathInfo.narSize;
}
act.setExpected(actCopyPath, bytesExpected);
auto source = sinkToSource([&](Sink & sink) { auto source = sinkToSource([&](Sink & sink) {
sink << pathsToCopy.size(); size_t nrTotal = pathsToCopy.size();
for (auto & [pathInfo, pathSource] : pathsToCopy) { sink << nrTotal;
// Reverse, so we can release memory at the original start
std::reverse(pathsToCopy.begin(), pathsToCopy.end());
while (!pathsToCopy.empty()) {
act.progress(nrTotal - pathsToCopy.size(), nrTotal, size_t(1), size_t(0));
auto & [pathInfo, pathSource] = pathsToCopy.back();
WorkerProto::Serialise<ValidPathInfo>::write(*this, WorkerProto::Serialise<ValidPathInfo>::write(*this,
WorkerProto::WriteConn { WorkerProto::WriteConn {
.to = sink, .to = sink,
@@ -549,6 +562,7 @@ void RemoteStore::addMultipleToStore(
}, },
pathInfo); pathInfo);
pathSource->drainInto(sink); pathSource->drainInto(sink);
pathsToCopy.pop_back();
} }
}); });
+1 -1
View File
@@ -102,7 +102,7 @@ public:
CheckSigsFlag checkSigs) override; CheckSigsFlag checkSigs) override;
void addMultipleToStore( void addMultipleToStore(
PathsSource & pathsToCopy, PathsSource && pathsToCopy,
Activity & act, Activity & act,
RepairFlag repair, RepairFlag repair,
CheckSigsFlag checkSigs) override; CheckSigsFlag checkSigs) override;
+10 -16
View File
@@ -223,7 +223,7 @@ StorePath Store::addToStore(
} }
void Store::addMultipleToStore( void Store::addMultipleToStore(
PathsSource & pathsToCopy, PathsSource && pathsToCopy,
Activity & act, Activity & act,
RepairFlag repair, RepairFlag repair,
CheckSigsFlag checkSigs) CheckSigsFlag checkSigs)
@@ -242,13 +242,11 @@ void Store::addMultipleToStore(
storePathsToAdd.insert(thingToAdd.first.path); storePathsToAdd.insert(thingToAdd.first.path);
} }
auto showProgress = [&]() { auto showProgress = [&, nrTotal = pathsToCopy.size()]() {
act.progress(nrDone, pathsToCopy.size(), nrRunning, nrFailed); act.progress(nrDone, nrTotal, nrRunning, nrFailed);
}; };
ThreadPool pool; processGraph<StorePath>(
processGraph<StorePath>(pool,
storePathsToAdd, storePathsToAdd,
[&](const StorePath & path) { [&](const StorePath & path) {
@@ -1028,12 +1026,10 @@ std::map<StorePath, StorePath> copyPaths(
} }
auto pathsMap = copyPaths(srcStore, dstStore, storePaths, repair, checkSigs, substitute); auto pathsMap = copyPaths(srcStore, dstStore, storePaths, repair, checkSigs, substitute);
ThreadPool pool;
try { try {
// Copy the realisation closure // Copy the realisation closure
processGraph<Realisation>( processGraph<Realisation>(
pool, Realisation::closure(srcStore, toplevelRealisations), Realisation::closure(srcStore, toplevelRealisations),
[&](const Realisation & current) -> std::set<Realisation> { [&](const Realisation & current) -> std::set<Realisation> {
std::set<Realisation> children; std::set<Realisation> children;
for (const auto & [drvOutput, _] : current.dependentRealisations) { for (const auto & [drvOutput, _] : current.dependentRealisations) {
@@ -1108,9 +1104,6 @@ std::map<StorePath, StorePath> copyPaths(
return storePathForDst; return storePathForDst;
}; };
// total is accessed by each copy, which are each handled in separate threads
std::atomic<uint64_t> total = 0;
for (auto & missingPath : sortedMissing) { for (auto & missingPath : sortedMissing) {
auto info = srcStore.queryPathInfo(missingPath); auto info = srcStore.queryPathInfo(missingPath);
@@ -1120,9 +1113,10 @@ std::map<StorePath, StorePath> copyPaths(
ValidPathInfo infoForDst = *info; ValidPathInfo infoForDst = *info;
infoForDst.path = storePathForDst; infoForDst.path = storePathForDst;
auto source = sinkToSource([&](Sink & sink) { auto source = sinkToSource([&, narSize = info->narSize](Sink & sink) {
// We can reasonably assume that the copy will happen whenever we // We can reasonably assume that the copy will happen whenever we
// read the path, so log something about that at that point // read the path, so log something about that at that point
uint64_t total = 0;
auto srcUri = srcStore.getUri(); auto srcUri = srcStore.getUri();
auto dstUri = dstStore.getUri(); auto dstUri = dstStore.getUri();
auto storePathS = srcStore.printStorePath(missingPath); auto storePathS = srcStore.printStorePath(missingPath);
@@ -1133,16 +1127,16 @@ std::map<StorePath, StorePath> copyPaths(
LambdaSink progressSink([&](std::string_view data) { LambdaSink progressSink([&](std::string_view data) {
total += data.size(); total += data.size();
act.progress(total, info->narSize); act.progress(total, narSize);
}); });
TeeSink tee { sink, progressSink }; TeeSink tee { sink, progressSink };
srcStore.narFromPath(missingPath, tee); srcStore.narFromPath(missingPath, tee);
}); });
pathsToCopy.push_back(std::pair{infoForDst, std::move(source)}); pathsToCopy.emplace_back(std::move(infoForDst), std::move(source));
} }
dstStore.addMultipleToStore(pathsToCopy, act, repair, checkSigs); dstStore.addMultipleToStore(std::move(pathsToCopy), act, repair, checkSigs);
return pathsMap; return pathsMap;
} }
+1 -1
View File
@@ -425,7 +425,7 @@ public:
CheckSigsFlag checkSigs = CheckSigs); CheckSigsFlag checkSigs = CheckSigs);
virtual void addMultipleToStore( virtual void addMultipleToStore(
PathsSource & pathsToCopy, PathsSource && pathsToCopy,
Activity & act, Activity & act,
RepairFlag repair = NoRepair, RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs); CheckSigsFlag checkSigs = CheckSigs);
@@ -2149,7 +2149,18 @@ void LocalDerivationGoal::runChild()
without file-write* allowed, access() incorrectly returns EPERM without file-write* allowed, access() incorrectly returns EPERM
*/ */
sandboxProfile += "(allow file-read* file-write* process-exec\n"; sandboxProfile += "(allow file-read* file-write* process-exec\n";
// We create multiple allow lists, to avoid exceeding a limit in the darwin sandbox interpreter.
// See https://github.com/NixOS/nix/issues/4119
// We split our allow groups approximately at half the actual limit, 1 << 16
const int breakpoint = sandboxProfile.length() + (1 << 14);
for (auto & i : pathsInChroot) { for (auto & i : pathsInChroot) {
if (sandboxProfile.length() >= breakpoint) {
debug("Sandbox break: %d %d", sandboxProfile.length(), breakpoint);
sandboxProfile += ")\n(allow file-read* file-write* process-exec\n";
}
if (i.first != i.second.source) if (i.first != i.second.source)
throw Error( throw Error(
"can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin", "can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin",
@@ -2657,10 +2668,14 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
wanted.to_string(HashFormat::SRI, true), wanted.to_string(HashFormat::SRI, true),
got.to_string(HashFormat::SRI, true))); got.to_string(HashFormat::SRI, true)));
} }
if (!newInfo0.references.empty()) if (!newInfo0.references.empty()) {
auto numViolations = newInfo.references.size();
delayedException = std::make_exception_ptr( delayedException = std::make_exception_ptr(
BuildError("illegal path references in fixed-output derivation '%s'", BuildError("fixed-output derivations must not reference store paths: '%s' references %d distinct paths, e.g. '%s'",
worker.store.printStorePath(drvPath))); worker.store.printStorePath(drvPath),
numViolations,
worker.store.printStorePath(*newInfo.references.begin())));
}
return newInfo0; return newInfo0;
}, },
@@ -2927,8 +2942,12 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
spec.insert(worker.store.parseStorePath(i)); spec.insert(worker.store.parseStorePath(i));
else if (auto output = get(outputs, i)) else if (auto output = get(outputs, i))
spec.insert(output->path); spec.insert(output->path);
else else {
throw BuildError("derivation contains an illegal reference specifier '%s'", i); std::string outputsListing = concatMapStringsSep(", ", outputs, [](auto & o) { return o.first; });
throw BuildError("derivation '%s' output check for '%s' contains an illegal reference specifier '%s',"
" expected store path or output name (one of [%s])",
worker.store.printStorePath(drvPath), outputName, i, outputsListing);
}
} }
auto used = recursive auto used = recursive
+13 -9
View File
@@ -1,12 +1,13 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -45,9 +46,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+14 -10
View File
@@ -1,14 +1,15 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -47,9 +48,12 @@ mkMesonLibrary (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+14
View File
@@ -261,4 +261,18 @@ TEST(pathExists, bogusPathDoesNotExist)
{ {
ASSERT_FALSE(pathExists("/schnitzel/darmstadt/pommes")); ASSERT_FALSE(pathExists("/schnitzel/darmstadt/pommes"));
} }
/* ----------------------------------------------------------------------------
* makeParentCanonical
* --------------------------------------------------------------------------*/
TEST(makeParentCanonical, noParent)
{
ASSERT_EQ(makeParentCanonical("file"), absPath(std::filesystem::path("file")));
}
TEST(makeParentCanonical, root)
{
ASSERT_EQ(makeParentCanonical("/"), "/");
}
} }
+18
View File
@@ -0,0 +1,18 @@
#include "util.hh"
#include "monitor-fd.hh"
#include <sys/file.h>
#include <gtest/gtest.h>
namespace nix {
TEST(MonitorFdHup, shouldNotBlock)
{
Pipe p;
p.create();
{
// when monitor gets destroyed it should cancel the
// background thread and do not block
MonitorFdHup monitor(p.readSide.get());
}
}
}
+35 -25
View File
@@ -1,19 +1,20 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-util nix-util,
, nix-util-c nix-util-c,
, nix-util-test-support nix-util-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -55,22 +56,31 @@ mkMesonExecutable (finalAttrs: {
mesonFlags = [ mesonFlags = [
]; ];
env = lib.optionalAttrs (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")) { env =
LDFLAGS = "-fuse-ld=gold"; lib.optionalAttrs
}; (stdenv.isLinux && !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux"))
{
LDFLAGS = "-fuse-ld=gold";
};
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+36
View File
@@ -81,6 +81,42 @@ TEST(concatStringsSep, buildSingleString)
ASSERT_EQ(concatStringsSep(",", strings), "this"); ASSERT_EQ(concatStringsSep(",", strings), "this");
} }
TEST(concatMapStringsSep, empty)
{
Strings strings;
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "");
}
TEST(concatMapStringsSep, justOne)
{
Strings strings;
strings.push_back("this");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this");
}
TEST(concatMapStringsSep, two)
{
Strings strings;
strings.push_back("this");
strings.push_back("that");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this,that");
}
TEST(concatMapStringsSep, map)
{
std::map<std::string, std::string> strings;
strings["this"] = "that";
strings["1"] = "one";
ASSERT_EQ(
concatMapStringsSep(
", ", strings, [](const std::pair<std::string, std::string> & s) { return s.first + " -> " + s.second; }),
"1 -> one, this -> that");
}
/* ---------------------------------------------------------------------------- /* ----------------------------------------------------------------------------
* dropEmptyInitThenConcatStringsSep * dropEmptyInitThenConcatStringsSep
* --------------------------------------------------------------------------*/ * --------------------------------------------------------------------------*/
Loaded 100 of 428 files, more files were not shown because too many files have changed in this diff. Show more