Compare commits

...
Author SHA1 Message Date
Eelco Dolstra 24fbae7755 Bump version 2025-06-24 19:26:27 +02:00
Eelco Dolstra 380cefacf0 Update the release script
(cherry picked from commit f472be29ab)
2025-06-24 19:19:15 +02:00
Eelco Dolstra 844ad565a8 Remove meta.maintainers
Some of the maintainer attribute names got changed in nixos-unstable
(e.g. "edolstra" is now "eelco") but we want this flake to work on
nixos-24.11. So just get rid of them.

(cherry picked from commit 93d8f62057)
(cherry picked from commit b3b4fc21da)
2025-06-24 18:00:01 +02:00
Eelco Dolstra 39b60f2e60 Merge remote-tracking branch 'cve/fod-cves-2.26' into 2.26-maintenance 2025-06-24 16:07:50 +02:00
mergify[bot] bbe7ffd4ca Merge pull request #13390 from NixOS/mergify/bp/2.26-maintenance/pr-13348
Make the repl test more robust (backport #13348)
2025-06-22 19:30:30 +00:00
Eelco Dolstra 0f1a5c519e Make the repl test more robust
Seen in https://github.com/DeterminateSystems/nix-src/actions/runs/15590867877/job/43909540271:

  nix-functional-tests> grep: repl_output: No such file or directory
  nix-functional-tests> +(repl.sh:174) cat repl_output

This is because there is a small possibility that the `nix repl` child
process hasn't created `repl_output` yet. So make sure it exists.

(cherry picked from commit 9eb46e9cc0)
2025-06-22 19:08:37 +00:00
Eelco Dolstra 0da3b18520 Fixes for GHSA-g948-229j-48j3
Squashed commit of the following:

commit 04fff3a637d455cbb1d75937a235950e43008db9
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:30:32 2025 +0200

    Chown structured attr files safely

commit 5417ad445e414c649d0cfc71a05661c7bf8f3ef5
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 12:14:04 2025 +0200

    Replace 'bool sync' with an enum for clarity

    And drop writeFileAndSync().

commit 7ae0141f328d8e8e1094be24665789c05f974ba6
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:35:28 2025 +0200

    Drop guessOrInventPathFromFD()

    No need to do hacky stuff like that when we already know the original path.

commit 45b05098bd019da7c57cd4227a89bfd0fa65bb08
Author: Eelco Dolstra <edolstra@gmail.com>
Date:   Thu Jun 12 11:15:58 2025 +0200

    Tweak comment

commit 0af15b31209d1b7ec8addfae9a1a6b60d8f35848
Author: Raito Bezarius <raito@lix.systems>
Date:   Thu Mar 27 12:22:26 2025 +0100

    libstore: ensure that temporary directory is always 0o000 before deletion

    In the case the deletion fails, we should ensure that the temporary
    directory cannot be used for nefarious purposes.

    Change-Id: I498a2dd0999a74195d13642f44a5de1e69d46120
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 2c20fa37b15cfa03ac6a1a6a47cdb2ed66c0827e
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 12:42:55 2025 +0100

    libutil: ensure that `_deletePath` does NOT use absolute paths with dirfds

    When calling `_deletePath` with a parent file descriptor, `openat` is
    made effective by using relative paths to the directory file descriptor.

    To avoid the problem, the signature is changed to resist misuse with an
    assert in the prologue of the function.

    Change-Id: I6b3fc766bad2afe54dc27d47d1df3873e188de96
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit d3c370bbcae48bb825ce19fd0f73bb4eefd2c9ea
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:07:47 2025 +0100

    libstore: ensure that `passAsFile` is created in the original temp dir

    This ensures that `passAsFile` data is created inside the expected
    temporary build directory by `openat()` from the parent directory file
    descriptor.

    This avoids a TOCTOU which is part of the attack chain of CVE-????.

    Change-Id: Ie5273446c4a19403088d0389ae8e3f473af8879a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 45d3598724f932d024ef6bc2ffb00c1bb90e6018
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:06:03 2025 +0100

    libutil: writeFile variant for file descriptors

    `writeFile` lose its `sync` boolean flag to make things simpler.

    A new `writeFileAndSync` function is created and all call sites are
    converted to it.

    Change-Id: Ib871a5283a9c047db1e4fe48a241506e4aab9192
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 732bd9b98cabf4aaf95a01fd318923de303f9996
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:05:34 2025 +0100

    libstore: chown to builder variant for file descriptors

    We use it immediately for the build temporary directory.

    Change-Id: I180193c63a2b98721f5fb8e542c4e39c099bb947
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit 962c65f8dcd5570dd92c72370a862c7b38942e0d
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:59 2025 +0100

    libstore: open build directory as a dirfd as well

    We now keep around a proper AutoCloseFD around the temporary directory
    which we plan to use for openat operations and avoiding the build
    directory being swapped out while we are doing something else.

    Change-Id: I18d387b0f123ebf2d20c6405cd47ebadc5505f2a
    Signed-off-by: Raito Bezarius <raito@lix.systems>

commit c9b42462b75b5a37ee6564c2b53cff186c8323da
Author: Raito Bezarius <raito@lix.systems>
Date:   Wed Mar 26 01:04:12 2025 +0100

    libutil: guess or invent a path from file descriptors

    This is useful for certain error recovery paths (no pun intended) that
    does not thread through the original path name.

    Change-Id: I2d800740cb4f9912e64c923120d3f977c58ccb7e
    Signed-off-by: Raito Bezarius <raito@lix.systems>
2025-06-19 16:41:46 +02:00
mergify[bot] 0bf8d076f5 Merge pull request #12980 from NixOS/mergify/bp/2.26-maintenance/pr-12979
Actually ignore system/user registries during locking (backport #12979)
2025-04-09 16:50:18 +00:00
Eelco Dolstra 43a3f55cda Actually ignore system/user registries during locking
Something went wrong in #12068 so this didn't work. Also added a test.

(cherry picked from commit 77d4316353)
2025-04-09 16:33:37 +00:00
mergify[bot] ec00ce9fbd Merge pull request #12953 from NixOS/mergify/bp/2.26-maintenance/pr-12951
tests/functional/repl: fix race condition (backport #12951)
2025-04-07 12:41:17 +00:00
Jörg Thalheim 6ce0e32766 tests/functional/repl: fix race condition
the sleep 1 is not enough in some circumstances. Switching to a fifo
helps.

(cherry picked from commit 1de951d31d)
2025-04-07 12:19:16 +00:00
mergify[bot] f8620b451b Merge pull request #12947 from NixOS/mergify/bp/2.26-maintenance/pr-12944
maintainers/release-notes: Let it fail (backport #12944)
2025-04-07 09:11:17 +00:00
Robert Hensing 4edcfa21f1 maintainers/release-notes: Let it fail
Fail when a command fails.

Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.

(cherry picked from commit da36c34db7)
2025-04-07 08:52:33 +00:00
Jörg Thalheim 2244aad4c0 Merge pull request #12927 from NixOS/mergify/bp/2.26-maintenance/pr-12911
create cache entry for paths already in the nix store (backport #12911)
2025-04-04 16:33:30 +02:00
Jörg Thalheim 720cdd59b9 create cache entry for paths already in the nix store
This allows path:/nix/store/* paths to not be copied twice to the nix
store.

(cherry picked from commit 61c6210dbf)
2025-04-04 16:15:12 +02:00
mergify[bot] 8e65d53de8 Merge pull request #12904 from NixOS/mergify/bp/2.26-maintenance/pr-12886
symlink_exists: wrap exceptions into nix exception (backport #12886)
2025-04-02 23:35:39 +00:00
Jörg Thalheim f17f3a22c6 symlink_exists: wrap exceptions into nix exception
(cherry picked from commit 779687854f)
2025-04-02 23:13:57 +00:00
mergify[bot] 06add7e551 Merge pull request #12864 from NixOS/mergify/bp/2.26-maintenance/pr-12810
Packaging improvements from Nixpkgs (backport #12810)
2025-04-01 18:19:38 +00:00
Robert Hensing 6b40d83ec9 packaging: finalAttrs.doCheck -> finalAttrs.finalPackage.doCheck
This includes the logic that disables checks on cross appropriately.

Co-authored-by: Peder Bergebakken Sundt <pbsds@hotmail.com>
(cherry picked from commit 27d71b21fc)
2025-04-01 17:49:43 +00:00
Robert Hensing 4767974d4f packaging: Various improvements
Co-authored-by: Mic92 <Mic92@users.noreply.github.com>
(cherry picked from commit 1172e49a3a)
2025-04-01 17:49:43 +00:00
mergify[bot] 4f85ceebbd Merge pull request #12855 from NixOS/mergify/bp/2.26-maintenance/pr-12765
Add various clang tidy fixes (backport #12765)
2025-04-01 14:17:26 +00:00
Jörg Thalheim c4e5df400d libstore/daemon: make sure monitor is not considered "unused"
(cherry picked from commit 5c3682d7a1)
2025-04-01 13:37:39 +00:00
Jörg Thalheim 11e7d82fc8 test/ca-fd-leak: fix clang-tidy lints
(cherry picked from commit b050db951b)
2025-04-01 13:37:39 +00:00
Jörg Thalheim ab7a1df6ed git/getStringUntil: fix uninitialized stack variable
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.

(cherry picked from commit 7e540059a3)
2025-04-01 13:37:39 +00:00
Jörg Thalheim 0178975340 libstore/local-store: fix linting warning about unused variable
(cherry picked from commit 05082ea1c5)
2025-04-01 13:37:39 +00:00
mergify[bot] 91349c465e Merge pull request #12842 from NixOS/mergify/bp/2.26-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:49:05 +00:00
Thomas Miedema 318929dfa9 nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:47:30 +00:00
John Ericson 14203d1e78 Merge pull request #12813 from NixOS/revert-12779-mergify/bp/2.26-maintenance/pr-12773
Revert "Unexpose config headers (low hanging fruit only) (backport #1…
2025-03-31 12:25:56 -04:00
John Ericson ce209f1cd7 Revert "Unexpose config headers (low hanging fruit only) (backport #12773)" 2025-03-31 11:57:46 -04:00
John Ericson 3e59cc0552 Merge pull request #12796 from NixOS/mergify/bp/2.26-maintenance/pr-12794
Fix windows build (backport #12794)
2025-03-28 21:48:14 -04:00
John Ericson 003584e59e Fix windows build
PR #12767 accidentally broke it.

(cherry picked from commit 99041b4d84)
2025-03-29 01:05:21 +00:00
mergify[bot] 004187e8cd Merge pull request #12792 from NixOS/mergify/bp/2.26-maintenance/pr-12789
nix daemon: Don't open the store (backport #12789)
2025-03-29 00:16:51 +00:00
Eelco Dolstra e6aad00570 nix daemon: Don't open the store
This makes it behave the same as nix-daemon. Opening the store in the
parent can cause a SIGBUS in libsqlite in the child:

  #0  0x00007f141cf6f789 in __memset_avx2_unaligned_erms () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
  #1  0x00007f141c322fe8 in walIndexAppend () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #2  0x00007f141c3711a2 in pagerWalFrames () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #3  0x00007f141c38317e in sqlite3PagerCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #4  0x00007f141c383555 in sqlite3BtreeCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #5  0x00007f141c384797 in sqlite3VdbeHalt () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #6  0x00007f141c3b8f60 in sqlite3VdbeExec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #7  0x00007f141c3bbfef in sqlite3_step () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #8  0x00007f141c3bd0e5 in sqlite3_exec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #9  0x00007f141da140e0 in nix::SQLiteTxn::commit() () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
  #10 0x00007f141d9ce69c in nix::LocalStore::registerValidPaths(std::map<nix::StorePath, nix::ValidPathInfo, std::less<nix::StorePath>, std::allocator<std::pair<nix::StorePath const, nix::ValidPathInfo> > > const&)::{lambda()#1}::operator()() const () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so

(cherry picked from commit 9590167290)
2025-03-29 00:02:39 +00:00
mergify[bot] 20e67b8bb6 Merge pull request #12784 from NixOS/mergify/bp/2.26-maintenance/pr-12758
libutil: Fix error message (backport #12758)
2025-03-28 19:18:47 +00:00
mergify[bot] e393b2c844 Merge pull request #12779 from NixOS/mergify/bp/2.26-maintenance/pr-12773
Unexpose config headers (low hanging fruit only) (backport #12773)
2025-03-28 18:43:43 +00:00
mergify[bot] 352e27772f Merge pull request #12777 from NixOS/mergify/bp/2.26-maintenance/pr-12767
use createDirs consistently everywhere (backport #12767)
2025-03-28 18:04:06 +00:00
Robert Hensing 33580356fb libutil: Fix error message
I encountered this with a misconfigured libutil. I doubt that a
non-lutimes config is viable, because tests were failing.

(cherry picked from commit 1cffcd91a9)
2025-03-28 17:08:19 +00:00
Robert Hensing 8f6392b777 Unexpose config headers (low hanging fruit only)
- Some headers were completely redundant and have been removed.
- Other headers have been turned private.
- Unnecessary meson.build code has been removed.
- libutil-tests now has a private config header, where previously
  it had none. This removes the need to expose a package version
  macro publicly.

(cherry picked from commit b86a76044e)
2025-03-28 16:44:26 +00:00
mergify[bot] 75710a809d Merge pull request #12775 from NixOS/mergify/bp/2.26-maintenance/pr-12772
Light packaging cleanups (backport #12772)
2025-03-28 16:43:09 +00:00
Jörg Thalheim d1fe15f43f use createDirs consistently everywhere
(cherry picked from commit a8217f2642)
2025-03-28 15:56:02 +00:00
John Ericson f6f89041c0 Add a withAWS flag to libstore
Nixpkgs wants this, at least.

(cherry picked from commit e4c571c2f1)
2025-03-28 15:54:57 +00:00
John Ericson 384415ddc0 Link the right issue about the bad AWS pkg-config
It is https://github.com/aws/aws-sdk-cpp/issues/2673

(cherry picked from commit fb9c9ee35a)
2025-03-28 15:54:57 +00:00
John Ericson 3d3b99e596 Remove boost env vars
https://github.com/NixOS/nixpkgs/issues/86131 is now fixed!
(cherry picked from commit 459fb59493)
2025-03-28 15:54:57 +00:00
John Ericson 45ff01a2d5 Fix a bunch of missing meson boilerplate
These other libraries need this too

(cherry picked from commit ffdce51cd5)
2025-03-28 15:54:56 +00:00
mergify[bot] ceec04f1e2 Merge pull request #12754 from NixOS/mergify/bp/2.26-maintenance/pr-12105
local-derivation-goal: improve "illegal reference" error (backport #12105)
2025-03-25 16:12:27 +00:00
Sergei Trofimovich fa33df1e76 tests/functional/check-refs.nix: format newly added test
(cherry picked from commit 4d72e0f73b)
2025-03-25 15:48:25 +00:00
Sergei Trofimovich 2fbebb6574 tests/functional/check-refs.sh: guard test12 against too old nix daemon
Otherwise without the change the test fails on nix-2.26 as:

    error: derivation contains an illegal reference specifier 'dev'

Note: the error message does not match intended change.
(cherry picked from commit 1e7c7244cf)
2025-03-25 15:48:25 +00:00
Robert Hensing 6e93311426 test illegal reference specifier error message
(cherry picked from commit f4def47c89)
2025-03-25 15:48:25 +00:00
Robert Hensing 7b3a78dbab checkRefs: use concatMapStringsSep
(cherry picked from commit 2b4d461c14)
2025-03-25 15:48:24 +00:00
Robert Hensing 0ddb8e21fe nix-util: Use small_vector in concatMapStringsSep
(cherry picked from commit 32898dc46a)
2025-03-25 15:48:24 +00:00
Robert Hensing beac7720a3 nix-util: Add concatMapStrings
(cherry picked from commit f3dbaa3f54)
2025-03-25 15:48:24 +00:00
Sergei Trofimovich 41a31db7c5 local-derivation-goal: improve "illegal reference" error
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.

Today's `nixpkgs` example:

Before the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation contains an illegal reference specifier 'man'

After the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])

(cherry picked from commit bbdc3197a9)
2025-03-25 15:48:23 +00:00
Jörg Thalheim ec2eedad26 Merge pull request #12742 from NixOS/mergify/bp/2.26-maintenance/pr-12734
packaging/everything.nix: make sure doc and manpage outputs are symlinks (backport #12734)
2025-03-24 14:42:17 +01:00
Kirens bdcefc6a64 make sure doc and manpage outputs are symlinks
Part of https://github.com/NixOS/nixpkgs/pull/392549
The doc and manpage fix already happend in 0ddfbc5939

(cherry picked from commit 15dfeb9182)
2025-03-24 13:09:44 +00:00
mergify[bot] d08afe373f Merge pull request #12739 from NixOS/mergify/bp/2.26-maintenance/pr-12736
`MonitorFdHup::~MonitorFdHup`: use proper close method instead of lib… (backport #12736)
2025-03-24 12:47:58 +00:00
Jörg Thalheim e29429488a MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)
2025-03-24 12:16:23 +00:00
John Ericson 65873d1e10 Merge pull request #12732 from NixOS/mergify/bp/2.26-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-23 20:23:50 -04:00
John Ericson b6a94d4a1a MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-23 23:54:09 +00:00
Jade Lovelace ff62e83989 daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-23 23:54:09 +00:00
Félix Baylac Jacqué fabf4a55c3 MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-23 23:54:09 +00:00
John Ericson b19db6f479 MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-23 23:54:09 +00:00
John Ericson e51dbf42bb MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-23 23:54:08 +00:00
Jörg Thalheim 68d8f09bd4 MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-23 23:54:08 +00:00
John Ericson a04bc3ed7a monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-23 23:54:08 +00:00
mergify[bot] b369ccf1d7 Merge pull request #12725 from NixOS/mergify/bp/2.26-maintenance/pr-12723
packaging/everything.nix: Fix doc and man outputs (backport #12723)
2025-03-23 11:59:53 +00:00
Sergei Zimmerman 10ef954f8a packaging/everything.nix: Fix doc and man outputs
We want the $doc, $man outputs to be symlinks pointing to nix-manual and
nix-manual.man. Creating the directories first makes the `ln` command
produce symlink $doc/${nix-manual} instead.

```
$file /nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man

/nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man:
  symbolic link to /nix/store/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
```

This is the reason `nix-env --help` is once again broken on 2.26/2.27/master
after 4108529.

(cherry picked from commit 0ddfbc5939)
2025-03-23 11:54:09 +00:00
mergify[bot] bcbc57a77e Merge pull request #12712 from NixOS/mergify/bp/2.26-maintenance/pr-12636
port crash-handler from lix to nix (backport #12636)
2025-03-21 15:44:28 +00:00
Jade Lovelace 045cff79b2 port crash-handler from lix to nix
It was first introduced in https://git.lix.systems/lix-project/lix/commit/19e0ce2c03d8e0baa16998b086665664c420c1df

In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.

Sample output:

Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]:  0# 0x000000000076876A in nix
                                   1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
                                   9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
                                  10# 0x000000000087A563 in nix
                                  11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  13# 0x00000000006F4DF5 in nix

Co-authored-by: eldritch horrors <pennae@lix.systems>
(cherry picked from commit 163f94412a)
2025-03-21 15:30:03 +00:00
mergify[bot] 3c937cf62c Merge pull request #12710 from NixOS/mergify/bp/2.26-maintenance/pr-12701
stack overflow is EvalBaseError (backport #12701)
2025-03-21 14:35:07 +00:00
Shahar "Dawn" Or 162081618f stack overflow is EvalBaseError
(cherry picked from commit 23c7a45a05)
2025-03-21 14:12:36 +00:00
mergify[bot] 70cb4b40e4 Merge pull request #12695 from NixOS/mergify/bp/2.26-maintenance/pr-12682
libfetchers: fix double quote in path printed in logger (backport #12682)
2025-03-19 22:42:55 +00:00
Jörg Thalheim 3176e024f6 libfetchers: fix double quote in path printed in logger
(cherry picked from commit 314e9fbeda)
2025-03-19 22:24:28 +00:00
mergify[bot] 7430d002bc Merge pull request #12685 from NixOS/mergify/bp/2.26-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 21:02:17 +00:00
Pierre-Etienne Meunier 2868a471d2 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:21:40 +00:00
mergify[bot] 74b786bf9a Merge pull request #12654 from NixOS/mergify/bp/2.26-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-14 20:40:50 +00:00
Sergei Zimmerman 9d54f34ed0 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-14 13:22:45 +00:00
Sergei Zimmerman bee21af2a1 libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 5a3b0d9c7a libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-14 13:22:44 +00:00
Sergei Zimmerman 6384a65baf {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:44 +00:00
Eelco Dolstra 3bd0406fa4 Bump version 2025-03-05 19:34:04 +01:00
mergify[bot] 42fc03dd1f Merge pull request #12601 from NixOS/mergify/bp/2.26-maintenance/pr-12600
packaging: Typo in setVersionLayer / preConfigure (backport #12600)
2025-03-04 17:08:09 +00:00
Robert Hensing 87bf338612 packaging: Typo in setVersionLayer / preConfigure
Apparently dead code in our use case, but good to keep nonetheless.
Credit: ztzg in https://github.com/NixOS/nix/pull/12498#pullrequestreview-2658031853

(cherry picked from commit dcaea8cb1c)
2025-03-04 16:46:15 +00:00
mergify[bot] cd3688101d Merge pull request #12585 from NixOS/mergify/bp/2.26-maintenance/pr-12582
packaging/everything.nix: Use a multi-output derivation (backport #12582)
2025-03-01 10:41:47 +00:00
Robert Hensing d8606f96ee packaging/everything.nix: Use a multi-output derivation
This should fix a few packaging regressions.

`dev` also includes a merged `includes/`, which may be helpful until
inter-component includes are fixed properly.

(cherry picked from commit 41085295ab)
2025-03-01 10:36:41 +00:00
mergify[bot] 1dda07eef2 Merge pull request #12562 from NixOS/mergify/bp/2.26-maintenance/pr-12559
nix flake archive: Recurse into relative path inputs (backport #12559)
2025-02-24 22:18:03 +00:00
Eelco Dolstra 827f760ad7 Fix test
We didn't backport `nix flake prefetch --out-link`.
2025-02-24 23:00:07 +01:00
Eelco Dolstra 91508de315 nix flake archive: Recurse into relative path inputs
We can't ignore them entirely, since we do want to archive their
transitive inputs.

Fixes #12438.

(cherry picked from commit 14c9755462)
2025-02-24 21:30:45 +00:00
605b2371f9 windows: fix compilation after recent changes (backport #12495) (#12561)
* windows: fix compilation after recent changes

Specifically last few week's merges involving legacy SSH options and
dynamic derivations.

(cherry picked from commit e0617d2545)

# Conflicts:
#	src/libstore/build/derivation-creation-and-realisation-goal.hh

* Remove unneeded

* Remove unneeded

---------

Co-authored-by: Brian McKenna <brian@brianmckenna.org>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2025-02-24 21:30:35 +00:00
Eelco Dolstra 6d9724a2b3 Merge pull request #12560 from NixOS/mergify/bp/2.26-maintenance/pr-12557
packaging: Use correct stdenv for x86_64-darwin (backport #12557)
2025-02-24 22:04:52 +01:00
Robert Hensing 9e87a58314 packaging: Use correct stdenv for x86_64-darwin
(cherry picked from commit 0772c2e3ab)
2025-02-24 20:55:05 +00:00
mergify[bot] 31a29e66ea Merge pull request #12554 from NixOS/mergify/bp/2.26-maintenance/pr-12553
nix flake archive: Skip relative path inputs (backport #12553)
2025-02-24 15:03:38 +00:00
Eelco Dolstra d670380bd9 nix flake archive: Skip relative path inputs
Fixes #12438.

(cherry picked from commit b4dfeafed5)
2025-02-24 14:49:11 +00:00
mergify[bot] 6d3ee99463 Merge pull request #12545 from NixOS/mergify/bp/2.26-maintenance/pr-12543
Fix perl store bindings (backport #12543)
2025-02-22 18:34:00 +00:00
mergify[bot] a30588365b Merge pull request #12547 from NixOS/mergify/bp/2.26-maintenance/pr-12510
libstore: fix expected bytes in progress bar (backport #12510)
2025-02-21 19:34:30 +00:00
Ivan Trubach c69d5af105 libstore: fix expected bytes in progress bar
(cherry picked from commit eb73bfcf73)
2025-02-21 19:12:02 +00:00
Fabian Möller cd1935468d Fix perl store bindings
When #9863 converted the `Nix::Store` free functions into member functions, the
implicit `this` argument was not accounted for when iterating over the variable
number of arguments in some functions.

(cherry picked from commit 5cf9e18167)
2025-02-21 14:47:37 +00:00
mergify[bot] 0d039d4abe Merge pull request #12537 from NixOS/mergify/bp/2.26-maintenance/pr-12391
Only try to chmod /nix/var/nix/profiles/per-user when necessary (backport #12391)
2025-02-20 16:45:41 +00:00
Sandro Jäckel 65583ca79b Only try to chmod /nix/var/nix/profiles/per-user when necessary
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
(cherry picked from commit dcbf4dcc09)
2025-02-20 16:11:27 +00:00
mergify[bot] 514f1dea44 Merge pull request #12530 from NixOS/mergify/bp/2.26-maintenance/pr-12439
Set FD_CLOEXEC on sockets created by curl (backport #12439)
2025-02-19 20:24:14 +00:00
MaxHearnden cd149b56c7 Set FD_CLOEXEC on sockets created by curl
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can
cause connections to remain open forever when using commands like `nix
shell`

This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION
callback.

(cherry picked from commit 12d2527276)
2025-02-19 19:47:45 +00:00
mergify[bot] 029dd96b8b Merge pull request #12523 from NixOS/mergify/bp/2.26-maintenance/pr-12518
packaging: Restore libgit2 USE_SSH=exec (backport #12518)
2025-02-19 09:14:56 +00:00
Robert Hensing 8028579060 packaging: Restore libgit2 USE_SSH=exec
... when nixpkgs is nixos-unstable or the overlay is used.

(cherry picked from commit 5488e29d2f)
2025-02-19 08:54:28 +00:00
mergify[bot] dae4dac76e Merge pull request #12519 from NixOS/mergify/bp/2.26-maintenance/pr-12503
Overriding the componentized build (backport #12503)
2025-02-19 00:59:46 +00:00
Robert Hensing 244735270a packaging: Remove dead code
... from nixpkgs, my bad.

(cherry picked from commit f0bdb65216)
2025-02-19 01:33:20 +01:00
Robert Hensing dade40573e packaging: Make patch count lazier
This makes `nix.version` quicker to evaluate, which should speed up
package listing operations.

If you want an accurate count, use `lib.optionals` in your override
instead of `null` values.

(cherry picked from commit d47e3c9576)
2025-02-19 01:33:20 +01:00
Robert Hensing bfb6f37b37 packaging: Add patch count to version
(cherry picked from commit 0dbe28ad9d)
2025-02-19 01:33:20 +01:00
Robert Hensing a4641be4e9 test: Ignore packaging-overriding check on darwin for now
(cherry picked from commit 03efba30da)
2025-02-19 01:33:20 +01:00
Robert Hensing cc3fb61249 packaging: Add source overriding "methods"
(cherry picked from commit 48fb6fdde9)
2025-02-19 01:33:20 +01:00
Robert Hensing 44fb647986 packaging: Add overrideAllMesonComponents
(cherry picked from commit f31d86284f)
2025-02-19 01:32:53 +01:00
Robert Hensing 540e8cb908 packaging: Move layers from dependencies to components
This makes it easier to implement batch overriding for the components.

(cherry picked from commit cca01407a7)
2025-02-19 01:32:40 +01:00
mergify[bot] 8a2bb811ac Merge pull request #12506 from NixOS/mergify/bp/2.26-maintenance/pr-12498
Restore detailed Nix CLI version (backport #12498)
2025-02-19 00:29:13 +00:00
Robert Hensing fd062585ac tests: Fix installTests
(cherry picked from commit 2b5365bcf7)
2025-02-19 01:04:04 +01:00
Robert Hensing b175e5bb6d Write just ./.version on all components
This way it's easier to get right. See previous commit.

(cherry picked from commit 3556f6bf4c)
2025-02-19 01:04:04 +01:00
Robert Hensing 21998464b7 Restore detailed Nix CLI version
... as intended.

Requirements:
- don't build fresh libraries for each git commit
- have git commit in the CLI

Bug:
- echo ${version} went into the wrong file => use the fact that it's
  a symlink, not just for reading but also for writing.

(cherry picked from commit bba4e6b061)
2025-02-19 01:04:02 +01:00
mergify[bot] d6a66b366e Merge pull request #12517 from NixOS/mergify/bp/2.26-maintenance/pr-12484
Support libgit2 1.9.0 (backport #12484)
2025-02-18 23:30:27 +00:00
Robert Hensing cc3ad9bd3a Support libgit2 1.9.0
For when the overlay is used with nixos-unstable.
1.9.0 has our patches.

(cherry picked from commit b0bbb1252a)
2025-02-18 23:16:12 +00:00
John Ericson e310c19a1a Merge pull request #12509 from NixOS/mergify/bp/2.26-maintenance/pr-12499
startDaemon(): Detect if the daemon crashes before creating the socket (backport #12499)
2025-02-18 12:23:20 -05:00
John Ericson 6c15761b0e Merge pull request #12505 from NixOS/mergify/bp/2.26-maintenance/pr-12497
Don't override default man search paths (backport #12497)
2025-02-18 12:17:08 -05:00
Eelco Dolstra f02a7b880e startDaemon(): Detect if the daemon crashes before creating the socket
This avoids timeouts like those seen in
https://github.com/NixOS/nix/actions/runs/13376958708/job/37358120348?pr=6962.

(cherry picked from commit 11c42cb2e1)
2025-02-18 16:39:48 +00:00
Sergei Zimmerman a004c84e85 Don't override default man search paths
By appending a colon to MANPATH NIX_MAN_DIR gets prepended to the
final MANPATH before default search paths.
This makes man still consider default search paths, but prefers
NIX_MAN_DIR (if it exists).

It still makes sense to point NIX_MAN_DIR to a correct location
by moving man pages build from nix-manual.man to nix-cli.man, but
this should fix most common use-cases where nix is installed globally.

(cherry picked from commit 95f16a3275)
2025-02-18 16:32:42 +00:00
Eelco Dolstra adc0f67130 Merge pull request #12502 from NixOS/mergify/bp/2.26-maintenance/pr-12492
Move code related to NIX_MAN_DIR from libstore to nix-cli (backport #12492)
2025-02-18 15:56:35 +01:00
Sergei Zimmerman 80db87bd4c Move code related to NIX_MAN_DIR from libstore to nix-cli
This is a prerequisite to properly fixing man-pages once and
for all [1]. Note that this patch leaves manpages for legacy
commands in a borked state, pending the movement of manpages from
nix-manual to nix-cli [2].

[1]: https://www.github.com/NixOS/nix/issues/12382
[2]: https://www.github.com/NixOS/nix/issues/12382#issuecomment-2663782043

(cherry picked from commit b1a38b3efe)
2025-02-18 14:37:28 +00:00
John Ericson a13149e03e Merge pull request #12486 from NixOS/mergify/bp/2.26-maintenance/pr-12481
resolveLookupPathPath(): Fix caching of negative lookups (backport #12481)
2025-02-17 12:32:58 -05:00
John Ericson bcbfdc1f28 Merge pull request #12485 from NixOS/mergify/bp/2.26-maintenance/pr-10765
Add `SSHMaster::Connection::trySetBufferSize` (backport #10765)
2025-02-17 12:28:47 -05:00
Eelco Dolstra 640ce50da1 resolveLookupPathPath(): Fix caching of negative lookups
This avoids spamming in case the missing search path entry does not
exist (#12480).

(cherry picked from commit df08e1e204)
2025-02-17 17:03:40 +00:00
Eelco Dolstra 7d168db83c getDefaultNixPath(): Don't add symlinks if the target doesn't exist
(cherry picked from commit 8ac49ea5de)
2025-02-17 17:03:40 +00:00
John Ericson fa7f0d6d07 Allow setting ssh:// pipe size
Exposed for Hydra. We could make it fancier but with (a) new store
settings (b) switch to `ssh-ng://` both in the works, it doesn't seem
worth it.

(cherry picked from commit 94a7c34b2f)
2025-02-17 16:56:35 +00:00
John Ericson 7112f8294c Add SSHMaster::Connection::trySetBufferSize
It is unused in Nix currently, but will be used in Hydra. This reflects
what Hydra does in https://github.com/NixOS/hydra/pull/1387.

We may probably to use it more widely for better SSH store performance,
but this needs to be subject to more testing before we do that.

(cherry picked from commit 0d25cc6541)
2025-02-17 16:56:35 +00:00
John Ericson 674a87462c Merge pull request #12478 from NixOS/mergify/bp/2.26-maintenance/pr-10748
Expose a bunch of things in the Legacy SSH Store for Hydra (backport #10748)
2025-02-16 17:56:54 -05:00
John Ericson 86ccad698e Expose a bunch of things in the Legacy SSH Store for Hydra
(cherry picked from commit 5eade48252)
2025-02-16 22:38:53 +00:00
Eelco Dolstra 970942f458 Bump version 2025-02-12 21:50:20 +01:00
mergify[bot] b3e9204833 Merge pull request #12459 from NixOS/mergify/bp/2.26-maintenance/pr-12458
lockFlake(): When refetching a locked flake, use the locked ref (backport #12458)
2025-02-12 16:09:58 +00:00
Eelco Dolstra 0ff190107f Resolve merge conflict 2025-02-12 16:54:48 +01:00
Eelco Dolstra dffcc184d7 lockFlake(): When refetching a locked flake, use the locked ref
Otherwise we may accidentally update a lock when we shouldn't.

Fixes #12445.

(cherry picked from commit 5c552b62fc)

# Conflicts:
#	src/libflake/flake/flake.cc
2025-02-12 15:46:52 +00:00
Eelco Dolstra 49f226e87b Merge pull request #12457 from NixOS/mergify/bp/2.26-maintenance/pr-12450
packaging/components.nix: Don't import a symlink (backport #12450)
2025-02-12 12:56:51 +01:00
Eelco Dolstra d308228497 Don't import a symlink
This is a workaround to avoid differing evaluation results between Nix
2.19 and >= 2.20 (#12449).

(cherry picked from commit 2e20a5f822)
2025-02-12 11:42:49 +00:00
Eelco Dolstra dfbd030d3f Merge pull request #12453 from NixOS/mergify/bp/2.26-maintenance/pr-12448
copyPathToStore(): Preserve symlinks (backport #12448)
2025-02-11 23:16:31 +01:00
Eelco Dolstra 83306bb841 copyPathToStore(): Preserve symlinks
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.

(cherry picked from commit 26b87e78b5)
2025-02-11 21:51:39 +00:00
mergify[bot] 4c3786dd40 Merge pull request #12436 from NixOS/mergify/bp/2.26-maintenance/pr-12373
pre-commit/check-merge-conflicts-2: fix use outside dev shell (backport #12373)
2025-02-10 15:15:29 +00:00
Robert Hensing 0531f1299c Resolve conflict 2025-02-10 16:01:13 +01:00
Robert Hensing df8d5e61ad test: Fix shellcheck by giving git-hashing scripts shebangs
This seems to be the way to do it now, even though I can't run them
without setting at least one env var.
I'll only fix shellcheck for now. Don't shoot the messenger.

It isn't quite clear to me why the previous commit masked this problem,
but I'm glad shellcheck has an effect or more effect now.
2025-02-10 15:14:53 +01:00
Robert Hensing 30435e0559 pre-commit/check-merge-conflicts-2: fix use outside dev shell
Note that this is just a script that is meant to run outside a
derivation (but also can be called by a derivation builder).
`touch $out` does not belong in it.

`touch $out` worked accidentally in the derivation-based check,
and also in the dev shell, but if pre-commit is invoked without
the dev shell it would fail.
2025-02-10 15:14:44 +01:00
Eelco Dolstra 31bdb2a8ff Merge pull request #12429 from NixOS/mergify/bp/2.26-maintenance/pr-12386
Git fetcher: Don't create refs when fetching by revision (backport #12386)
2025-02-06 15:24:14 +01:00
Eelco Dolstra 8e4cd2f537 readHead(): Make sure we're returning the HEAD ref line
If we previously fetched by revision, the output of "git ls-remote"
won't start with the expected line like

  ref: refs/heads/master HEAD

but will be something like

  5c4410e3b9891c05ab40d723de78c6f0be45ad30        refs/heads/5c4410e3b9891c05ab40d723de78c6f0be45ad30

This then causes Nix to treat that revision as a refname, which then
leads to warnings like

  warning: could not update cached head '5c4410e3b9891c05ab40d723de78c6f0be45ad30' for 'file:///tmp/repo'

(cherry picked from commit c8b22643ba)
2025-02-06 14:05:03 +00:00
Eelco Dolstra 1fe33c13d9 Git fetcher: Don't use refspec <rev>:<rev>
This causes Git to create a local ref named refs/head/<rev>, e.g.

  $ git -C ~/.cache/nix/gitv3/11irpim06vj4h6c0w8yls6kx4hvl0qd0gr1fvk47n76g6wf1s1vk ls-remote --symref .
  5c4410e3b9891c05ab40d723de78c6f0be45ad30        refs/heads/5c4410e3b9891c05ab40d723de78c6f0be45ad30
  7f6bde8a20de4cccc2256f088bc5af9dbe38881d        refs/heads/7f6bde8a20de4cccc2256f088bc5af9dbe38881d

which confuses readHead(), leading to errors like

  fatal: Refusing to point HEAD outside of refs/
  warning: could not update cached head 'd275d93aa0bb8a004939b2f1e87f559f989453be' for 'file:///tmp/repo'

(cherry picked from commit ee9fa0d360)
2025-02-06 14:05:03 +00:00
Eelco Dolstra 727cf59997 Git fetcher: Don't pass URL query parameters for file:// URLs
Git interprets them as part of the file name, so passing parameters
like 'rev' breaks. Only relevant for testing (when _NIX_FORCE_HTTP is
set) and local bare repos.

(cherry picked from commit 9f72d5bce9)
2025-02-06 14:05:02 +00:00
mergify[bot] acba2c6f4f Merge pull request #12406 from NixOS/mergify/bp/2.26-maintenance/pr-12405
nix-profile.fish: Typo NIX_SS{H => L}_CERT_FILE (backport #12405)
2025-02-03 12:50:41 +00:00
Illia Bobyr 491aaaf116 nix-profile.fish: Typo NIX_SS{H => L}_CERT_FILE
(cherry picked from commit 803fb83f7f)

# Conflicts:
#	scripts/nix-profile.fish.in
2025-02-03 12:36:51 +00:00
mergify[bot] 862b3f41a4 Merge pull request #12396 from NixOS/mergify/bp/2.26-maintenance/pr-12385
Fix duplicate setPathDisplay() (backport #12385)
2025-02-02 03:05:49 +00:00
Eelco Dolstra 28684af74b GitExportIgnoreSourceAccessor: Don't show «unknown»
In general we should set the path display prefix on the inner
accessor, so we now pass the display prefix to getAccessor().

(cherry picked from commit 3032512425)
2025-02-02 02:10:53 +00:00
Eelco Dolstra 1c1f8b2343 Fix duplicate setPathDisplay()
Fixes messages like 'copying /tmp/repo/tmp/repo to the store'. The
PosixSourceAccessor already sets the prefix. Setting the prefix twice
shouldn't be a problem, but GitRepoImpl::getAccessor() returns a
wrapped accessor so it's not actually idempotent.

(cherry picked from commit 102d90ebf0)
2025-02-02 02:10:53 +00:00
mergify[bot] a02e814b4b Merge pull request #12371 from NixOS/mergify/bp/2.26-maintenance/pr-12363
Issue #12161, add `meta.mainProgram` (backport #12363)
2025-01-28 08:39:15 +00:00
Robert Hensing a75cf57702 packages.nix-cli: Add meta.mainProgram
(cherry picked from commit 850329dea5)
2025-01-28 08:16:00 +00:00
Robert Hensing 605bd06ca4 packages.default: Add meta.mainProgram
(cherry picked from commit 0d7418b4fe)
2025-01-28 08:16:00 +00:00
mergify[bot] 84efd3ecf2 Merge pull request #12367 from NixOS/mergify/bp/2.26-maintenance/pr-12362
refactor: Extract EvalState::realiseString (backport #12362)
2025-01-27 16:08:41 +00:00
Robert Hensing 527e68ac3e refactor: Extract EvalState::realiseString
(cherry picked from commit 7465fbe926)
2025-01-27 15:51:30 +00:00
mergify[bot] 2fce659c7d Merge pull request #12359 from NixOS/mergify/bp/2.26-maintenance/pr-12356
Improve "illegal path references in fixed output derivation" error (backport #12356)
2025-01-26 20:10:39 +00:00
Ben Millwood 791d6cf433 Improve "illegal path references in fixed output derivation" error
The main improvement is that the new message gives an example of a path
that is referenced, which should make it easier to track down. While
there, I also clarified the wording, saying exactly why the paths in
question were illegal.

(cherry picked from commit 4e5d1b281e)
2025-01-26 19:39:33 +00:00
Robert Hensing 3e0d9ff937 Merge pull request #12354 from NixOS/nixfmt-on-2.26
Apply nixfmt (backport 2.26)
2025-01-24 22:43:06 +01:00
Robert Hensing f629d81df0 test: Fix shifted source positions after formatting 2025-01-24 22:21:27 +01:00
Robert Hensing 32aed360b8 Format .nix files
This does not include any automation for the release branch, but
is based on the configuration of https://github.com/NixOS/nix/pull/12349

    pre-commit run -a nixfmt-rfc-style
2025-01-24 20:40:21 +01:00
Eelco Dolstra 6cb17fd836 Bump version 2025-01-24 16:20:43 +01:00
mergify[bot] 36bd92736f Merge pull request #12350 from NixOS/mergify/bp/2.26-maintenance/pr-12347
EvalState::resolveLookupPathPath(): Call resolveSymlinks() before pathExists() (backport #12347)
2025-01-24 14:02:15 +00:00
Eelco Dolstra 90159cb197 EvalState::resolveLookupPathPath(): Call resolveSymlinks() before pathExists()
Fixes #12339.

(cherry picked from commit 00d9e7e1f4)
2025-01-24 13:46:16 +00:00
mergify[bot] 4cfeb8d1bb Merge pull request #12346 from NixOS/mergify/bp/2.26-maintenance/pr-12336
libstore: Fix progress bars (backport #12336)
2025-01-24 11:46:16 +00:00
Philipp Otterbein 9cf3d3368e libstore: fix progress bars
(cherry picked from commit be97dc1efc)
2025-01-24 11:10:50 +00:00
Eelco Dolstra aab6419b63 Merge pull request #12334 from NixOS/mergify/bp/2.26-maintenance/pr-12331
GitRepo::fetch(): Ignore $GIT_DIR (backport #12331)
2025-01-22 22:00:16 +01:00
Eelco Dolstra 832221650b GitRepo::fetch(): Ignore $GIT_DIR
Fixes #12325.

(cherry picked from commit 41983dba8f)
2025-01-22 20:30:29 +00:00
Eelco Dolstra 2301d86f32 GitRepo::fetch(): Cleanup
(cherry picked from commit bd10b859f7)
2025-01-22 20:30:28 +00:00
Eelco Dolstra 91e60321f6 Bump version 2025-01-22 13:36:30 +01:00
Eelco Dolstra 28752fe288 Mark official release 2025-01-22 12:16:44 +01:00
410 changed files with 9343 additions and 6110 deletions

No files matched your search

+1 -1
View File
@@ -1 +1 @@
2.26.0 2.26.5
+9 -10
View File
@@ -1,10 +1,9 @@
(import (import (
( let
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in lock = builtins.fromJSON (builtins.readFile ./flake.lock);
fetchTarball { in
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz"; fetchTarball {
sha256 = lock.nodes.flake-compat.locked.narHash; url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
} sha256 = lock.nodes.flake-compat.locked.narHash;
) }
{ src = ./.; } ) { src = ./.; }).defaultNix
).defaultNix
+9 -1
View File
@@ -5,7 +5,15 @@ in
builtinsInfo: builtinsInfo:
let let
showBuiltin = name: { doc, type ? null, args ? [ ], experimental-feature ? null, impure-only ? false }: showBuiltin =
name:
{
doc,
type ? null,
args ? [ ],
experimental-feature ? null,
impure-only ? false,
}:
let let
type' = optionalString (type != null) " (${type})"; type' = optionalString (type != null) " (${type})";
+90 -62
View File
@@ -32,7 +32,13 @@ let
commandInfo = fromJSON commandDump; commandInfo = fromJSON commandDump;
showCommand = { command, details, filename, toplevel }: showCommand =
{
command,
details,
filename,
toplevel,
}:
let let
result = '' result = ''
@@ -56,26 +62,27 @@ let
${maybeOptions} ${maybeOptions}
''; '';
showSynopsis = command: args: showSynopsis =
command: args:
let let
showArgument = arg: "*${arg.label}*" + optionalString (! arg ? arity) "..."; showArgument = arg: "*${arg.label}*" + optionalString (!arg ? arity) "...";
arguments = concatStringsSep " " (map showArgument args); arguments = concatStringsSep " " (map showArgument args);
in '' in
''
`${command}` [*option*...] ${arguments} `${command}` [*option*...] ${arguments}
''; '';
maybeSubcommands = optionalString (details ? commands && details.commands != {}) maybeSubcommands = optionalString (details ? commands && details.commands != { }) ''
'' where *subcommand* is one of the following:
where *subcommand* is one of the following:
${subcommands} ${subcommands}
''; '';
subcommands = if length categories > 1 subcommands = if length categories > 1 then listCategories else listSubcommands details.commands;
then listCategories
else listSubcommands details.commands;
categories = sort (x: y: x.id < y.id) (unique (map (cmd: cmd.category) (attrValues details.commands))); categories = sort (x: y: x.id < y.id) (
unique (map (cmd: cmd.category) (attrValues details.commands))
);
listCategories = concatStrings (map showCategory categories); listCategories = concatStrings (map showCategory categories);
@@ -99,38 +106,39 @@ let
${allStores} ${allStores}
''; '';
index = replaceStrings index =
[ "@store-types@" "./local-store.md" "./local-daemon-store.md" ] replaceStrings
[ storesOverview "#local-store" "#local-daemon-store" ] [ "@store-types@" "./local-store.md" "./local-daemon-store.md" ]
details.doc; [ storesOverview "#local-store" "#local-daemon-store" ]
details.doc;
storesOverview = storesOverview =
let let
showEntry = store: showEntry = store: "- [${store.name}](#${store.slug})";
"- [${store.name}](#${store.slug})";
in in
concatStringsSep "\n" (map showEntry storesList) + "\n"; concatStringsSep "\n" (map showEntry storesList) + "\n";
allStores = concatStringsSep "\n" (attrValues storePages); allStores = concatStringsSep "\n" (attrValues storePages);
storePages = listToAttrs storePages = listToAttrs (
(map (s: { name = s.filename; value = s.page; }) storesList); map (s: {
name = s.filename;
value = s.page;
}) storesList
);
storesList = showStoreDocs { storesList = showStoreDocs {
storeInfo = commandInfo.stores; storeInfo = commandInfo.stores;
inherit inlineHTML; inherit inlineHTML;
}; };
hasInfix = infix: content: hasInfix =
infix: content:
builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content); builtins.stringLength content != builtins.stringLength (replaceStrings [ infix ] [ "" ] content);
in in
optionalString (details ? doc) ( optionalString (details ? doc) (
# An alternate implementation with builtins.match stack overflowed on some systems. # An alternate implementation with builtins.match stack overflowed on some systems.
if hasInfix "@store-types@" details.doc if hasInfix "@store-types@" details.doc then help-stores else details.doc
then help-stores
else details.doc
); );
maybeOptions = maybeOptions =
let let
allVisibleOptions = filterAttrs allVisibleOptions = filterAttrs (_: o: !o.hiddenCategory) (details.flags // toplevel.flags);
(_: o: ! o.hiddenCategory)
(details.flags // toplevel.flags);
in in
optionalString (allVisibleOptions != { }) '' optionalString (allVisibleOptions != { }) ''
# Options # Options
@@ -142,55 +150,73 @@ let
> See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags. > See [`man nix.conf`](@docroot@/command-ref/conf-file.md#command-line-flags) for overriding configuration settings with command line flags.
''; '';
showOptions = inlineHTML: allOptions: showOptions =
inlineHTML: allOptions:
let let
showCategory = cat: opts: '' showCategory = cat: opts: ''
${optionalString (cat != "") "## ${cat}"} ${optionalString (cat != "") "## ${cat}"}
${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))} ${concatStringsSep "\n" (attrValues (mapAttrs showOption opts))}
''; '';
showOption = name: option: showOption =
name: option:
let let
result = trim '' result = trim ''
- ${item} - ${item}
${option.description} ${option.description}
''; '';
item = if inlineHTML item =
then ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}'' if inlineHTML then
else "`--${name}` ${shortName} ${labels}"; ''<span id="opt-${name}">[`--${name}`](#opt-${name})</span> ${shortName} ${labels}''
shortName = optionalString else
(option ? shortName) "`--${name}` ${shortName} ${labels}";
("/ `-${option.shortName}`"); shortName = optionalString (option ? shortName) ("/ `-${option.shortName}`");
labels = optionalString labels = optionalString (option ? labels) (concatStringsSep " " (map (s: "*${s}*") option.labels));
(option ? labels) in
(concatStringsSep " " (map (s: "*${s}*") option.labels)); result;
in result; categories =
categories = mapAttrs mapAttrs
# Convert each group from a list of key-value pairs back to an attrset # Convert each group from a list of key-value pairs back to an attrset
(_: listToAttrs) (_: listToAttrs)
(groupBy (groupBy (cmd: cmd.value.category) (attrsToList allOptions));
(cmd: cmd.value.category) in
(attrsToList allOptions)); concatStrings (attrValues (mapAttrs showCategory categories));
in concatStrings (attrValues (mapAttrs showCategory categories)); in
in squash result; squash result;
appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name; appendName = filename: name: (if filename == "nix" then "nix3" else filename) + "-" + name;
processCommand = { command, details, filename, toplevel }: processCommand =
{
command,
details,
filename,
toplevel,
}:
let let
cmd = { cmd = {
inherit command; inherit command;
name = filename + ".md"; name = filename + ".md";
value = showCommand { inherit command details filename toplevel; }; value = showCommand {
inherit
command
details
filename
toplevel
;
};
}; };
subcommand = subCmd: processCommand { subcommand =
command = command + " " + subCmd; subCmd:
details = details.commands.${subCmd}; processCommand {
filename = appendName filename subCmd; command = command + " " + subCmd;
inherit toplevel; details = details.commands.${subCmd};
}; filename = appendName filename subCmd;
in [ cmd ] ++ concatMap subcommand (attrNames details.commands or {}); inherit toplevel;
};
in
[ cmd ] ++ concatMap subcommand (attrNames details.commands or { });
manpages = processCommand { manpages = processCommand {
command = "nix"; command = "nix";
@@ -199,9 +225,11 @@ let
toplevel = commandInfo.args; toplevel = commandInfo.args;
}; };
tableOfContents = let tableOfContents =
showEntry = page: let
" - [${page.command}](command-ref/new-cli/${page.name})"; showEntry = page: " - [${page.command}](command-ref/new-cli/${page.name})";
in concatStringsSep "\n" (map showEntry manpages) + "\n"; in
concatStringsSep "\n" (map showEntry manpages) + "\n";
in (listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; } in
(listToAttrs manpages) // { "SUMMARY.md" = tableOfContents; }
+74 -42
View File
@@ -1,67 +1,99 @@
let let
inherit (builtins) attrValues concatStringsSep isAttrs isBool mapAttrs; inherit (builtins)
inherit (import <nix/utils.nix>) concatStrings indent optionalString squash; attrValues
concatStringsSep
isAttrs
isBool
mapAttrs
;
inherit (import <nix/utils.nix>)
concatStrings
indent
optionalString
squash
;
in in
# `inlineHTML` is a hack to accommodate inconsistent output from `lowdown` # `inlineHTML` is a hack to accommodate inconsistent output from `lowdown`
{ prefix, inlineHTML ? true }: settingsInfo: {
prefix,
inlineHTML ? true,
}:
settingsInfo:
let let
showSetting = prefix: setting: { description, documentDefault, defaultValue, aliases, value, experimentalFeature }: showSetting =
prefix: setting:
{
description,
documentDefault,
defaultValue,
aliases,
value,
experimentalFeature,
}:
let let
result = squash '' result = squash ''
- ${item} - ${item}
${indent " " body} ${indent " " body}
''; '';
item = if inlineHTML item =
then ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>'' if inlineHTML then
else "`${setting}`"; ''<span id="${prefix}-${setting}">[`${setting}`](#${prefix}-${setting})</span>''
else
"`${setting}`";
# separate body to cleanly handle indentation # separate body to cleanly handle indentation
body = '' body = ''
${experimentalFeatureNote} ${experimentalFeatureNote}
${description} ${description}
**Default:** ${showDefault documentDefault defaultValue} **Default:** ${showDefault documentDefault defaultValue}
${showAliases aliases} ${showAliases aliases}
''; '';
experimentalFeatureNote = optionalString (experimentalFeature != null) '' experimentalFeatureNote = optionalString (experimentalFeature != null) ''
> **Warning** > **Warning**
> >
> This setting is part of an > This setting is part of an
> [experimental feature](@docroot@/development/experimental-features.md). > [experimental feature](@docroot@/development/experimental-features.md).
> >
> To change this setting, make sure the > To change this setting, make sure the
> [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature}) > [`${experimentalFeature}` experimental feature](@docroot@/development/experimental-features.md#xp-feature-${experimentalFeature})
> is enabled. > is enabled.
> For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md): > For example, include the following in [`nix.conf`](@docroot@/command-ref/conf-file.md):
> >
> ``` > ```
> extra-experimental-features = ${experimentalFeature} > extra-experimental-features = ${experimentalFeature}
> ${setting} = ... > ${setting} = ...
> ``` > ```
''; '';
showDefault = documentDefault: defaultValue: showDefault =
documentDefault: defaultValue:
if documentDefault then if documentDefault then
# a StringMap value type is specified as a string, but # a StringMap value type is specified as a string, but
# this shows the value type. The empty stringmap is `null` in # this shows the value type. The empty stringmap is `null` in
# JSON, but that converts to `{ }` here. # JSON, but that converts to `{ }` here.
if defaultValue == "" || defaultValue == [] || isAttrs defaultValue if defaultValue == "" || defaultValue == [ ] || isAttrs defaultValue then
then "*empty*" "*empty*"
else if isBool defaultValue then else if isBool defaultValue then
if defaultValue then "`true`" else "`false`" if defaultValue then "`true`" else "`false`"
else "`${toString defaultValue}`" else
else "*machine-specific*"; "`${toString defaultValue}`"
else
"*machine-specific*";
showAliases = aliases: showAliases =
optionalString (aliases != []) aliases:
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}"; optionalString (aliases != [ ])
"**Deprecated alias:** ${(concatStringsSep ", " (map (s: "`${s}`") aliases))}";
in result; in
result;
in concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo)) in
concatStrings (attrValues (mapAttrs (showSetting prefix) settingsInfo))
+37 -14
View File
@@ -1,6 +1,20 @@
let let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings; inherit (builtins)
inherit (import <nix/utils.nix>) optionalString filterAttrs trim squash toLower unique indent; attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
inherit (import <nix/utils.nix>)
optionalString
filterAttrs
trim
squash
toLower
unique
indent
;
showSettings = import <nix/generate-settings.nix>; showSettings = import <nix/generate-settings.nix>;
in in
@@ -14,7 +28,13 @@ in
let let
showStore = { name, slug }: { settings, doc, experimentalFeature }: showStore =
{ name, slug }:
{
settings,
doc,
experimentalFeature,
}:
let let
result = squash '' result = squash ''
# ${name} # ${name}
@@ -25,7 +45,10 @@ let
## Settings ## Settings
${showSettings { prefix = "store-${slug}"; inherit inlineHTML; } settings} ${showSettings {
prefix = "store-${slug}";
inherit inlineHTML;
} settings}
''; '';
experimentalFeatureNote = optionalString (experimentalFeature != null) '' experimentalFeatureNote = optionalString (experimentalFeature != null) ''
@@ -43,15 +66,15 @@ let
> extra-experimental-features = ${experimentalFeature} > extra-experimental-features = ${experimentalFeature}
> ``` > ```
''; '';
in result; in
result;
storesList = map storesList = map (name: rec {
(name: rec { inherit name;
inherit name; slug = replaceStrings [ " " ] [ "-" ] (toLower name);
slug = replaceStrings [ " " ] [ "-" ] (toLower name); filename = "${slug}.md";
filename = "${slug}.md"; page = showStore { inherit name slug; } storeInfo.${name};
page = showStore { inherit name slug; } storeInfo.${name}; }) (attrNames storeInfo);
})
(attrNames storeInfo);
in storesList in
storesList
+18 -10
View File
@@ -1,5 +1,11 @@
let let
inherit (builtins) attrNames listToAttrs concatStringsSep readFile replaceStrings; inherit (builtins)
attrNames
listToAttrs
concatStringsSep
readFile
replaceStrings
;
showSettings = import <nix/generate-settings.nix>; showSettings = import <nix/generate-settings.nix>;
showStoreDocs = import <nix/generate-store-info.nix>; showStoreDocs = import <nix/generate-store-info.nix>;
in in
@@ -14,26 +20,28 @@ let
index = index =
let let
showEntry = store: showEntry = store: "- [${store.name}](./${store.filename})";
"- [${store.name}](./${store.filename})";
in in
concatStringsSep "\n" (map showEntry storesList); concatStringsSep "\n" (map showEntry storesList);
"index.md" = replaceStrings "index.md" =
[ "@store-types@" ] [ index ] replaceStrings [ "@store-types@" ] [ index ]
(readFile ./source/store/types/index.md.in); (readFile ./source/store/types/index.md.in);
tableOfContents = tableOfContents =
let let
showEntry = store: showEntry = store: " - [${store.name}](store/types/${store.filename})";
" - [${store.name}](store/types/${store.filename})";
in in
concatStringsSep "\n" (map showEntry storesList) + "\n"; concatStringsSep "\n" (map showEntry storesList) + "\n";
"SUMMARY.md" = tableOfContents; "SUMMARY.md" = tableOfContents;
storePages = listToAttrs storePages = listToAttrs (
(map (s: { name = s.filename; value = s.page; }) storesList); map (s: {
name = s.filename;
value = s.page;
}) storesList
);
in in
storePages // { inherit "index.md" "SUMMARY.md"; } storePages // { inherit "index.md" "SUMMARY.md"; }
@@ -2,8 +2,8 @@ with builtins;
with import <nix/utils.nix>; with import <nix/utils.nix>;
let let
showExperimentalFeature = name: doc: showExperimentalFeature = name: doc: ''
'' - [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name})
- [`${name}`](@docroot@/development/experimental-features.md#xp-feature-${name}) '';
''; in
in xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps))) xps: indent " " (concatStrings (attrValues (mapAttrs showExperimentalFeature xps)))
+2 -1
View File
@@ -2,7 +2,8 @@ with builtins;
with import <nix/utils.nix>; with import <nix/utils.nix>;
let let
showExperimentalFeature = name: doc: showExperimentalFeature =
name: doc:
squash '' squash ''
## [`${name}`]{#xp-feature-${name}} ## [`${name}`]{#xp-feature-${name}}
+32 -28
View File
@@ -1,19 +1,20 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, meson meson,
, ninja ninja,
, lowdown-unsandboxed lowdown-unsandboxed,
, mdbook mdbook,
, mdbook-linkcheck mdbook-linkcheck,
, jq jq,
, python3 python3,
, rsync rsync,
, nix-cli nix-cli,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -25,18 +26,22 @@ mkMesonDerivation (finalAttrs: {
inherit version; inherit version;
workDir = ./.; workDir = ./.;
fileset = fileset.difference fileset =
(fileset.unions [ fileset.difference
../../.version (fileset.unions [
# Too many different types of files to filter for now ../../.version
../../doc/manual # Too many different types of files to filter for now
./. ../../doc/manual
]) ./.
# Do a blacklist instead ])
../../doc/manual/package.nix; # Do a blacklist instead
../../doc/manual/package.nix;
# TODO the man pages should probably be separate # TODO the man pages should probably be separate
outputs = [ "out" "man" ]; outputs = [
"out"
"man"
];
# Hack for sake of the dev shell # Hack for sake of the dev shell
passthru.externalNativeBuildInputs = [ passthru.externalNativeBuildInputs = [
@@ -54,11 +59,10 @@ mkMesonDerivation (finalAttrs: {
nix-cli nix-cli
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''$out/nix-support mkdir -p ''$out/nix-support
+10
View File
@@ -0,0 +1,10 @@
---
synopsis: Set FD_CLOEXEC on sockets created by curl
issues: []
prs: [12439]
---
Curl creates sockets without setting FD_CLOEXEC/SOCK_CLOEXEC, this can cause connections to remain open forever when using commands like `nix shell`
This change sets the FD_CLOEXEC flag using a CURLOPT_SOCKOPTFUNCTION callback.
+31 -13
View File
@@ -11,10 +11,15 @@ rec {
concatStrings = concatStringsSep ""; concatStrings = concatStringsSep "";
attrsToList = a: attrsToList =
map (name: { inherit name; value = a.${name}; }) (builtins.attrNames a); a:
map (name: {
inherit name;
value = a.${name};
}) (builtins.attrNames a);
replaceStringsRec = from: to: string: replaceStringsRec =
from: to: string:
# recursively replace occurrences of `from` with `to` within `string` # recursively replace occurrences of `from` with `to` within `string`
# example: # example:
# replaceStringRec "--" "-" "hello-----world" # replaceStringRec "--" "-" "hello-----world"
@@ -22,16 +27,18 @@ rec {
let let
replaced = replaceStrings [ from ] [ to ] string; replaced = replaceStrings [ from ] [ to ] string;
in in
if replaced == string then string else replaceStringsRec from to replaced; if replaced == string then string else replaceStringsRec from to replaced;
toLower = replaceStrings upperChars lowerChars; toLower = replaceStrings upperChars lowerChars;
squash = replaceStringsRec "\n\n\n" "\n\n"; squash = replaceStringsRec "\n\n\n" "\n\n";
trim = string: trim =
string:
# trim trailing spaces and squash non-leading spaces # trim trailing spaces and squash non-leading spaces
let let
trimLine = line: trimLine =
line:
let let
# separate leading spaces from the rest # separate leading spaces from the rest
parts = split "(^ *)" line; parts = split "(^ *)" line;
@@ -39,19 +46,30 @@ rec {
rest = elemAt parts 2; rest = elemAt parts 2;
# drop trailing spaces # drop trailing spaces
body = head (split " *$" rest); body = head (split " *$" rest);
in spaces + replaceStringsRec " " " " body; in
in concatStringsSep "\n" (map trimLine (splitLines string)); spaces + replaceStringsRec " " " " body;
in
concatStringsSep "\n" (map trimLine (splitLines string));
# FIXME: O(n^2) # FIXME: O(n^2)
unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) []; unique = foldl' (acc: e: if elem e acc then acc else acc ++ [ e ]) [ ];
nameValuePair = name: value: { inherit name value; }; nameValuePair = name: value: { inherit name value; };
filterAttrs = pred: set: filterAttrs =
listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set)); pred: set:
listToAttrs (
concatMap (
name:
let
v = set.${name};
in
if pred name v then [ (nameValuePair name v) ] else [ ]
) (attrNames set)
);
optionalString = cond: string: if cond then string else ""; optionalString = cond: string: if cond then string else "";
indent = prefix: s: indent =
concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s)); prefix: s: concatStringsSep "\n" (map (x: if x == "" then x else "${prefix}${x}") (splitLines s));
} }
+234 -204
View File
@@ -1,112 +1,113 @@
{ pkgs ? import <nixpkgs> { } {
, lib ? pkgs.lib pkgs ? import <nixpkgs> { },
, name ? "nix" lib ? pkgs.lib,
, tag ? "latest" name ? "nix",
, bundleNixpkgs ? true tag ? "latest",
, channelName ? "nixpkgs" bundleNixpkgs ? true,
, channelURL ? "https://nixos.org/channels/nixpkgs-unstable" channelName ? "nixpkgs",
, extraPkgs ? [] channelURL ? "https://nixos.org/channels/nixpkgs-unstable",
, maxLayers ? 100 extraPkgs ? [ ],
, nixConf ? {} maxLayers ? 100,
, flake-registry ? null nixConf ? { },
, uid ? 0 flake-registry ? null,
, gid ? 0 uid ? 0,
, uname ? "root" gid ? 0,
, gname ? "root" uname ? "root",
gname ? "root",
}: }:
let let
defaultPkgs = with pkgs; [ defaultPkgs =
nix with pkgs;
bashInteractive [
coreutils-full nix
gnutar bashInteractive
gzip coreutils-full
gnugrep gnutar
which gzip
curl gnugrep
less which
wget curl
man less
cacert.out wget
findutils man
iana-etc cacert.out
git findutils
openssh iana-etc
] ++ extraPkgs; git
openssh
]
++ extraPkgs;
users = { users =
{
root = { root = {
uid = 0; uid = 0;
shell = "${pkgs.bashInteractive}/bin/bash"; shell = "${pkgs.bashInteractive}/bin/bash";
home = "/root"; home = "/root";
gid = 0; gid = 0;
groups = [ "root" ]; groups = [ "root" ];
description = "System administrator"; description = "System administrator";
};
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.optionalAttrs (uid != 0) {
"${uname}" = {
uid = uid;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/home/${uname}";
gid = gid;
groups = [ "${gname}" ];
description = "Nix user";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups =
{
root.gid = 0;
nixbld.gid = 30000;
nobody.gid = 65534;
}
// lib.optionalAttrs (gid != 0) {
"${gname}".gid = gid;
}; };
nobody = {
uid = 65534;
shell = "${pkgs.shadow}/bin/nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
} // lib.optionalAttrs (uid != 0) {
"${uname}" = {
uid = uid;
shell = "${pkgs.bashInteractive}/bin/bash";
home = "/home/${uname}";
gid = gid;
groups = [ "${gname}" ];
description = "Nix user";
};
} // lib.listToAttrs (
map
(
n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}
)
(lib.lists.range 1 32)
);
groups = {
root.gid = 0;
nixbld.gid = 30000;
nobody.gid = 65534;
} // lib.optionalAttrs (gid != 0) {
"${gname}".gid = gid;
};
userToPasswd = ( userToPasswd = (
k: k:
{ uid {
, gid ? 65534 uid,
, home ? "/var/empty" gid ? 65534,
, description ? "" home ? "/var/empty",
, shell ? "/bin/false" description ? "",
, groups ? [ ] shell ? "/bin/false",
}: "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}" groups ? [ ],
); }:
passwdContents = ( "${k}:x:${toString uid}:${toString gid}:${description}:${home}:${shell}"
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToPasswd users))
); );
passwdContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToPasswd users)));
userToShadow = k: { ... }: "${k}:!:1::::::"; userToShadow = k: { ... }: "${k}:!:1::::::";
shadowContents = ( shadowContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs userToShadow users)));
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs userToShadow users))
);
# Map groups to members # Map groups to members
# { # {
@@ -116,42 +117,35 @@ let
let let
# Create a flat list of user/group mappings # Create a flat list of user/group mappings
mappings = ( mappings = (
builtins.foldl' builtins.foldl' (
( acc: user:
acc: user: let
let groups = users.${user}.groups or [ ];
groups = users.${user}.groups or [ ]; in
in acc
acc ++ map ++ map (group: {
(group: { inherit user group;
inherit user group; }) groups
}) ) [ ] (lib.attrNames users)
groups
)
[ ]
(lib.attrNames users)
); );
in in
( (builtins.foldl' (
builtins.foldl' acc: v:
( acc
acc: v: acc // { // {
${v.group} = acc.${v.group} or [ ] ++ [ v.user ]; ${v.group} = acc.${v.group} or [ ] ++ [ v.user ];
} }
) ) { } mappings)
{ }
mappings)
); );
groupToGroup = k: { gid }: groupToGroup =
k:
{ gid }:
let let
members = groupMemberMap.${k} or [ ]; members = groupMemberMap.${k} or [ ];
in in
"${k}:x:${toString gid}:${lib.concatStringsSep "," members}"; "${k}:x:${toString gid}:${lib.concatStringsSep "," members}";
groupContents = ( groupContents = (lib.concatStringsSep "\n" (lib.attrValues (lib.mapAttrs groupToGroup groups)));
lib.concatStringsSep "\n"
(lib.attrValues (lib.mapAttrs groupToGroup groups))
);
defaultNixConf = { defaultNixConf = {
sandbox = "false"; sandbox = "false";
@@ -159,11 +153,17 @@ let
trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ]; trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
}; };
nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v: nixConfContents =
let (lib.concatStringsSep "\n" (
vStr = if builtins.isList v then lib.concatStringsSep " " v else v; lib.mapAttrsFlatten (
in n: v:
"${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n"; let
vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
in
"${n} = ${vStr}"
) (defaultNixConf // nixConf)
))
+ "\n";
userHome = if uid == 0 then "/root" else "/home/${uname}"; userHome = if uid == 0 then "/root" else "/home/${uname}";
@@ -184,21 +184,29 @@ let
manifest = pkgs.buildPackages.runCommand "manifest.nix" { } '' manifest = pkgs.buildPackages.runCommand "manifest.nix" { } ''
cat > $out <<EOF cat > $out <<EOF
[ [
${lib.concatStringsSep "\n" (builtins.map (drv: let ${lib.concatStringsSep "\n" (
outputs = drv.outputsToInstall or [ "out" ]; builtins.map (
in '' drv:
{ let
${lib.concatStringsSep "\n" (builtins.map (output: '' outputs = drv.outputsToInstall or [ "out" ];
${output} = { outPath = "${lib.getOutput output drv}"; }; in
'') outputs)} ''
outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ]; {
name = "${drv.name}"; ${lib.concatStringsSep "\n" (
outPath = "${drv}"; builtins.map (output: ''
system = "${drv.system}"; ${output} = { outPath = "${lib.getOutput output drv}"; };
type = "derivation"; '') outputs
meta = { }; )}
} outputs = [ ${lib.concatStringsSep " " (builtins.map (x: "\"${x}\"") outputs)} ];
'') defaultPkgs)} name = "${drv.name}";
outPath = "${drv}";
system = "${drv.system}";
type = "derivation";
meta = { };
}
''
) defaultPkgs
)}
] ]
EOF EOF
''; '';
@@ -207,16 +215,22 @@ let
cp -a ${rootEnv}/* $out/ cp -a ${rootEnv}/* $out/
ln -s ${manifest} $out/manifest.nix ln -s ${manifest} $out/manifest.nix
''; '';
flake-registry-path = if (flake-registry == null) then flake-registry-path =
null if (flake-registry == null) then
else if (builtins.readFileType (toString flake-registry)) == "directory" then null
"${flake-registry}/flake-registry.json" else if (builtins.readFileType (toString flake-registry)) == "directory" then
else "${flake-registry}/flake-registry.json"
flake-registry; else
flake-registry;
in in
pkgs.runCommand "base-system" pkgs.runCommand "base-system"
{ {
inherit passwdContents groupContents shadowContents nixConfContents; inherit
passwdContents
groupContents
shadowContents
nixConfContents
;
passAsFile = [ passAsFile = [
"passwdContents" "passwdContents"
"groupContents" "groupContents"
@@ -225,67 +239,79 @@ let
]; ];
allowSubstitutes = false; allowSubstitutes = false;
preferLocalBuild = true; preferLocalBuild = true;
} ('' }
env (
set -x ''
mkdir -p $out/etc env
set -x
mkdir -p $out/etc
mkdir -p $out/etc/ssl/certs mkdir -p $out/etc/ssl/certs
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
cat $passwdContentsPath > $out/etc/passwd cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd echo "" >> $out/etc/passwd
cat $groupContentsPath > $out/etc/group cat $groupContentsPath > $out/etc/group
echo "" >> $out/etc/group echo "" >> $out/etc/group
cat $shadowContentsPath > $out/etc/shadow cat $shadowContentsPath > $out/etc/shadow
echo "" >> $out/etc/shadow echo "" >> $out/etc/shadow
mkdir -p $out/usr mkdir -p $out/usr
ln -s /nix/var/nix/profiles/share $out/usr/ ln -s /nix/var/nix/profiles/share $out/usr/
mkdir -p $out/nix/var/nix/gcroots mkdir -p $out/nix/var/nix/gcroots
mkdir $out/tmp mkdir $out/tmp
mkdir -p $out/var/tmp mkdir -p $out/var/tmp
mkdir -p $out/etc/nix mkdir -p $out/etc/nix
cat $nixConfContentsPath > $out/etc/nix/nix.conf cat $nixConfContentsPath > $out/etc/nix/nix.conf
mkdir -p $out${userHome} mkdir -p $out${userHome}
mkdir -p $out/nix/var/nix/profiles/per-user/${uname} mkdir -p $out/nix/var/nix/profiles/per-user/${uname}
ln -s ${profile} $out/nix/var/nix/profiles/default-1-link ln -s ${profile} $out/nix/var/nix/profiles/default-1-link
ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default ln -s /nix/var/nix/profiles/default-1-link $out/nix/var/nix/profiles/default
ln -s /nix/var/nix/profiles/default $out${userHome}/.nix-profile ln -s /nix/var/nix/profiles/default $out${userHome}/.nix-profile
ln -s ${channel} $out/nix/var/nix/profiles/per-user/${uname}/channels-1-link ln -s ${channel} $out/nix/var/nix/profiles/per-user/${uname}/channels-1-link
ln -s /nix/var/nix/profiles/per-user/${uname}/channels-1-link $out/nix/var/nix/profiles/per-user/${uname}/channels ln -s /nix/var/nix/profiles/per-user/${uname}/channels-1-link $out/nix/var/nix/profiles/per-user/${uname}/channels
mkdir -p $out${userHome}/.nix-defexpr mkdir -p $out${userHome}/.nix-defexpr
ln -s /nix/var/nix/profiles/per-user/${uname}/channels $out${userHome}/.nix-defexpr/channels ln -s /nix/var/nix/profiles/per-user/${uname}/channels $out${userHome}/.nix-defexpr/channels
echo "${channelURL} ${channelName}" > $out${userHome}/.nix-channels echo "${channelURL} ${channelName}" > $out${userHome}/.nix-channels
mkdir -p $out/bin $out/usr/bin mkdir -p $out/bin $out/usr/bin
ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env ln -s ${pkgs.coreutils}/bin/env $out/usr/bin/env
ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh ln -s ${pkgs.bashInteractive}/bin/bash $out/bin/sh
'' + (lib.optionalString (flake-registry-path != null) '' ''
nixCacheDir="${userHome}/.cache/nix" + (lib.optionalString (flake-registry-path != null) ''
mkdir -p $out$nixCacheDir nixCacheDir="${userHome}/.cache/nix"
globalFlakeRegistryPath="$nixCacheDir/flake-registry.json" mkdir -p $out$nixCacheDir
ln -s ${flake-registry-path} $out$globalFlakeRegistryPath globalFlakeRegistryPath="$nixCacheDir/flake-registry.json"
mkdir -p $out/nix/var/nix/gcroots/auto ln -s ${flake-registry-path} $out$globalFlakeRegistryPath
rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath)) mkdir -p $out/nix/var/nix/gcroots/auto
ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName rootName=$(${pkgs.nix}/bin/nix --extra-experimental-features nix-command hash file --type sha1 --base32 <(echo -n $globalFlakeRegistryPath))
'')); ln -s $globalFlakeRegistryPath $out/nix/var/nix/gcroots/auto/$rootName
'')
);
in in
pkgs.dockerTools.buildLayeredImageWithNixDb { pkgs.dockerTools.buildLayeredImageWithNixDb {
inherit name tag maxLayers uid gid uname gname; inherit
name
tag
maxLayers
uid
gid
uname
gname
;
contents = [ baseSystem ]; contents = [ baseSystem ];
@@ -305,15 +331,19 @@ pkgs.dockerTools.buildLayeredImageWithNixDb {
User = "${toString uid}:${toString gid}"; User = "${toString uid}:${toString gid}";
Env = [ Env = [
"USER=${uname}" "USER=${uname}"
"PATH=${lib.concatStringsSep ":" [ "PATH=${
"${userHome}/.nix-profile/bin" lib.concatStringsSep ":" [
"/nix/var/nix/profiles/default/bin" "${userHome}/.nix-profile/bin"
"/nix/var/nix/profiles/default/sbin" "/nix/var/nix/profiles/default/bin"
]}" "/nix/var/nix/profiles/default/sbin"
"MANPATH=${lib.concatStringsSep ":" [ ]
"${userHome}/.nix-profile/share/man" }"
"/nix/var/nix/profiles/default/share/man" "MANPATH=${
]}" lib.concatStringsSep ":" [
"${userHome}/.nix-profile/share/man"
"/nix/var/nix/profiles/default/share/man"
]
}"
"SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "GIT_SSL_CAINFO=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
"NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt" "NIX_SSL_CERT_FILE=/nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt"
+285 -161
View File
@@ -5,7 +5,10 @@
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2"; inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446"; inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; }; inputs.flake-compat = {
url = "github:edolstra/flake-compat";
flake = false;
};
# dev tooling # dev tooling
inputs.flake-parts.url = "github:hercules-ci/flake-parts"; inputs.flake-parts.url = "github:hercules-ci/flake-parts";
@@ -18,18 +21,29 @@
inputs.git-hooks-nix.inputs.flake-compat.follows = ""; inputs.git-hooks-nix.inputs.flake-compat.follows = "";
inputs.git-hooks-nix.inputs.gitignore.follows = ""; inputs.git-hooks-nix.inputs.gitignore.follows = "";
outputs = inputs@{ self, nixpkgs, nixpkgs-regression, ... }: outputs =
inputs@{
self,
nixpkgs,
nixpkgs-regression,
...
}:
let let
inherit (nixpkgs) lib; inherit (nixpkgs) lib;
officialRelease = false; officialRelease = true;
linux32BitSystems = [ "i686-linux" ]; linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [ "x86_64-linux" "aarch64-linux" ]; linux64BitSystems = [
"x86_64-linux"
"aarch64-linux"
];
linuxSystems = linux32BitSystems ++ linux64BitSystems; linuxSystems = linux32BitSystems ++ linux64BitSystems;
darwinSystems = [ "x86_64-darwin" "aarch64-darwin" ]; darwinSystems = [
"x86_64-darwin"
"aarch64-darwin"
];
systems = linuxSystems ++ darwinSystems; systems = linuxSystems ++ darwinSystems;
crossSystems = [ crossSystems = [
@@ -59,63 +73,77 @@
(Provided that the names are unique.) (Provided that the names are unique.)
See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs See https://nixos.org/manual/nixpkgs/stable/index.html#function-library-lib.attrsets.concatMapAttrs
*/ */
flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs; flatMapAttrs = attrs: f: lib.concatMapAttrs f attrs;
forAllSystems = lib.genAttrs systems; forAllSystems = lib.genAttrs systems;
forAllCrossSystems = lib.genAttrs crossSystems; forAllCrossSystems = lib.genAttrs crossSystems;
forAllStdenvs = f: forAllStdenvs =
lib.listToAttrs f:
(map lib.listToAttrs (
(stdenvName: { map (stdenvName: {
name = "${stdenvName}Packages"; name = "${stdenvName}Packages";
value = f stdenvName; value = f stdenvName;
}) }) stdenvs
stdenvs); );
# We don't apply flake-parts to the whole flake so that non-development attributes # We don't apply flake-parts to the whole flake so that non-development attributes
# load without fetching any development inputs. # load without fetching any development inputs.
devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } { devFlake = inputs.flake-parts.lib.mkFlake { inherit inputs; } {
imports = [ ./maintainers/flake-module.nix ]; imports = [ ./maintainers/flake-module.nix ];
systems = lib.subtractLists crossSystems systems; systems = lib.subtractLists crossSystems systems;
perSystem = { system, ... }: { perSystem =
_module.args.pkgs = nixpkgsFor.${system}.native; { system, ... }:
}; {
_module.args.pkgs = nixpkgsFor.${system}.native;
};
}; };
# Memoize nixpkgs for different platforms for efficiency. # Memoize nixpkgs for different platforms for efficiency.
nixpkgsFor = forAllSystems nixpkgsFor = forAllSystems (
(system: let system:
make-pkgs = crossSystem: stdenv: import nixpkgs { let
localSystem = { make-pkgs =
inherit system; crossSystem: stdenv:
import nixpkgs {
localSystem = {
inherit system;
};
crossSystem =
if crossSystem == null then
null
else
{
config = crossSystem;
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
}; };
crossSystem = if crossSystem == null then null else {
config = crossSystem;
} // lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
};
overlays = [
(overlayFor (p: p.${stdenv}))
];
};
stdenvs = forAllStdenvs (make-pkgs null); stdenvs = forAllStdenvs (make-pkgs null);
native = stdenvs.stdenvPackages; native = stdenvs.stdenvPackages;
in { in
{
inherit stdenvs native; inherit stdenvs native;
static = native.pkgsStatic; static = native.pkgsStatic;
llvm = native.pkgsLLVM; llvm = native.pkgsLLVM;
cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv"); cross = forAllCrossSystems (crossSystem: make-pkgs crossSystem "stdenv");
}); }
);
binaryTarball = nix: pkgs: pkgs.callPackage ./scripts/binary-tarball.nix { binaryTarball =
inherit nix; nix: pkgs:
}; pkgs.callPackage ./scripts/binary-tarball.nix {
inherit nix;
};
overlayFor = getStdenv: final: prev: overlayFor =
getStdenv: final: prev:
let let
stdenv = getStdenv final; stdenv = getStdenv final;
in in
@@ -137,7 +165,9 @@
f = import ./packaging/components.nix { f = import ./packaging/components.nix {
inherit (final) lib; inherit (final) lib;
inherit officialRelease; inherit officialRelease;
pkgs = final;
src = self; src = self;
maintainers = [ ];
}; };
}; };
@@ -162,12 +192,19 @@
# See https://github.com/NixOS/nixpkgs/pull/214409 # See https://github.com/NixOS/nixpkgs/pull/214409
# Remove when fixed in this flake's nixpkgs # Remove when fixed in this flake's nixpkgs
pre-commit = pre-commit =
if prev.stdenv.hostPlatform.system == "i686-linux" if prev.stdenv.hostPlatform.system == "i686-linux" then
then (prev.pre-commit.override (o: { dotnet-sdk = ""; })).overridePythonAttrs (o: { doCheck = false; }) (prev.pre-commit.override (o: {
else prev.pre-commit; dotnet-sdk = "";
})).overridePythonAttrs
(o: {
doCheck = false;
})
else
prev.pre-commit;
}; };
in { in
{
# A Nixpkgs overlay that overrides the 'nix' and # A Nixpkgs overlay that overrides the 'nix' and
# 'nix-perl-bindings' packages. # 'nix-perl-bindings' packages.
overlays.default = overlayFor (p: p.stdenv); overlays.default = overlayFor (p: p.stdenv);
@@ -186,53 +223,93 @@
; ;
}; };
checks = forAllSystems (system: { checks = forAllSystems (
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system}; system:
installTests = self.hydraJobs.installTests.${system}; {
nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system}; installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
rl-next = installTests = self.hydraJobs.installTests.${system};
let pkgs = nixpkgsFor.${system}.native; nixpkgsLibTests = self.hydraJobs.tests.nixpkgsLibTests.${system};
in pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } '' rl-next =
LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out let
''; pkgs = nixpkgsFor.${system}.native;
repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { }; in
} // (lib.optionalAttrs (builtins.elem system linux64BitSystems)) { pkgs.buildPackages.runCommand "test-rl-next-release-notes" { } ''
dockerImage = self.hydraJobs.dockerImage.${system}; LANG=C.UTF-8 ${pkgs.changelog-d}/bin/changelog-d ${./doc/manual/rl-next} >$out
} // (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) { '';
# Some perl dependencies are broken on i686-linux. repl-completion = nixpkgsFor.${system}.native.callPackage ./tests/repl-completion.nix { };
# Since the support is only best-effort there, disable the perl
# bindings /**
perlBindings = self.hydraJobs.perlBindings.${system}; Checks for our packaging expressions.
} This shouldn't build anything significant; just check that things
# Add "passthru" tests (including derivations) are _set up_ correctly.
// flatMapAttrs ({ */
"" = nixpkgsFor.${system}.native; packaging-overriding =
} // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) { let
# TODO: enable static builds for darwin, blocked on: pkgs = nixpkgsFor.${system}.native;
# https://github.com/NixOS/nixpkgs/issues/320448 nix = self.packages.${system}.nix;
# TODO: disabled to speed up GHA CI. in
#"static-" = nixpkgsFor.${system}.static; assert (nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src.patches == [ pkgs.emptyFile ];
}) if pkgs.stdenv.buildPlatform.isDarwin then
(nixpkgsPrefix: nixpkgs: lib.warn "packaging-overriding check currently disabled because of a permissions issue on macOS" pkgs.emptyFile
flatMapAttrs nixpkgs.nixComponents else
(pkgName: pkg: # If this fails, something might be wrong with how we've wired the scope,
flatMapAttrs pkg.tests or {} # or something could be broken in Nixpkgs.
(testName: test: { pkgs.testers.testEqualContents {
"${nixpkgsPrefix}${pkgName}-${testName}" = test; assertion = "trivial patch does not change source contents";
}) expected = "${./.}";
actual =
# Same for all components; nix-util is an arbitrary pick
(nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src;
};
}
// (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
}
// (lib.optionalAttrs (!(builtins.elem system linux32BitSystems))) {
# Some perl dependencies are broken on i686-linux.
# Since the support is only best-effort there, disable the perl
# bindings
perlBindings = self.hydraJobs.perlBindings.${system};
}
# Add "passthru" tests
//
flatMapAttrs
(
{
"" = nixpkgsFor.${system}.native;
}
// lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
# TODO: disabled to speed up GHA CI.
#"static-" = nixpkgsFor.${system}.static;
}
) )
// lib.optionalAttrs (nixpkgs.stdenv.hostPlatform == nixpkgs.stdenv.buildPlatform) { (
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests; nixpkgsPrefix: nixpkgs:
} flatMapAttrs nixpkgs.nixComponents (
) pkgName: pkg:
// devFlake.checks.${system} or {} flatMapAttrs pkg.tests or { } (
testName: test: {
"${nixpkgsPrefix}${pkgName}-${testName}" = test;
}
)
)
// lib.optionalAttrs (nixpkgs.stdenv.hostPlatform == nixpkgs.stdenv.buildPlatform) {
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests;
}
)
// devFlake.checks.${system} or { }
); );
packages = forAllSystems (system: packages = forAllSystems (
{ # Here we put attributes that map 1:1 into packages.<system>, ie system:
{
# Here we put attributes that map 1:1 into packages.<system>, ie
# for which we don't apply the full build matrix such as cross or static. # for which we don't apply the full build matrix such as cross or static.
inherit (nixpkgsFor.${system}.native) inherit (nixpkgsFor.${system}.native)
changelog-d; changelog-d
;
default = self.packages.${system}.nix; default = self.packages.${system}.nix;
installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system}; installerScriptForGHA = self.hydraJobs.installerScriptForGHA.${system};
binaryTarball = self.hydraJobs.binaryTarball.${system}; binaryTarball = self.hydraJobs.binaryTarball.${system};
@@ -243,96 +320,143 @@
nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs; nix-external-api-docs = nixpkgsFor.${system}.native.nixComponents.nix-external-api-docs;
} }
# We need to flatten recursive attribute sets of derivations to pass `flake check`. # We need to flatten recursive attribute sets of derivations to pass `flake check`.
// flatMapAttrs //
{ # Components we'll iterate over in the upcoming lambda flatMapAttrs
"nix-util" = { }; {
"nix-util-c" = { }; # Components we'll iterate over in the upcoming lambda
"nix-util-test-support" = { }; "nix-util" = { };
"nix-util-tests" = { }; "nix-util-c" = { };
"nix-util-test-support" = { };
"nix-util-tests" = { };
"nix-store" = { }; "nix-store" = { };
"nix-store-c" = { }; "nix-store-c" = { };
"nix-store-test-support" = { }; "nix-store-test-support" = { };
"nix-store-tests" = { }; "nix-store-tests" = { };
"nix-fetchers" = { }; "nix-fetchers" = { };
"nix-fetchers-tests" = { }; "nix-fetchers-tests" = { };
"nix-expr" = { }; "nix-expr" = { };
"nix-expr-c" = { }; "nix-expr-c" = { };
"nix-expr-test-support" = { }; "nix-expr-test-support" = { };
"nix-expr-tests" = { }; "nix-expr-tests" = { };
"nix-flake" = { }; "nix-flake" = { };
"nix-flake-tests" = { }; "nix-flake-tests" = { };
"nix-main" = { }; "nix-main" = { };
"nix-main-c" = { }; "nix-main-c" = { };
"nix-cmd" = { }; "nix-cmd" = { };
"nix-cli" = { }; "nix-cli" = { };
"nix-everything" = { }; "nix-everything" = { };
"nix-functional-tests" = { supportsCross = false; }; "nix-functional-tests" = {
supportsCross = false;
};
"nix-perl-bindings" = { supportsCross = false; }; "nix-perl-bindings" = {
} supportsCross = false;
(pkgName: { supportsCross ? true }: { };
# These attributes go right into `packages.<system>`.
"${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
"${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
"${pkgName}-llvm" = nixpkgsFor.${system}.llvm.nixComponents.${pkgName};
} }
// lib.optionalAttrs supportsCross (flatMapAttrs (lib.genAttrs crossSystems (_: { })) (crossSystem: {}: { (
# These attributes go right into `packages.<system>`. pkgName:
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}; {
})) supportsCross ? true,
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (stdenvName: {}: { }:
# These attributes go right into `packages.<system>`. {
"${pkgName}-${stdenvName}" = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName}; # These attributes go right into `packages.<system>`.
}) "${pkgName}" = nixpkgsFor.${system}.native.nixComponents.${pkgName};
) "${pkgName}-static" = nixpkgsFor.${system}.static.nixComponents.${pkgName};
"${pkgName}-llvm" = nixpkgsFor.${system}.llvm.nixComponents.${pkgName};
}
// lib.optionalAttrs supportsCross (
flatMapAttrs (lib.genAttrs crossSystems (_: { })) (
crossSystem:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${crossSystem}" = nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName};
}
)
)
// flatMapAttrs (lib.genAttrs stdenvs (_: { })) (
stdenvName:
{ }:
{
# These attributes go right into `packages.<system>`.
"${pkgName}-${stdenvName}" =
nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".nixComponents.${pkgName};
}
)
)
// lib.optionalAttrs (builtins.elem system linux64BitSystems) { // lib.optionalAttrs (builtins.elem system linux64BitSystems) {
dockerImage = dockerImage =
let let
pkgs = nixpkgsFor.${system}.native; pkgs = nixpkgsFor.${system}.native;
image = import ./docker.nix { inherit pkgs; tag = pkgs.nix.version; }; image = import ./docker.nix {
in inherit pkgs;
pkgs.runCommand tag = pkgs.nix.version;
"docker-image-tarball-${pkgs.nix.version}" };
{ meta.description = "Docker image with Nix for ${system}"; } in
'' pkgs.runCommand "docker-image-tarball-${pkgs.nix.version}"
mkdir -p $out/nix-support { meta.description = "Docker image with Nix for ${system}"; }
image=$out/image.tar.gz ''
ln -s ${image} $image mkdir -p $out/nix-support
echo "file binary-dist $image" >> $out/nix-support/hydra-build-products image=$out/image.tar.gz
''; ln -s ${image} $image
}); echo "file binary-dist $image" >> $out/nix-support/hydra-build-products
'';
}
);
devShells = let devShells =
makeShell = import ./packaging/dev-shell.nix { inherit lib devFlake; }; let
prefixAttrs = prefix: lib.concatMapAttrs (k: v: { "${prefix}-${k}" = v; }); makeShell = import ./packaging/dev-shell.nix { inherit lib devFlake; };
in prefixAttrs = prefix: lib.concatMapAttrs (k: v: { "${prefix}-${k}" = v; });
forAllSystems (system: in
prefixAttrs "native" (forAllStdenvs (stdenvName: makeShell { forAllSystems (
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages"; system:
})) // prefixAttrs "native" (
lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) ( forAllStdenvs (
prefixAttrs "static" (forAllStdenvs (stdenvName: makeShell { stdenvName:
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsStatic; makeShell {
})) // pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages";
prefixAttrs "llvm" (forAllStdenvs (stdenvName: makeShell { }
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsLLVM; )
})) // )
prefixAttrs "cross" (forAllCrossSystems (crossSystem: makeShell { // lib.optionalAttrs (!nixpkgsFor.${system}.native.stdenv.isDarwin) (
pkgs = nixpkgsFor.${system}.cross.${crossSystem}; prefixAttrs "static" (
})) forAllStdenvs (
) // stdenvName:
{ makeShell {
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsStatic;
}
)
)
// prefixAttrs "llvm" (
forAllStdenvs (
stdenvName:
makeShell {
pkgs = nixpkgsFor.${system}.stdenvs."${stdenvName}Packages".pkgsLLVM;
}
)
)
// prefixAttrs "cross" (
forAllCrossSystems (
crossSystem:
makeShell {
pkgs = nixpkgsFor.${system}.cross.${crossSystem};
}
)
)
)
// {
default = self.devShells.${system}.native-stdenvPackages; default = self.devShells.${system}.native-stdenvPackages;
} }
); );
}; };
} }
File diff suppressed because it is too large. Load diff
+2
View File
@@ -2,6 +2,8 @@
# vim: set filetype=bash: # vim: set filetype=bash:
#!nix shell .#changelog-d --command bash #!nix shell .#changelog-d --command bash
set -euo pipefail
# --- CONFIGURATION --- # --- CONFIGURATION ---
# This does double duty for # This does double duty for
+8 -8
View File
@@ -42,7 +42,7 @@ my $flakeUrl = $evalInfo->{flake};
my $flakeInfo = decode_json(`nix flake metadata --json "$flakeUrl"` or die) if $flakeUrl; my $flakeInfo = decode_json(`nix flake metadata --json "$flakeUrl"` or die) if $flakeUrl;
my $nixRev = ($flakeInfo ? $flakeInfo->{revision} : $evalInfo->{jobsetevalinputs}->{nix}->{revision}) or die; my $nixRev = ($flakeInfo ? $flakeInfo->{revision} : $evalInfo->{jobsetevalinputs}->{nix}->{revision}) or die;
my $buildInfo = decode_json(fetch("$evalUrl/job/build.nix.x86_64-linux", 'application/json')); my $buildInfo = decode_json(fetch("$evalUrl/job/build.nix-everything.x86_64-linux", 'application/json'));
#print Dumper($buildInfo); #print Dumper($buildInfo);
my $releaseName = $buildInfo->{nixname}; my $releaseName = $buildInfo->{nixname};
@@ -91,7 +91,7 @@ sub getStorePath {
sub copyManual { sub copyManual {
my $manual; my $manual;
eval { eval {
$manual = getStorePath("build.nix.x86_64-linux", "doc"); $manual = getStorePath("manual");
}; };
if ($@) { if ($@) {
warn "$@"; warn "$@";
@@ -240,12 +240,12 @@ if ($haveDocker) {
# Upload nix-fallback-paths.nix. # Upload nix-fallback-paths.nix.
write_file("$tmpDir/fallback-paths.nix", write_file("$tmpDir/fallback-paths.nix",
"{\n" . "{\n" .
" x86_64-linux = \"" . getStorePath("build.nix.x86_64-linux") . "\";\n" . " x86_64-linux = \"" . getStorePath("build.nix-everything.x86_64-linux") . "\";\n" .
" i686-linux = \"" . getStorePath("build.nix.i686-linux") . "\";\n" . " i686-linux = \"" . getStorePath("build.nix-everything.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.nix.aarch64-linux") . "\";\n" . " aarch64-linux = \"" . getStorePath("build.nix-everything.aarch64-linux") . "\";\n" .
" riscv64-linux = \"" . getStorePath("buildCross.nix.riscv64-unknown-linux-gnu.x86_64-linux") . "\";\n" . " riscv64-linux = \"" . getStorePath("buildCross.nix-everything.riscv64-unknown-linux-gnu.x86_64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.nix.x86_64-darwin") . "\";\n" . " x86_64-darwin = \"" . getStorePath("build.nix-everything.x86_64-darwin") . "\";\n" .
" aarch64-darwin = \"" . getStorePath("build.nix.aarch64-darwin") . "\";\n" . " aarch64-darwin = \"" . getStorePath("build.nix-everything.aarch64-darwin") . "\";\n" .
"}\n"); "}\n");
# Upload release files to S3. # Upload release files to S3.
+364 -6
View File
@@ -1,29 +1,333 @@
{ {
lib, lib,
pkgs,
src, src,
officialRelease, officialRelease,
maintainers,
}: }:
scope: scope:
let let
inherit (scope) callPackage; inherit (scope)
callPackage
;
inherit
(scope.callPackage (
{ stdenv }:
{
inherit stdenv;
}
) { })
stdenv
;
inherit (pkgs.buildPackages)
meson
ninja
pkg-config
;
baseVersion = lib.fileContents ../.version; baseVersion = lib.fileContents ../.version;
versionSuffix = lib.optionalString (!officialRelease) "pre"; versionSuffix = lib.optionalString (!officialRelease) "pre";
fineVersionSuffix = lib.optionalString fineVersionSuffix =
(!officialRelease) lib.optionalString (!officialRelease)
"pre${builtins.substring 0 8 (src.lastModifiedDate or src.lastModified or "19700101")}_${src.shortRev or "dirty"}"; "pre${
builtins.substring 0 8 (src.lastModifiedDate or src.lastModified or "19700101")
}_${src.shortRev or "dirty"}";
fineVersion = baseVersion + fineVersionSuffix; fineVersion = baseVersion + fineVersionSuffix;
root = ../.;
# Indirection for Nixpkgs to override when package.nix files are vendored
filesetToSource = lib.fileset.toSource;
/**
Given a set of layers, create a mkDerivation-like function
*/
mkPackageBuilder =
exts: userFn: stdenv.mkDerivation (lib.extends (lib.composeManyExtensions exts) userFn);
setVersionLayer = finalAttrs: prevAttrs: {
preConfigure =
prevAttrs.preConfigure or ""
+
# Update the repo-global .version file.
# Symlink ./.version points there, but by default only workDir is writable.
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
};
localSourceLayer =
finalAttrs: prevAttrs:
let
workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
# the requirement that everything except passthru and meta must be
# serialized by mkDerivation, which doesn't work for this.
prevAttrs.workDir;
workDirSubpath = lib.path.removePrefix root workDirPath;
sources =
assert prevAttrs.fileset._type == "fileset";
prevAttrs.fileset;
src = lib.fileset.toSource {
fileset = sources;
inherit root;
};
in
{
sourceRoot = "${src.name}/" + workDirSubpath;
inherit src;
# Clear what `derivation` can't/shouldn't serialize; see prevAttrs.workDir.
fileset = null;
workDir = null;
};
resolveRelPath = p: lib.path.removePrefix root p;
makeFetchedSourceLayer =
finalScope: finalAttrs: prevAttrs:
let
workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
# the requirement that everything except passthru and meta must be
# serialized by mkDerivation, which doesn't work for this.
prevAttrs.workDir;
workDirSubpath = resolveRelPath workDirPath;
in
{
sourceRoot = "${finalScope.patchedSrc.name}/" + workDirSubpath;
src = finalScope.patchedSrc;
version =
let
n = lib.length finalScope.patches;
in
if n == 0 then prevAttrs.version else prevAttrs.version + "+${toString n}";
# Clear what `derivation` can't/shouldn't serialize; see prevAttrs.workDir.
fileset = null;
workDir = null;
};
mesonLayer = finalAttrs: prevAttrs: {
# NOTE:
# As of https://github.com/NixOS/nixpkgs/blob/8baf8241cea0c7b30e0b8ae73474cb3de83c1a30/pkgs/by-name/me/meson/setup-hook.sh#L26,
# `mesonBuildType` defaults to `plain` if not specified. We want our Nix-built binaries to be optimized by default.
# More on build types here: https://mesonbuild.com/Builtin-options.html#details-for-buildtype.
mesonBuildType = "release";
# NOTE:
# Users who are debugging Nix builds are expected to set the environment variable `mesonBuildType`, per the
# guidance in https://github.com/NixOS/nix/blob/8a3fc27f1b63a08ac983ee46435a56cf49ebaf4a/doc/manual/source/development/debugging.md?plain=1#L10.
# For this reason, we don't want to refer to `finalAttrs.mesonBuildType` here, but rather use the environment variable.
preConfigure =
prevAttrs.preConfigure or ""
+
lib.optionalString
(
!stdenv.hostPlatform.isWindows
# build failure
&& !stdenv.hostPlatform.isStatic
# LTO breaks exception handling on x86-64-darwin.
&& stdenv.system != "x86_64-darwin"
)
''
case "$mesonBuildType" in
release|minsize) appendToVar mesonFlags "-Db_lto=true" ;;
*) appendToVar mesonFlags "-Db_lto=false" ;;
esac
'';
nativeBuildInputs = [
meson
ninja
] ++ prevAttrs.nativeBuildInputs or [ ];
mesonCheckFlags = prevAttrs.mesonCheckFlags or [ ] ++ [
"--print-errorlogs"
];
};
mesonBuildLayer = finalAttrs: prevAttrs: {
nativeBuildInputs = prevAttrs.nativeBuildInputs or [ ] ++ [
pkg-config
];
separateDebugInfo = !stdenv.hostPlatform.isStatic;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
env =
prevAttrs.env or { }
// lib.optionalAttrs (
stdenv.isLinux
&& !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")
&& !(stdenv.hostPlatform.useLLVM or false)
) { LDFLAGS = "-fuse-ld=gold"; };
};
mesonLibraryLayer = finalAttrs: prevAttrs: {
outputs = prevAttrs.outputs or [ "out" ] ++ [ "dev" ];
};
# Work around weird `--as-needed` linker behavior with BSD, see
# https://github.com/mesonbuild/meson/issues/3593
bsdNoLinkAsNeeded =
finalAttrs: prevAttrs:
lib.optionalAttrs stdenv.hostPlatform.isBSD {
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
};
nixDefaultsLayer = finalAttrs: prevAttrs: {
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
pos = builtins.unsafeGetAttrPos "pname" prevAttrs;
meta = prevAttrs.meta or { } // {
homepage = prevAttrs.meta.homepage or "https://nixos.org/nix";
longDescription =
prevAttrs.longDescription or ''
Nix is a powerful package manager for mainly Linux and other Unix systems that
makes package management reliable and reproducible. It provides atomic
upgrades and rollbacks, side-by-side installation of multiple versions of
a package, multi-user package management and easy setup of build
environments.
'';
license = prevAttrs.meta.license or lib.licenses.lgpl21Plus;
maintainers = prevAttrs.meta.maintainers or [ ] ++ scope.maintainers;
platforms = prevAttrs.meta.platforms or (lib.platforms.unix ++ lib.platforms.windows);
};
};
/**
Append patches to the source layer.
*/
appendPatches =
scope: patches:
scope.overrideScope (
finalScope: prevScope: {
patches = prevScope.patches ++ patches;
}
);
in in
# This becomes the pkgs.nixComponents attribute set # This becomes the pkgs.nixComponents attribute set
{ {
version = baseVersion + versionSuffix; version = baseVersion + versionSuffix;
inherit versionSuffix; inherit versionSuffix;
inherit maintainers;
inherit filesetToSource;
/**
A user-provided extension function to apply to each component derivation.
*/
mesonComponentOverrides = finalAttrs: prevAttrs: { };
/**
An overridable derivation layer for handling the sources.
*/
sourceLayer = localSourceLayer;
/**
Resolve a path value to either itself or a path in the `src`, depending
whether `overrideSource` was called.
*/
resolvePath = p: p;
/**
Apply an extension function (i.e. overlay-shaped) to all component derivations.
*/
overrideAllMesonComponents =
f:
scope.overrideScope (
finalScope: prevScope: {
mesonComponentOverrides = lib.composeExtensions scope.mesonComponentOverrides f;
}
);
/**
Provide an alternate source. This allows the expressions to be vendored without copying the sources,
but it does make the build non-granular; all components will use a complete source.
Packaging expressions will be ignored.
Single argument: the source to use.
See also `appendPatches`
*/
overrideSource =
src:
scope.overrideScope (
finalScope: prevScope: {
sourceLayer = makeFetchedSourceLayer finalScope;
/**
Unpatched source for the build of Nix. Packaging expressions will be ignored.
*/
src = src;
/**
Patches for the whole Nix source. Changes to packaging expressions will be ignored.
*/
patches = [ ];
/**
Fetched and patched source to be used in component derivations.
*/
patchedSrc =
if finalScope.patches == [ ] then
src
else
pkgs.buildPackages.srcOnly (
pkgs.buildPackages.stdenvNoCC.mkDerivation {
name = "${finalScope.src.name or "nix-source"}-patched";
inherit (finalScope) src patches;
}
);
resolvePath = p: finalScope.patchedSrc + "/${resolveRelPath p}";
filesetToSource = { root, fileset }: finalScope.resolvePath root;
appendPatches = appendPatches finalScope;
}
);
/**
Append patches to be applied to the whole Nix source.
This affects all components.
Changes to the packaging expressions will be ignored.
*/
appendPatches =
patches:
# switch to "fetched" source first, so that patches apply to the whole tree.
(scope.overrideSource "${./..}").appendPatches patches;
mkMesonDerivation = mkPackageBuilder [
nixDefaultsLayer
scope.sourceLayer
setVersionLayer
mesonLayer
scope.mesonComponentOverrides
];
mkMesonExecutable = mkPackageBuilder [
nixDefaultsLayer
bsdNoLinkAsNeeded
scope.sourceLayer
setVersionLayer
mesonLayer
mesonBuildLayer
scope.mesonComponentOverrides
];
mkMesonLibrary = mkPackageBuilder [
nixDefaultsLayer
bsdNoLinkAsNeeded
scope.sourceLayer
mesonLayer
setVersionLayer
mesonBuildLayer
mesonLibraryLayer
scope.mesonComponentOverrides
];
nix-util = callPackage ../src/libutil/package.nix { }; nix-util = callPackage ../src/libutil/package.nix { };
nix-util-c = callPackage ../src/libutil-c/package.nix { }; nix-util-c = callPackage ../src/libutil-c/package.nix { };
@@ -54,7 +358,9 @@ in
nix-cli = callPackage ../src/nix/package.nix { version = fineVersion; }; nix-cli = callPackage ../src/nix/package.nix { version = fineVersion; };
nix-functional-tests = callPackage ../src/nix-functional-tests/package.nix { version = fineVersion; }; nix-functional-tests = callPackage ../tests/functional/package.nix {
version = fineVersion;
};
nix-manual = callPackage ../doc/manual/package.nix { version = fineVersion; }; nix-manual = callPackage ../doc/manual/package.nix { version = fineVersion; };
nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { version = fineVersion; }; nix-internal-api-docs = callPackage ../src/internal-api-docs/package.nix { version = fineVersion; };
@@ -62,5 +368,57 @@ in
nix-perl-bindings = callPackage ../src/perl/package.nix { }; nix-perl-bindings = callPackage ../src/perl/package.nix { };
nix-everything = callPackage ../packaging/everything.nix { }; nix-everything = callPackage ../packaging/everything.nix { } // {
# Note: no `passthru.overrideAllMesonComponents` etc
# This would propagate into `nix.overrideAttrs f`, but then discard
# `f` when `.overrideAllMesonComponents` is used.
# Both "methods" should be views on the same fixpoint overriding mechanism
# for that to work. For now, we intentionally don't support the broken
# two-fixpoint solution.
/**
Apply an extension function (i.e. overlay-shaped) to all component derivations, and return the nix package.
Single argument: the extension function to apply (finalAttrs: prevAttrs: { ... })
*/
overrideAllMesonComponents = f: (scope.overrideAllMesonComponents f).nix-everything;
/**
Append patches to be applied to the whole Nix source.
This affects all components.
Changes to the packaging expressions will be ignored.
Single argument: list of patches to apply
See also `overrideSource`
*/
appendPatches = ps: (scope.appendPatches ps).nix-everything;
/**
Provide an alternate source. This allows the expressions to be vendored without copying the sources,
but it does make the build non-granular; all components will use a complete source.
Packaging expressions will be ignored.
Filesets in the packaging expressions will be ignored.
Single argument: the source to use.
See also `appendPatches`
*/
overrideSource = src: (scope.overrideSource src).nix-everything;
/**
Override any internals of the Nix package set.
Single argument: the extension function to apply to the package set (finalScope: prevScope: { ... })
Example:
```
overrideScope (finalScope: prevScope: { aws-sdk-cpp = null; })
```
*/
overrideScope = f: (scope.overrideScope f).nix-everything;
};
} }
+61 -172
View File
@@ -17,11 +17,7 @@ in
let let
inherit (pkgs) lib; inherit (pkgs) lib;
root = ../.; stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64 then darwinStdenv else prevStdenv;
stdenv = if prevStdenv.isDarwin && prevStdenv.isx86_64
then darwinStdenv
else prevStdenv;
# Fix the following error with the default x86_64-darwin SDK: # Fix the following error with the default x86_64-darwin SDK:
# #
@@ -32,183 +28,76 @@ let
# all the way back to 10.6. # all the way back to 10.6.
darwinStdenv = pkgs.overrideSDK prevStdenv { darwinMinVersion = "10.13"; }; darwinStdenv = pkgs.overrideSDK prevStdenv { darwinMinVersion = "10.13"; };
# Nixpkgs implements this by returning a subpath into the fetched Nix sources.
resolvePath = p: p;
# Indirection for Nixpkgs to override when package.nix files are vendored
filesetToSource = lib.fileset.toSource;
/** Given a set of layers, create a mkDerivation-like function */
mkPackageBuilder = exts: userFn:
stdenv.mkDerivation (lib.extends (lib.composeManyExtensions exts) userFn);
localSourceLayer = finalAttrs: prevAttrs:
let
workDirPath =
# Ideally we'd pick finalAttrs.workDir, but for now `mkDerivation` has
# the requirement that everything except passthru and meta must be
# serialized by mkDerivation, which doesn't work for this.
prevAttrs.workDir;
workDirSubpath = lib.path.removePrefix root workDirPath;
sources = assert prevAttrs.fileset._type == "fileset"; prevAttrs.fileset;
src = lib.fileset.toSource { fileset = sources; inherit root; };
in
{
sourceRoot = "${src.name}/" + workDirSubpath;
inherit src;
# Clear what `derivation` can't/shouldn't serialize; see prevAttrs.workDir.
fileset = null;
workDir = null;
};
mesonLayer = finalAttrs: prevAttrs:
{
# NOTE:
# As of https://github.com/NixOS/nixpkgs/blob/8baf8241cea0c7b30e0b8ae73474cb3de83c1a30/pkgs/by-name/me/meson/setup-hook.sh#L26,
# `mesonBuildType` defaults to `plain` if not specified. We want our Nix-built binaries to be optimized by default.
# More on build types here: https://mesonbuild.com/Builtin-options.html#details-for-buildtype.
mesonBuildType = "release";
# NOTE:
# Users who are debugging Nix builds are expected to set the environment variable `mesonBuildType`, per the
# guidance in https://github.com/NixOS/nix/blob/8a3fc27f1b63a08ac983ee46435a56cf49ebaf4a/doc/manual/source/development/debugging.md?plain=1#L10.
# For this reason, we don't want to refer to `finalAttrs.mesonBuildType` here, but rather use the environment variable.
preConfigure = prevAttrs.preConfigure or "" + lib.optionalString (
!stdenv.hostPlatform.isWindows
# build failure
&& !stdenv.hostPlatform.isStatic
# LTO breaks exception handling on x86-64-darwin.
&& stdenv.system != "x86_64-darwin"
) ''
case "$mesonBuildType" in
release|minsize) appendToVar mesonFlags "-Db_lto=true" ;;
*) appendToVar mesonFlags "-Db_lto=false" ;;
esac
'';
nativeBuildInputs = [
pkgs.buildPackages.meson
pkgs.buildPackages.ninja
] ++ prevAttrs.nativeBuildInputs or [];
mesonCheckFlags = prevAttrs.mesonCheckFlags or [] ++ [
"--print-errorlogs"
];
};
mesonBuildLayer = finalAttrs: prevAttrs:
{
nativeBuildInputs = prevAttrs.nativeBuildInputs or [] ++ [
pkgs.buildPackages.pkg-config
];
separateDebugInfo = !stdenv.hostPlatform.isStatic;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
env = prevAttrs.env or {}
// lib.optionalAttrs
(stdenv.isLinux
&& !(stdenv.hostPlatform.isStatic && stdenv.system == "aarch64-linux")
&& !(stdenv.hostPlatform.useLLVM or false))
{ LDFLAGS = "-fuse-ld=gold"; };
};
mesonLibraryLayer = finalAttrs: prevAttrs:
{
outputs = prevAttrs.outputs or [ "out" ] ++ [ "dev" ];
};
# Work around weird `--as-needed` linker behavior with BSD, see
# https://github.com/mesonbuild/meson/issues/3593
bsdNoLinkAsNeeded = finalAttrs: prevAttrs:
lib.optionalAttrs stdenv.hostPlatform.isBSD {
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [];
};
miscGoodPractice = finalAttrs: prevAttrs:
{
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
};
in in
scope: { scope: {
inherit stdenv; inherit stdenv;
aws-sdk-cpp = (pkgs.aws-sdk-cpp.override { aws-sdk-cpp =
apis = [ "s3" "transfer" ]; (pkgs.aws-sdk-cpp.override {
customMemoryManagement = false; apis = [
}).overrideAttrs { "s3"
# only a stripped down version is built, which takes a lot less resources "transfer"
# to build, so we don't need a "big-parallel" machine. ];
requiredSystemFeatures = [ ]; customMemoryManagement = false;
}; }).overrideAttrs
{
# only a stripped down version is built, which takes a lot less resources
# to build, so we don't need a "big-parallel" machine.
requiredSystemFeatures = [ ];
};
boehmgc = pkgs.boehmgc.override { boehmgc = pkgs.boehmgc.override {
enableLargeConfig = true; enableLargeConfig = true;
}; };
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed. # TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost = (pkgs.boost.override { boost =
extraB2Args = [ (pkgs.boost.override {
"--with-container" extraB2Args = [
"--with-context" "--with-container"
"--with-coroutine" "--with-context"
]; "--with-coroutine"
}).overrideAttrs (old: {
# Need to remove `--with-*` to use `--with-libraries=...`
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 = pkgs.libgit2.overrideAttrs (attrs: {
cmakeFlags = attrs.cmakeFlags or []
++ [ "-DUSE_SSH=exec" ];
nativeBuildInputs = attrs.nativeBuildInputs or []
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
]; ];
# Only `git apply` can handle git binary patches }).overrideAttrs
prePatch = attrs.prePatch or "" (old: {
+ lib.optionalString (!stdenv.hostPlatform.isWindows) '' # Need to remove `--with-*` to use `--with-libraries=...`
patch() { buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
git apply installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
} });
'';
patches = attrs.patches or []
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
inherit resolvePath filesetToSource; libgit2 = pkgs.libgit2.overrideAttrs (
attrs:
mkMesonDerivation = {
mkPackageBuilder [ cmakeFlags = attrs.cmakeFlags or [ ] ++ [ "-DUSE_SSH=exec" ];
miscGoodPractice }
localSourceLayer # libgit2: Nixpkgs 24.11 has < 1.9.0, which needs our patches
mesonLayer // lib.optionalAttrs (!lib.versionAtLeast pkgs.libgit2.version "1.9.0") {
]; nativeBuildInputs =
mkMesonExecutable = attrs.nativeBuildInputs or [ ]
mkPackageBuilder [ # gitMinimal does not build on Windows. See packbuilder patch.
miscGoodPractice ++ lib.optionals (!stdenv.hostPlatform.isWindows) [
bsdNoLinkAsNeeded # Needed for `git apply`; see `prePatch`
localSourceLayer pkgs.buildPackages.gitMinimal
mesonLayer ];
mesonBuildLayer # Only `git apply` can handle git binary patches
]; prePatch =
mkMesonLibrary = attrs.prePatch or ""
mkPackageBuilder [ + lib.optionalString (!stdenv.hostPlatform.isWindows) ''
miscGoodPractice patch() {
bsdNoLinkAsNeeded git apply
localSourceLayer }
mesonLayer '';
mesonBuildLayer patches =
mesonLibraryLayer attrs.patches or [ ]
]; ++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
}
);
} }
+112 -108
View File
@@ -2,127 +2,131 @@
{ pkgs }: { pkgs }:
pkgs.nixComponents.nix-util.overrideAttrs (attrs: pkgs.nixComponents.nix-util.overrideAttrs (
attrs:
let let
stdenv = pkgs.nixDependencies.stdenv; stdenv = pkgs.nixDependencies.stdenv;
buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform; buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
modular = devFlake.getSystem stdenv.buildPlatform.system; modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag = prefix: flag: transformFlag =
assert builtins.isString flag; prefix: flag:
let assert builtins.isString flag;
rest = builtins.substring 2 (builtins.stringLength flag) flag; let
in rest = builtins.substring 2 (builtins.stringLength flag) flag;
in
"-D${prefix}:${rest}"; "-D${prefix}:${rest}";
havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix; havePerl = stdenv.buildPlatform == stdenv.hostPlatform && stdenv.hostPlatform.isUnix;
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags; ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
in { in
pname = "shell-for-" + attrs.pname; {
pname = "shell-for-" + attrs.pname;
# Remove the version suffix to avoid unnecessary attempts to substitute in nix develop # Remove the version suffix to avoid unnecessary attempts to substitute in nix develop
version = lib.fileContents ../.version; version = lib.fileContents ../.version;
name = attrs.pname; name = attrs.pname;
installFlags = "sysconfdir=$(out)/etc"; installFlags = "sysconfdir=$(out)/etc";
shellHook = '' shellHook = ''
PATH=$prefix/bin:$PATH PATH=$prefix/bin:$PATH
unset PYTHONPATH unset PYTHONPATH
export MANPATH=$out/share/man:$MANPATH export MANPATH=$out/share/man:$MANPATH
# Make bash completion work. # Make bash completion work.
XDG_DATA_DIRS+=:$out/share XDG_DATA_DIRS+=:$out/share
# Make the default phases do the right thing. # Make the default phases do the right thing.
# FIXME: this wouldn't be needed if the ninja package set buildPhase() instead of $buildPhase. # FIXME: this wouldn't be needed if the ninja package set buildPhase() instead of $buildPhase.
# FIXME: mesonConfigurePhase shouldn't cd to the build directory. It would be better to pass '-C <dir>' to ninja. # FIXME: mesonConfigurePhase shouldn't cd to the build directory. It would be better to pass '-C <dir>' to ninja.
cdToBuildDir() { cdToBuildDir() {
if [[ ! -e build.ninja ]]; then if [[ ! -e build.ninja ]]; then
cd build cd build
fi fi
} }
configurePhase() { configurePhase() {
mesonConfigurePhase mesonConfigurePhase
} }
buildPhase() { buildPhase() {
cdToBuildDir cdToBuildDir
ninjaBuildPhase ninjaBuildPhase
} }
checkPhase() { checkPhase() {
cdToBuildDir cdToBuildDir
mesonCheckPhase mesonCheckPhase
} }
installPhase() { installPhase() {
cdToBuildDir cdToBuildDir
ninjaInstallPhase ninjaInstallPhase
} }
''; '';
# We use this shell with the local checkout, not unpackPhase. # We use this shell with the local checkout, not unpackPhase.
src = null; src = null;
env = { env = {
# Needed for Meson to find Boost. # For `make format`, to work without installing pre-commit
# https://github.com/NixOS/nixpkgs/issues/86131. _NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include"; modular.pre-commit.settings.rawConfig
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib"; }";
# For `make format`, to work without installing pre-commit };
_NIX_PRE_COMMIT_HOOKS_CONFIG =
"${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml" modular.pre-commit.settings.rawConfig}";
};
mesonFlags = mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags) map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents.nix-util.mesonFlags)
++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags) ++ map (transformFlag "libstore") (ignoreCrossFile pkgs.nixComponents.nix-store.mesonFlags)
++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags) ++ map (transformFlag "libfetchers") (ignoreCrossFile pkgs.nixComponents.nix-fetchers.mesonFlags)
++ lib.optionals havePerl (map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)) ++ lib.optionals havePerl (
++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags) map (transformFlag "perl") (ignoreCrossFile pkgs.nixComponents.nix-perl-bindings.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags) )
; ++ map (transformFlag "libexpr") (ignoreCrossFile pkgs.nixComponents.nix-expr.mesonFlags)
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents.nix-cmd.mesonFlags);
nativeBuildInputs = attrs.nativeBuildInputs or [] nativeBuildInputs =
++ pkgs.nixComponents.nix-util.nativeBuildInputs attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents.nix-store.nativeBuildInputs ++ pkgs.nixComponents.nix-util.nativeBuildInputs
++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs ++ pkgs.nixComponents.nix-store.nativeBuildInputs
++ pkgs.nixComponents.nix-expr.nativeBuildInputs ++ pkgs.nixComponents.nix-fetchers.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs ++ pkgs.nixComponents.nix-expr.nativeBuildInputs
++ lib.optionals buildCanExecuteHost pkgs.nixComponents.nix-manual.externalNativeBuildInputs ++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.nativeBuildInputs
++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs ++ lib.optionals buildCanExecuteHost pkgs.nixComponents.nix-manual.externalNativeBuildInputs
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs ++ pkgs.nixComponents.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-functional-tests.externalNativeBuildInputs ++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ lib.optional ++ pkgs.nixComponents.nix-functional-tests.externalNativeBuildInputs
(!buildCanExecuteHost ++ lib.optional (
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479 !buildCanExecuteHost
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin) # Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages && !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)) && stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
pkgs.buildPackages.mesonEmulatorHook && lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
++ [ ) pkgs.buildPackages.mesonEmulatorHook
pkgs.buildPackages.cmake ++ [
pkgs.buildPackages.shellcheck pkgs.buildPackages.cmake
pkgs.buildPackages.changelog-d pkgs.buildPackages.shellcheck
modular.pre-commit.settings.package pkgs.buildPackages.changelog-d
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.package
modular.pre-commit.settings.installationScript) (pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
] ]
# TODO: Remove the darwin check once # TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available # https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear ++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (lib.hiPrio pkgs.buildPackages.clang-tools); ++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
);
buildInputs = attrs.buildInputs or [] buildInputs =
++ pkgs.nixComponents.nix-util.buildInputs attrs.buildInputs or [ ]
++ pkgs.nixComponents.nix-store.buildInputs ++ pkgs.nixComponents.nix-util.buildInputs
++ pkgs.nixComponents.nix-store-tests.externalBuildInputs ++ pkgs.nixComponents.nix-store.buildInputs
++ pkgs.nixComponents.nix-fetchers.buildInputs ++ pkgs.nixComponents.nix-store-tests.externalBuildInputs
++ pkgs.nixComponents.nix-expr.buildInputs ++ pkgs.nixComponents.nix-fetchers.buildInputs
++ pkgs.nixComponents.nix-expr.externalPropagatedBuildInputs ++ pkgs.nixComponents.nix-expr.buildInputs
++ pkgs.nixComponents.nix-cmd.buildInputs ++ pkgs.nixComponents.nix-expr.externalPropagatedBuildInputs
++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.externalBuildInputs ++ pkgs.nixComponents.nix-cmd.buildInputs
++ lib.optional havePerl pkgs.perl ++ lib.optionals havePerl pkgs.nixComponents.nix-perl-bindings.externalBuildInputs
; ++ lib.optional havePerl pkgs.perl;
}) }
)
+157 -105
View File
@@ -1,8 +1,11 @@
{ {
lib, lib,
stdenv, stdenv,
lndir,
buildEnv, buildEnv,
maintainers,
nix-util, nix-util,
nix-util-c, nix-util-c,
nix-util-tests, nix-util-tests,
@@ -38,74 +41,37 @@
nix-perl-bindings, nix-perl-bindings,
testers, testers,
runCommand,
patchedSrc ? null,
}: }:
let let
libs = { libs =
inherit {
nix-util inherit
nix-util-c nix-util
nix-store nix-util-c
nix-store-c nix-store
nix-fetchers nix-store-c
nix-expr nix-fetchers
nix-expr-c nix-expr
nix-flake nix-expr-c
nix-flake-c nix-flake
nix-main nix-flake-c
nix-main-c nix-main
nix-cmd nix-main-c
; nix-cmd
} // lib.optionalAttrs (!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform) { ;
# Currently fails in static build }
inherit // lib.optionalAttrs
nix-perl-bindings (!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform)
; {
}; # Currently fails in static build
inherit
dev = stdenv.mkDerivation (finalAttrs: { nix-perl-bindings
name = "nix-${nix-cli.version}-dev"; ;
pname = "nix";
version = nix-cli.version;
dontUnpack = true;
dontBuild = true;
libs = map lib.getDev (lib.attrValues libs);
installPhase = ''
mkdir -p $out/nix-support
echo $libs >> $out/nix-support/propagated-build-inputs
'';
passthru = {
tests = {
pkg-config =
testers.hasPkgConfigModules {
package = finalAttrs.finalPackage;
};
}; };
# If we were to fully emulate output selection here, we'd confuse the Nix CLIs,
# because they rely on `drvPath`.
dev = finalAttrs.finalPackage.out;
libs = throw "`nix.dev.libs` is not meant to be used; use `nix.libs` instead.";
};
meta = {
pkgConfigModules = [
"nix-cmd"
"nix-expr"
"nix-expr-c"
"nix-fetchers"
"nix-flake"
"nix-flake-c"
"nix-main"
"nix-main-c"
"nix-store"
"nix-store-c"
"nix-util"
"nix-util-c"
];
};
});
devdoc = buildEnv { devdoc = buildEnv {
name = "nix-${nix-cli.version}-devdoc"; name = "nix-${nix-cli.version}-devdoc";
paths = [ paths = [
@@ -115,39 +81,100 @@ let
}; };
in in
(buildEnv { stdenv.mkDerivation (finalAttrs: {
name = "nix-${nix-cli.version}"; pname = "nix";
paths = [ version = nix-cli.version;
nix-cli
nix-manual.man /**
This package uses a multi-output derivation, even though some outputs could
have been provided directly by the constituent component that provides it.
This is because not all tooling handles packages composed of arbitrary
outputs yet. This includes nix itself, https://github.com/NixOS/nix/issues/6507.
`devdoc` is also available, but not listed here, because this attribute is
not an output of the same derivation that provides `out`, `dev`, etc.
*/
outputs = [
"out"
"dev"
"doc"
"man"
]; ];
meta.mainProgram = "nix"; /**
}).overrideAttrs (finalAttrs: prevAttrs: { Unpacking is handled in this package's constituent components
*/
dontUnpack = true;
/**
Building is handled in this package's constituent components
*/
dontBuild = true;
/**
`doCheck` controles whether tests are added as build gate for the combined package.
This includes both the unit tests and the functional tests, but not the
integration tests that run in CI (the flake's `hydraJobs` and some of the `checks`).
*/
doCheck = true; doCheck = true;
doInstallCheck = true;
checkInputs = [ /**
# Make sure the unit tests have passed `fixupPhase` currently doesn't understand that a symlink output isn't writable.
nix-util-tests.tests.run
nix-store-tests.tests.run
nix-expr-tests.tests.run
nix-fetchers-tests.tests.run
nix-flake-tests.tests.run
# Make sure the functional tests have passed We don't compile or link anything in this derivation, so fixups aren't needed.
nix-functional-tests */
dontFixup = true;
# dev bundle is ok checkInputs =
# (checkInputs must be empty paths??) [
(runCommand "check-pkg-config" { checked = dev.tests.pkg-config; } "mkdir $out") # Make sure the unit tests have passed
] ++ lib.optionals (!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform) [ nix-util-tests.tests.run
# Perl currently fails in static build nix-store-tests.tests.run
# TODO: Split out tests into a separate derivation? nix-expr-tests.tests.run
nix-perl-bindings nix-fetchers-tests.tests.run
nix-flake-tests.tests.run
# Make sure the functional tests have passed
nix-functional-tests
]
++ lib.optionals
(!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform)
[
# Perl currently fails in static build
# TODO: Split out tests into a separate derivation?
nix-perl-bindings
];
nativeBuildInputs = [
lndir
]; ];
passthru = prevAttrs.passthru // {
installPhase =
let
devPaths = lib.mapAttrsToList (_k: lib.getDev) finalAttrs.finalPackage.libs;
in
''
mkdir -p $out $dev/nix-support
# Custom files
echo $libs >> $dev/nix-support/propagated-build-inputs
echo ${nix-cli} ${lib.escapeShellArgs devPaths} >> $dev/nix-support/propagated-build-inputs
# Merged outputs
lndir ${nix-cli} $out
for lib in ${lib.escapeShellArgs devPaths}; do
lndir $lib $dev
done
# Forwarded outputs
ln -sT ${nix-manual} $doc
ln -sT ${nix-manual.man} $man
'';
passthru = {
inherit (nix-cli) version; inherit (nix-cli) version;
src = patchedSrc;
/** /**
These are the libraries that are part of the Nix project. They are used These are the libraries that are part of the Nix project. They are used
@@ -165,28 +192,53 @@ in
unusedInputsForTests = [ nix ]; unusedInputsForTests = [ nix ];
disallowedReferences = nix.all; disallowedReferences = nix.all;
``` ```
*/ */
inherit libs; inherit libs;
tests = prevAttrs.passthru.tests or {} // { /**
# TODO: create a proper fixpoint and: Developer documentation for `nix`, in `share/doc/nix/{internal,external}-api/`.
# pkg-config =
# testers.hasPkgConfigModules { This is not a proper output; see `outputs` for context.
# package = finalPackage; */
# }; inherit devdoc;
};
/** /**
A derivation referencing the `dev` outputs of the Nix libraries. Extra tests that test this package, but do not run as part of the build.
*/ See <https://nixos.org/manual/nixpkgs/stable/index.html#var-passthru-tests>
inherit dev; */
inherit devdoc; tests = {
doc = nix-manual; pkg-config = testers.hasPkgConfigModules {
outputs = [ "out" "dev" "devdoc" "doc" ]; package = finalAttrs.finalPackage;
all = lib.attrValues (lib.genAttrs finalAttrs.passthru.outputs (outName: finalAttrs.finalPackage.${outName})); };
};
}; };
meta = prevAttrs.meta // {
meta = {
mainProgram = "nix";
description = "The Nix package manager"; description = "The Nix package manager";
pkgConfigModules = dev.meta.pkgConfigModules; longDescription = nix-cli.meta.longDescription;
homepage = nix-cli.meta.homepage;
license = nix-cli.meta.license;
maintainers = maintainers;
platforms = nix-cli.meta.platforms;
outputsToInstall = [
"out"
"man"
];
pkgConfigModules = [
"nix-cmd"
"nix-expr"
"nix-expr-c"
"nix-fetchers"
"nix-flake"
"nix-flake-c"
"nix-main"
"nix-main-c"
"nix-store"
"nix-store-c"
"nix-util"
"nix-util-c"
];
}; };
}) })
+125 -80
View File
@@ -1,22 +1,25 @@
{ inputs {
, binaryTarball inputs,
, forAllCrossSystems binaryTarball,
, forAllSystems forAllCrossSystems,
, lib forAllSystems,
, linux64BitSystems lib,
, nixpkgsFor linux64BitSystems,
, self nixpkgsFor,
, officialRelease self,
officialRelease,
}: }:
let let
inherit (inputs) nixpkgs nixpkgs-regression; inherit (inputs) nixpkgs nixpkgs-regression;
installScriptFor = tarballs: installScriptFor =
tarballs:
nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix { nixpkgsFor.x86_64-linux.native.callPackage ../scripts/installer.nix {
inherit tarballs; inherit tarballs;
}; };
testNixVersions = pkgs: daemon: testNixVersions =
pkgs: daemon:
pkgs.nixComponents.nix-functional-tests.override { pkgs.nixComponents.nix-functional-tests.override {
pname = "nix-daemon-compat-tests"; pname = "nix-daemon-compat-tests";
version = "${pkgs.nix.version}-with-daemon-${daemon.version}"; version = "${pkgs.nix.version}-with-daemon-${daemon.version}";
@@ -54,44 +57,70 @@ let
in in
{ {
# Binary package for various platforms. # Binary package for various platforms.
build = forAllPackages (pkgName: build = forAllPackages (
forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})); pkgName: forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.${pkgName})
);
shellInputs = removeAttrs shellInputs = removeAttrs (forAllSystems (
(forAllSystems (system: self.devShells.${system}.default.inputDerivation)) system: self.devShells.${system}.default.inputDerivation
[ "i686-linux" ]; )) [ "i686-linux" ];
buildStatic = forAllPackages (pkgName: buildStatic = forAllPackages (
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})); pkgName:
lib.genAttrs linux64BitSystems (system: nixpkgsFor.${system}.static.nixComponents.${pkgName})
);
buildCross = forAllPackages (pkgName: buildCross = forAllPackages (
pkgName:
# Hack to avoid non-evaling package # Hack to avoid non-evaling package
(if pkgName == "nix-functional-tests" then lib.flip builtins.removeAttrs ["x86_64-w64-mingw32"] else lib.id) (
(forAllCrossSystems (crossSystem: if pkgName == "nix-functional-tests" then
lib.genAttrs [ "x86_64-linux" ] (system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName})))); lib.flip builtins.removeAttrs [ "x86_64-w64-mingw32" ]
else
lib.id
)
(
forAllCrossSystems (
crossSystem:
lib.genAttrs [ "x86_64-linux" ] (
system: nixpkgsFor.${system}.cross.${crossSystem}.nixComponents.${pkgName}
)
)
)
);
buildNoGc = let buildNoGc =
components = forAllSystems (system: let
nixpkgsFor.${system}.native.nixComponents.overrideScope (self: super: { components = forAllSystems (
nix-expr = super.nix-expr.override { enableGC = false; }; system:
}) nixpkgsFor.${system}.native.nixComponents.overrideScope (
); self: super: {
in forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName})); nix-expr = super.nix-expr.override { enableGC = false; };
}
)
);
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-cli); buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-cli);
# Toggles some settings for better coverage. Windows needs these # Toggles some settings for better coverage. Windows needs these
# library combinations, and Debian build Nix with GNU readline too. # library combinations, and Debian build Nix with GNU readline too.
buildReadlineNoMarkdown = let buildReadlineNoMarkdown =
components = forAllSystems (system: let
nixpkgsFor.${system}.native.nixComponents.overrideScope (self: super: { components = forAllSystems (
nix-cmd = super.nix-cmd.override { system:
enableMarkdown = false; nixpkgsFor.${system}.native.nixComponents.overrideScope (
readlineFlavor = "readline"; self: super: {
}; nix-cmd = super.nix-cmd.override {
}) enableMarkdown = false;
); readlineFlavor = "readline";
in forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName})); };
}
)
);
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
# Perl bindings for various platforms. # Perl bindings for various platforms.
perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-perl-bindings); perlBindings = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents.nix-perl-bindings);
@@ -99,13 +128,18 @@ in
# Binary tarball for various platforms, containing a Nix store # Binary tarball for various platforms, containing a Nix store
# with the closure of 'nix' package, and the second half of # with the closure of 'nix' package, and the second half of
# the installation script. # the installation script.
binaryTarball = forAllSystems (system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native); binaryTarball = forAllSystems (
system: binaryTarball nixpkgsFor.${system}.native.nix nixpkgsFor.${system}.native
);
binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (system: binaryTarballCross = lib.genAttrs [ "x86_64-linux" ] (
forAllCrossSystems (crossSystem: system:
binaryTarball forAllCrossSystems (
nixpkgsFor.${system}.cross.${crossSystem}.nix crossSystem:
nixpkgsFor.${system}.cross.${crossSystem})); binaryTarball nixpkgsFor.${system}.cross.${crossSystem}.nix
nixpkgsFor.${system}.cross.${crossSystem}
)
);
# The first half of the installation script. This is uploaded # The first half of the installation script. This is uploaded
# to https://nixos.org/nix/install. It downloads the binary # to https://nixos.org/nix/install. It downloads the binary
@@ -124,9 +158,12 @@ in
self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu" self.hydraJobs.binaryTarballCross."x86_64-linux"."riscv64-unknown-linux-gnu"
]; ];
installerScriptForGHA = forAllSystems (system: nixpkgsFor.${system}.native.callPackage ../scripts/installer.nix { installerScriptForGHA = forAllSystems (
tarballs = [ self.hydraJobs.binaryTarball.${system} ]; system:
}); nixpkgsFor.${system}.native.callPackage ../scripts/installer.nix {
tarballs = [ self.hydraJobs.binaryTarball.${system} ];
}
);
# docker image with Nix inside # docker image with Nix inside
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage); dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
@@ -147,16 +184,24 @@ in
external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs; external-api-docs = nixpkgsFor.x86_64-linux.native.nixComponents.nix-external-api-docs;
# System tests. # System tests.
tests = import ../tests/nixos { inherit lib nixpkgs nixpkgsFor self; } // { tests =
import ../tests/nixos {
inherit
lib
nixpkgs
nixpkgsFor
self
;
}
// {
# Make sure that nix-env still produces the exact same result # Make sure that nix-env still produces the exact same result
# on a particular version of Nixpkgs. # on a particular version of Nixpkgs.
evalNixpkgs = evalNixpkgs =
let let
inherit (nixpkgsFor.x86_64-linux.native) runCommand nix; inherit (nixpkgsFor.x86_64-linux.native) runCommand nix;
in in
runCommand "eval-nixos" { buildInputs = [ nix ]; } runCommand "eval-nixos" { buildInputs = [ nix ]; } ''
''
type -p nix-env type -p nix-env
# Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593. # Note: we're filtering out nixos-install-tools because https://github.com/NixOS/nixpkgs/pull/153594#issuecomment-1020530593.
( (
@@ -167,36 +212,36 @@ in
mkdir $out mkdir $out
''; '';
nixpkgsLibTests = nixpkgsLibTests = forAllSystems (
forAllSystems (system: system:
import (nixpkgs + "/lib/tests/test-with-nix.nix") import (nixpkgs + "/lib/tests/test-with-nix.nix") {
{ lib = nixpkgsFor.${system}.native.lib;
lib = nixpkgsFor.${system}.native.lib; nix = self.packages.${system}.nix-cli;
nix = self.packages.${system}.nix-cli; pkgs = nixpkgsFor.${system}.native;
pkgs = nixpkgsFor.${system}.native; }
}
); );
}; };
metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" { metrics.nixpkgs = import "${nixpkgs-regression}/pkgs/top-level/metrics.nix" {
pkgs = nixpkgsFor.x86_64-linux.native; pkgs = nixpkgsFor.x86_64-linux.native;
nixpkgs = nixpkgs-regression; nixpkgs = nixpkgs-regression;
}; };
installTests = forAllSystems (system: installTests = forAllSystems (
let pkgs = nixpkgsFor.${system}.native; in system:
pkgs.runCommand "install-tests" let
{ pkgs = nixpkgsFor.${system}.native;
againstSelf = testNixVersions pkgs pkgs.nix; in
againstCurrentLatest = pkgs.runCommand "install-tests" {
# FIXME: temporarily disable this on macOS because of #3605. againstSelf = testNixVersions pkgs pkgs.nix;
if system == "x86_64-linux" againstCurrentLatest =
then testNixVersions pkgs pkgs.nixVersions.latest # FIXME: temporarily disable this on macOS because of #3605.
else null; if system == "x86_64-linux" then testNixVersions pkgs pkgs.nixVersions.latest else null;
# Disabled because the latest stable version doesn't handle # Disabled because the latest stable version doesn't handle
# `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work # `NIX_DAEMON_SOCKET_PATH` which is required for the tests to work
# againstLatestStable = testNixVersions pkgs pkgs.nixStable; # againstLatestStable = testNixVersions pkgs pkgs.nixStable;
} "touch $out"); } "touch $out"
);
installerTests = import ../tests/installer { installerTests = import ../tests/installer {
binaryTarballs = self.hydraJobs.binaryTarball; binaryTarballs = self.hydraJobs.binaryTarball;
+10 -6
View File
@@ -1,14 +1,18 @@
{ runCommand {
, system runCommand,
, buildPackages system,
, cacert buildPackages,
, nix cacert,
nix,
}: }:
let let
installerClosureInfo = buildPackages.closureInfo { installerClosureInfo = buildPackages.closureInfo {
rootPaths = [ nix cacert ]; rootPaths = [
nix
cacert
];
}; };
inherit (nix) version; inherit (nix) version;
+37 -31
View File
@@ -1,36 +1,42 @@
{ lib {
, runCommand lib,
, nix runCommand,
, tarballs nix,
tarballs,
}: }:
runCommand "installer-script" { runCommand "installer-script"
buildInputs = [ nix ]; {
} '' buildInputs = [ nix ];
mkdir -p $out/nix-support
# Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
tarballPath() {
# Remove the store prefix
local path=''${1#${builtins.storeDir}/}
# Get the path relative to the derivation root
local rest=''${path#*/}
# Get the derivation hash
local drvHash=''${path%%-*}
echo "$drvHash/$rest"
} }
''
mkdir -p $out/nix-support
substitute ${./install.in} $out/install \ # Converts /nix/store/50p3qk8k...-nix-2.4pre20201102_550e11f/bin/nix to 50p3qk8k.../bin/nix.
${lib.concatMapStrings tarballPath() {
(tarball: let # Remove the store prefix
inherit (tarball.stdenv.hostPlatform) system; local path=''${1#${builtins.storeDir}/}
in '' \ # Get the path relative to the derivation root
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \ local rest=''${path#*/}
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \ # Get the derivation hash
'' local drvHash=''${path%%-*}
) echo "$drvHash/$rest"
tarballs }
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products substitute ${./install.in} $out/install \
'' ${
lib.concatMapStrings (
tarball:
let
inherit (tarball.stdenv.hostPlatform) system;
in
''
\
--replace '@tarballHash_${system}@' $(nix --experimental-features nix-command hash-file --base16 --type sha256 ${tarball}/*.tar.xz) \
--replace '@tarballPath_${system}@' $(tarballPath ${tarball}/*.tar.xz) \
''
) tarballs
} --replace '@nixVersion@' ${nix.version}
echo "file installer $out/install" >> $out/nix-support/hydra-build-products
''
+1 -1
View File
@@ -1,7 +1,7 @@
# Only execute this file once per shell. # Only execute this file once per shell.
# This file is tested by tests/installer/default.nix. # This file is tested by tests/installer/default.nix.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1 export __ETC_PROFILE_NIX_SOURCED=1
NIX_LINK=$HOME/.nix-profile NIX_LINK=$HOME/.nix-profile
if [ -n "${XDG_STATE_HOME-}" ]; then if [ -n "${XDG_STATE_HOME-}" ]; then
+1 -1
View File
@@ -29,7 +29,7 @@ if test -n "$HOME" && test -n "$USER"
end end
# Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work. # Set $NIX_SSL_CERT_FILE so that Nixpkgs applications like curl work.
if test -n "$NIX_SSH_CERT_FILE" if test -n "$NIX_SSL_CERT_FILE"
: # Allow users to override the NIX_SSL_CERT_FILE : # Allow users to override the NIX_SSL_CERT_FILE
else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch else if test -e /etc/ssl/certs/ca-certificates.crt # NixOS, Ubuntu, Debian, Gentoo, Arch
set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt set --export NIX_SSL_CERT_FILE /etc/ssl/certs/ca-certificates.crt
+10 -10
View File
@@ -1,11 +1,12 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, doxygen doxygen,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -39,11 +40,10 @@ mkMesonDerivation (finalAttrs: {
doxygen doxygen
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''${!outputDoc}/nix-support mkdir -p ''${!outputDoc}/nix-support
+23 -21
View File
@@ -1,11 +1,12 @@
{ lib {
, mkMesonDerivation lib,
mkMesonDerivation,
, doxygen doxygen,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -17,27 +18,28 @@ mkMesonDerivation (finalAttrs: {
inherit version; inherit version;
workDir = ./.; workDir = ./.;
fileset = let fileset =
cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh"); let
in fileset.unions [ cpp = fileset.fileFilter (file: file.hasExt "cc" || file.hasExt "hh");
./.version in
../../.version fileset.unions [
./meson.build ./.version
./doxygen.cfg.in ../../.version
# Source is not compiled, but still must be available for Doxygen ./meson.build
# to gather comments. ./doxygen.cfg.in
(cpp ../.) # Source is not compiled, but still must be available for Doxygen
]; # to gather comments.
(cpp ../.)
];
nativeBuildInputs = [ nativeBuildInputs = [
doxygen doxygen
]; ];
preConfigure = preConfigure = ''
'' chmod u+w ./.version
chmod u+w ./.version echo ${finalAttrs.version} > ./.version
echo ${finalAttrs.version} > ./.version '';
'';
postInstall = '' postInstall = ''
mkdir -p ''${!outputDoc}/nix-support mkdir -p ''${!outputDoc}/nix-support
+5
View File
@@ -113,10 +113,15 @@ headers = [config_h] + files(
'repl.hh', 'repl.hh',
) )
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library( this_library = library(
'nixcmd', 'nixcmd',
sources, sources,
dependencies : deps_public + deps_private + deps_other, dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers prelink : true, # For C++ static initializers
install : true, install : true,
) )
+25 -32
View File
@@ -1,32 +1,33 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, nix-expr nix-expr,
, nix-flake nix-flake,
, nix-main nix-main,
, editline editline,
, readline readline,
, lowdown lowdown,
, nlohmann_json nlohmann_json,
# Configuration Options # Configuration Options
, version version,
# Whether to enable Markdown rendering in the Nix binary. # Whether to enable Markdown rendering in the Nix binary.
, enableMarkdown ? !stdenv.hostPlatform.isWindows enableMarkdown ? !stdenv.hostPlatform.isWindows,
# Which interactive line editor library to use for Nix's repl. # Which interactive line editor library to use for Nix's repl.
# #
# Currently supported choices are: # Currently supported choices are:
# #
# - editline (default) # - editline (default)
# - readline # - readline
, readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline" readlineFlavor ? if stdenv.hostPlatform.isWindows then "readline" else "editline",
}: }:
let let
@@ -63,14 +64,6 @@ mkMesonLibrary (finalAttrs: {
nlohmann_json nlohmann_json
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
(lib.mesonEnable "markdown" enableMarkdown) (lib.mesonEnable "markdown" enableMarkdown)
(lib.mesonOption "readline-flavor" readlineFlavor) (lib.mesonOption "readline-flavor" readlineFlavor)
+4 -7
View File
@@ -138,16 +138,13 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
out << ANSI_RED "error: " << ANSI_NORMAL; out << ANSI_RED "error: " << ANSI_NORMAL;
out << dt.hint.str() << "\n"; out << dt.hint.str() << "\n";
// prefer direct pos, but if noPos then try the expr. auto pos = dt.getPos(positions);
auto pos = dt.pos
? dt.pos
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
if (pos) { if (pos) {
out << *pos; out << pos;
if (auto loc = pos->getCodeLines()) { if (auto loc = pos.getCodeLines()) {
out << "\n"; out << "\n";
printCodeLines(out, "", *pos, *loc); printCodeLines(out, "", pos, *loc);
out << "\n"; out << "\n";
} }
} }
+1 -5
View File
@@ -613,12 +613,8 @@ nix_realised_string * nix_string_realise(nix_c_context * context, EvalState * st
context->last_err_code = NIX_OK; context->last_err_code = NIX_OK;
try { try {
auto & v = check_value_in(value); auto & v = check_value_in(value);
nix::NixStringContext stringContext;
auto rawStr = state->state.coerceToString(nix::noPos, v, stringContext, "while realising a string").toOwned();
nix::StorePathSet storePaths; nix::StorePathSet storePaths;
auto rewrites = state->state.realiseContext(stringContext, &storePaths); auto s = state->state.realiseString(v, &storePaths, isIFD);
auto s = nix::rewriteStrings(rawStr, rewrites);
// Convert to the C API StorePath type and convert to vector for index-based access // Convert to the C API StorePath type and convert to vector for index-based access
std::vector<StorePath> vec; std::vector<StorePath> vec;
+7 -14
View File
@@ -1,12 +1,13 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-store-c nix-store-c,
, nix-expr nix-expr,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -35,14 +36,6 @@ mkMesonLibrary (finalAttrs: {
nix-expr nix-expr
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+9 -16
View File
@@ -1,15 +1,16 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-store-test-support nix-store-test-support,
, nix-expr nix-expr,
, nix-expr-c nix-expr-c,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -39,14 +40,6 @@ mkMesonLibrary (finalAttrs: {
rapidcheck rapidcheck
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+30 -31
View File
@@ -1,20 +1,21 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-expr nix-expr,
, nix-expr-c nix-expr-c,
, nix-expr-test-support nix-expr-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -45,29 +46,27 @@ mkMesonExecutable (finalAttrs: {
gtest gtest
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+54 -54
View File
@@ -20,77 +20,77 @@ let
# Resolve a input spec into a node name. An input spec is # Resolve a input spec into a node name. An input spec is
# either a node name, or a 'follows' path from the root # either a node name, or a 'follows' path from the root
# node. # node.
resolveInput = inputSpec: resolveInput =
if builtins.isList inputSpec inputSpec: if builtins.isList inputSpec then getInputByPath lockFile.root inputSpec else inputSpec;
then getInputByPath lockFile.root inputSpec
else inputSpec;
# Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the # Follow an input path (e.g. ["dwarffs" "nixpkgs"]) from the
# root node, returning the final node. # root node, returning the final node.
getInputByPath = nodeName: path: getInputByPath =
if path == [] nodeName: path:
then nodeName if path == [ ] then
nodeName
else else
getInputByPath getInputByPath
# Since this could be a 'follows' input, call resolveInput. # Since this could be a 'follows' input, call resolveInput.
(resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path}) (resolveInput lockFile.nodes.${nodeName}.inputs.${builtins.head path})
(builtins.tail path); (builtins.tail path);
allNodes = allNodes = builtins.mapAttrs (
builtins.mapAttrs key: node:
(key: node: let
let
parentNode = allNodes.${getInputByPath lockFile.root node.parent}; parentNode = allNodes.${getInputByPath lockFile.root node.parent};
sourceInfo = sourceInfo =
if overrides ? ${key} if overrides ? ${key} then
then overrides.${key}.sourceInfo
overrides.${key}.sourceInfo else if node.locked.type == "path" && builtins.substring 0 1 node.locked.path != "/" then
else if node.locked.type == "path" && builtins.substring 0 1 node.locked.path != "/" parentNode.sourceInfo
then // {
parentNode.sourceInfo // { outPath = parentNode.outPath + ("/" + node.locked.path);
outPath = parentNode.outPath + ("/" + node.locked.path); }
} else
else # FIXME: remove obsolete node.info.
# FIXME: remove obsolete node.info. # Note: lock file entries are always final.
# Note: lock file entries are always final. fetchTreeFinal (node.info or { } // removeAttrs node.locked [ "dir" ]);
fetchTreeFinal (node.info or {} // removeAttrs node.locked ["dir"]);
subdir = overrides.${key}.dir or node.locked.dir or ""; subdir = overrides.${key}.dir or node.locked.dir or "";
outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir); outPath = sourceInfo + ((if subdir == "" then "" else "/") + subdir);
flake = import (outPath + "/flake.nix"); flake = import (outPath + "/flake.nix");
inputs = builtins.mapAttrs inputs = builtins.mapAttrs (inputName: inputSpec: allNodes.${resolveInput inputSpec}) (
(inputName: inputSpec: allNodes.${resolveInput inputSpec}) node.inputs or { }
(node.inputs or {}); );
outputs = flake.outputs (inputs // { self = result; }); outputs = flake.outputs (inputs // { self = result; });
result = result =
outputs outputs
# We add the sourceInfo attribute for its metadata, as they are # We add the sourceInfo attribute for its metadata, as they are
# relevant metadata for the flake. However, the outPath of the # relevant metadata for the flake. However, the outPath of the
# sourceInfo does not necessarily match the outPath of the flake, # sourceInfo does not necessarily match the outPath of the flake,
# as the flake may be in a subdirectory of a source. # as the flake may be in a subdirectory of a source.
# This is shadowed in the next // # This is shadowed in the next //
// sourceInfo // sourceInfo
// { // {
# This shadows the sourceInfo.outPath # This shadows the sourceInfo.outPath
inherit outPath; inherit outPath;
inherit inputs; inherit outputs; inherit sourceInfo; _type = "flake"; inherit inputs;
}; inherit outputs;
inherit sourceInfo;
_type = "flake";
};
in in
if node.flake or true then if node.flake or true then
assert builtins.isFunction flake.outputs; assert builtins.isFunction flake.outputs;
result result
else else
sourceInfo sourceInfo
) ) lockFile.nodes;
lockFile.nodes;
in allNodes.${lockFile.root} in
allNodes.${lockFile.root}
+1 -1
View File
@@ -45,7 +45,7 @@ EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr
// TODO: check compatibility with nested debugger calls. // TODO: check compatibility with nested debugger calls.
// TODO: What side-effects?? // TODO: What side-effects??
error.state.debugTraces.push_front(DebugTrace{ error.state.debugTraces.push_front(DebugTrace{
.pos = error.state.positions[expr.getPos()], .pos = expr.getPos(),
.expr = expr, .expr = expr,
.env = env, .env = env,
.hint = HintFmt("Fake frame for debugging purposes"), .hint = HintFmt("Fake frame for debugging purposes"),
+1 -1
View File
@@ -146,7 +146,7 @@ inline void EvalState::forceList(Value & v, const PosIdx pos, std::string_view e
[[gnu::always_inline]] [[gnu::always_inline]]
inline CallDepth EvalState::addCallDepth(const PosIdx pos) { inline CallDepth EvalState::addCallDepth(const PosIdx pos) {
if (callDepth > settings.maxCallDepth) if (callDepth > settings.maxCallDepth)
error<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow(); error<EvalBaseError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
return CallDepth(callDepth); return CallDepth(callDepth);
}; };
+1 -1
View File
@@ -57,7 +57,7 @@ Strings EvalSettings::getDefaultNixPath()
{ {
Strings res; Strings res;
auto add = [&](const Path & p, const std::string & s = std::string()) { auto add = [&](const Path & p, const std::string & s = std::string()) {
if (pathAccessible(p)) { if (std::filesystem::exists(p)) {
if (s.empty()) { if (s.empty()) {
res.push_back(p); res.push_back(p);
} else { } else {
+32 -26
View File
@@ -754,18 +754,26 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
if (!debugRepl || inDebugger) if (!debugRepl || inDebugger)
return; return;
auto dts = auto dts = [&]() -> std::unique_ptr<DebugTraceStacker> {
error && expr.getPos() if (error && expr.getPos()) {
? std::make_unique<DebugTraceStacker>( auto trace = DebugTrace{
*this, .pos = [&]() -> std::variant<Pos, PosIdx> {
DebugTrace { if (error->info().pos) {
.pos = error->info().pos ? error->info().pos : positions[expr.getPos()], if (auto * pos = error->info().pos.get())
return *pos;
return noPos;
}
return expr.getPos();
}(),
.expr = expr, .expr = expr,
.env = env, .env = env,
.hint = error->info().msg, .hint = error->info().msg,
.isError = true .isError = true};
})
: nullptr; return std::make_unique<DebugTraceStacker>(*this, std::move(trace));
}
return nullptr;
}();
if (error) if (error)
{ {
@@ -810,7 +818,7 @@ static std::unique_ptr<DebugTraceStacker> makeDebugTraceStacker(
EvalState & state, EvalState & state,
Expr & expr, Expr & expr,
Env & env, Env & env,
std::shared_ptr<Pos> && pos, std::variant<Pos, PosIdx> pos,
const Args & ... formatArgs) const Args & ... formatArgs)
{ {
return std::make_unique<DebugTraceStacker>(state, return std::make_unique<DebugTraceStacker>(state,
@@ -1087,7 +1095,7 @@ void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
*this, *this,
*e, *e,
this->baseEnv, this->baseEnv,
e->getPos() ? std::make_shared<Pos>(positions[e->getPos()]) : nullptr, e->getPos(),
"while evaluating the file '%1%':", resolvedPath.to_string()) "while evaluating the file '%1%':", resolvedPath.to_string())
: nullptr; : nullptr;
@@ -1313,9 +1321,7 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env2, env2,
getPos() getPos(),
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while evaluating a '%1%' expression", "while evaluating a '%1%' expression",
"let" "let"
) )
@@ -1384,7 +1390,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env, env,
state.positions[getPos()], getPos(),
"while evaluating the attribute '%1%'", "while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath)) showAttrPath(state, env, attrPath))
: nullptr; : nullptr;
@@ -1585,7 +1591,7 @@ void EvalState::callFunction(Value & fun, std::span<Value *> args, Value & vRes,
try { try {
auto dts = debugRepl auto dts = debugRepl
? makeDebugTraceStacker( ? makeDebugTraceStacker(
*this, *lambda.body, env2, positions[lambda.pos], *this, *lambda.body, env2, lambda.pos,
"while calling %s", "while calling %s",
lambda.name lambda.name
? concatStrings("'", symbols[lambda.name], "'") ? concatStrings("'", symbols[lambda.name], "'")
@@ -1720,9 +1726,7 @@ void ExprCall::eval(EvalState & state, Env & env, Value & v)
state, state,
*this, *this,
env, env,
getPos() getPos(),
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
"while calling a function" "while calling a function"
) )
: nullptr; : nullptr;
@@ -2106,7 +2110,7 @@ void EvalState::forceValueDeep(Value & v)
try { try {
// If the value is a thunk, we're evaling. Otherwise no trace necessary. // If the value is a thunk, we're evaling. Otherwise no trace necessary.
auto dts = debugRepl && i.value->isThunk() auto dts = debugRepl && i.value->isThunk()
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, positions[i.pos], ? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, i.pos,
"while evaluating the attribute '%1%'", symbols[i.name]) "while evaluating the attribute '%1%'", symbols[i.name])
: nullptr; : nullptr;
@@ -2384,7 +2388,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
: [&]() { : [&]() {
auto dstPath = fetchToStore( auto dstPath = fetchToStore(
*store, *store,
path.resolveSymlinks(), path.resolveSymlinks(SymlinkResolution::Ancestors),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy, settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(), path.baseName(),
ContentAddressMethod::Raw::NixArchive, ContentAddressMethod::Raw::NixArchive,
@@ -3070,8 +3074,11 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
auto i = lookupPathResolved.find(value); auto i = lookupPathResolved.find(value);
if (i != lookupPathResolved.end()) return i->second; if (i != lookupPathResolved.end()) return i->second;
auto finish = [&](SourcePath res) { auto finish = [&](std::optional<SourcePath> res) {
debug("resolved search path element '%s' to '%s'", value, res); if (res)
debug("resolved search path element '%s' to '%s'", value, *res);
else
debug("failed to resolve search path element '%s'", value);
lookupPathResolved.emplace(value, res); lookupPathResolved.emplace(value, res);
return res; return res;
}; };
@@ -3114,7 +3121,7 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
} }
} }
if (path.pathExists()) if (path.resolveSymlinks().pathExists())
return finish(std::move(path)); return finish(std::move(path));
else { else {
logWarning({ logWarning({
@@ -3123,8 +3130,7 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
} }
} }
debug("failed to resolve search path element '%s'", value); return finish(std::nullopt);
return std::nullopt;
} }
+27 -1
View File
@@ -171,11 +171,28 @@ struct RegexCache;
std::shared_ptr<RegexCache> makeRegexCache(); std::shared_ptr<RegexCache> makeRegexCache();
struct DebugTrace { struct DebugTrace {
std::shared_ptr<Pos> pos; /* WARNING: Converting PosIdx -> Pos should be done with extra care. This is
due to the fact that operator[] of PosTable is incredibly expensive. */
std::variant<Pos, PosIdx> pos;
const Expr & expr; const Expr & expr;
const Env & env; const Env & env;
HintFmt hint; HintFmt hint;
bool isError; bool isError;
Pos getPos(const PosTable & table) const
{
return std::visit(
overloaded{
[&](PosIdx idx) {
// Prefer direct pos, but if noPos then try the expr.
if (!idx)
idx = expr.getPos();
return table[idx];
},
[&](Pos pos) { return pos; },
},
pos);
}
}; };
class EvalState : public std::enable_shared_from_this<EvalState> class EvalState : public std::enable_shared_from_this<EvalState>
@@ -820,6 +837,15 @@ public:
*/ */
[[nodiscard]] StringMap realiseContext(const NixStringContext & context, StorePathSet * maybePaths = nullptr, bool isIFD = true); [[nodiscard]] StringMap realiseContext(const NixStringContext & context, StorePathSet * maybePaths = nullptr, bool isIFD = true);
/**
* Realise the given string with context, and return the string with outputs instead of downstream output placeholders.
* @param[in] str the string to realise
* @param[out] paths all referenced store paths will be added to this set
* @return the realised string
* @throw EvalError if the value is not a string, path or derivation (see `coerceToString`)
*/
std::string realiseString(Value & str, StorePathSet * storePathsOutMaybe, bool isIFD = true, const PosIdx pos = noPos);
/* Call the binary path filter predicate used builtins.path etc. */ /* Call the binary path filter predicate used builtins.path etc. */
bool callPathFilter( bool callPathFilter(
Value * filterFun, Value * filterFun,
+62 -30
View File
@@ -1,40 +1,72 @@
{ system ? "" # obsolete {
, url system ? "", # obsolete
, hash ? "" # an SRI hash url,
hash ? "", # an SRI hash
# Legacy hash specification # Legacy hash specification
, md5 ? "", sha1 ? "", sha256 ? "", sha512 ? "" md5 ? "",
, outputHash ? sha1 ? "",
if hash != "" then hash else if sha512 != "" then sha512 else if sha1 != "" then sha1 else if md5 != "" then md5 else sha256 sha256 ? "",
, outputHashAlgo ? sha512 ? "",
if hash != "" then "" else if sha512 != "" then "sha512" else if sha1 != "" then "sha1" else if md5 != "" then "md5" else "sha256" outputHash ?
if hash != "" then
hash
else if sha512 != "" then
sha512
else if sha1 != "" then
sha1
else if md5 != "" then
md5
else
sha256,
outputHashAlgo ?
if hash != "" then
""
else if sha512 != "" then
"sha512"
else if sha1 != "" then
"sha1"
else if md5 != "" then
"md5"
else
"sha256",
, executable ? false executable ? false,
, unpack ? false unpack ? false,
, name ? baseNameOf (toString url) name ? baseNameOf (toString url),
, impure ? false impure ? false,
}: }:
derivation ({ derivation (
builder = "builtin:fetchurl"; {
builder = "builtin:fetchurl";
# New-style output content requirements. # New-style output content requirements.
outputHashMode = if unpack || executable then "recursive" else "flat"; outputHashMode = if unpack || executable then "recursive" else "flat";
inherit name url executable unpack; inherit
name
url
executable
unpack
;
system = "builtin"; system = "builtin";
# No need to double the amount of network traffic # No need to double the amount of network traffic
preferLocalBuild = true; preferLocalBuild = true;
# This attribute does nothing; it's here to avoid changing evaluation results. # This attribute does nothing; it's here to avoid changing evaluation results.
impureEnvVars = [ impureEnvVars = [
"http_proxy" "https_proxy" "ftp_proxy" "all_proxy" "no_proxy" "http_proxy"
]; "https_proxy"
"ftp_proxy"
"all_proxy"
"no_proxy"
];
# To make "nix-prefetch-url" work. # To make "nix-prefetch-url" work.
urls = [ url ]; urls = [ url ];
} // (if impure }
then { __impure = true; } // (if impure then { __impure = true; } else { inherit outputHashAlgo outputHash; })
else { inherit outputHashAlgo outputHash; })) )
+21 -15
View File
@@ -1,21 +1,27 @@
attrs @ { drvPath, outputs, name, ... }: attrs@{
drvPath,
outputs,
name,
...
}:
let let
commonAttrs = (builtins.listToAttrs outputsList) // commonAttrs = (builtins.listToAttrs outputsList) // {
{ all = map (x: x.value) outputsList; all = map (x: x.value) outputsList;
inherit drvPath name; inherit drvPath name;
type = "derivation"; type = "derivation";
}; };
outputToAttrListElement = outputName: outputToAttrListElement = outputName: {
{ name = outputName; name = outputName;
value = commonAttrs // { value = commonAttrs // {
outPath = builtins.getAttr outputName attrs; outPath = builtins.getAttr outputName attrs;
inherit outputName; inherit outputName;
};
}; };
};
outputsList = map outputToAttrListElement outputs; outputsList = map outputToAttrListElement outputs;
in (builtins.head outputsList).value in
(builtins.head outputsList).value
+5 -2
View File
@@ -171,8 +171,6 @@ headers = [config_h] + files(
# internal: 'lexer-helpers.hh', # internal: 'lexer-helpers.hh',
'nixexpr.hh', 'nixexpr.hh',
'parser-state.hh', 'parser-state.hh',
'pos-idx.hh',
'pos-table.hh',
'primops.hh', 'primops.hh',
'print-ambiguous.hh', 'print-ambiguous.hh',
'print-options.hh', 'print-options.hh',
@@ -188,6 +186,9 @@ headers = [config_h] + files(
subdir('primops') subdir('primops')
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library( this_library = library(
'nixexpr', 'nixexpr',
sources, sources,
@@ -195,6 +196,8 @@ this_library = library(
lexer_tab, lexer_tab,
generated_headers, generated_headers,
dependencies : deps_public + deps_private + deps_other, dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers prelink : true, # For C++ static initializers
install : true, install : true,
) )
-35
View File
@@ -601,41 +601,6 @@ void ExprLambda::setDocComment(DocComment docComment) {
} }
}; };
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(
std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
/* Symbol table. */ /* Symbol table. */
size_t SymbolTable::totalSize() const size_t SymbolTable::totalSize() const
+27 -44
View File
@@ -1,33 +1,34 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, bison bison,
, flex flex,
, cmake # for resolving toml11 dep cmake, # for resolving toml11 dep
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, boost boost,
, boehmgc boehmgc,
, nlohmann_json nlohmann_json,
, toml11 toml11,
# Configuration Options # Configuration Options
, version version,
# Whether to use garbage collection for the Nix language evaluator. # Whether to use garbage collection for the Nix language evaluator.
# #
# If it is disabled, we just leak memory, but this is not as bad as it # If it is disabled, we just leak memory, but this is not as bad as it
# sounds so long as evaluation just takes places within short-lived # sounds so long as evaluation just takes places within short-lived
# processes. (When the process exits, the memory is reclaimed; it is # processes. (When the process exits, the memory is reclaimed; it is
# only leaked *within* the process.) # only leaked *within* the process.)
# #
# Temporarily disabled on Windows because the `GC_throw_bad_alloc` # Temporarily disabled on Windows because the `GC_throw_bad_alloc`
# symbol is missing during linking. # symbol is missing during linking.
, enableGC ? !stdenv.hostPlatform.isWindows enableGC ? !stdenv.hostPlatform.isWindows,
}: }:
let let
@@ -51,10 +52,7 @@ mkMesonLibrary (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "hh") ./.) (fileset.fileFilter (file: file.hasExt "hh") ./.)
./lexer.l ./lexer.l
./parser.y ./parser.y
(fileset.difference (fileset.difference (fileset.fileFilter (file: file.hasExt "nix") ./.) ./package.nix)
(fileset.fileFilter (file: file.hasExt "nix") ./.)
./package.nix
)
]; ];
nativeBuildInputs = [ nativeBuildInputs = [
@@ -79,25 +77,10 @@ mkMesonLibrary (finalAttrs: {
nlohmann_json nlohmann_json
] ++ lib.optional enableGC boehmgc; ] ++ lib.optional enableGC boehmgc;
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
(lib.mesonEnable "gc" enableGC) (lib.mesonEnable "gc" enableGC)
]; ];
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
}; };
+9
View File
@@ -47,6 +47,15 @@ static inline Value * mkString(EvalState & state, const std::csub_match & match)
return v; return v;
} }
std::string EvalState::realiseString(Value & s, StorePathSet * storePathsOutMaybe, bool isIFD, const PosIdx pos)
{
nix::NixStringContext stringContext;
auto rawStr = coerceToString(pos, s, stringContext, "while realising a string").toOwned();
auto rewrites = realiseContext(stringContext, storePathsOutMaybe, isIFD);
return nix::rewriteStrings(rawStr, rewrites);
}
StringMap EvalState::realiseContext(const NixStringContext & context, StorePathSet * maybePathsOut, bool isIFD) StringMap EvalState::realiseContext(const NixStringContext & context, StorePathSet * maybePathsOut, bool isIFD)
{ {
std::vector<DerivedPath::Built> drvs; std::vector<DerivedPath::Built> drvs;
+19 -12
View File
@@ -26,27 +26,34 @@
Note that `derivation` is very bare-bones, and provides almost no commands during the build. Note that `derivation` is very bare-bones, and provides almost no commands during the build.
Most likely, you'll want to use functions like `stdenv.mkDerivation` in Nixpkgs to set up a basic environment. Most likely, you'll want to use functions like `stdenv.mkDerivation` in Nixpkgs to set up a basic environment.
*/ */
drvAttrs @ { outputs ? [ "out" ], ... }: drvAttrs@{
outputs ? [ "out" ],
...
}:
let let
strict = derivationStrict drvAttrs; strict = derivationStrict drvAttrs;
commonAttrs = drvAttrs // (builtins.listToAttrs outputsList) // commonAttrs =
{ all = map (x: x.value) outputsList; drvAttrs
// (builtins.listToAttrs outputsList)
// {
all = map (x: x.value) outputsList;
inherit drvAttrs; inherit drvAttrs;
}; };
outputToAttrListElement = outputName: outputToAttrListElement = outputName: {
{ name = outputName; name = outputName;
value = commonAttrs // { value = commonAttrs // {
outPath = builtins.getAttr outputName strict; outPath = builtins.getAttr outputName strict;
drvPath = strict.drvPath; drvPath = strict.drvPath;
type = "derivation"; type = "derivation";
inherit outputName; inherit outputName;
};
}; };
};
outputsList = map outputToAttrListElement outputs; outputsList = map outputToAttrListElement outputs;
in (builtins.head outputsList).value in
(builtins.head outputsList).value
+29 -30
View File
@@ -1,19 +1,20 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-fetchers nix-fetchers,
, nix-store-test-support nix-store-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -43,29 +44,27 @@ mkMesonExecutable (finalAttrs: {
gtest gtest
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+16 -6
View File
@@ -4,6 +4,21 @@
namespace nix { namespace nix {
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string &name,
const std::string &fingerprint,
ContentAddressMethod method,
const std::string &path)
{
return fetchers::Cache::Key{"fetchToStore", {
{"name", name},
{"fingerprint", fingerprint},
{"method", std::string{method.render()}},
{"path", path}
}};
}
StorePath fetchToStore( StorePath fetchToStore(
Store & store, Store & store,
const SourcePath & path, const SourcePath & path,
@@ -19,12 +34,7 @@ StorePath fetchToStore(
std::optional<fetchers::Cache::Key> cacheKey; std::optional<fetchers::Cache::Key> cacheKey;
if (!filter && path.accessor->fingerprint) { if (!filter && path.accessor->fingerprint) {
cacheKey = fetchers::Cache::Key{"fetchToStore", { cacheKey = makeFetchToStoreCacheKey(std::string{name}, *path.accessor->fingerprint, method, path.path.abs());
{"name", std::string{name}},
{"fingerprint", *path.accessor->fingerprint},
{"method", std::string{method.render()}},
{"path", path.path.abs()}
}};
if (auto res = fetchers::getCache()->lookupStorePath(*cacheKey, store)) { if (auto res = fetchers::getCache()->lookupStorePath(*cacheKey, store)) {
debug("store path cache hit for '%s'", path); debug("store path cache hit for '%s'", path);
return res->storePath; return res->storePath;
+4
View File
@@ -5,6 +5,7 @@
#include "file-system.hh" #include "file-system.hh"
#include "repair-flag.hh" #include "repair-flag.hh"
#include "file-content-address.hh" #include "file-content-address.hh"
#include "cache.hh"
namespace nix { namespace nix {
@@ -22,4 +23,7 @@ StorePath fetchToStore(
PathFilter * filter = nullptr, PathFilter * filter = nullptr,
RepairFlag repair = NoRepair); RepairFlag repair = NoRepair);
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string & name, const std::string & fingerprint, ContentAddressMethod method, const std::string & path);
} }
+19 -16
View File
@@ -206,7 +206,8 @@ static git_packbuilder_progress PACKBUILDER_PROGRESS_CHECK_INTERRUPT = &packBuil
} // extern "C" } // extern "C"
static void initRepoAtomically(std::filesystem::path &path, bool bare) { static void initRepoAtomically(std::filesystem::path &path, bool bare)
{
if (pathExists(path.string())) return; if (pathExists(path.string())) return;
Path tmpDir = createTempDir(os_string_to_string(PathViewNG { std::filesystem::path(path).parent_path() })); Path tmpDir = createTempDir(os_string_to_string(PathViewNG { std::filesystem::path(path).parent_path() }));
@@ -507,7 +508,10 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
*/ */
ref<GitSourceAccessor> getRawAccessor(const Hash & rev); ref<GitSourceAccessor> getRawAccessor(const Hash & rev);
ref<SourceAccessor> getAccessor(const Hash & rev, bool exportIgnore) override; ref<SourceAccessor> getAccessor(
const Hash & rev,
bool exportIgnore,
std::string displayPrefix) override;
ref<SourceAccessor> getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError e) override; ref<SourceAccessor> getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError e) override;
@@ -544,13 +548,10 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
// then use code that was removed in this commit (see blame) // then use code that was removed in this commit (see blame)
auto dir = this->path; auto dir = this->path;
Strings gitArgs; Strings gitArgs{"-C", dir.string(), "--git-dir", ".", "fetch", "--quiet", "--force"};
if (shallow) { if (shallow)
gitArgs = { "-C", dir.string(), "fetch", "--quiet", "--force", "--depth", "1", "--", url, refspec }; append(gitArgs, {"--depth", "1"});
} append(gitArgs, {std::string("--"), url, refspec});
else {
gitArgs = { "-C", dir.string(), "fetch", "--quiet", "--force", "--", url, refspec };
}
runProgram(RunOptions { runProgram(RunOptions {
.program = "git", .program = "git",
@@ -629,7 +630,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Hash treeHashToNarHash(const Hash & treeHash) override Hash treeHashToNarHash(const Hash & treeHash) override
{ {
auto accessor = getAccessor(treeHash, false); auto accessor = getAccessor(treeHash, false, "");
fetchers::Cache::Key cacheKey{"treeHashToNarHash", {{"treeHash", treeHash.gitRev()}}}; fetchers::Cache::Key cacheKey{"treeHashToNarHash", {{"treeHash", treeHash.gitRev()}}};
@@ -1196,16 +1197,18 @@ ref<GitSourceAccessor> GitRepoImpl::getRawAccessor(const Hash & rev)
return make_ref<GitSourceAccessor>(self, rev); return make_ref<GitSourceAccessor>(self, rev);
} }
ref<SourceAccessor> GitRepoImpl::getAccessor(const Hash & rev, bool exportIgnore) ref<SourceAccessor> GitRepoImpl::getAccessor(
const Hash & rev,
bool exportIgnore,
std::string displayPrefix)
{ {
auto self = ref<GitRepoImpl>(shared_from_this()); auto self = ref<GitRepoImpl>(shared_from_this());
ref<GitSourceAccessor> rawGitAccessor = getRawAccessor(rev); ref<GitSourceAccessor> rawGitAccessor = getRawAccessor(rev);
if (exportIgnore) { rawGitAccessor->setPathDisplay(std::move(displayPrefix));
if (exportIgnore)
return make_ref<GitExportIgnoreSourceAccessor>(self, rawGitAccessor, rev); return make_ref<GitExportIgnoreSourceAccessor>(self, rawGitAccessor, rev);
} else
else {
return rawGitAccessor; return rawGitAccessor;
}
} }
ref<SourceAccessor> GitRepoImpl::getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError) ref<SourceAccessor> GitRepoImpl::getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError)
@@ -1238,7 +1241,7 @@ std::vector<std::tuple<GitRepoImpl::Submodule, Hash>> GitRepoImpl::getSubmodules
/* Read the .gitmodules files from this revision. */ /* Read the .gitmodules files from this revision. */
CanonPath modulesFile(".gitmodules"); CanonPath modulesFile(".gitmodules");
auto accessor = getAccessor(rev, exportIgnore); auto accessor = getAccessor(rev, exportIgnore, "");
if (!accessor->pathExists(modulesFile)) return {}; if (!accessor->pathExists(modulesFile)) return {};
/* Parse it and get the revision of each submodule. */ /* Parse it and get the revision of each submodule. */
+4 -1
View File
@@ -86,7 +86,10 @@ struct GitRepo
virtual bool hasObject(const Hash & oid) = 0; virtual bool hasObject(const Hash & oid) = 0;
virtual ref<SourceAccessor> getAccessor(const Hash & rev, bool exportIgnore) = 0; virtual ref<SourceAccessor> getAccessor(
const Hash & rev,
bool exportIgnore,
std::string displayPrefix) = 0;
virtual ref<SourceAccessor> getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError) = 0; virtual ref<SourceAccessor> getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError) = 0;
+13 -11
View File
@@ -69,7 +69,7 @@ std::optional<std::string> readHead(const Path & path)
std::string_view line = output; std::string_view line = output;
line = line.substr(0, line.find("\n")); line = line.substr(0, line.find("\n"));
if (const auto parseResult = git::parseLsRemoteLine(line)) { if (const auto parseResult = git::parseLsRemoteLine(line); parseResult && parseResult->reference == "HEAD") {
switch (parseResult->kind) { switch (parseResult->kind) {
case git::LsRemoteRefLine::Kind::Symbolic: case git::LsRemoteRefLine::Kind::Symbolic:
debug("resolved HEAD ref '%s' for repo '%s'", parseResult->target, path); debug("resolved HEAD ref '%s' for repo '%s'", parseResult->target, path);
@@ -459,8 +459,14 @@ struct GitInputScheme : InputScheme
url); url);
} }
repoInfo.location = std::filesystem::absolute(url.path); repoInfo.location = std::filesystem::absolute(url.path);
} else } else {
if (url.scheme == "file")
/* Query parameters are meaningless for file://, but
Git interprets them as part of the file name. So get
rid of them. */
url.query.clear();
repoInfo.location = url; repoInfo.location = url;
}
// If this is a local directory and no ref or revision is // If this is a local directory and no ref or revision is
// given, then allow the use of an unclean working tree. // given, then allow the use of an unclean working tree.
@@ -605,16 +611,16 @@ struct GitInputScheme : InputScheme
try { try {
auto fetchRef = auto fetchRef =
getAllRefsAttr(input) getAllRefsAttr(input)
? "refs/*" ? "refs/*:refs/*"
: input.getRev() : input.getRev()
? input.getRev()->gitRev() ? input.getRev()->gitRev()
: ref.compare(0, 5, "refs/") == 0 : ref.compare(0, 5, "refs/") == 0
? ref ? fmt("%1%:%1%", ref)
: ref == "HEAD" : ref == "HEAD"
? ref ? ref
: "refs/heads/" + ref; : fmt("%1%:%1%", "refs/heads/" + ref);
repo->fetch(repoUrl.to_string(), fmt("%s:%s", fetchRef, fetchRef), getShallowAttr(input)); repo->fetch(repoUrl.to_string(), fetchRef, getShallowAttr(input));
} catch (Error & e) { } catch (Error & e) {
if (!std::filesystem::exists(localRefFile)) throw; if (!std::filesystem::exists(localRefFile)) throw;
logError(e.info()); logError(e.info());
@@ -672,9 +678,7 @@ struct GitInputScheme : InputScheme
verifyCommit(input, repo); verifyCommit(input, repo);
bool exportIgnore = getExportIgnoreAttr(input); bool exportIgnore = getExportIgnoreAttr(input);
auto accessor = repo->getAccessor(rev, exportIgnore); auto accessor = repo->getAccessor(rev, exportIgnore, "«" + input.to_string() + "»");
accessor->setPathDisplay("«" + input.to_string() + "»");
/* If the repo has submodules, fetch them and return a mounted /* If the repo has submodules, fetch them and return a mounted
input accessor consisting of the accessor for the top-level input accessor consisting of the accessor for the top-level
@@ -737,8 +741,6 @@ struct GitInputScheme : InputScheme
exportIgnore, exportIgnore,
makeNotAllowedError(repoInfo.locationToArg())); makeNotAllowedError(repoInfo.locationToArg()));
accessor->setPathDisplay(repoInfo.locationToArg());
/* If the repo has submodules, return a mounted input accessor /* If the repo has submodules, return a mounted input accessor
consisting of the accessor for the top-level repo and the consisting of the accessor for the top-level repo and the
accessors for the submodule workdirs. */ accessors for the submodule workdirs. */
+4 -3
View File
@@ -294,9 +294,10 @@ struct GitArchiveInputScheme : InputScheme
#endif #endif
input.attrs.insert_or_assign("lastModified", uint64_t(tarballInfo.lastModified)); input.attrs.insert_or_assign("lastModified", uint64_t(tarballInfo.lastModified));
auto accessor = getTarballCache()->getAccessor(tarballInfo.treeHash, false); auto accessor = getTarballCache()->getAccessor(
tarballInfo.treeHash,
accessor->setPathDisplay("«" + input.to_string() + "»"); false,
"«" + input.to_string() + "»");
return {accessor, input}; return {accessor, input};
} }
+5
View File
@@ -76,10 +76,15 @@ headers = files(
'tarball.hh', 'tarball.hh',
) )
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library( this_library = library(
'nixfetchers', 'nixfetchers',
sources, sources,
dependencies : deps_public + deps_private + deps_other, dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers prelink : true, # For C++ static initializers
install : true, install : true,
) )
+9 -16
View File
@@ -1,14 +1,15 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nlohmann_json nlohmann_json,
, libgit2 libgit2,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -40,14 +41,6 @@ mkMesonLibrary (finalAttrs: {
nlohmann_json nlohmann_json
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
}; };
+11 -1
View File
@@ -2,6 +2,8 @@
#include "store-api.hh" #include "store-api.hh"
#include "archive.hh" #include "archive.hh"
#include "store-path-accessor.hh" #include "store-path-accessor.hh"
#include "fetch-to-store.hh"
#include "cache.hh"
namespace nix::fetchers { namespace nix::fetchers {
@@ -125,7 +127,7 @@ struct PathInputScheme : InputScheme
auto absPath = getAbsPath(input); auto absPath = getAbsPath(input);
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying '%s' to the store", absPath)); Activity act(*logger, lvlTalkative, actUnknown, fmt("copying %s to the store", absPath));
// FIXME: check whether access to 'path' is allowed. // FIXME: check whether access to 'path' is allowed.
auto storePath = store->maybeParseStorePath(absPath.string()); auto storePath = store->maybeParseStorePath(absPath.string());
@@ -142,6 +144,14 @@ struct PathInputScheme : InputScheme
storePath = store->addToStoreFromDump(*src, "source"); storePath = store->addToStoreFromDump(*src, "source");
} }
// To avoid copying the path again to the /nix/store, we need to add a cache entry.
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive;
auto fp = getFingerprint(store, input);
if (fp) {
auto cacheKey = makeFetchToStoreCacheKey(input.getName(), *fp, method, "/");
fetchers::getCache()->upsert(cacheKey, *store, {}, *storePath);
}
/* Trust the lastModified value supplied by the user, if /* Trust the lastModified value supplied by the user, if
any. It's not a "secure" attribute so we don't care. */ any. It's not a "secure" attribute so we don't care. */
if (!input.getLastModified()) if (!input.getLastModified())
+7 -5
View File
@@ -105,7 +105,8 @@ DownloadFileResult downloadFile(
static DownloadTarballResult downloadTarball_( static DownloadTarballResult downloadTarball_(
const std::string & url, const std::string & url,
const Headers & headers) const Headers & headers,
const std::string & displayPrefix)
{ {
Cache::Key cacheKey{"tarball", {{"url", url}}}; Cache::Key cacheKey{"tarball", {{"url", url}}};
@@ -118,7 +119,7 @@ static DownloadTarballResult downloadTarball_(
.treeHash = treeHash, .treeHash = treeHash,
.lastModified = (time_t) getIntAttr(infoAttrs, "lastModified"), .lastModified = (time_t) getIntAttr(infoAttrs, "lastModified"),
.immutableUrl = maybeGetStrAttr(infoAttrs, "immutableUrl"), .immutableUrl = maybeGetStrAttr(infoAttrs, "immutableUrl"),
.accessor = getTarballCache()->getAccessor(treeHash, false), .accessor = getTarballCache()->getAccessor(treeHash, false, displayPrefix),
}; };
}; };
@@ -371,9 +372,10 @@ struct TarballInputScheme : CurlInputScheme
{ {
auto input(_input); auto input(_input);
auto result = downloadTarball_(getStrAttr(input.attrs, "url"), {}); auto result = downloadTarball_(
getStrAttr(input.attrs, "url"),
result.accessor->setPathDisplay("«" + input.to_string() + "»"); {},
"«" + input.to_string() + "»");
if (result.immutableUrl) { if (result.immutableUrl) {
auto immutableInput = Input::fromURL(*input.settings, *result.immutableUrl); auto immutableInput = Input::fromURL(*input.settings, *result.immutableUrl);
+8 -15
View File
@@ -1,13 +1,14 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-store-c nix-store-c,
, nix-expr-c nix-expr-c,
, nix-flake nix-flake,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -37,14 +38,6 @@ mkMesonLibrary (finalAttrs: {
nix-flake nix-flake
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+31 -32
View File
@@ -1,20 +1,21 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-flake nix-flake,
, nix-flake-c nix-flake-c,
, nix-expr-test-support nix-expr-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, resolvePath resolvePath,
}: }:
let let
@@ -45,30 +46,28 @@ mkMesonExecutable (finalAttrs: {
gtest gtest
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${resolvePath ./data} lib.optionalString stdenv.hostPlatform.isWindows ''
export NIX_CONFIG="extra-experimental-features = flakes" export HOME="$PWD/home-dir"
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} mkdir -p "$HOME"
touch $out ''
''); + ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
export NIX_CONFIG="extra-experimental-features = flakes"
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+5 -5
View File
@@ -554,12 +554,12 @@ LockedFlake lockFlake(
/* Get the input flake, resolve 'path:./...' /* Get the input flake, resolve 'path:./...'
flakerefs relative to the parent flake. */ flakerefs relative to the parent flake. */
auto getInputFlake = [&]() auto getInputFlake = [&](const FlakeRef & ref)
{ {
if (auto resolvedPath = resolveRelativePath()) { if (auto resolvedPath = resolveRelativePath()) {
return readFlake(state, *input.ref, *input.ref, *input.ref, *resolvedPath, inputPath); return readFlake(state, ref, ref, ref, *resolvedPath, inputPath);
} else { } else {
return getFlake(state, *input.ref, useRegistries, flakeCache, inputPath); return getFlake(state, ref, useRegistries, flakeCache, inputPath);
} }
}; };
@@ -640,7 +640,7 @@ LockedFlake lockFlake(
} }
if (mustRefetch) { if (mustRefetch) {
auto inputFlake = getInputFlake(); auto inputFlake = getInputFlake(oldLock->lockedRef);
nodePaths.emplace(childNode, inputFlake.path.parent()); nodePaths.emplace(childNode, inputFlake.path.parent());
computeLocks(inputFlake.inputs, childNode, inputPath, oldLock, followsPrefix, computeLocks(inputFlake.inputs, childNode, inputPath, oldLock, followsPrefix,
inputFlake.path, false); inputFlake.path, false);
@@ -668,7 +668,7 @@ LockedFlake lockFlake(
auto ref = (input2.ref && explicitCliOverrides.contains(inputPath)) ? *input2.ref : *input.ref; auto ref = (input2.ref && explicitCliOverrides.contains(inputPath)) ? *input2.ref : *input.ref;
if (input.isFlake) { if (input.isFlake) {
auto inputFlake = getInputFlake(); auto inputFlake = getInputFlake(*input.ref);
auto childNode = make_ref<LockedNode>( auto childNode = make_ref<LockedNode>(
inputFlake.lockedRef, inputFlake.lockedRef,
+1 -1
View File
@@ -39,7 +39,7 @@ FlakeRef FlakeRef::resolve(
ref<Store> store, ref<Store> store,
const fetchers::RegistryFilter & filter) const const fetchers::RegistryFilter & filter) const
{ {
auto [input2, extraAttrs] = lookupInRegistries(store, input); auto [input2, extraAttrs] = lookupInRegistries(store, input, filter);
return FlakeRef(std::move(input2), fetchers::maybeGetStrAttr(extraAttrs, "dir").value_or(subdir)); return FlakeRef(std::move(input2), fetchers::maybeGetStrAttr(extraAttrs, "dir").value_or(subdir));
} }
+5
View File
@@ -58,10 +58,15 @@ headers = files(
'flake/url-name.hh', 'flake/url-name.hh',
) )
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library( this_library = library(
'nixflake', 'nixflake',
sources, sources,
dependencies : deps_public + deps_private + deps_other, dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers prelink : true, # For C++ static initializers
install : true, install : true,
) )
+10 -17
View File
@@ -1,15 +1,16 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util nix-util,
, nix-store nix-store,
, nix-fetchers nix-fetchers,
, nix-expr nix-expr,
, nlohmann_json nlohmann_json,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -39,14 +40,6 @@ mkMesonLibrary (finalAttrs: {
nlohmann_json nlohmann_json
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
}; };
+9 -16
View File
@@ -1,14 +1,15 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util-c nix-util-c,
, nix-store nix-store,
, nix-store-c nix-store-c,
, nix-main nix-main,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -39,14 +40,6 @@ mkMesonLibrary (finalAttrs: {
nix-main nix-main
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+5
View File
@@ -82,10 +82,15 @@ headers = [config_h] + files(
'shared.hh', 'shared.hh',
) )
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library( this_library = library(
'nixmain', 'nixmain',
sources, sources,
dependencies : deps_public + deps_private + deps_other, dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers prelink : true, # For C++ static initializers
install : true, install : true,
) )
+8 -15
View File
@@ -1,14 +1,15 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, openssl openssl,
, nix-util nix-util,
, nix-store nix-store,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -36,14 +37,6 @@ mkMesonLibrary (finalAttrs: {
openssl openssl
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
}; };
-14
View File
@@ -315,20 +315,6 @@ void printVersion(const std::string & programName)
throw Exit(); throw Exit();
} }
void showManPage(const std::string & name)
{
restoreProcessContext();
setEnv("MANPATH", settings.nixManDir.c_str());
execlp("man", "man", name.c_str(), nullptr);
if (errno == ENOENT) {
// Not SysError because we don't want to suffix the errno, aka No such file or directory.
throw Error("The '%1%' command was not found, but it is needed for '%2%' and some other '%3%' commands' help text. Perhaps you could install the '%1%' command?", "man", name.c_str(), "nix-*");
}
throw SysError("command 'man %1%' failed", name.c_str());
}
int handleExceptions(const std::string & programName, std::function<void()> fun) int handleExceptions(const std::string & programName, std::function<void()> fun)
{ {
ReceiveInterrupts receiveInterrupts; // FIXME: need better place for this ReceiveInterrupts receiveInterrupts; // FIXME: need better place for this
-5
View File
@@ -70,11 +70,6 @@ struct LegacyArgs : public MixCommonArgs, public RootArgs
}; };
/**
* Show the manual page for the specified program.
*/
void showManPage(const std::string & name);
/** /**
* The constructor of this class starts a pager if standard output is a * The constructor of this class starts a pager if standard output is a
* terminal and $PAGER is set. Standard output is redirected to the * terminal and $PAGER is set. Standard output is redirected to the
+7 -14
View File
@@ -1,12 +1,13 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util-c nix-util-c,
, nix-store nix-store,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -35,14 +36,6 @@ mkMesonLibrary (finalAttrs: {
nix-store nix-store
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+9 -16
View File
@@ -1,15 +1,16 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util-test-support nix-util-test-support,
, nix-store nix-store,
, nix-store-c nix-store-c,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -39,14 +40,6 @@ mkMesonLibrary (finalAttrs: {
rapidcheck rapidcheck
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+42 -42
View File
@@ -1,21 +1,22 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-store nix-store,
, nix-store-c nix-store-c,
, nix-store-test-support nix-store-test-support,
, sqlite sqlite,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
, filesetToSource filesetToSource,
}: }:
let let
@@ -51,39 +52,38 @@ mkMesonExecutable (finalAttrs: {
nix-store-test-support nix-store-test-support
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
passthru = { passthru = {
tests = { tests = {
run = let run =
# Some data is shared with the functional tests: they create it, let
# we consume it. # Some data is shared with the functional tests: they create it,
data = filesetToSource { # we consume it.
root = ../..; data = filesetToSource {
fileset = lib.fileset.unions [ root = ../..;
./data fileset = lib.fileset.unions [
../../tests/functional/derivation ./data
]; ../../tests/functional/derivation
}; ];
in runCommand "${finalAttrs.pname}-run" { };
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; in
} (lib.optionalString stdenv.hostPlatform.isWindows '' runCommand "${finalAttrs.pname}-run"
export HOME="$PWD/home-dir" {
mkdir -p "$HOME" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
'' + '' }
export _NIX_TEST_UNIT_DATA=${data + "/src/libstore-tests/data"} (
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} lib.optionalString stdenv.hostPlatform.isWindows ''
touch $out export HOME="$PWD/home-dir"
''); mkdir -p "$HOME"
''
+ ''
export _NIX_TEST_UNIT_DATA=${data + "/src/libstore-tests/data"}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+1 -5
View File
@@ -23,11 +23,7 @@ void builtinUnpackChannel(
throw Error("channelName is not allowed to contain filesystem separators, got %1%", channelName); throw Error("channelName is not allowed to contain filesystem separators, got %1%", channelName);
} }
try { createDirs(out);
fs::create_directories(out);
} catch (fs::filesystem_error &) {
throw SysError("creating directory '%1%'", out.string());
}
unpackTarfile(src, out); unpackTarfile(src, out);
+1
View File
@@ -1025,6 +1025,7 @@ void processConnection(
{ {
#ifndef _WIN32 // TODO need graceful async exit support on Windows? #ifndef _WIN32 // TODO need graceful async exit support on Windows?
auto monitor = !recursive ? std::make_unique<MonitorFdHup>(from.fd) : nullptr; auto monitor = !recursive ? std::make_unique<MonitorFdHup>(from.fd) : nullptr;
(void) monitor; // suppress warning
#endif #endif
/* Exchange the greeting. */ /* Exchange the greeting. */
+12
View File
@@ -300,6 +300,14 @@ struct curlFileTransfer : public FileTransfer
return ((TransferItem *) userp)->readCallback(buffer, size, nitems); return ((TransferItem *) userp)->readCallback(buffer, size, nitems);
} }
#if !defined(_WIN32) && LIBCURL_VERSION_NUM >= 0x071000
static int cloexec_callback(void *, curl_socket_t curlfd, curlsocktype purpose) {
unix::closeOnExec(curlfd);
vomit("cloexec set for fd %i", curlfd);
return CURL_SOCKOPT_OK;
}
#endif
void init() void init()
{ {
if (!req) req = curl_easy_init(); if (!req) req = curl_easy_init();
@@ -359,6 +367,10 @@ struct curlFileTransfer : public FileTransfer
curl_easy_setopt(req, CURLOPT_SSL_VERIFYHOST, 0); curl_easy_setopt(req, CURLOPT_SSL_VERIFYHOST, 0);
} }
#if !defined(_WIN32) && LIBCURL_VERSION_NUM >= 0x071000
curl_easy_setopt(req, CURLOPT_SOCKOPTFUNCTION, cloexec_callback);
#endif
curl_easy_setopt(req, CURLOPT_CONNECTTIMEOUT, fileTransferSettings.connectTimeout.get()); curl_easy_setopt(req, CURLOPT_CONNECTTIMEOUT, fileTransferSettings.connectTimeout.get());
curl_easy_setopt(req, CURLOPT_LOW_SPEED_LIMIT, 1L); curl_easy_setopt(req, CURLOPT_LOW_SPEED_LIMIT, 1L);
+1 -2
View File
@@ -65,7 +65,6 @@ Settings::Settings()
, nixStateDir(canonPath(getEnvNonEmpty("NIX_STATE_DIR").value_or(NIX_STATE_DIR))) , nixStateDir(canonPath(getEnvNonEmpty("NIX_STATE_DIR").value_or(NIX_STATE_DIR)))
, nixConfDir(canonPath(getEnvNonEmpty("NIX_CONF_DIR").value_or(NIX_CONF_DIR))) , nixConfDir(canonPath(getEnvNonEmpty("NIX_CONF_DIR").value_or(NIX_CONF_DIR)))
, nixUserConfFiles(getUserConfigFiles()) , nixUserConfFiles(getUserConfigFiles())
, nixManDir(canonPath(NIX_MAN_DIR))
, nixDaemonSocketFile(canonPath(getEnvNonEmpty("NIX_DAEMON_SOCKET_PATH").value_or(nixStateDir + DEFAULT_SOCKET_PATH))) , nixDaemonSocketFile(canonPath(getEnvNonEmpty("NIX_DAEMON_SOCKET_PATH").value_or(nixStateDir + DEFAULT_SOCKET_PATH)))
{ {
#ifndef _WIN32 #ifndef _WIN32
@@ -243,7 +242,7 @@ Path Settings::getDefaultSSLCertFile()
return ""; return "";
} }
const std::string nixVersion = PACKAGE_VERSION; std::string nixVersion = PACKAGE_VERSION;
NLOHMANN_JSON_SERIALIZE_ENUM(SandboxMode, { NLOHMANN_JSON_SERIALIZE_ENUM(SandboxMode, {
{SandboxMode::smEnabled, true}, {SandboxMode::smEnabled, true},
+9 -6
View File
@@ -84,11 +84,6 @@ public:
*/ */
std::vector<Path> nixUserConfFiles; std::vector<Path> nixUserConfFiles;
/**
* The directory where the man pages are stored.
*/
Path nixManDir;
/** /**
* File name of the socket the daemon listens to. * File name of the socket the daemon listens to.
*/ */
@@ -1253,7 +1248,15 @@ void loadConfFile(AbstractConfig & config);
// Used by the Settings constructor // Used by the Settings constructor
std::vector<Path> getUserConfigFiles(); std::vector<Path> getUserConfigFiles();
extern const std::string nixVersion; /**
* The version of Nix itself.
*
* This is not `const`, so that the Nix CLI can provide a more detailed version
* number including the git revision, without having to "re-compile" the entire
* set of Nix libraries to include that version, even when those libraries are
* not affected by the change.
*/
extern std::string nixVersion;
/** /**
* @param loadConfig Whether to load configuration from `nix.conf`, `NIX_CONFIG`, etc. May be disabled for unit tests. * @param loadConfig Whether to load configuration from `nix.conf`, `NIX_CONFIG`, etc. May be disabled for unit tests.
+93 -14
View File
@@ -69,7 +69,10 @@ ref<LegacySSHStore::Connection> LegacySSHStore::openConnection()
command.push_back("--store"); command.push_back("--store");
command.push_back(remoteStore.get()); command.push_back(remoteStore.get());
} }
conn->sshConn = master.startCommand(std::move(command)); conn->sshConn = master.startCommand(std::move(command), std::list{extraSshArgs});
if (connPipeSize) {
conn->sshConn->trySetBufferSize(*connPipeSize);
}
conn->to = FdSink(conn->sshConn->in.get()); conn->to = FdSink(conn->sshConn->in.get());
conn->from = FdSource(conn->sshConn->out.get()); conn->from = FdSource(conn->sshConn->out.get());
@@ -100,19 +103,31 @@ std::string LegacySSHStore::getUri()
return *uriSchemes().begin() + "://" + host; return *uriSchemes().begin() + "://" + host;
} }
std::map<StorePath, UnkeyedValidPathInfo> LegacySSHStore::queryPathInfosUncached(
const StorePathSet & paths)
{
auto conn(connections->get());
/* No longer support missing NAR hash */
assert(GET_PROTOCOL_MINOR(conn->remoteVersion) >= 4);
debug("querying remote host '%s' for info on '%s'", host, concatStringsSep(", ", printStorePathSet(paths)));
auto infos = conn->queryPathInfos(*this, paths);
for (const auto & [_, info] : infos) {
if (info.narHash == Hash::dummy)
throw Error("NAR hash is now mandatory");
}
return infos;
}
void LegacySSHStore::queryPathInfoUncached(const StorePath & path, void LegacySSHStore::queryPathInfoUncached(const StorePath & path,
Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept
{ {
try { try {
auto conn(connections->get()); auto infos = queryPathInfosUncached({path});
/* No longer support missing NAR hash */
assert(GET_PROTOCOL_MINOR(conn->remoteVersion) >= 4);
debug("querying remote host '%s' for info on '%s'", host, printStorePath(path));
auto infos = conn->queryPathInfos(*this, {path});
switch (infos.size()) { switch (infos.size()) {
case 0: case 0:
@@ -120,9 +135,6 @@ void LegacySSHStore::queryPathInfoUncached(const StorePath & path,
case 1: { case 1: {
auto & [path2, info] = *infos.begin(); auto & [path2, info] = *infos.begin();
if (info.narHash == Hash::dummy)
throw Error("NAR hash is now mandatory");
assert(path == path2); assert(path == path2);
return callback(std::make_shared<ValidPathInfo>( return callback(std::make_shared<ValidPathInfo>(
std::move(path), std::move(path),
@@ -193,13 +205,19 @@ void LegacySSHStore::addToStore(const ValidPathInfo & info, Source & source,
void LegacySSHStore::narFromPath(const StorePath & path, Sink & sink) void LegacySSHStore::narFromPath(const StorePath & path, Sink & sink)
{ {
auto conn(connections->get()); narFromPath(path, [&](auto & source) {
conn->narFromPath(*this, path, [&](auto & source) {
copyNAR(source, sink); copyNAR(source, sink);
}); });
} }
void LegacySSHStore::narFromPath(const StorePath & path, std::function<void(Source &)> fun)
{
auto conn(connections->get());
conn->narFromPath(*this, path, fun);
}
static ServeProto::BuildOptions buildSettings() static ServeProto::BuildOptions buildSettings()
{ {
return { return {
@@ -223,6 +241,19 @@ BuildResult LegacySSHStore::buildDerivation(const StorePath & drvPath, const Bas
return conn->getBuildDerivationResponse(*this); return conn->getBuildDerivationResponse(*this);
} }
std::function<BuildResult()> LegacySSHStore::buildDerivationAsync(
const StorePath & drvPath, const BasicDerivation & drv,
const ServeProto::BuildOptions & options)
{
// Until we have C++23 std::move_only_function
auto conn = std::make_shared<Pool<Connection>::Handle>(connections->get());
(*conn)->putBuildDerivationRequest(*this, drvPath, drv, options);
return [this,conn]() -> BuildResult {
return (*conn)->getBuildDerivationResponse(*this);
};
}
void LegacySSHStore::buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore) void LegacySSHStore::buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore)
{ {
@@ -294,6 +325,32 @@ StorePathSet LegacySSHStore::queryValidPaths(const StorePathSet & paths,
} }
StorePathSet LegacySSHStore::queryValidPaths(const StorePathSet & paths,
bool lock, SubstituteFlag maybeSubstitute)
{
auto conn(connections->get());
return conn->queryValidPaths(*this,
lock, paths, maybeSubstitute);
}
void LegacySSHStore::addMultipleToStoreLegacy(Store & srcStore, const StorePathSet & paths)
{
auto conn(connections->get());
conn->to << ServeProto::Command::ImportPaths;
try {
srcStore.exportPaths(paths, conn->to);
} catch (...) {
conn->good = false;
throw;
}
conn->to.flush();
if (readInt(conn->from) != 1)
throw Error("remote machine failed to import closure");
}
void LegacySSHStore::connect() void LegacySSHStore::connect()
{ {
auto conn(connections->get()); auto conn(connections->get());
@@ -307,6 +364,28 @@ unsigned int LegacySSHStore::getProtocol()
} }
pid_t LegacySSHStore::getConnectionPid()
{
auto conn(connections->get());
#ifndef _WIN32
return conn->sshConn->sshPid;
#else
// TODO: Implement
return 0;
#endif
}
LegacySSHStore::ConnectionStats LegacySSHStore::getConnectionStats()
{
auto conn(connections->get());
return {
.bytesReceived = conn->from.read,
.bytesSent = conn->to.written,
};
}
/** /**
* The legacy ssh protocol doesn't support checking for trusted-user. * The legacy ssh protocol doesn't support checking for trusted-user.
* Try using ssh-ng:// instead if you want to know. * Try using ssh-ng:// instead if you want to know.
+60
View File
@@ -6,6 +6,7 @@
#include "ssh.hh" #include "ssh.hh"
#include "callback.hh" #include "callback.hh"
#include "pool.hh" #include "pool.hh"
#include "serve-protocol.hh"
namespace nix { namespace nix {
@@ -24,6 +25,16 @@ struct LegacySSHStoreConfig : virtual CommonSSHStoreConfig
const Setting<int> maxConnections{this, 1, "max-connections", const Setting<int> maxConnections{this, 1, "max-connections",
"Maximum number of concurrent SSH connections."}; "Maximum number of concurrent SSH connections."};
/**
* Hack for hydra
*/
Strings extraSshArgs = {};
/**
* Exposed for hydra
*/
std::optional<size_t> connPipeSize;
const std::string name() override { return "SSH Store"; } const std::string name() override { return "SSH Store"; }
static std::set<std::string> uriSchemes() { return {"ssh"}; } static std::set<std::string> uriSchemes() { return {"ssh"}; }
@@ -60,11 +71,24 @@ struct LegacySSHStore : public virtual LegacySSHStoreConfig, public virtual Stor
void queryPathInfoUncached(const StorePath & path, void queryPathInfoUncached(const StorePath & path,
Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept override; Callback<std::shared_ptr<const ValidPathInfo>> callback) noexcept override;
std::map<StorePath, UnkeyedValidPathInfo> queryPathInfosUncached(
const StorePathSet & paths);
void addToStore(const ValidPathInfo & info, Source & source, void addToStore(const ValidPathInfo & info, Source & source,
RepairFlag repair, CheckSigsFlag checkSigs) override; RepairFlag repair, CheckSigsFlag checkSigs) override;
void narFromPath(const StorePath & path, Sink & sink) override; void narFromPath(const StorePath & path, Sink & sink) override;
/**
* Hands over the connection temporarily as source to the given
* function. The function must not consume beyond the NAR; it can
* not just blindly try to always read more bytes until it is
* cut-off.
*
* This is exposed for sake of Hydra.
*/
void narFromPath(const StorePath & path, std::function<void(Source &)> fun);
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override
{ unsupported("queryPathFromHashPart"); } { unsupported("queryPathFromHashPart"); }
@@ -93,6 +117,16 @@ public:
BuildResult buildDerivation(const StorePath & drvPath, const BasicDerivation & drv, BuildResult buildDerivation(const StorePath & drvPath, const BasicDerivation & drv,
BuildMode buildMode) override; BuildMode buildMode) override;
/**
* Note, the returned function must only be called once, or we'll
* try to read from the connection twice.
*
* @todo Use C++23 `std::move_only_function`.
*/
std::function<BuildResult()> buildDerivationAsync(
const StorePath & drvPath, const BasicDerivation & drv,
const ServeProto::BuildOptions & options);
void buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore) override; void buildPaths(const std::vector<DerivedPath> & drvPaths, BuildMode buildMode, std::shared_ptr<Store> evalStore) override;
void ensurePath(const StorePath & path) override void ensurePath(const StorePath & path) override
@@ -119,10 +153,36 @@ public:
StorePathSet queryValidPaths(const StorePathSet & paths, StorePathSet queryValidPaths(const StorePathSet & paths,
SubstituteFlag maybeSubstitute = NoSubstitute) override; SubstituteFlag maybeSubstitute = NoSubstitute) override;
/**
* Custom variation that atomically creates temp locks on the remote
* side.
*
* This exists to prevent a race where the remote host
* garbage-collects paths that are already there. Optionally, ask
* the remote host to substitute missing paths.
*/
StorePathSet queryValidPaths(const StorePathSet & paths,
bool lock,
SubstituteFlag maybeSubstitute = NoSubstitute);
/**
* Just exists because this is exactly what Hydra was doing, and we
* don't yet want an algorithmic change.
*/
void addMultipleToStoreLegacy(Store & srcStore, const StorePathSet & paths);
void connect() override; void connect() override;
unsigned int getProtocol() override; unsigned int getProtocol() override;
struct ConnectionStats {
size_t bytesReceived, bytesSent;
};
ConnectionStats getConnectionStats();
pid_t getConnectionPid();
/** /**
* The legacy ssh protocol doesn't support checking for trusted-user. * The legacy ssh protocol doesn't support checking for trusted-user.
* Try using ssh-ng:// instead if you want to know. * Try using ssh-ng:// instead if you want to know.
+9 -4
View File
@@ -116,7 +116,7 @@ LocalStore::LocalStore(
state->stmts = std::make_unique<State::Stmts>(); state->stmts = std::make_unique<State::Stmts>();
/* Create missing state directories if they don't already exist. */ /* Create missing state directories if they don't already exist. */
createDirs(realStoreDir); createDirs(realStoreDir.get());
if (readOnly) { if (readOnly) {
experimentalFeatureSettings.require(Xp::ReadOnlyLocalStore); experimentalFeatureSettings.require(Xp::ReadOnlyLocalStore);
} else { } else {
@@ -136,7 +136,12 @@ LocalStore::LocalStore(
for (auto & perUserDir : {profilesDir + "/per-user", gcRootsDir + "/per-user"}) { for (auto & perUserDir : {profilesDir + "/per-user", gcRootsDir + "/per-user"}) {
createDirs(perUserDir); createDirs(perUserDir);
if (!readOnly) { if (!readOnly) {
if (chmod(perUserDir.c_str(), 0755) == -1) auto st = lstat(perUserDir);
// Skip chmod call if the directory already has the correct permissions (0755).
// This is to avoid failing when the executing user lacks permissions to change the directory's permissions
// even if it would be no-op.
if ((st.st_mode & (S_IRWXU | S_IRWXG | S_IRWXO)) != 0755 && chmod(perUserDir.c_str(), 0755) == -1)
throw SysError("could not set permissions on '%s' to 755", perUserDir); throw SysError("could not set permissions on '%s' to 755", perUserDir);
} }
} }
@@ -243,7 +248,7 @@ LocalStore::LocalStore(
else if (curSchema == 0) { /* new store */ else if (curSchema == 0) { /* new store */
curSchema = nixSchemaVersion; curSchema = nixSchemaVersion;
openDB(*state, true); openDB(*state, true);
writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true); writeFile(schemaPath, fmt("%1%", curSchema), 0666, FsSync::Yes);
} }
else if (curSchema < nixSchemaVersion) { else if (curSchema < nixSchemaVersion) {
@@ -294,7 +299,7 @@ LocalStore::LocalStore(
txn.commit(); txn.commit();
} }
writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true); writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, FsSync::Yes);
lockFile(globalLock.get(), ltRead, true); lockFile(globalLock.get(), ltRead, true);
} }
+2 -1
View File
@@ -112,7 +112,8 @@ deps_public += nlohmann_json
sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19') sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19')
deps_private += sqlite deps_private += sqlite
# AWS C++ SDK has bad pkg-config # AWS C++ SDK has bad pkg-config. See
# https://github.com/aws/aws-sdk-cpp/issues/2673 for details.
aws_s3 = dependency('aws-cpp-sdk-s3', required : false) aws_s3 = dependency('aws-cpp-sdk-s3', required : false)
configdata.set('ENABLE_S3', aws_s3.found().to_int()) configdata.set('ENABLE_S3', aws_s3.found().to_int())
if aws_s3.found() if aws_s3.found()
+38 -47
View File
@@ -1,25 +1,30 @@
{ lib {
, stdenv lib,
, mkMesonLibrary stdenv,
mkMesonLibrary,
, unixtools unixtools,
, darwin darwin,
, nix-util nix-util,
, boost boost,
, curl curl,
, aws-sdk-cpp aws-sdk-cpp,
, libseccomp libseccomp,
, nlohmann_json nlohmann_json,
, sqlite sqlite,
, busybox-sandbox-shell ? null busybox-sandbox-shell ? null,
# Configuration Options # Configuration Options
, version version,
, embeddedSandboxShell ? stdenv.hostPlatform.isStatic embeddedSandboxShell ? stdenv.hostPlatform.isStatic,
withAWS ?
# Default is this way because there have been issues building this dependency
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin),
}: }:
let let
@@ -48,46 +53,32 @@ mkMesonLibrary (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "sql") ./.) (fileset.fileFilter (file: file.hasExt "sql") ./.)
]; ];
nativeBuildInputs = nativeBuildInputs = lib.optional embeddedSandboxShell unixtools.hexdump;
lib.optional embeddedSandboxShell unixtools.hexdump;
buildInputs = [ buildInputs =
boost [
curl boost
sqlite curl
] ++ lib.optional stdenv.hostPlatform.isLinux libseccomp sqlite
]
++ lib.optional stdenv.hostPlatform.isLinux libseccomp
# There have been issues building these dependencies # There have been issues building these dependencies
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox ++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional (stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)) ++ lib.optional withAWS aws-sdk-cpp;
aws-sdk-cpp
;
propagatedBuildInputs = [ propagatedBuildInputs = [
nix-util nix-util
nlohmann_json nlohmann_json
]; ];
preConfigure = mesonFlags =
# "Inline" .version so it's not a symlink, and includes the suffix. [
# Do the meson utils, without modification. (lib.mesonEnable "seccomp-sandboxing" stdenv.hostPlatform.isLinux)
'' (lib.mesonBool "embedded-sandbox-shell" embeddedSandboxShell)
chmod u+w ./.version ]
echo ${version} > ../../.version ++ lib.optionals stdenv.hostPlatform.isLinux [
''; (lib.mesonOption "sandbox-shell" "${busybox-sandbox-shell}/bin/busybox")
];
mesonFlags = [
(lib.mesonEnable "seccomp-sandboxing" stdenv.hostPlatform.isLinux)
(lib.mesonBool "embedded-sandbox-shell" embeddedSandboxShell)
] ++ lib.optionals stdenv.hostPlatform.isLinux [
(lib.mesonOption "sandbox-shell" "${busybox-sandbox-shell}/bin/busybox")
];
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
};
meta = { meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows; platforms = lib.platforms.unix ++ lib.platforms.windows;
+11 -1
View File
@@ -539,11 +539,21 @@ void RemoteStore::addMultipleToStore(
RepairFlag repair, RepairFlag repair,
CheckSigsFlag checkSigs) CheckSigsFlag checkSigs)
{ {
// `addMultipleToStore` is single threaded
size_t bytesExpected = 0;
for (auto & [pathInfo, _] : pathsToCopy) {
bytesExpected += pathInfo.narSize;
}
act.setExpected(actCopyPath, bytesExpected);
auto source = sinkToSource([&](Sink & sink) { auto source = sinkToSource([&](Sink & sink) {
sink << pathsToCopy.size(); size_t nrTotal = pathsToCopy.size();
sink << nrTotal;
// Reverse, so we can release memory at the original start // Reverse, so we can release memory at the original start
std::reverse(pathsToCopy.begin(), pathsToCopy.end()); std::reverse(pathsToCopy.begin(), pathsToCopy.end());
while (!pathsToCopy.empty()) { while (!pathsToCopy.empty()) {
act.progress(nrTotal - pathsToCopy.size(), nrTotal, size_t(1), size_t(0));
auto & [pathInfo, pathSource] = pathsToCopy.back(); auto & [pathInfo, pathSource] = pathsToCopy.back();
WorkerProto::Serialise<ValidPathInfo>::write(*this, WorkerProto::Serialise<ValidPathInfo>::write(*this,
WorkerProto::WriteConn { WorkerProto::WriteConn {
+15
View File
@@ -240,4 +240,19 @@ Path SSHMaster::startMaster()
#endif #endif
void SSHMaster::Connection::trySetBufferSize(size_t size)
{
#ifdef F_SETPIPE_SZ
/* This `fcntl` method of doing this takes a positive `int`. Check
and convert accordingly.
The function overall still takes `size_t` because this is more
portable for a platform-agnostic interface. */
assert(size <= INT_MAX);
int pipesize = size;
fcntl(in.get(), F_SETPIPE_SZ, pipesize);
fcntl(out.get(), F_SETPIPE_SZ, pipesize);
#endif
}
} }
+12
View File
@@ -54,6 +54,18 @@ public:
Pid sshPid; Pid sshPid;
#endif #endif
AutoCloseFD out, in; AutoCloseFD out, in;
/**
* Try to set the buffer size in both directions to the
* designated amount, if possible. If not possible, does
* nothing.
*
* Current implementation is to use `fcntl` with `F_SETPIPE_SZ`,
* which is Linux-only. For this implementation, `size` must
* convertable to an `int`. In other words, it must be within
* `[0, INT_MAX]`.
*/
void trySetBufferSize(size_t size);
}; };
/** /**
+11 -12
View File
@@ -230,20 +230,24 @@ void Store::addMultipleToStore(
{ {
std::atomic<size_t> nrDone{0}; std::atomic<size_t> nrDone{0};
std::atomic<size_t> nrFailed{0}; std::atomic<size_t> nrFailed{0};
std::atomic<uint64_t> bytesExpected{0};
std::atomic<uint64_t> nrRunning{0}; std::atomic<uint64_t> nrRunning{0};
using PathWithInfo = std::pair<ValidPathInfo, std::unique_ptr<Source>>; using PathWithInfo = std::pair<ValidPathInfo, std::unique_ptr<Source>>;
uint64_t bytesExpected = 0;
std::map<StorePath, PathWithInfo *> infosMap; std::map<StorePath, PathWithInfo *> infosMap;
StorePathSet storePathsToAdd; StorePathSet storePathsToAdd;
for (auto & thingToAdd : pathsToCopy) { for (auto & thingToAdd : pathsToCopy) {
bytesExpected += thingToAdd.first.narSize;
infosMap.insert_or_assign(thingToAdd.first.path, &thingToAdd); infosMap.insert_or_assign(thingToAdd.first.path, &thingToAdd);
storePathsToAdd.insert(thingToAdd.first.path); storePathsToAdd.insert(thingToAdd.first.path);
} }
auto showProgress = [&]() { act.setExpected(actCopyPath, bytesExpected);
act.progress(nrDone, pathsToCopy.size(), nrRunning, nrFailed);
auto showProgress = [&, nrTotal = pathsToCopy.size()]() {
act.progress(nrDone, nrTotal, nrRunning, nrFailed);
}; };
processGraph<StorePath>( processGraph<StorePath>(
@@ -259,9 +263,6 @@ void Store::addMultipleToStore(
return StorePathSet(); return StorePathSet();
} }
bytesExpected += info.narSize;
act.setExpected(actCopyPath, bytesExpected);
return info.references; return info.references;
}, },
@@ -1104,9 +1105,6 @@ std::map<StorePath, StorePath> copyPaths(
return storePathForDst; return storePathForDst;
}; };
// total is accessed by each copy, which are each handled in separate threads
std::atomic<uint64_t> total = 0;
for (auto & missingPath : sortedMissing) { for (auto & missingPath : sortedMissing) {
auto info = srcStore.queryPathInfo(missingPath); auto info = srcStore.queryPathInfo(missingPath);
@@ -1116,9 +1114,10 @@ std::map<StorePath, StorePath> copyPaths(
ValidPathInfo infoForDst = *info; ValidPathInfo infoForDst = *info;
infoForDst.path = storePathForDst; infoForDst.path = storePathForDst;
auto source = sinkToSource([&](Sink & sink) { auto source = sinkToSource([&, narSize = info->narSize](Sink & sink) {
// We can reasonably assume that the copy will happen whenever we // We can reasonably assume that the copy will happen whenever we
// read the path, so log something about that at that point // read the path, so log something about that at that point
uint64_t total = 0;
auto srcUri = srcStore.getUri(); auto srcUri = srcStore.getUri();
auto dstUri = dstStore.getUri(); auto dstUri = dstStore.getUri();
auto storePathS = srcStore.printStorePath(missingPath); auto storePathS = srcStore.printStorePath(missingPath);
@@ -1129,13 +1128,13 @@ std::map<StorePath, StorePath> copyPaths(
LambdaSink progressSink([&](std::string_view data) { LambdaSink progressSink([&](std::string_view data) {
total += data.size(); total += data.size();
act.progress(total, info->narSize); act.progress(total, narSize);
}); });
TeeSink tee { sink, progressSink }; TeeSink tee { sink, progressSink };
srcStore.narFromPath(missingPath, tee); srcStore.narFromPath(missingPath, tee);
}); });
pathsToCopy.push_back(std::pair{infoForDst, std::move(source)}); pathsToCopy.emplace_back(std::move(infoForDst), std::move(source));
} }
dstStore.addMultipleToStore(std::move(pathsToCopy), act, repair, checkSigs); dstStore.addMultipleToStore(std::move(pathsToCopy), act, repair, checkSigs);
@@ -559,7 +559,14 @@ void LocalDerivationGoal::startBuilder()
} else { } else {
tmpDir = topTmpDir; tmpDir = topTmpDir;
} }
chownToBuilder(tmpDir);
/* The TOCTOU between the previous mkdir call and this open call is unavoidable due to
POSIX semantics.*/
tmpDirFd = AutoCloseFD{open(tmpDir.c_str(), O_RDONLY | O_NOFOLLOW | O_DIRECTORY)};
if (!tmpDirFd)
throw SysError("failed to open the build temporary directory descriptor '%1%'", tmpDir);
chownToBuilder(tmpDirFd.get(), tmpDir);
for (auto & [outputName, status] : initialOutputs) { for (auto & [outputName, status] : initialOutputs) {
/* Set scratch path we'll actually use during the build. /* Set scratch path we'll actually use during the build.
@@ -1157,9 +1164,7 @@ void LocalDerivationGoal::initTmpDir()
} else { } else {
auto hash = hashString(HashAlgorithm::SHA256, i.first); auto hash = hashString(HashAlgorithm::SHA256, i.first);
std::string fn = ".attr-" + hash.to_string(HashFormat::Nix32, false); std::string fn = ".attr-" + hash.to_string(HashFormat::Nix32, false);
Path p = tmpDir + "/" + fn; writeBuilderFile(fn, rewriteStrings(i.second, inputRewrites));
writeFile(p, rewriteStrings(i.second, inputRewrites));
chownToBuilder(p);
env[i.first + "Path"] = tmpDirInSandbox + "/" + fn; env[i.first + "Path"] = tmpDirInSandbox + "/" + fn;
} }
} }
@@ -1264,11 +1269,9 @@ void LocalDerivationGoal::writeStructuredAttrs()
auto jsonSh = writeStructuredAttrsShell(json); auto jsonSh = writeStructuredAttrsShell(json);
writeFile(tmpDir + "/.attrs.sh", rewriteStrings(jsonSh, inputRewrites)); writeBuilderFile(".attrs.sh", rewriteStrings(jsonSh, inputRewrites));
chownToBuilder(tmpDir + "/.attrs.sh");
env["NIX_ATTRS_SH_FILE"] = tmpDirInSandbox + "/.attrs.sh"; env["NIX_ATTRS_SH_FILE"] = tmpDirInSandbox + "/.attrs.sh";
writeFile(tmpDir + "/.attrs.json", rewriteStrings(json.dump(), inputRewrites)); writeBuilderFile(".attrs.json", rewriteStrings(json.dump(), inputRewrites));
chownToBuilder(tmpDir + "/.attrs.json");
env["NIX_ATTRS_JSON_FILE"] = tmpDirInSandbox + "/.attrs.json"; env["NIX_ATTRS_JSON_FILE"] = tmpDirInSandbox + "/.attrs.json";
} }
} }
@@ -1779,6 +1782,24 @@ void setupSeccomp()
#endif #endif
} }
void LocalDerivationGoal::chownToBuilder(int fd, const Path & path)
{
if (!buildUser) return;
if (fchown(fd, buildUser->getUID(), buildUser->getGID()) == -1)
throw SysError("cannot change ownership of file '%1%'", path);
}
void LocalDerivationGoal::writeBuilderFile(
const std::string & name,
std::string_view contents)
{
auto path = std::filesystem::path(tmpDir) / name;
AutoCloseFD fd{openat(tmpDirFd.get(), name.c_str(), O_WRONLY | O_TRUNC | O_CREAT | O_CLOEXEC | O_EXCL | O_NOFOLLOW, 0666)};
if (!fd)
throw SysError("creating file %s", path);
writeFile(fd, path, contents);
chownToBuilder(fd.get(), path);
}
void LocalDerivationGoal::runChild() void LocalDerivationGoal::runChild()
{ {
@@ -2149,7 +2170,18 @@ void LocalDerivationGoal::runChild()
without file-write* allowed, access() incorrectly returns EPERM without file-write* allowed, access() incorrectly returns EPERM
*/ */
sandboxProfile += "(allow file-read* file-write* process-exec\n"; sandboxProfile += "(allow file-read* file-write* process-exec\n";
// We create multiple allow lists, to avoid exceeding a limit in the darwin sandbox interpreter.
// See https://github.com/NixOS/nix/issues/4119
// We split our allow groups approximately at half the actual limit, 1 << 16
const int breakpoint = sandboxProfile.length() + (1 << 14);
for (auto & i : pathsInChroot) { for (auto & i : pathsInChroot) {
if (sandboxProfile.length() >= breakpoint) {
debug("Sandbox break: %d %d", sandboxProfile.length(), breakpoint);
sandboxProfile += ")\n(allow file-read* file-write* process-exec\n";
}
if (i.first != i.second.source) if (i.first != i.second.source)
throw Error( throw Error(
"can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin", "can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin",
@@ -2657,10 +2689,14 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
wanted.to_string(HashFormat::SRI, true), wanted.to_string(HashFormat::SRI, true),
got.to_string(HashFormat::SRI, true))); got.to_string(HashFormat::SRI, true)));
} }
if (!newInfo0.references.empty()) if (!newInfo0.references.empty()) {
auto numViolations = newInfo.references.size();
delayedException = std::make_exception_ptr( delayedException = std::make_exception_ptr(
BuildError("illegal path references in fixed-output derivation '%s'", BuildError("fixed-output derivations must not reference store paths: '%s' references %d distinct paths, e.g. '%s'",
worker.store.printStorePath(drvPath))); worker.store.printStorePath(drvPath),
numViolations,
worker.store.printStorePath(*newInfo.references.begin())));
}
return newInfo0; return newInfo0;
}, },
@@ -2927,8 +2963,12 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
spec.insert(worker.store.parseStorePath(i)); spec.insert(worker.store.parseStorePath(i));
else if (auto output = get(outputs, i)) else if (auto output = get(outputs, i))
spec.insert(output->path); spec.insert(output->path);
else else {
throw BuildError("derivation contains an illegal reference specifier '%s'", i); std::string outputsListing = concatMapStringsSep(", ", outputs, [](auto & o) { return o.first; });
throw BuildError("derivation '%s' output check for '%s' contains an illegal reference specifier '%s',"
" expected store path or output name (one of [%s])",
worker.store.printStorePath(drvPath), outputName, i, outputsListing);
}
} }
auto used = recursive auto used = recursive
@@ -3034,6 +3074,15 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
void LocalDerivationGoal::deleteTmpDir(bool force) void LocalDerivationGoal::deleteTmpDir(bool force)
{ {
if (topTmpDir != "") { if (topTmpDir != "") {
/* As an extra precaution, even in the event of `deletePath` failing to
* clean up, the `tmpDir` will be chowned as if we were to move
* it inside the Nix store.
*
* This hardens against an attack which smuggles a file descriptor
* to make use of the temporary directory.
*/
chmod(topTmpDir.c_str(), 0000);
/* Don't keep temporary directories for builtins because they /* Don't keep temporary directories for builtins because they
might have privileged stuff (like a copy of netrc). */ might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv->isBuiltin()) { if (settings.keepFailed && !force && !drv->isBuiltin()) {
@@ -37,6 +37,11 @@ struct LocalDerivationGoal : public DerivationGoal
*/ */
Path topTmpDir; Path topTmpDir;
/**
* The file descriptor of the temporary directory.
*/
AutoCloseFD tmpDirFd;
/** /**
* The path of the temporary directory in the sandbox. * The path of the temporary directory in the sandbox.
*/ */
@@ -244,9 +249,24 @@ struct LocalDerivationGoal : public DerivationGoal
/** /**
* Make a file owned by the builder. * Make a file owned by the builder.
*
* SAFETY: this function is prone to TOCTOU as it receives a path and not a descriptor.
* It's only safe to call in a child of a directory only visible to the owner.
*/ */
void chownToBuilder(const Path & path); void chownToBuilder(const Path & path);
/**
* Make a file owned by the builder addressed by its file descriptor.
*/
void chownToBuilder(int fd, const Path & path);
/**
* Create a file in `tmpDir` owned by the builder.
*/
void writeBuilderFile(
const std::string & name,
std::string_view contents);
int getChildStatus() override; int getChildStatus() override;
/** /**
+6 -13
View File
@@ -1,11 +1,12 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util nix-util,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -33,14 +34,6 @@ mkMesonLibrary (finalAttrs: {
nix-util nix-util
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+8 -15
View File
@@ -1,14 +1,15 @@
{ lib {
, mkMesonLibrary lib,
mkMesonLibrary,
, nix-util nix-util,
, nix-util-c nix-util-c,
, rapidcheck rapidcheck,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -37,14 +38,6 @@ mkMesonLibrary (finalAttrs: {
rapidcheck rapidcheck
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
+18
View File
@@ -0,0 +1,18 @@
#include "util.hh"
#include "monitor-fd.hh"
#include <sys/file.h>
#include <gtest/gtest.h>
namespace nix {
TEST(MonitorFdHup, shouldNotBlock)
{
Pipe p;
p.create();
{
// when monitor gets destroyed it should cancel the
// background thread and do not block
MonitorFdHup monitor(p.readSide.get());
}
}
}
+29 -30
View File
@@ -1,19 +1,20 @@
{ lib {
, buildPackages lib,
, stdenv buildPackages,
, mkMesonExecutable stdenv,
mkMesonExecutable,
, nix-util nix-util,
, nix-util-c nix-util-c,
, nix-util-test-support nix-util-test-support,
, rapidcheck rapidcheck,
, gtest gtest,
, runCommand runCommand,
# Configuration Options # Configuration Options
, version version,
}: }:
let let
@@ -44,29 +45,27 @@ mkMesonExecutable (finalAttrs: {
gtest gtest
]; ];
preConfigure =
# "Inline" .version so it's not a symlink, and includes the suffix.
# Do the meson utils, without modification.
''
chmod u+w ./.version
echo ${version} > ../../.version
'';
mesonFlags = [ mesonFlags = [
]; ];
passthru = { passthru = {
tests = { tests = {
run = runCommand "${finalAttrs.pname}-run" { run =
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages; runCommand "${finalAttrs.pname}-run"
} (lib.optionalString stdenv.hostPlatform.isWindows '' {
export HOME="$PWD/home-dir" meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
mkdir -p "$HOME" }
'' + '' (
export _NIX_TEST_UNIT_DATA=${./data} lib.optionalString stdenv.hostPlatform.isWindows ''
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage} export HOME="$PWD/home-dir"
touch $out mkdir -p "$HOME"
''); ''
+ ''
export _NIX_TEST_UNIT_DATA=${./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
}; };
}; };
+36
View File
@@ -80,6 +80,42 @@ TEST(concatStringsSep, buildSingleString)
ASSERT_EQ(concatStringsSep(",", strings), "this"); ASSERT_EQ(concatStringsSep(",", strings), "this");
} }
TEST(concatMapStringsSep, empty)
{
Strings strings;
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "");
}
TEST(concatMapStringsSep, justOne)
{
Strings strings;
strings.push_back("this");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this");
}
TEST(concatMapStringsSep, two)
{
Strings strings;
strings.push_back("this");
strings.push_back("that");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this,that");
}
TEST(concatMapStringsSep, map)
{
std::map<std::string, std::string> strings;
strings["this"] = "that";
strings["1"] = "one";
ASSERT_EQ(
concatMapStringsSep(
", ", strings, [](const std::pair<std::string, std::string> & s) { return s.first + " -> " + s.second; }),
"1 -> one, this -> that");
}
/* ---------------------------------------------------------------------------- /* ----------------------------------------------------------------------------
* dropEmptyInitThenConcatStringsSep * dropEmptyInitThenConcatStringsSep
* --------------------------------------------------------------------------*/ * --------------------------------------------------------------------------*/
+8
View File
@@ -50,6 +50,14 @@ struct LinesOfCode {
std::optional<std::string> nextLineOfCode; std::optional<std::string> nextLineOfCode;
}; };
/* NOTE: position.hh recursively depends on source-path.hh -> source-accessor.hh
-> hash.hh -> config.hh -> experimental-features.hh -> error.hh -> Pos.
There are other such cycles.
Thus, Pos has to be an incomplete type in this header. But since ErrorInfo/Trace
have to refer to Pos, they have to use pointer indirection via std::shared_ptr
to break the recursive header dependency.
FIXME: Untangle this mess. Should there be AbstractPos as there used to be before
4feb7d9f71? */
struct Pos; struct Pos;
void printCodeLines(std::ostream & out, void printCodeLines(std::ostream & out,
+1 -1
View File
@@ -93,7 +93,7 @@ void restorePath(
{ {
switch (method) { switch (method) {
case FileSerialisationMethod::Flat: case FileSerialisationMethod::Flat:
writeFile(path, source, 0666, startFsync); writeFile(path, source, 0666, startFsync ? FsSync::Yes : FsSync::No);
break; break;
case FileSerialisationMethod::NixArchive: case FileSerialisationMethod::NixArchive:
restorePath(path, source, startFsync); restorePath(path, source, startFsync);
+44 -27
View File
@@ -29,7 +29,17 @@
namespace nix { namespace nix {
namespace fs { using namespace std::filesystem; } namespace fs {
using namespace std::filesystem;
bool symlink_exists(const std::filesystem::path & path) {
try {
return std::filesystem::exists(std::filesystem::symlink_status(path));
} catch (const std::filesystem::filesystem_error & e) {
throw SysError("cannot check existence of %1%", path);
}
}
}
bool isAbsolute(PathView path) bool isAbsolute(PathView path)
{ {
@@ -258,7 +268,7 @@ void readFile(const Path & path, Sink & sink)
} }
void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync) void writeFile(const Path & path, std::string_view s, mode_t mode, FsSync sync)
{ {
AutoCloseFD fd = toDescriptor(open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT AutoCloseFD fd = toDescriptor(open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT
// TODO // TODO
@@ -268,22 +278,29 @@ void writeFile(const Path & path, std::string_view s, mode_t mode, bool sync)
, mode)); , mode));
if (!fd) if (!fd)
throw SysError("opening file '%1%'", path); throw SysError("opening file '%1%'", path);
try {
writeFull(fd.get(), s); writeFile(fd, path, s, mode, sync);
} catch (Error & e) {
e.addTrace({}, "writing file '%1%'", path); /* Close explicitly to propagate the exceptions. */
throw;
}
if (sync)
fd.fsync();
// Explicitly close to make sure exceptions are propagated.
fd.close(); fd.close();
if (sync)
syncParent(path);
} }
void writeFile(AutoCloseFD & fd, const Path & origPath, std::string_view s, mode_t mode, FsSync sync)
{
assert(fd);
try {
writeFull(fd.get(), s);
void writeFile(const Path & path, Source & source, mode_t mode, bool sync) if (sync == FsSync::Yes)
fd.fsync();
} catch (Error & e) {
e.addTrace({}, "writing file '%1%'", origPath);
throw;
}
}
void writeFile(const Path & path, Source & source, mode_t mode, FsSync sync)
{ {
AutoCloseFD fd = toDescriptor(open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT AutoCloseFD fd = toDescriptor(open(path.c_str(), O_WRONLY | O_TRUNC | O_CREAT
// TODO // TODO
@@ -307,11 +324,11 @@ void writeFile(const Path & path, Source & source, mode_t mode, bool sync)
e.addTrace({}, "writing file '%1%'", path); e.addTrace({}, "writing file '%1%'", path);
throw; throw;
} }
if (sync) if (sync == FsSync::Yes)
fd.fsync(); fd.fsync();
// Explicitly close to make sure exceptions are propagated. // Explicitly close to make sure exceptions are propagated.
fd.close(); fd.close();
if (sync) if (sync == FsSync::Yes)
syncParent(path); syncParent(path);
} }
@@ -374,7 +391,8 @@ static void _deletePath(Descriptor parentfd, const fs::path & path, uint64_t & b
#ifndef _WIN32 #ifndef _WIN32
checkInterrupt(); checkInterrupt();
std::string name(baseNameOf(path.native())); std::string name(path.filename());
assert(name != "." && name != ".." && !name.empty());
struct stat st; struct stat st;
if (fstatat(parentfd, name.c_str(), &st, if (fstatat(parentfd, name.c_str(), &st,
@@ -415,7 +433,7 @@ static void _deletePath(Descriptor parentfd, const fs::path & path, uint64_t & b
throw SysError("chmod %1%", path); throw SysError("chmod %1%", path);
} }
int fd = openat(parentfd, path.c_str(), O_RDONLY); int fd = openat(parentfd, name.c_str(), O_RDONLY | O_DIRECTORY | O_NOFOLLOW);
if (fd == -1) if (fd == -1)
throw SysError("opening directory %1%", path); throw SysError("opening directory %1%", path);
AutoCloseDir dir(fdopendir(fd)); AutoCloseDir dir(fdopendir(fd));
@@ -427,7 +445,7 @@ static void _deletePath(Descriptor parentfd, const fs::path & path, uint64_t & b
checkInterrupt(); checkInterrupt();
std::string childName = dirent->d_name; std::string childName = dirent->d_name;
if (childName == "." || childName == "..") continue; if (childName == "." || childName == "..") continue;
_deletePath(dirfd(dir.get()), path + "/" + childName, bytesFreed); _deletePath(dirfd(dir.get()), path / childName, bytesFreed);
} }
if (errno) throw SysError("reading directory %1%", path); if (errno) throw SysError("reading directory %1%", path);
} }
@@ -445,14 +463,13 @@ static void _deletePath(Descriptor parentfd, const fs::path & path, uint64_t & b
static void _deletePath(const fs::path & path, uint64_t & bytesFreed) static void _deletePath(const fs::path & path, uint64_t & bytesFreed)
{ {
Path dir = dirOf(path.string()); assert(path.is_absolute());
if (dir == "") assert(path.parent_path() != path);
dir = "/";
AutoCloseFD dirfd = toDescriptor(open(dir.c_str(), O_RDONLY)); AutoCloseFD dirfd = toDescriptor(open(path.parent_path().string().c_str(), O_RDONLY));
if (!dirfd) { if (!dirfd) {
if (errno == ENOENT) return; if (errno == ENOENT) return;
throw SysError("opening directory '%1%'", path); throw SysError("opening directory %s", path.parent_path());
} }
_deletePath(dirfd.get(), path, bytesFreed); _deletePath(dirfd.get(), path, bytesFreed);
@@ -475,12 +492,12 @@ void createDir(const Path & path, mode_t mode)
throw SysError("creating directory '%1%'", path); throw SysError("creating directory '%1%'", path);
} }
void createDirs(const Path & path) void createDirs(const fs::path & path)
{ {
try { try {
fs::create_directories(path); fs::create_directories(path);
} catch (fs::filesystem_error & e) { } catch (fs::filesystem_error & e) {
throw SysError("creating directory '%1%'", path); throw SysError("creating directory '%1%'", path.string());
} }
} }
@@ -680,7 +697,7 @@ void setWriteTime(
if (utimes(path.c_str(), times) == -1) if (utimes(path.c_str(), times) == -1)
throw SysError("changing modification time of %s (not a symlink)", path); throw SysError("changing modification time of %s (not a symlink)", path);
} else { } else {
throw Error("Cannot modification time of symlink %s", path); throw Error("Cannot change modification time of symlink %s", path);
} }
#endif #endif
#endif #endif
+14 -14
View File
@@ -134,6 +134,7 @@ bool pathExists(const Path & path);
namespace fs { namespace fs {
/** /**
* TODO: we may actually want to use pathExists instead of this function
* ``` * ```
* symlink_exists(p) = std::filesystem::exists(std::filesystem::symlink_status(p)) * symlink_exists(p) = std::filesystem::exists(std::filesystem::symlink_status(p))
* ``` * ```
@@ -142,9 +143,7 @@ namespace fs {
* std::filesystem::exists(p) = std::filesystem::exists(std::filesystem::status(p)) * std::filesystem::exists(p) = std::filesystem::exists(std::filesystem::status(p))
* ``` * ```
*/ */
inline bool symlink_exists(const std::filesystem::path & path) { bool symlink_exists(const std::filesystem::path & path);
return std::filesystem::exists(std::filesystem::symlink_status(path));
}
} // namespace fs } // namespace fs
@@ -194,21 +193,27 @@ std::string readFile(const Path & path);
std::string readFile(const std::filesystem::path & path); std::string readFile(const std::filesystem::path & path);
void readFile(const Path & path, Sink & sink); void readFile(const Path & path, Sink & sink);
enum struct FsSync { Yes, No };
/** /**
* Write a string to a file. * Write a string to a file.
*/ */
void writeFile(const Path & path, std::string_view s, mode_t mode = 0666, bool sync = false); void writeFile(const Path & path, std::string_view s, mode_t mode = 0666, FsSync sync = FsSync::No);
static inline void writeFile(const std::filesystem::path & path, std::string_view s, mode_t mode = 0666, bool sync = false)
static inline void writeFile(const std::filesystem::path & path, std::string_view s, mode_t mode = 0666, FsSync sync = FsSync::No)
{ {
return writeFile(path.string(), s, mode, sync); return writeFile(path.string(), s, mode, sync);
} }
void writeFile(const Path & path, Source & source, mode_t mode = 0666, bool sync = false); void writeFile(const Path & path, Source & source, mode_t mode = 0666, FsSync sync = FsSync::No);
static inline void writeFile(const std::filesystem::path & path, Source & source, mode_t mode = 0666, bool sync = false)
static inline void writeFile(const std::filesystem::path & path, Source & source, mode_t mode = 0666, FsSync sync = FsSync::No)
{ {
return writeFile(path.string(), source, mode, sync); return writeFile(path.string(), source, mode, sync);
} }
void writeFile(AutoCloseFD & fd, const Path & origPath, std::string_view s, mode_t mode = 0666, FsSync sync = FsSync::No);
/** /**
* Flush a path's parent directory to disk. * Flush a path's parent directory to disk.
*/ */
@@ -231,14 +236,9 @@ void deletePath(const std::filesystem::path & path, uint64_t & bytesFreed);
/** /**
* Create a directory and all its parents, if necessary. * Create a directory and all its parents, if necessary.
* *
* In the process of being deprecated for * Wrapper around `std::filesystem::create_directories` to handle exceptions.
* `std::filesystem::create_directories`.
*/ */
void createDirs(const Path & path); void createDirs(const std::filesystem::path & path);
inline void createDirs(PathView path)
{
return createDirs(Path(path));
}
/** /**
* Create a single directory. * Create a single directory.
+1 -1
View File
@@ -33,7 +33,7 @@ std::optional<Mode> decodeMode(RawMode m) {
static std::string getStringUntil(Source & source, char byte) static std::string getStringUntil(Source & source, char byte)
{ {
std::string s; std::string s;
char n[1]; char n[1] = { 0 };
source(std::string_view { n, 1 }); source(std::string_view { n, 1 });
while (*n != byte) { while (*n != byte) {
s += *n; s += *n;
+3
View File
@@ -148,6 +148,7 @@ sources = files(
'logging.cc', 'logging.cc',
'memory-source-accessor.cc', 'memory-source-accessor.cc',
'position.cc', 'position.cc',
'pos-table.cc',
'posix-source-accessor.cc', 'posix-source-accessor.cc',
'references.cc', 'references.cc',
'serialise.cc', 'serialise.cc',
@@ -217,6 +218,8 @@ headers = [config_h] + files(
'muxable-pipe.hh', 'muxable-pipe.hh',
'os-string.hh', 'os-string.hh',
'pool.hh', 'pool.hh',
'pos-idx.hh',
'pos-table.hh',
'position.hh', 'position.hh',
'posix-source-accessor.hh', 'posix-source-accessor.hh',
'processes.hh', 'processes.hh',
Loaded 100 of 410 files, more files were not shown because too many files have changed in this diff. Show more