Compare commits

..
Author SHA1 Message Date
mergify[bot] 920f14a360 Merge pull request #13208 from NixOS/mergify/bp/2.27-maintenance/pr-13207
dev-shell: Drop bear dependency (backport #13207)
2025-05-15 21:58:20 +00:00
Sergei Zimmerman 775c045a9e dev-shell: Drop bear dependency
Since the autotools-based build system has been removed
and meson already generates compile database there's no
need to have it in the devshell.

(cherry picked from commit 67535263a5)
2025-05-15 21:49:07 +00:00
Eelco Dolstra 9febd9f84b Merge pull request #13020 from NixOS/mergify/bp/2.27-maintenance/pr-13019
Mention BLAKE3 in the Nix 2.27 release notes (backport #13019)
2025-04-14 15:57:09 +02:00
Eelco Dolstra 064c141fc8 Mention BLAKE3 in the Nix 2.27 release notes
(cherry picked from commit c0ed07755a)
2025-04-14 13:47:54 +00:00
mergify[bot] af298ee53f Merge pull request #12981 from NixOS/mergify/bp/2.27-maintenance/pr-12979
Actually ignore system/user registries during locking (backport #12979)
2025-04-09 16:53:03 +00:00
Eelco Dolstra 4dfe5eb587 Actually ignore system/user registries during locking
Something went wrong in #12068 so this didn't work. Also added a test.

(cherry picked from commit 77d4316353)
2025-04-09 16:33:40 +00:00
mergify[bot] b23189061e Merge pull request #12971 from NixOS/mergify/bp/2.27-maintenance/pr-12967
Add trailing commas on addFlag incantations (backport #12967)
2025-04-07 23:37:16 +00:00
Graham Christensen 025c935ba2 format as required
(cherry picked from commit 9b47b2b217)
2025-04-07 23:16:22 +00:00
Graham Christensen 29d37eb2db Add trailing commas on addFlag incantations
(cherry picked from commit 06acbd37bd)
2025-04-07 23:16:22 +00:00
mergify[bot] d0f98c76f9 Merge pull request #12954 from NixOS/mergify/bp/2.27-maintenance/pr-12951
tests/functional/repl: fix race condition (backport #12951)
2025-04-07 12:39:59 +00:00
Jörg Thalheim eb97d69f0d tests/functional/repl: fix race condition
the sleep 1 is not enough in some circumstances. Switching to a fifo
helps.

(cherry picked from commit 1de951d31d)
2025-04-07 12:19:27 +00:00
mergify[bot] 92b4578a65 Merge pull request #12948 from NixOS/mergify/bp/2.27-maintenance/pr-12944
maintainers/release-notes: Let it fail (backport #12944)
2025-04-07 09:11:32 +00:00
Robert Hensing 6733f72ea4 maintainers/release-notes: Let it fail
Fail when a command fails.

Basic error handling was missing, which would lead to errors getting
obscured a bit by subsequent successful logging.

(cherry picked from commit da36c34db7)
2025-04-07 08:52:36 +00:00
Jörg Thalheim ad814e7b93 Merge pull request #12923 from NixOS/mergify/bp/2.27-maintenance/pr-12911
create cache entry for paths already in the nix store (backport #12911)
2025-04-04 16:23:54 +02:00
Jörg Thalheim 82a47978db create cache entry for paths already in the nix store
This allows path:/nix/store/* paths to not be copied twice to the nix
store.

(cherry picked from commit 61c6210dbf)
2025-04-04 16:04:14 +02:00
mergify[bot] f02ce07682 Merge pull request #12920 from NixOS/mergify/bp/2.27-maintenance/pr-12919
Reinstate Darwin minversion (backport #12919)
2025-04-04 06:46:44 +00:00
Robert Hensing f0bf94fe29 packaging/dependency: Clarify darwinMinVersion
(cherry picked from commit 4be92e7b82)
2025-04-04 06:39:08 +00:00
Robert Hensing 22168ff57d Revert "remove obsolete stdenv darwinMinVersion override"
This reverts commit d91310bb32.

> Some packages require setting a non-default deployment target
> (or minimum version) to gain access to certain APIs. You do
> that using the darwinMinVersionHook, which takes the deployment
> target version as a parameter.

-- https://github.com/NixOS/nixpkgs/blob/60b54c7aee3c0cefde72d1a151bb7d3a46361ca2/doc/stdenv/platform-notes.chapter.md#what-is-a-deployment-target-or-minimum-version-sec-darwin-troubleshooting-using-deployment-targets

This will again solve error:

    ../nix_api_expr.cc:38:18: error: aligned allocation function of type 'void *(std::size_t, std::align_val_t)' is only available on macOS 10.13 or newer

-- https://hydra.nixos.org/build/294088946

(cherry picked from commit 5c4a4aeed7)
2025-04-04 06:39:08 +00:00
mergify[bot] bfa581fcff Merge pull request #12913 from NixOS/mergify/bp/2.27-maintenance/pr-12912
Set path display for substituted inputs (backport #12912)
2025-04-03 13:07:34 +00:00
Eelco Dolstra 8d1c97a55a Set path display for substituted inputs
(cherry picked from commit 4a397cfb80)
2025-04-03 12:50:23 +00:00
mergify[bot] 71aad3ad44 Merge pull request #12865 from NixOS/mergify/bp/2.27-maintenance/pr-12810
Packaging improvements from Nixpkgs (backport #12810)
2025-04-03 08:19:38 +00:00
Robert Hensing aa5ffba310 packaging: finalAttrs.doCheck -> finalAttrs.finalPackage.doCheck
This includes the logic that disables checks on cross appropriately.

Co-authored-by: Peder Bergebakken Sundt <pbsds@hotmail.com>
(cherry picked from commit 27d71b21fc)
2025-04-03 09:52:57 +02:00
Robert Hensing bb104e9b28 packaging: Various improvements
Co-authored-by: Mic92 <Mic92@users.noreply.github.com>
(cherry picked from commit 1172e49a3a)
2025-04-03 09:52:57 +02:00
Eelco Dolstra c73c503d5f Merge pull request #12881 from NixOS/mergify/bp/2.27-maintenance/pr-12869
Apply makeNotAllowedError to empty Git repos (backport #12869)
2025-04-03 09:52:46 +02:00
Eelco Dolstra 95cd6c1e48 Merge remote-tracking branch 'origin/2.27-maintenance' into mergify/bp/2.27-maintenance/pr-12869 2025-04-03 09:32:40 +02:00
Eelco Dolstra 635a9975fd Merge pull request #12900 from NixOS/revert-12892-mergify/bp/2.27-maintenance/pr-12876
Revert "decrease connect-timeout to 5s (backport #12876)"
2025-04-03 09:26:45 +02:00
mergify[bot] 45aba8d640 Merge pull request #12905 from NixOS/mergify/bp/2.27-maintenance/pr-12886
symlink_exists: wrap exceptions into nix exception (backport #12886)
2025-04-02 23:37:30 +00:00
Jörg Thalheim fa140349b8 symlink_exists: wrap exceptions into nix exception
(cherry picked from commit 779687854f)
2025-04-02 23:14:01 +00:00
mergify[bot] 2c02b58617 Merge pull request #12894 from NixOS/mergify/bp/2.27-maintenance/pr-12863
libgit2: use upstream version if possible (backport #12863)
2025-04-02 22:28:06 +00:00
Robert Hensing cb3c8bba70 Revert "decrease connect-timeout to 5s (backport #12876)" 2025-04-02 23:53:21 +02:00
mergify[bot] 5ecb8a12ee Merge pull request #12892 from NixOS/mergify/bp/2.27-maintenance/pr-12876
decrease connect-timeout to 5s (backport #12876)
2025-04-02 21:39:44 +00:00
Jörg Thalheim a6a621c8ff remove obsolete stdenv darwinMinVersion override
we are more up-to-date now:

nix-repl> stdenv.hostPlatform.darwinMinVersion
"11.3"

(cherry picked from commit d91310bb32)
2025-04-02 20:45:31 +00:00
Jörg Thalheim f1cb0e6ddb libgit2: use upstream version if possible
we don't seem to use libgit2 for fetching via ssh, hence it shouldn't
matter if it's using libssh or the ssh binary.

(cherry picked from commit 0b61b758fb)
2025-04-02 20:45:30 +00:00
Jörg Thalheim 7ef0eddfdc decrease connect-timeout to 5s
For people self-hosting caches that can be occasionally down, the
default timeout is very long. This is annoying if you are trying to
update your binary cache at the same time you are trying to update
another machine. Same if cachix has one of its rare hiccups.

We tested this value of 5s in srvos now for years and we like to travel
around the world with shitty internet, so it should be still reasonable
high.

(cherry picked from commit bef91a618a)
2025-04-02 20:43:28 +00:00
Eelco Dolstra c0b4fb0de8 Merge pull request #12883 from NixOS/mergify/bp/2.27-maintenance/pr-12870
Git fetcher: Improve error message for untracked files (backport #12870)
2025-04-02 21:30:50 +02:00
Eelco Dolstra 2b4ddbbf47 Make Git error messages more consistent
(cherry picked from commit f15681df26)
2025-04-02 19:15:12 +00:00
Eelco Dolstra d653225a24 Tweak error message
(cherry picked from commit 277c29a64b)
2025-04-02 19:15:12 +00:00
Graham Christensen 398c4136c2 Improve and fix the error message when a file is not tracked by Git
(cherry picked from commit 62e2304891)
2025-04-02 19:15:11 +00:00
Eelco Dolstra 084e75ed2f Apply makeNotAllowedError to empty repos
(cherry picked from commit 67e957b636)
2025-04-02 18:33:01 +00:00
mergify[bot] ae2bd40d06 Merge pull request #12861 from NixOS/mergify/bp/2.27-maintenance/pr-12853
flake: nixpkgs: 24.11 -> nixos-unstable (backport #12853)
2025-04-01 17:18:43 +00:00
Robert Hensing 9b87c4f806 tests/nixos: Work around network-online.target inactivity
(cherry picked from commit 58b657b976)
2025-04-01 16:50:27 +00:00
Robert Hensing 5671175a5e Format
clang-format: 18.1.8 -> 19.1.7
(cherry picked from commit 55297f865c)

# Conflicts:
#	src/libstore-test-support/outputs-spec.cc
2025-04-01 16:50:27 +00:00
Robert Hensing f33a435329 flake: nixpkgs: 24.11 -> nixos-unstable
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/48d12d5e70ee91fe8481378e540433a7303dbf6a?narHash=sha256-1Noao/H%2BN8nFB4Beoy8fgwrcOQLVm9o4zKW1ODaqK9E%3D' (2024-12-16)
  → 'github:NixOS/nixpkgs/52faf482a3889b7619003c0daec593a1912fddc1?narHash=sha256-6hl6L/tRnwubHcA4pfUUtk542wn2Om%2BD4UnDhlDW9BE%3D' (2025-03-30)

(cherry picked from commit c212035d94)
2025-04-01 16:50:26 +00:00
mergify[bot] 07309b497a Merge pull request #12856 from NixOS/mergify/bp/2.27-maintenance/pr-12765
Add various clang tidy fixes (backport #12765)
2025-04-01 14:27:32 +00:00
Jörg Thalheim a42345a1ff libstore/daemon: make sure monitor is not considered "unused"
(cherry picked from commit 5c3682d7a1)
2025-04-01 13:37:46 +00:00
Jörg Thalheim 9ee7318a23 test/ca-fd-leak: fix clang-tidy lints
(cherry picked from commit b050db951b)
2025-04-01 13:37:45 +00:00
Jörg Thalheim b1826f720c git/getStringUntil: fix uninitialized stack variable
at least clang-tidy is not convinced that this initialized.
If this is not the case, the impact should be small and hopefully also
more robust if changed.

(cherry picked from commit 7e540059a3)
2025-04-01 13:37:45 +00:00
Jörg Thalheim da1b5599b7 libstore/local-store: fix linting warning about unused variable
(cherry picked from commit 05082ea1c5)
2025-04-01 13:37:45 +00:00
mergify[bot] 8c5107a640 Merge pull request #12843 from NixOS/mergify/bp/2.27-maintenance/pr-12805
nix-daemon: source nix-profile-daemon.sh only once (backport #12805)
2025-04-01 07:49:39 +00:00
Thomas Miedema a5f003555e nix-daemon: source nix-profile-daemon.sh only once
On my system (Ubuntu 24.04 with nix installed using
https://zero-to-nix.com/), I noticed that my PATH
contained multiple times the following entries:

  /home/thomas/.nix-profile/bin
  /nix/var/nix/profiles/default/bin

Fix it by inserting a missing `export`, to make
sure `nix-daemon.sh` is really only executed once.

(cherry picked from commit 2b4e3fa144)
2025-04-01 06:48:29 +00:00
mergify[bot] 668e09f74a Merge pull request #12816 from NixOS/mergify/bp/2.27-maintenance/pr-12812
Disable packaging-overriding test (backport #12812)
2025-03-31 16:48:51 +00:00
Eelco Dolstra 67aa608856 Disable packaging-overriding
Fixes #12690.

(cherry picked from commit a4be66828a)
2025-03-31 16:31:11 +00:00
John Ericson 4688590733 Merge pull request #12797 from NixOS/mergify/bp/2.27-maintenance/pr-12794
Fix windows build (backport #12794)
2025-03-28 21:50:47 -04:00
John Ericson 44c2bd35e0 Fix windows build
PR #12767 accidentally broke it.

(cherry picked from commit 99041b4d84)
2025-03-29 01:05:25 +00:00
mergify[bot] eb7d96a8ac Merge pull request #12793 from NixOS/mergify/bp/2.27-maintenance/pr-12789
nix daemon: Don't open the store (backport #12789)
2025-03-29 00:24:30 +00:00
Eelco Dolstra 84a25dc846 nix daemon: Don't open the store
This makes it behave the same as nix-daemon. Opening the store in the
parent can cause a SIGBUS in libsqlite in the child:

  #0  0x00007f141cf6f789 in __memset_avx2_unaligned_erms () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
  #1  0x00007f141c322fe8 in walIndexAppend () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #2  0x00007f141c3711a2 in pagerWalFrames () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #3  0x00007f141c38317e in sqlite3PagerCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #4  0x00007f141c383555 in sqlite3BtreeCommitPhaseOne.part.0 () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #5  0x00007f141c384797 in sqlite3VdbeHalt () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #6  0x00007f141c3b8f60 in sqlite3VdbeExec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #7  0x00007f141c3bbfef in sqlite3_step () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #8  0x00007f141c3bd0e5 in sqlite3_exec () from /nix/store/bbd59cqw259149r2ddk4w1q0lr2fch8c-sqlite-3.46.1/lib/libsqlite3.so.0
  #9  0x00007f141da140e0 in nix::SQLiteTxn::commit() () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so
  #10 0x00007f141d9ce69c in nix::LocalStore::registerValidPaths(std::map<nix::StorePath, nix::ValidPathInfo, std::less<nix::StorePath>, std::allocator<std::pair<nix::StorePath const, nix::ValidPathInfo> > > const&)::{lambda()#1}::operator()() const () from /nix/store/1m4r8s7s1v54zq9isncvjgia02bffxlz-determinate-nix-store-3.1.0/lib/libnixstore.so

(cherry picked from commit 9590167290)
2025-03-29 00:02:50 +00:00
mergify[bot] d891f233cb Merge pull request #12785 from NixOS/mergify/bp/2.27-maintenance/pr-12758
libutil: Fix error message (backport #12758)
2025-03-28 19:23:41 +00:00
mergify[bot] 84c4049e3c Merge pull request #12778 from NixOS/mergify/bp/2.27-maintenance/pr-12767
use createDirs consistently everywhere (backport #12767)
2025-03-28 18:09:59 +00:00
Robert Hensing 9c20bb18de libutil: Fix error message
I encountered this with a misconfigured libutil. I doubt that a
non-lutimes config is viable, because tests were failing.

(cherry picked from commit 1cffcd91a9)
2025-03-28 17:08:23 +00:00
mergify[bot] 52049bc76e Merge pull request #12776 from NixOS/mergify/bp/2.27-maintenance/pr-12772
Light packaging cleanups (backport #12772)
2025-03-28 16:44:06 +00:00
Jörg Thalheim e308524097 use createDirs consistently everywhere
(cherry picked from commit a8217f2642)
2025-03-28 15:56:02 +00:00
John Ericson 5056aae63a Add a withAWS flag to libstore
Nixpkgs wants this, at least.

(cherry picked from commit e4c571c2f1)
2025-03-28 15:54:57 +00:00
John Ericson 602840bfd2 Link the right issue about the bad AWS pkg-config
It is https://github.com/aws/aws-sdk-cpp/issues/2673

(cherry picked from commit fb9c9ee35a)
2025-03-28 15:54:56 +00:00
John Ericson 71b0edcfe3 Remove boost env vars
https://github.com/NixOS/nixpkgs/issues/86131 is now fixed!
(cherry picked from commit 459fb59493)
2025-03-28 15:54:56 +00:00
John Ericson 1763cf115b Fix a bunch of missing meson boilerplate
These other libraries need this too

(cherry picked from commit ffdce51cd5)
2025-03-28 15:54:56 +00:00
mergify[bot] 6f3dd6527d Merge pull request #12755 from NixOS/mergify/bp/2.27-maintenance/pr-12105
local-derivation-goal: improve "illegal reference" error (backport #12105)
2025-03-25 16:22:19 +00:00
Sergei Trofimovich cadfed692c tests/functional/check-refs.nix: format newly added test
(cherry picked from commit 4d72e0f73b)
2025-03-25 15:49:31 +00:00
Sergei Trofimovich c2cffe6249 tests/functional/check-refs.sh: guard test12 against too old nix daemon
Otherwise without the change the test fails on nix-2.26 as:

    error: derivation contains an illegal reference specifier 'dev'

Note: the error message does not match intended change.
(cherry picked from commit 1e7c7244cf)
2025-03-25 15:49:31 +00:00
Robert Hensing 6f54b90f36 test illegal reference specifier error message
(cherry picked from commit f4def47c89)
2025-03-25 15:49:31 +00:00
Robert Hensing db1950e768 checkRefs: use concatMapStringsSep
(cherry picked from commit 2b4d461c14)
2025-03-25 15:49:30 +00:00
Robert Hensing aa7433982b nix-util: Use small_vector in concatMapStringsSep
(cherry picked from commit 32898dc46a)
2025-03-25 15:49:30 +00:00
Robert Hensing f9d1f36169 nix-util: Add concatMapStrings
(cherry picked from commit f3dbaa3f54)
2025-03-25 15:49:29 +00:00
Sergei Trofimovich 79122c66b1 local-derivation-goal: improve "illegal reference" error
Before the change "illegal reference" was hard to interpret as it did
not mention what derivation actually hits it.

Today's `nixpkgs` example:

Before the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation contains an illegal reference specifier 'man'

After the change:

    $ nix build --no-link -f. postgresql_14
    ...
    error: derivation '/nix/store/bxp6g57limvwiga61vdlyvhy7i8rp6wd-postgresql-14.15.drv' output check for 'lib' contains an illegal reference specifier 'man', expected store path or output name (one of [debug, dev, doc, lib, out])

(cherry picked from commit bbdc3197a9)
2025-03-25 15:49:29 +00:00
Eelco Dolstra 01ffee0033 Bump version 2025-03-24 16:29:59 +01:00
Jörg Thalheim 9cb662df74 Merge pull request #12743 from NixOS/mergify/bp/2.27-maintenance/pr-12734
packaging/everything.nix: make sure doc and manpage outputs are symlinks (backport #12734)
2025-03-24 14:47:47 +01:00
Kirens 53ac3c8ba9 make sure doc and manpage outputs are symlinks
Part of https://github.com/NixOS/nixpkgs/pull/392549
The doc and manpage fix already happend in 0ddfbc5939

(cherry picked from commit 15dfeb9182)
2025-03-24 13:10:27 +00:00
Jörg Thalheim 9178731176 Merge pull request #12740 from NixOS/mergify/bp/2.27-maintenance/pr-12736
`MonitorFdHup::~MonitorFdHup`: use proper close method instead of lib… (backport #12736)
2025-03-24 13:50:33 +01:00
Jörg Thalheim 2e9a36a8fe MonitorFdHup::~MonitorFdHup: use proper close method instead of libc close()
Otherwise closing it again will cause an EBADF in the AutoCloseFd class.

(cherry picked from commit 87a34a45ff)
2025-03-24 12:17:04 +00:00
John Ericson 1a87f122f4 Merge pull request #12733 from NixOS/mergify/bp/2.27-maintenance/pr-12714
`MonitorFdHup`: replace `pthread_cancel` trick with a notification pipe (backport #12714)
2025-03-23 20:24:23 -04:00
John Ericson 490e7c0984 MonitorFdHup: Don't sleep anymore
After the previous commit it should not be necessary. Furthermore, if we
*do* sleep, we'll exacerbate a race condition (in conjunction with
getting rid of the thread cancellation) that will cause test failures.

(cherry picked from commit 49f486d8e0)
2025-03-23 23:54:11 +00:00
Jade Lovelace 27f29ff6ed daemon: remove workaround for macOS kernel bug that seems fixed
This was filed as https://github.com/nixos/nix/issues/7584, but as far
as I can tell, the previous solution of POLLHUP works just fine on macOS
14. I've also tested on an ancient machine with macOS 10.15.7, which
also has POLLHUP work correctly.

It's possible this might regress some older versions of macOS that have
a kernel bug, but I went looking through the history on the sources and
didn't find anything that looked terribly convincingly like a bug fix
between 2020 and today. If such a broken version exists, it seems pretty
reasonable to suggest simply updating the OS.

Change-Id: I178a038baa000f927ea2cbc4587d69d8ab786843

Based off of commit 69e2ee5b25752ba5fd8644cef56fb9d627ca4a64. Ericson2314 added
additional other information.

(cherry picked from commit 9b3352c3c8)
2025-03-23 23:54:10 +00:00
Félix Baylac Jacqué ea19cb2f50 MonitorFdHup: replace pthread_cancel trick with a notification pipe
On https://github.com/NixOS/nix/issues/8946, we faced a surprising
behaviour wrt. exception when using pthread_cancel. In a nutshell when
a thread is inside a catch block and it's getting pthread_cancel by
another one, then the original exception is bubbled up and crashes the
process.

We now poll on the notification pipe from the thread and exit when the
main thread closes its end. This solution does not exhibit surprising
behaviour wrt. exceptions.

Co-authored-by: Mic92 <joerg@thalheim.io>

Fixes https://github.com/NixOS/nix/issues/8946

See also Lix https://gerrit.lix.systems/c/lix/+/1605 which is very
similar by coincidence. Pulled a comment from that.

(cherry picked from commit 1c636284a3)
2025-03-23 23:54:10 +00:00
John Ericson df18c9b2ed MonitorFdHup: introduce a num_fds variable
Better than just putting `1` in multiple spots.

(cherry picked from commit cb95791198)
2025-03-23 23:54:10 +00:00
John Ericson 1a461baee1 MonitorFdHup: Cleanup a bit with designated initializers
(cherry picked from commit d028bb4c4a)
2025-03-23 23:54:10 +00:00
Jörg Thalheim 709e228589 MonitorFdHup: raise explicit SysError rather unreachable
Syscalls can fail for many reasons and we don't want to loose the errno
and error context.

(cherry picked from commit 8e0bc2c3a8)
2025-03-23 23:54:09 +00:00
John Ericson 7d7508fb7a monitor-fd.hh: Format
It's a pretty small diff, so let's just start formatting before we make
other changes.

(cherry picked from commit 041394b741)
2025-03-23 23:54:09 +00:00
mergify[bot] 991b297011 Merge pull request #12726 from NixOS/mergify/bp/2.27-maintenance/pr-12723
packaging/everything.nix: Fix doc and man outputs (backport #12723)
2025-03-23 11:59:19 +00:00
Sergei Zimmerman c73f672afd packaging/everything.nix: Fix doc and man outputs
We want the $doc, $man outputs to be symlinks pointing to nix-manual and
nix-manual.man. Creating the directories first makes the `ln` command
produce symlink $doc/${nix-manual} instead.

```
$file /nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man

/nix/store/q4dwlnd36gpfajgfcp6hca2xwy068wjq-nix-2.27.1-man/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man:
  symbolic link to /nix/store/rwh8ky3k040wyrywl8k2v5b3csdfbdg7-nix-manual-2.27.1-man
```

This is the reason `nix-env --help` is once again broken on 2.26/2.27/master
after 4108529.

(cherry picked from commit 0ddfbc5939)
2025-03-23 11:54:17 +00:00
mergify[bot] e39c497cee Merge pull request #12720 from NixOS/mergify/bp/2.27-maintenance/pr-12719
rl-2.27.md: Fix GitHub links (backport #12719)
2025-03-21 19:59:15 +00:00
Eelco Dolstra d000f5943a rl-2.27.md: Fix GitHub links
https://discourse.nixos.org/t/nix-2-27-0-released/62003/2?u=edolstra
(cherry picked from commit be5a455a1a)
2025-03-21 19:48:51 +00:00
Eelco Dolstra 624b54a392 Bump version 2025-03-21 19:31:47 +01:00
mergify[bot] d72fc01ffd Merge pull request #12713 from NixOS/mergify/bp/2.27-maintenance/pr-12636
port crash-handler from lix to nix (backport #12636)
2025-03-21 15:45:09 +00:00
Jade Lovelace b1f0f1c5a1 port crash-handler from lix to nix
It was first introduced in https://git.lix.systems/lix-project/lix/commit/19e0ce2c03d8e0baa16998b086665664c420c1df

In Nix we only register the crash handler in main instead of initNix,
because library user may want to use their own crash handler.

Sample output:

Mar 12 08:38:06 eve nix[2303762]: Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:
Mar 12 08:38:06 eve nix[2303762]: Exception: nix::SysError: error: writing to file: Resource temporarily unavailable
Mar 12 08:38:06 eve nix[2303762]: Stack trace:
Mar 12 08:38:06 eve nix[2303762]:  0# 0x000000000076876A in nix
                                   1# 0x00007FDA40E9F20A in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   2# std::unexpected() in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   3# 0x00007FDA40E9F487 in /nix/store/2lhklm5aizx30qbw49acnrrzkj9lbmij-gcc-14-20241116-lib/lib/libstdc++.so.6
                                   4# nix::writeFull(int, std::basic_string_view<char, std::char_traits<char> >, bool) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   5# nix::writeLine(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   6# nix::JSONLogger::write(nlohmann::json_abi_v3_11_3::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::json_abi_v3_11_3::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> >, void> const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   7# nix::JSONLogger::logEI(nix::ErrorInfo const&) in /home/joerg/git/nix/inst/lib/libnixutil.so
                                   8# nix::Logger::logEI(nix::Verbosity, nix::ErrorInfo) in nix
                                   9# nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) in /home/joerg/git/nix/inst/lib/libnixmain.so
                                  10# 0x000000000087A563 in nix
                                  11# 0x00007FDA40BD41FE in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  12# __libc_start_main in /nix/store/6q2mknq81cyscjmkv72fpcsvan56qhmg-glibc-2.40-66/lib/libc.so.6
                                  13# 0x00000000006F4DF5 in nix

Co-authored-by: eldritch horrors <pennae@lix.systems>
(cherry picked from commit 163f94412a)
2025-03-21 15:31:03 +00:00
mergify[bot] 616109a040 Merge pull request #12711 from NixOS/mergify/bp/2.27-maintenance/pr-12701
stack overflow is EvalBaseError (backport #12701)
2025-03-21 14:37:25 +00:00
Shahar "Dawn" Or 3f56dd9927 stack overflow is EvalBaseError
(cherry picked from commit 23c7a45a05)
2025-03-21 14:13:40 +00:00
mergify[bot] 54bb3f54e8 Merge pull request #12696 from NixOS/mergify/bp/2.27-maintenance/pr-12682
libfetchers: fix double quote in path printed in logger (backport #12682)
2025-03-19 22:48:41 +00:00
Jörg Thalheim 0ec28acef9 libfetchers: fix double quote in path printed in logger
(cherry picked from commit 314e9fbeda)
2025-03-19 22:25:06 +00:00
mergify[bot] b5dc44c586 Merge pull request #12691 from NixOS/mergify/bp/2.27-maintenance/pr-12687
libutil/file-descriptor: handle EAGAIN in read/write operations (backport #12687)
2025-03-19 22:08:46 +00:00
Jörg Thalheim 32ab3ef598 libutil/file-descriptor: handle EAGAIN in read/write operations
We now see exception beeing thrown when remote building in master
because of writing to a non-blocking file descriptor from our json logger.

> #0  0x00007f2ea97aea9c in __pthread_kill_implementation () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #1  0x00007f2ea975c576 in raise () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #2  0x00007f2ea9744935 in abort () from /nix/store/wn7v2vhyyyi6clcyn0s9ixvl7d4d87ic-glibc-2.40-36/lib/libc.so.6
> #3  0x00007f2ea99e8c2b in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #4  0x00007f2ea99f820a in __cxxabiv1::__terminate(void (*)()) () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #5  0x00007f2ea99f8275 in std::terminate() () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #6  0x00007f2ea99f84c7 in __cxa_throw () from /nix/store/ybjcla5bhj8g1y84998pn4a2drfxybkv-gcc-13.3.0-lib/lib/libstdc++.so.6
> #7  0x00007f2eaa5035c2 in nix::writeFull (fd=2, s=..., allowInterrupts=true) at ../unix/file-descriptor.cc:43
> #8  0x00007f2eaa5633c4 in nix::JSONLogger::write (this=this@entry=0x249a7d40, json=...) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/char_traits.h:358
> #9  0x00007f2eaa5658d7 in nix::JSONLogger::logEI (this=<optimized out>, ei=...) at ../logging.cc:242
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> #11 nix::handleExceptions (programName="nix", fun=...) at ../shared.cc:343
> #12 0x0000000000465b1f in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/4krab2h0hd4wvxxmscxrw21pl77j4i7j-gcc-13.3.0/include/c++/13.3.0/bits/allocator.h:163
> (gdb) frame 10
> #10 0x00007f2ea9c5d048 in nix::Logger::logEI (ei=..., lvl=nix::lvlError, this=0x249a7d40) at /nix/store/a7cq5bqh0ryvnkv4m19ffchnvi8l9qx6-nix-util-2.27.0-dev/include/nix/logging.hh:108
> 108             logEI(ei);

So far only drainFD sets the non-blocking flag on a "readable" file descriptor,
while this is a "writeable" file descriptor.
It's not clear to me yet, why we see logs after that point, but it's
also not that bad to handle EAGAIN in read/write functions after all.

(cherry picked from commit 2790f5f9ae)
2025-03-19 21:46:21 +00:00
mergify[bot] 8705feb7cc Merge pull request #12686 from NixOS/mergify/bp/2.27-maintenance/pr-12570
Fix macos sandbox issue (backport #12570)
2025-03-19 21:23:43 +00:00
Pierre-Etienne Meunier aecd2b5d92 Fix macos sandbox issue
Co-authored-by: John Ericson <git@JohnEricson.me>

Co-authored-by: Poliorcetics <poliorcetics@users.noreply.github.com>
(cherry picked from commit 300465c7b8)
2025-03-19 20:21:53 +00:00
mergify[bot] bdaa8c55d2 Merge pull request #12655 from NixOS/mergify/bp/2.27-maintenance/pr-12645
Make debugger significantly faster (backport #12645)
2025-03-14 13:54:47 +00:00
Sergei Zimmerman 11919bc471 {libexpr,libcmd}: Make debugger significantly faster
The underlying issue is that debugger code path was
calling PosTable::operator[] in each eval method.
This has become incredibly expensive since 5d9fdab3de.

While we are it it, I've reworked the code to
not use std::shared_ptr where it really isn't necessary.

As I've documented in previous commits, this is actually
more a workaround for recursive header dependencies now
and is only necessary in `error.hh` code.

Some ad-hoc benchmarking:

After this commit:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     784.2 ms ±   7.1 ms    [User: 561.4 ms, System: 147.7 ms]
  Range (min … max):   773.5 ms … 792.6 ms    10 runs
```

On master 3604c7c51:

```
Benchmark 1: nix eval nixpkgs#hello --impure --ignore-try --no-eval-cache --debugger
  Time (mean ± σ):     22.914 s ±  0.178 s    [User: 18.524 s, System: 4.151 s]
  Range (min … max):   22.738 s … 23.290 s    10 runs
```

(cherry picked from commit adbd08399c)
2025-03-14 13:22:53 +00:00
Sergei Zimmerman 593e0eebeb libutil: Fix Pos::getSourcePath
Previous implementation didn't actually check if
std::get_if returned a nullptr:

std::optional<SourcePath> getSourcePath() const {
    return *std::get_if<SourcePath>(&origin);
}

(cherry picked from commit 50123f2a56)
2025-03-14 13:22:52 +00:00
Sergei Zimmerman 8c2a792d2b libutil: Document hacks and problems around Pos class
This should provide context for follow-up commits in
the patch series.

(cherry picked from commit bf12aedf2e)
2025-03-14 13:22:52 +00:00
Sergei Zimmerman b540c2419f {libutil,libexpr}: Move pos-idx,pos-table code to libutil
All of this code doesn't actually depend on anything from
libexpr. Because Pos is so tigtly coupled with Error, it
makes sense to have in the same library.

(cherry picked from commit a53b184e63)
2025-03-14 13:22:52 +00:00
mergify[bot] 96f0fd3ce0 Merge pull request #12605 from NixOS/mergify/bp/2.27-maintenance/pr-12604
Fix chopped up repl output (backport #12604)
2025-03-04 19:33:00 +00:00
Robert Hensing dc6a936d97 repl: Fix :print corruption
See preceding commits.

(cherry picked from commit 1e1c587073)
2025-03-04 19:10:02 +00:00
Robert Hensing ac3fc8876c refactor: RAII logger suspension
(cherry picked from commit 30694b5d8a)
2025-03-04 19:10:01 +00:00
Robert Hensing 375df6c086 repl: Fix value printing corruption
The resume call would get some non-flushed(?) data.
Extending the pause to include the newline makes the complete flush
part of the pause.

(cherry picked from commit 880489051a)
2025-03-04 19:10:01 +00:00
Robert Hensing 88e6b7d6b2 progress-bar: Make pause/resume nestable
(cherry picked from commit 49d8ee5359)
2025-03-04 19:10:01 +00:00
mergify[bot] 181ffe30be Merge pull request #12602 from NixOS/mergify/bp/2.27-maintenance/pr-12600
packaging: Typo in setVersionLayer / preConfigure (backport #12600)
2025-03-04 17:09:29 +00:00
Robert Hensing afad1e536e packaging: Typo in setVersionLayer / preConfigure
Apparently dead code in our use case, but good to keep nonetheless.
Credit: ztzg in https://github.com/NixOS/nix/pull/12498#pullrequestreview-2658031853

(cherry picked from commit dcaea8cb1c)
2025-03-04 16:46:18 +00:00
John Ericson fffb54d8f7 Merge pull request #12594 from NixOS/mergify/bp/2.27-maintenance/pr-12442
Expand manual on derivation outputs (backport #12442)
2025-03-03 17:28:32 -05:00
Robert Hensing 8b9d401fe4 manual: Edit
(cherry picked from commit 1e00d14c29)
2025-03-03 22:13:09 +00:00
John Ericson f278a631b0 Expand manual on derivation outputs
Note, this includes some text adapted from from Eelco's dissertation

(cherry picked from commit 2aa6e0f084)
2025-03-03 22:13:08 +00:00
mergify[bot] 7e61e9650c Merge pull request #12590 from NixOS/mergify/bp/2.27-maintenance/pr-12582
packaging/everything.nix: Use a multi-output derivation (backport #12582)
2025-03-02 13:16:07 +00:00
Robert Hensing 7e7e9d9eab packaging/everything.nix: Use a multi-output derivation
This should fix a few packaging regressions.

`dev` also includes a merged `includes/`, which may be helpful until
inter-component includes are fixed properly.

(cherry picked from commit 41085295ab)
2025-03-02 13:09:49 +00:00
Robert Hensing 863c1b60d2 Merge pull request #12576 from obsidiansystems/2.27-no-dyn-drv
[2.27] Once again revert "Adapt scheduler to work with dynamic derivations"
2025-02-27 20:56:16 +01:00
John Ericson f636ced7d2 Revert "Revert "Revert "Adapt scheduler to work with dynamic derivations"""
The bug reappeared after all, and the fix introduced a different bug. We
want to release 2.27 imminently so there is no time to do a proper fix,
which appears to require a larger reworking. Hopefully we will have it
for 2.28, however.

This reverts commit c98525235f.
2025-02-27 13:42:57 -05:00
Eelco Dolstra 494953cfb6 Mark official release 2025-02-27 17:03:27 +01:00
118 changed files with 1006 additions and 640 deletions

No files matched your search

+10 -1
View File
@@ -11,7 +11,16 @@
.github/CODEOWNERS @edolstra
# Documentation of built-in functions
src/libexpr/primops.cc @roberth
src/libexpr/primops.cc @roberth @fricklerhandwerk
# Documentation of settings
src/libexpr/eval-settings.hh @fricklerhandwerk
src/libstore/globals.hh @fricklerhandwerk
# Documentation
doc/manual @fricklerhandwerk
maintainers/*.md @fricklerhandwerk
src/**/*.md @fricklerhandwerk
# Libstore layer
/src/libstore @ericson2314
-11
View File
@@ -117,14 +117,3 @@ pull_request_rules:
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.27
conditions:
- label=backport 2.27-maintenance
actions:
backport:
branches:
- "2.27-maintenance"
labels:
- automatic backport
- merge-queue
+1 -1
View File
@@ -1 +1 @@
2.27.0
2.27.2
+1 -40
View File
@@ -443,7 +443,7 @@ three kinds of patterns:
This works on any set that contains at least the three named
attributes.
- It is possible to provide *default values* for attributes, in
It is possible to provide *default values* for attributes, in
which case they are allowed to be missing. A default value is
specified by writing `name ? e`, where *e* is an arbitrary
expression. For example,
@@ -503,45 +503,6 @@ three kinds of patterns:
> [ 23 {} ]
> ```
- All bindings introduced by the function are in scope in the entire function expression; not just in the body.
It can therefore be used in default values.
> **Example**
>
> A parameter (`x`), is used in the default value for another parameter (`y`):
>
> ```nix
> let
> f = { x, y ? [x] }: { inherit y; };
> in
> f { x = 3; }
> ```
>
> This evaluates to:
>
> ```nix
> {
> y = [ 3 ];
> }
> ```
> **Example**
>
> The binding of an `@` pattern, `args`, is used in the default value for a parameter, `x`:
>
> ```nix
> let
> f = args@{ x ? args.a, ... }: x;
> in
> f { a = 1; }
> ```
>
> This evaluates to:
>
> ```nix
> 1
> ```
Note that functions do not have names. If you want to give them a name,
you can bind them to an attribute, e.g.,
+11 -2
View File
@@ -30,14 +30,23 @@
The evaluator now presents a "union" filesystem view of the `/nix/store` in the host and the chroot.
This change also removes some hacks that broke `builtins.{path,filterSource}` in chroot stores [#11503](https://github.com/NixOS/nix/issue/11503).
This change also removes some hacks that broke `builtins.{path,filterSource}` in chroot stores [#11503](https://github.com/NixOS/nix/issues/11503).
- `nix flake prefetch` now has a `--out-link` option [#12443](https://github.com/NixOS/nix/issue/12443)
- `nix flake prefetch` now has a `--out-link` option [#12443](https://github.com/NixOS/nix/pull/12443)
- Set `FD_CLOEXEC` on sockets created by curl [#12439](https://github.com/NixOS/nix/pull/12439)
Curl created sockets without setting `FD_CLOEXEC`/`SOCK_CLOEXEC`. This could previously cause connections to remain open forever when using commands like `nix shell`. This change sets the `FD_CLOEXEC` flag using a `CURLOPT_SOCKOPTFUNCTION` callback.
- Add BLAKE3 hash algorithm [#12379](https://github.com/NixOS/nix/pull/12379)
Nix now supports the BLAKE3 hash algorithm as an experimental feature (`blake3-hashes`):
```console
# nix hash file ./file --type blake3 --extra-experimental-features blake3-hashes
blake3-34P4p+iZXcbbyB1i4uoF7eWCGcZHjmaRn6Y7QdynLwU=
```
# Contributors
This release was made possible by the following 21 contributors:
Generated
+4 -4
View File
@@ -63,16 +63,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1734359947,
"narHash": "sha256-1Noao/H+N8nFB4Beoy8fgwrcOQLVm9o4zKW1ODaqK9E=",
"lastModified": 1743315132,
"narHash": "sha256-6hl6L/tRnwubHcA4pfUUtk542wn2Om+D4UnDhlDW9BE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "48d12d5e70ee91fe8481378e540433a7303dbf6a",
"rev": "52faf482a3889b7619003c0daec593a1912fddc1",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "release-24.11",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
+31 -20
View File
@@ -1,7 +1,7 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-24.11";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
@@ -32,7 +32,7 @@
let
inherit (nixpkgs) lib;
officialRelease = false;
officialRelease = true;
linux32BitSystems = [ "i686-linux" ];
linux64BitSystems = [
@@ -156,6 +156,13 @@
inherit officialRelease;
pkgs = final;
src = self;
maintainers = with lib.maintainers; [
edolstra
Ericson2314
Mic92
roberth
tomberek
];
};
};
@@ -230,24 +237,28 @@
This shouldn't build anything significant; just check that things
(including derivations) are _set up_ correctly.
*/
packaging-overriding =
let
pkgs = nixpkgsFor.${system}.native;
nix = self.packages.${system}.nix;
in
assert (nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src.patches == [ pkgs.emptyFile ];
if pkgs.stdenv.buildPlatform.isDarwin then
lib.warn "packaging-overriding check currently disabled because of a permissions issue on macOS" pkgs.emptyFile
else
# If this fails, something might be wrong with how we've wired the scope,
# or something could be broken in Nixpkgs.
pkgs.testers.testEqualContents {
assertion = "trivial patch does not change source contents";
expected = "${./.}";
actual =
# Same for all components; nix-util is an arbitrary pick
(nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src;
};
# Disabled due to a bug in `testEqualContents` (see
# https://github.com/NixOS/nix/issues/12690).
/*
packaging-overriding =
let
pkgs = nixpkgsFor.${system}.native;
nix = self.packages.${system}.nix;
in
assert (nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src.patches == [ pkgs.emptyFile ];
if pkgs.stdenv.buildPlatform.isDarwin then
lib.warn "packaging-overriding check currently disabled because of a permissions issue on macOS" pkgs.emptyFile
else
# If this fails, something might be wrong with how we've wired the scope,
# or something could be broken in Nixpkgs.
pkgs.testers.testEqualContents {
assertion = "trivial patch does not change source contents";
expected = "${./.}";
actual =
# Same for all components; nix-util is an arbitrary pick
(nix.appendPatches [ pkgs.emptyFile ]).libs.nix-util.src;
};
*/
}
// (lib.optionalAttrs (builtins.elem system linux64BitSystems)) {
dockerImage = self.hydraJobs.dockerImage.${system};
+2 -2
View File
@@ -37,7 +37,7 @@ The team is on Github as [@NixOS/nix-team](https://github.com/orgs/NixOS/teams/n
The team meets twice a week (times are denoted in the [Europe/Amsterdam](https://en.m.wikipedia.org/wiki/Time_in_the_Netherlands) time zone):
- Discussion meeting: Wednesday 21:00-22:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
- Discussion meeting: [Wednesday 21:00-22:00 Europe/Amsterdam](https://www.google.com/calendar/event?eid=ZG5rZzNyajRjajducGV2NGY5aGkzYWIwdnJfMjAyNDA1MDhUMTkwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
1. Triage issues and pull requests from the [No Status](#no-status) column (30 min)
2. Discuss issues and pull requests from the [To discuss](#to-discuss) column (30 min).
@@ -46,7 +46,7 @@ The team meets twice a week (times are denoted in the [Europe/Amsterdam](https:/
- mark it as draft if it is blocked on the contributor
- escalate it back to the team by moving it to To discuss, and leaving a comment as to why the issue needs to be discussed again.
- Work meeting: Mondays 14:00-16:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
- Work meeting: [Mondays 14:00-16:00 Europe/Amsterdam](https://www.google.com/calendar/event?eid=Ym52NDdzYnRic2NzcDcybjZiNDhpNzhpa3NfMjAyNDA1MTNUMTIwMDAwWiBiOW81MmZvYnFqYWs4b3E4bGZraGczdDBxZ0Bn)
1. Code review on pull requests from [In review](#in-review).
2. Other chores and tasks.
-2
View File
@@ -127,7 +127,6 @@
''^src/libexpr/nixexpr\.cc$''
''^src/libexpr/nixexpr\.hh$''
''^src/libexpr/parser-state\.hh$''
''^src/libexpr/pos-table\.hh$''
''^src/libexpr/primops\.cc$''
''^src/libexpr/primops\.hh$''
''^src/libexpr/primops/context\.cc$''
@@ -397,7 +396,6 @@
''^src/libutil/types\.hh$''
''^src/libutil/unix/file-descriptor\.cc$''
''^src/libutil/unix/file-path\.cc$''
''^src/libutil/unix/monitor-fd\.hh$''
''^src/libutil/unix/processes\.cc$''
''^src/libutil/unix/signals-impl\.hh$''
''^src/libutil/unix/signals\.cc$''
+2
View File
@@ -2,6 +2,8 @@
# vim: set filetype=bash:
#!nix shell .#changelog-d --command bash
set -euo pipefail
# --- CONFIGURATION ---
# This does double duty for
+52 -6
View File
@@ -3,6 +3,7 @@
pkgs,
src,
officialRelease,
maintainers,
}:
scope:
@@ -110,7 +111,7 @@ let
let
n = lib.length finalScope.patches;
in
if n == 0 then finalAttrs.version else finalAttrs.version + "+${toString n}";
if n == 0 then prevAttrs.version else prevAttrs.version + "+${toString n}";
# Clear what `derivation` can't/shouldn't serialize; see prevAttrs.workDir.
fileset = null;
@@ -180,9 +181,24 @@ let
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
};
miscGoodPractice = finalAttrs: prevAttrs: {
nixDefaultsLayer = finalAttrs: prevAttrs: {
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
pos = builtins.unsafeGetAttrPos "pname" prevAttrs;
meta = prevAttrs.meta or { } // {
homepage = prevAttrs.meta.homepage or "https://nixos.org/nix";
longDescription =
prevAttrs.longDescription or ''
Nix is a powerful package manager for mainly Linux and other Unix systems that
makes package management reliable and reproducible. It provides atomic
upgrades and rollbacks, side-by-side installation of multiple versions of
a package, multi-user package management and easy setup of build
environments.
'';
license = prevAttrs.meta.license or lib.licenses.lgpl21Plus;
maintainers = prevAttrs.meta.maintainers or [ ] ++ scope.maintainers;
platforms = prevAttrs.meta.platforms or (lib.platforms.unix ++ lib.platforms.windows);
};
};
/**
@@ -202,6 +218,7 @@ in
{
version = baseVersion + versionSuffix;
inherit versionSuffix;
inherit maintainers;
inherit filesetToSource;
@@ -237,6 +254,10 @@ in
but it does make the build non-granular; all components will use a complete source.
Packaging expressions will be ignored.
Single argument: the source to use.
See also `appendPatches`
*/
overrideSource =
src:
@@ -265,6 +286,7 @@ in
}
);
resolvePath = p: finalScope.patchedSrc + "/${resolveRelPath p}";
filesetToSource = { root, fileset }: finalScope.resolvePath root;
appendPatches = appendPatches finalScope;
}
);
@@ -281,14 +303,14 @@ in
(scope.overrideSource "${./..}").appendPatches patches;
mkMesonDerivation = mkPackageBuilder [
miscGoodPractice
nixDefaultsLayer
scope.sourceLayer
setVersionLayer
mesonLayer
scope.mesonComponentOverrides
];
mkMesonExecutable = mkPackageBuilder [
miscGoodPractice
nixDefaultsLayer
bsdNoLinkAsNeeded
scope.sourceLayer
setVersionLayer
@@ -297,7 +319,7 @@ in
scope.mesonComponentOverrides
];
mkMesonLibrary = mkPackageBuilder [
miscGoodPractice
nixDefaultsLayer
bsdNoLinkAsNeeded
scope.sourceLayer
mesonLayer
@@ -347,7 +369,7 @@ in
nix-perl-bindings = callPackage ../src/perl/package.nix { };
nix-everything = callPackage ../packaging/everything.nix { } // {
# Note: no `passthru.overrideAllMesonComponents`
# Note: no `passthru.overrideAllMesonComponents` etc
# This would propagate into `nix.overrideAttrs f`, but then discard
# `f` when `.overrideAllMesonComponents` is used.
# Both "methods" should be views on the same fixpoint overriding mechanism
@@ -355,6 +377,8 @@ in
# two-fixpoint solution.
/**
Apply an extension function (i.e. overlay-shaped) to all component derivations, and return the nix package.
Single argument: the extension function to apply (finalAttrs: prevAttrs: { ... })
*/
overrideAllMesonComponents = f: (scope.overrideAllMesonComponents f).nix-everything;
@@ -363,6 +387,10 @@ in
This affects all components.
Changes to the packaging expressions will be ignored.
Single argument: list of patches to apply
See also `overrideSource`
*/
appendPatches = ps: (scope.appendPatches ps).nix-everything;
@@ -371,8 +399,26 @@ in
but it does make the build non-granular; all components will use a complete source.
Packaging expressions will be ignored.
Filesets in the packaging expressions will be ignored.
Single argument: the source to use.
See also `appendPatches`
*/
overrideSource = src: (scope.overrideSource src).nix-everything;
/**
Override any internals of the Nix package set.
Single argument: the extension function to apply to the package set (finalScope: prevScope: { ... })
Example:
```
overrideScope (finalScope: prevScope: { aws-sdk-cpp = null; })
```
*/
overrideScope = f: (scope.overrideScope f).nix-everything;
};
}
+36 -35
View File
@@ -26,6 +26,9 @@ let
# Despite the use of the 10.13 deployment target here, the aligned
# allocation function Clang uses with this setting actually works
# all the way back to 10.6.
# NOTE: this is not just a version constraint, but a request to make Darwin
# provide this version level of support. Removing this minimum version
# request will regress the above error.
darwinStdenv = pkgs.overrideSDK prevStdenv { darwinMinVersion = "10.13"; };
in
@@ -65,39 +68,37 @@ scope: {
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 = pkgs.libgit2.overrideAttrs (
attrs:
{
cmakeFlags = attrs.cmakeFlags or [ ] ++ [ "-DUSE_SSH=exec" ];
}
# libgit2: Nixpkgs 24.11 has < 1.9.0, which needs our patches
// lib.optionalAttrs (!lib.versionAtLeast pkgs.libgit2.version "1.9.0") {
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch =
attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
'';
patches =
attrs.patches or [ ]
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
}
);
libgit2 =
if lib.versionAtLeast pkgs.libgit2.version "1.9.0" then
pkgs.libgit2
else
pkgs.libgit2.overrideAttrs (attrs: {
# libgit2: Nixpkgs 24.11 has < 1.9.0, which needs our patches
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch =
attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
'';
patches =
attrs.patches or [ ]
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
}
-7
View File
@@ -72,10 +72,6 @@ pkgs.nixComponents.nix-util.overrideAttrs (
src = null;
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev pkgs.nixDependencies.boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib pkgs.nixDependencies.boost}/lib";
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
modular.pre-commit.settings.rawConfig
@@ -118,9 +114,6 @@ pkgs.nixComponents.nix-util.overrideAttrs (
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
]
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
);
+36 -46
View File
@@ -4,6 +4,8 @@
lndir,
buildEnv,
maintainers,
nix-util,
nix-util-c,
nix-util-tests,
@@ -39,6 +41,8 @@
nix-perl-bindings,
testers,
patchedSrc ? null,
}:
let
@@ -68,48 +72,6 @@ let
;
};
dev = stdenv.mkDerivation (finalAttrs: {
name = "nix-${nix-cli.version}-dev";
pname = "nix";
version = nix-cli.version;
dontUnpack = true;
dontBuild = true;
libs = map lib.getDev (lib.attrValues libs);
installPhase = ''
mkdir -p $out/nix-support
echo $libs >> $out/nix-support/propagated-build-inputs
'';
passthru = {
tests = {
pkg-config = testers.hasPkgConfigModules {
package = finalAttrs.finalPackage;
};
};
# If we were to fully emulate output selection here, we'd confuse the Nix CLIs,
# because they rely on `drvPath`.
dev = finalAttrs.finalPackage.out;
libs = throw "`nix.dev.libs` is not meant to be used; use `nix.libs` instead.";
};
meta = {
mainProgram = "nix";
pkgConfigModules = [
"nix-cmd"
"nix-expr"
"nix-expr-c"
"nix-fetchers"
"nix-flake"
"nix-flake-c"
"nix-main"
"nix-main-c"
"nix-store"
"nix-store-c"
"nix-util"
"nix-util-c"
];
};
});
devdoc = buildEnv {
name = "nix-${nix-cli.version}-devdoc";
paths = [
@@ -192,21 +154,27 @@ stdenv.mkDerivation (finalAttrs: {
devPaths = lib.mapAttrsToList (_k: lib.getDev) finalAttrs.finalPackage.libs;
in
''
mkdir -p $out $dev $doc $man
mkdir -p $out $dev/nix-support
# Custom files
echo $libs >> $dev/nix-support/propagated-build-inputs
echo ${nix-cli} ${lib.escapeShellArgs devPaths} >> $dev/nix-support/propagated-build-inputs
# Merged outputs
lndir ${nix-cli} $out
for lib in ${lib.escapeShellArgs devPaths}; do
lndir $lib $dev
done
# Forwarded outputs
ln -s ${nix-manual} $doc
ln -s ${nix-manual.man} $man
ln -sT ${nix-manual} $doc
ln -sT ${nix-manual.man} $man
'';
passthru = {
inherit (nix-cli) version;
src = patchedSrc;
/**
These are the libraries that are part of the Nix project. They are used
@@ -248,7 +216,29 @@ stdenv.mkDerivation (finalAttrs: {
meta = {
mainProgram = "nix";
description = "The Nix package manager";
pkgConfigModules = dev.meta.pkgConfigModules;
longDescription = nix-cli.meta.longDescription;
homepage = nix-cli.meta.homepage;
license = nix-cli.meta.license;
maintainers = maintainers;
platforms = nix-cli.meta.platforms;
outputsToInstall = [
"out"
"man"
];
pkgConfigModules = [
"nix-cmd"
"nix-expr"
"nix-expr-c"
"nix-fetchers"
"nix-flake"
"nix-flake-c"
"nix-main"
"nix-main-c"
"nix-store"
"nix-store-c"
"nix-util"
"nix-util-c"
];
};
})
+1 -1
View File
@@ -1,7 +1,7 @@
# Only execute this file once per shell.
# This file is tested by tests/installer/default.nix.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1
export __ETC_PROFILE_NIX_SOURCED=1
NIX_LINK=$HOME/.nix-profile
if [ -n "${XDG_STATE_HOME-}" ]; then
+7 -6
View File
@@ -237,12 +237,13 @@ void StorePathCommand::run(ref<Store> store, StorePaths && storePaths)
MixProfile::MixProfile()
{
addFlag(
{.longName = "profile",
.description = "The profile to operate on.",
.labels = {"path"},
.handler = {&profile},
.completer = completePath});
addFlag({
.longName = "profile",
.description = "The profile to operate on.",
.labels = {"path"},
.handler = {&profile},
.completer = completePath,
});
}
void MixProfile::updateProfile(const StorePath & storePath)
+4 -4
View File
@@ -63,7 +63,7 @@ MixEvalArgs::MixEvalArgs()
.description = "Pass the value *expr* as the argument *name* to Nix functions.",
.category = category,
.labels = {"name", "expr"},
.handler = {[&](std::string name, std::string expr) { autoArgs.insert_or_assign(name, AutoArg{AutoArgExpr{expr}}); }}
.handler = {[&](std::string name, std::string expr) { autoArgs.insert_or_assign(name, AutoArg{AutoArgExpr{expr}}); }},
});
addFlag({
@@ -80,7 +80,7 @@ MixEvalArgs::MixEvalArgs()
.category = category,
.labels = {"name", "path"},
.handler = {[&](std::string name, std::string path) { autoArgs.insert_or_assign(name, AutoArg{AutoArgFile{path}}); }},
.completer = completePath
.completer = completePath,
});
addFlag({
@@ -105,7 +105,7 @@ MixEvalArgs::MixEvalArgs()
.labels = {"path"},
.handler = {[&](std::string s) {
lookupPath.elements.emplace_back(LookupPath::Elem::parse(s));
}}
}},
});
addFlag({
@@ -131,7 +131,7 @@ MixEvalArgs::MixEvalArgs()
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, openStore(), prefix);
}}
}},
});
addFlag({
+13 -13
View File
@@ -64,21 +64,21 @@ MixFlakeOptions::MixFlakeOptions()
.handler = {[&]() {
lockFlags.recreateLockFile = true;
warn("'--recreate-lock-file' is deprecated and will be removed in a future version; use 'nix flake update' instead.");
}}
}},
});
addFlag({
.longName = "no-update-lock-file",
.description = "Do not allow any updates to the flake's lock file.",
.category = category,
.handler = {&lockFlags.updateLockFile, false}
.handler = {&lockFlags.updateLockFile, false},
});
addFlag({
.longName = "no-write-lock-file",
.description = "Do not write the flake's newly generated lock file.",
.category = category,
.handler = {&lockFlags.writeLockFile, false}
.handler = {&lockFlags.writeLockFile, false},
});
addFlag({
@@ -94,14 +94,14 @@ MixFlakeOptions::MixFlakeOptions()
.handler = {[&]() {
lockFlags.useRegistries = false;
warn("'--no-registries' is deprecated; use '--no-use-registries'");
}}
}},
});
addFlag({
.longName = "commit-lock-file",
.description = "Commit changes to the flake's lock file.",
.category = category,
.handler = {&lockFlags.commitLockFile, true}
.handler = {&lockFlags.commitLockFile, true},
});
addFlag({
@@ -121,7 +121,7 @@ MixFlakeOptions::MixFlakeOptions()
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeInputAttrPath(completions, getEvalState(), getFlakeRefsForCompletion(), prefix);
}}
}},
});
addFlag({
@@ -141,7 +141,7 @@ MixFlakeOptions::MixFlakeOptions()
} else if (n == 1) {
completeFlakeRef(completions, getEvalState()->store, prefix);
}
}}
}},
});
addFlag({
@@ -152,7 +152,7 @@ MixFlakeOptions::MixFlakeOptions()
.handler = {[&](std::string lockFilePath) {
lockFlags.referenceLockFilePath = {getFSSourceAccessor(), CanonPath(absPath(lockFilePath))};
}},
.completer = completePath
.completer = completePath,
});
addFlag({
@@ -163,7 +163,7 @@ MixFlakeOptions::MixFlakeOptions()
.handler = {[&](std::string lockFilePath) {
lockFlags.outputLockFilePath = lockFilePath;
}},
.completer = completePath
.completer = completePath,
});
addFlag({
@@ -190,7 +190,7 @@ MixFlakeOptions::MixFlakeOptions()
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, getEvalState()->store, prefix);
}}
}},
});
}
@@ -206,7 +206,7 @@ SourceExprCommand::SourceExprCommand()
.category = installablesCategory,
.labels = {"file"},
.handler = {&file},
.completer = completePath
.completer = completePath,
});
addFlag({
@@ -214,7 +214,7 @@ SourceExprCommand::SourceExprCommand()
.description = "Interpret [*installables*](@docroot@/command-ref/new-cli/nix.md#installables) as attribute paths relative to the Nix expression *expr*.",
.category = installablesCategory,
.labels = {"expr"},
.handler = {&expr}
.handler = {&expr},
});
}
@@ -834,7 +834,7 @@ RawInstallablesCommand::RawInstallablesCommand()
addFlag({
.longName = "stdin",
.description = "Read installables from the standard input. No default installable applied.",
.handler = {&readFromStdIn, true}
.handler = {&readFromStdIn, true},
});
expectArgs({
+5
View File
@@ -113,10 +113,15 @@ headers = [config_h] + files(
'repl.hh',
)
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixcmd',
sources,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
)
+4 -66
View File
@@ -3,7 +3,6 @@
#include <cstring>
#include "error.hh"
#include "eval-error.hh"
#include "repl-interacter.hh"
#include "repl.hh"
@@ -141,16 +140,13 @@ static std::ostream & showDebugTrace(std::ostream & out, const PosTable & positi
out << ANSI_RED "error: " << ANSI_NORMAL;
out << dt.hint.str() << "\n";
// prefer direct pos, but if noPos then try the expr.
auto pos = dt.pos
? dt.pos
: positions[dt.expr.getPos() ? dt.expr.getPos() : noPos];
auto pos = dt.getPos(positions);
if (pos) {
out << *pos;
if (auto loc = pos->getCodeLines()) {
out << pos;
if (auto loc = pos.getCodeLines()) {
out << "\n";
printCodeLines(out, "", *pos, *loc);
printCodeLines(out, "", pos, *loc);
out << "\n";
}
}
@@ -345,58 +341,6 @@ void NixRepl::loadDebugTraceEnv(DebugTrace & dt)
}
}
static Value * lazyProbe(Expr * e, EvalState &state, Env &env)
{
if (auto select = dynamic_cast<ExprSelect *>(e)) {
if (select->attrPath.size() != 1) {
throw Unsupported(":probe: Nested attribute selection not supported yet. Workaround: use parentheses?");
}
if (select->def) {
// This seems like nonsensical input, and we only really support simple attribute paths
throw Unsupported(":probe: %1% keyword is not supported.", "or");
}
auto attr = select->attrPath[0];
if (!attr.symbol) {
throw Unsupported(":probe: Computed attribute selection not supported yet.");
}
auto name = state.symbols[attr.symbol];
auto v = lazyProbe(select->e, state, env);
if (v->type() != nAttrs) {
state.error<TypeError>("can't look up '%1%', expecting the left hand side to be an attribute set, but it is %2%", name, v->type()).atPos(select->e->getPos()).debugThrow();
}
auto it = v->attrs()->find(attr.symbol);
if (it == v->attrs()->end()) {
state.error<EvalError>("attribute '%1%' missing", state.symbols[attr.symbol]).atPos(select->e->getPos()).debugThrow();
}
return it->value;
}
if (auto ref = dynamic_cast<ExprVar *>(e)) {
auto name = state.symbols[ref->name];
// TODO: subtly different intent, so factor out common logic from maybeThunk instead
Value * v = ref->maybeThunk(state, env);
if (!v) {
state.error<EvalError>("Variable '%1%' not found in scope", name).atPos(ref->getPos()).debugThrow();
}
return v;
}
state.error<EvalError>("'%s' is not an expression I can probe. Use variables and attribute selection").atPos(e->getPos()).debugThrow();
}
static void printLazyProbe(Expr * e, EvalState &state, Env &env)
{
Value * v = lazyProbe(e, state, env);
auto suspension = logger->suspend();
std::cout << (v->type() == nThunk ? ANSI_GREEN : ANSI_BLUE);
std::cout << v->type() << std::endl;
std::cout << ANSI_NORMAL;
}
ProcessLineResult NixRepl::processLine(std::string line)
{
line = trim(line);
@@ -433,7 +377,6 @@ ProcessLineResult NixRepl::processLine(std::string line)
<< " :lf, :load-flake <ref> Load Nix flake and add it to scope\n"
<< " :p, :print <expr> Evaluate and print expression recursively\n"
<< " Strings are printed directly, without escaping.\n"
<< " :probe <select> Do not evaluate, but report type or thunk.\n"
<< " :q, :quit Exit nix-repl\n"
<< " :r, :reload Reload all files\n"
<< " :sh <expr> Build dependencies of derivation, then start\n"
@@ -650,11 +593,6 @@ ProcessLineResult NixRepl::processLine(std::string line)
std::cout << std::endl;
}
else if (command == ":probe") {
auto expr = parseString(arg);
printLazyProbe(expr, *state, *env);
}
else if (command == ":q" || command == ":quit") {
state->debugStop = false;
return ProcessLineResult::Quit;
+1 -3
View File
@@ -199,9 +199,7 @@ EvalState * nix_state_create(nix_c_context * context, const char ** lookupPath_c
!= NIX_OK)
return nullptr;
auto *state = nix_eval_state_build(context, builder);
nix_eval_state_builder_free(builder);
return state;
return nix_eval_state_build(context, builder);
}
void nix_state_free(EvalState * state)
+12 -21
View File
@@ -8,32 +8,23 @@ using namespace nix;
Gen<NixStringContextElem::DrvDeep> Arbitrary<NixStringContextElem::DrvDeep>::arbitrary()
{
return gen::map(gen::arbitrary<StorePath>(), [](StorePath drvPath) {
return NixStringContextElem::DrvDeep{
.drvPath = drvPath,
};
return gen::just(NixStringContextElem::DrvDeep {
.drvPath = *gen::arbitrary<StorePath>(),
});
}
Gen<NixStringContextElem> Arbitrary<NixStringContextElem>::arbitrary()
{
return gen::mapcat(
gen::inRange<uint8_t>(0, std::variant_size_v<NixStringContextElem::Raw>),
[](uint8_t n) -> Gen<NixStringContextElem> {
switch (n) {
case 0:
return gen::map(
gen::arbitrary<NixStringContextElem::Opaque>(), [](NixStringContextElem a) { return a; });
case 1:
return gen::map(
gen::arbitrary<NixStringContextElem::DrvDeep>(), [](NixStringContextElem a) { return a; });
case 2:
return gen::map(
gen::arbitrary<NixStringContextElem::Built>(), [](NixStringContextElem a) { return a; });
default:
assert(false);
}
});
switch (*gen::inRange<uint8_t>(0, std::variant_size_v<NixStringContextElem::Raw>)) {
case 0:
return gen::just<NixStringContextElem>(*gen::arbitrary<NixStringContextElem::Opaque>());
case 1:
return gen::just<NixStringContextElem>(*gen::arbitrary<NixStringContextElem::DrvDeep>());
case 2:
return gen::just<NixStringContextElem>(*gen::arbitrary<NixStringContextElem::Built>());
default:
assert(false);
}
}
}
+4 -7
View File
@@ -44,11 +44,11 @@ RC_GTEST_FIXTURE_PROP(
* to worry about race conditions if the tests run concurrently.
*/
ExperimentalFeatureSettings mockXpSettings;
mockXpSettings.set("experimental-features", "ca-derivations dynamic-derivations");
mockXpSettings.set("experimental-features", "ca-derivations");
auto * v = state.allocValue();
state.mkOutputString(*v, b, std::nullopt, mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "");
RC_ASSERT(SingleDerivedPath { b } == d);
}
@@ -57,12 +57,9 @@ RC_GTEST_FIXTURE_PROP(
prop_derived_path_built_out_path_round_trip,
(const SingleDerivedPath::Built & b, const StorePath & outPath))
{
ExperimentalFeatureSettings mockXpSettings;
mockXpSettings.set("experimental-features", "dynamic-derivations");
auto * v = state.allocValue();
state.mkOutputString(*v, b, outPath, mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
state.mkOutputString(*v, b, outPath);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "");
RC_ASSERT(SingleDerivedPath { b } == d);
}
-3
View File
@@ -63,9 +63,6 @@ TEST_F(nix_api_expr_test, nix_expr_eval_external)
std::string string_value;
nix_get_string(nullptr, valueResult, OBSERVE_STRING(string_value));
ASSERT_STREQ("nix-external<MyExternalValueDesc( 42 )>", string_value.c_str());
nix_state_free(stateResult);
nix_state_free(stateFn);
}
}
+1 -3
View File
@@ -124,9 +124,7 @@ RC_GTEST_PROP(
prop_round_rip,
(const NixStringContextElem & o))
{
ExperimentalFeatureSettings xpSettings;
xpSettings.set("experimental-features", "dynamic-derivations");
RC_ASSERT(o == NixStringContextElem::parse(o.to_string(), xpSettings));
RC_ASSERT(o == NixStringContextElem::parse(o.to_string()));
}
#endif
+1 -1
View File
@@ -45,7 +45,7 @@ EvalErrorBuilder<T> & EvalErrorBuilder<T>::withFrame(const Env & env, const Expr
// TODO: check compatibility with nested debugger calls.
// TODO: What side-effects??
error.state.debugTraces.push_front(DebugTrace{
.pos = error.state.positions[expr.getPos()],
.pos = expr.getPos(),
.expr = expr,
.env = env,
.hint = HintFmt("Fake frame for debugging purposes"),
+1 -1
View File
@@ -146,7 +146,7 @@ inline void EvalState::forceList(Value & v, const PosIdx pos, std::string_view e
[[gnu::always_inline]]
inline CallDepth EvalState::addCallDepth(const PosIdx pos) {
if (callDepth > settings.maxCallDepth)
error<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
error<EvalBaseError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
return CallDepth(callDepth);
};
+31 -27
View File
@@ -272,7 +272,7 @@ EvalState::EvalState(
/* Apply access control if needed. */
if (settings.restrictEval || settings.pureEval)
accessor = AllowListSourceAccessor::create(accessor, {},
accessor = AllowListSourceAccessor::create(accessor, {}, {},
[&settings](const CanonPath & path) -> RestrictedPathError {
auto modeInformation = settings.pureEval
? "in pure evaluation mode (use '--impure' to override)"
@@ -771,18 +771,26 @@ void EvalState::runDebugRepl(const Error * error, const Env & env, const Expr &
if (!debugRepl || inDebugger)
return;
auto dts =
error && expr.getPos()
? std::make_unique<DebugTraceStacker>(
*this,
DebugTrace {
.pos = error->info().pos ? error->info().pos : positions[expr.getPos()],
auto dts = [&]() -> std::unique_ptr<DebugTraceStacker> {
if (error && expr.getPos()) {
auto trace = DebugTrace{
.pos = [&]() -> std::variant<Pos, PosIdx> {
if (error->info().pos) {
if (auto * pos = error->info().pos.get())
return *pos;
return noPos;
}
return expr.getPos();
}(),
.expr = expr,
.env = env,
.hint = error->info().msg,
.isError = true
})
: nullptr;
.isError = true};
return std::make_unique<DebugTraceStacker>(*this, std::move(trace));
}
return nullptr;
}();
if (error)
{
@@ -827,7 +835,7 @@ static std::unique_ptr<DebugTraceStacker> makeDebugTraceStacker(
EvalState & state,
Expr & expr,
Env & env,
std::shared_ptr<Pos> && pos,
std::variant<Pos, PosIdx> pos,
const Args & ... formatArgs)
{
return std::make_unique<DebugTraceStacker>(state,
@@ -1104,7 +1112,7 @@ void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
*this,
*e,
this->baseEnv,
e->getPos() ? std::make_shared<Pos>(positions[e->getPos()]) : nullptr,
e->getPos(),
"while evaluating the file '%1%':", resolvedPath.to_string())
: nullptr;
@@ -1330,9 +1338,7 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env2,
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
getPos(),
"while evaluating a '%1%' expression",
"let"
)
@@ -1401,7 +1407,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
state.positions[getPos()],
getPos(),
"while evaluating the attribute '%1%'",
showAttrPath(state, env, attrPath))
: nullptr;
@@ -1602,7 +1608,7 @@ void EvalState::callFunction(Value & fun, std::span<Value *> args, Value & vRes,
try {
auto dts = debugRepl
? makeDebugTraceStacker(
*this, *lambda.body, env2, positions[lambda.pos],
*this, *lambda.body, env2, lambda.pos,
"while calling %s",
lambda.name
? concatStrings("'", symbols[lambda.name], "'")
@@ -1737,9 +1743,7 @@ void ExprCall::eval(EvalState & state, Env & env, Value & v)
state,
*this,
env,
getPos()
? std::make_shared<Pos>(state.positions[getPos()])
: nullptr,
getPos(),
"while calling a function"
)
: nullptr;
@@ -2123,7 +2127,7 @@ void EvalState::forceValueDeep(Value & v)
try {
// If the value is a thunk, we're evaling. Otherwise no trace necessary.
auto dts = debugRepl && i.value->isThunk()
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, positions[i.pos],
? makeDebugTraceStacker(*this, *i.value->payload.thunk.expr, *i.value->payload.thunk.env, i.pos,
"while evaluating the attribute '%1%'", symbols[i.name])
: nullptr;
@@ -2245,18 +2249,18 @@ std::string_view EvalState::forceString(Value & v, const PosIdx pos, std::string
}
void copyContext(const Value & v, NixStringContext & context, const ExperimentalFeatureSettings & xpSettings)
void copyContext(const Value & v, NixStringContext & context)
{
if (v.payload.string.context)
for (const char * * p = v.payload.string.context; *p; ++p)
context.insert(NixStringContextElem::parse(*p, xpSettings));
context.insert(NixStringContextElem::parse(*p));
}
std::string_view EvalState::forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings)
std::string_view EvalState::forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx)
{
auto s = forceString(v, pos, errorCtx);
copyContext(v, context, xpSettings);
copyContext(v, context);
return s;
}
@@ -2462,10 +2466,10 @@ StorePath EvalState::coerceToStorePath(const PosIdx pos, Value & v, NixStringCon
}
std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedPathUnchecked(const PosIdx pos, Value & v, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings)
std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedPathUnchecked(const PosIdx pos, Value & v, std::string_view errorCtx)
{
NixStringContext context;
auto s = forceString(v, context, pos, errorCtx, xpSettings);
auto s = forceString(v, context, pos, errorCtx);
auto csize = context.size();
if (csize != 1)
error<EvalError>(
+21 -4
View File
@@ -159,7 +159,7 @@ void printEnvBindings(const SymbolTable & st, const StaticEnv & se, const Env &
std::unique_ptr<ValMap> mapStaticEnvBindings(const SymbolTable & st, const StaticEnv & se, const Env & env);
void copyContext(const Value & v, NixStringContext & context, const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
void copyContext(const Value & v, NixStringContext & context);
std::string printValue(EvalState & state, Value & v);
@@ -171,11 +171,28 @@ struct RegexCache;
std::shared_ptr<RegexCache> makeRegexCache();
struct DebugTrace {
std::shared_ptr<Pos> pos;
/* WARNING: Converting PosIdx -> Pos should be done with extra care. This is
due to the fact that operator[] of PosTable is incredibly expensive. */
std::variant<Pos, PosIdx> pos;
const Expr & expr;
const Env & env;
HintFmt hint;
bool isError;
Pos getPos(const PosTable & table) const
{
return std::visit(
overloaded{
[&](PosIdx idx) {
// Prefer direct pos, but if noPos then try the expr.
if (!idx)
idx = expr.getPos();
return table[idx];
},
[&](Pos pos) { return pos; },
},
pos);
}
};
class EvalState : public std::enable_shared_from_this<EvalState>
@@ -510,7 +527,7 @@ public:
*/
void forceFunction(Value & v, const PosIdx pos, std::string_view errorCtx);
std::string_view forceString(Value & v, const PosIdx pos, std::string_view errorCtx);
std::string_view forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
std::string_view forceString(Value & v, NixStringContext & context, const PosIdx pos, std::string_view errorCtx);
std::string_view forceStringNoCtx(Value & v, const PosIdx pos, std::string_view errorCtx);
template<typename... Args>
@@ -562,7 +579,7 @@ public:
/**
* Part of `coerceToSingleDerivedPath()` without any store IO which is exposed for unit testing only.
*/
std::pair<SingleDerivedPath, std::string_view> coerceToSingleDerivedPathUnchecked(const PosIdx pos, Value & v, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
std::pair<SingleDerivedPath, std::string_view> coerceToSingleDerivedPathUnchecked(const PosIdx pos, Value & v, std::string_view errorCtx);
/**
* Coerce to `SingleDerivedPath`.
+5 -2
View File
@@ -172,8 +172,6 @@ headers = [config_h] + files(
# internal: 'lexer-helpers.hh',
'nixexpr.hh',
'parser-state.hh',
'pos-idx.hh',
'pos-table.hh',
'primops.hh',
'print-ambiguous.hh',
'print-options.hh',
@@ -189,6 +187,9 @@ headers = [config_h] + files(
subdir('primops')
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixexpr',
sources,
@@ -196,6 +197,8 @@ this_library = library(
lexer_tab,
generated_headers,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
)
-35
View File
@@ -601,41 +601,6 @@ void ExprLambda::setDocComment(DocComment docComment) {
}
};
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(
std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
/* Symbol table. */
size_t SymbolTable::totalSize() const
-7
View File
@@ -81,13 +81,6 @@ mkMesonLibrary (finalAttrs: {
(lib.mesonEnable "gc" enableGC)
];
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
};
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
};
+16 -7
View File
@@ -1,9 +1,23 @@
#include "fetch-to-store.hh"
#include "fetchers.hh"
#include "cache.hh"
namespace nix {
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string &name,
const std::string &fingerprint,
ContentAddressMethod method,
const std::string &path)
{
return fetchers::Cache::Key{"fetchToStore", {
{"name", name},
{"fingerprint", fingerprint},
{"method", std::string{method.render()}},
{"path", path}
}};
}
StorePath fetchToStore(
Store & store,
const SourcePath & path,
@@ -19,12 +33,7 @@ StorePath fetchToStore(
std::optional<fetchers::Cache::Key> cacheKey;
if (!filter && path.accessor->fingerprint) {
cacheKey = fetchers::Cache::Key{"fetchToStore", {
{"name", std::string{name}},
{"fingerprint", *path.accessor->fingerprint},
{"method", std::string{method.render()}},
{"path", path.path.abs()}
}};
cacheKey = makeFetchToStoreCacheKey(std::string{name}, *path.accessor->fingerprint, method, path.path.abs());
if (auto res = fetchers::getCache()->lookupStorePath(*cacheKey, store)) {
debug("store path cache hit for '%s'", path);
return res->storePath;
+4
View File
@@ -5,6 +5,7 @@
#include "file-system.hh"
#include "repair-flag.hh"
#include "file-content-address.hh"
#include "cache.hh"
namespace nix {
@@ -22,4 +23,7 @@ StorePath fetchToStore(
PathFilter * filter = nullptr,
RepairFlag repair = NoRepair);
fetchers::Cache::Key makeFetchToStoreCacheKey(
const std::string & name, const std::string & fingerprint, ContentAddressMethod method, const std::string & path);
}
+2
View File
@@ -323,6 +323,8 @@ std::pair<ref<SourceAccessor>, Input> Input::getAccessorUnchecked(ref<Store> sto
accessor->fingerprint = getFingerprint(store);
accessor->setPathDisplay("«" + to_string() + "»");
return {accessor, *this};
} catch (Error & e) {
debug("substitution of input '%s' failed: %s", to_string(), e.what());
+12 -2
View File
@@ -58,18 +58,23 @@ void FilteringSourceAccessor::checkAccess(const CanonPath & path)
struct AllowListSourceAccessorImpl : AllowListSourceAccessor
{
std::set<CanonPath> allowedPrefixes;
std::unordered_set<CanonPath> allowedPaths;
AllowListSourceAccessorImpl(
ref<SourceAccessor> next,
std::set<CanonPath> && allowedPrefixes,
std::unordered_set<CanonPath> && allowedPaths,
MakeNotAllowedError && makeNotAllowedError)
: AllowListSourceAccessor(SourcePath(next), std::move(makeNotAllowedError))
, allowedPrefixes(std::move(allowedPrefixes))
, allowedPaths(std::move(allowedPaths))
{ }
bool isAllowed(const CanonPath & path) override
{
return path.isAllowed(allowedPrefixes);
return
allowedPaths.contains(path)
|| path.isAllowed(allowedPrefixes);
}
void allowPrefix(CanonPath prefix) override
@@ -81,9 +86,14 @@ struct AllowListSourceAccessorImpl : AllowListSourceAccessor
ref<AllowListSourceAccessor> AllowListSourceAccessor::create(
ref<SourceAccessor> next,
std::set<CanonPath> && allowedPrefixes,
std::unordered_set<CanonPath> && allowedPaths,
MakeNotAllowedError && makeNotAllowedError)
{
return make_ref<AllowListSourceAccessorImpl>(next, std::move(allowedPrefixes), std::move(makeNotAllowedError));
return make_ref<AllowListSourceAccessorImpl>(
next,
std::move(allowedPrefixes),
std::move(allowedPaths),
std::move(makeNotAllowedError));
}
bool CachingFilteringSourceAccessor::isAllowed(const CanonPath & path)
@@ -2,6 +2,8 @@
#include "source-path.hh"
#include <unordered_set>
namespace nix {
/**
@@ -70,6 +72,7 @@ struct AllowListSourceAccessor : public FilteringSourceAccessor
static ref<AllowListSourceAccessor> create(
ref<SourceAccessor> next,
std::set<CanonPath> && allowedPrefixes,
std::unordered_set<CanonPath> && allowedPaths,
MakeNotAllowedError && makeNotAllowedError);
using FilteringSourceAccessor::FilteringSourceAccessor;
+5 -4
View File
@@ -44,10 +44,11 @@ static void downloadToSink(
static std::string getLfsApiToken(const ParsedURL & url)
{
auto [status, output] = runProgram(RunOptions{
.program = "ssh",
.args = {*url.authority, "git-lfs-authenticate", url.path, "download"},
});
auto [status, output] = runProgram(
RunOptions{
.program = "ssh",
.args = {*url.authority, "git-lfs-authenticate", url.path, "download"},
});
if (output.empty())
throw Error(
+4 -8
View File
@@ -1215,16 +1215,12 @@ ref<SourceAccessor> GitRepoImpl::getAccessor(
ref<SourceAccessor> GitRepoImpl::getAccessor(const WorkdirInfo & wd, bool exportIgnore, MakeNotAllowedError makeNotAllowedError)
{
auto self = ref<GitRepoImpl>(shared_from_this());
/* In case of an empty workdir, return an empty in-memory tree. We
cannot use AllowListSourceAccessor because it would return an
error for the root (and we can't add the root to the allow-list
since that would allow access to all its children). */
ref<SourceAccessor> fileAccessor =
wd.files.empty()
? makeEmptySourceAccessor()
: AllowListSourceAccessor::create(
AllowListSourceAccessor::create(
makeFSSourceAccessor(path),
std::set<CanonPath> { wd.files },
std::set<CanonPath>{ wd.files },
// Always allow access to the root, but not its children.
std::unordered_set<CanonPath>{CanonPath::root},
std::move(makeNotAllowedError)).cast<SourceAccessor>();
if (exportIgnore)
return make_ref<GitExportIgnoreSourceAccessor>(self, fileAccessor, std::nullopt);
+14 -8
View File
@@ -532,14 +532,20 @@ struct GitInputScheme : InputScheme
return *head;
}
static MakeNotAllowedError makeNotAllowedError(std::string url)
static MakeNotAllowedError makeNotAllowedError(std::filesystem::path repoPath)
{
return [url{std::move(url)}](const CanonPath & path) -> RestrictedPathError
{
if (nix::pathExists(path.abs()))
return RestrictedPathError("access to path '%s' is forbidden because it is not under Git control; maybe you should 'git add' it to the repository '%s'?", path, url);
return [repoPath{std::move(repoPath)}](const CanonPath & path) -> RestrictedPathError {
if (nix::pathExists(repoPath / path.rel()))
return RestrictedPathError(
"Path '%1%' in the repository %2% is not tracked by Git.\n"
"\n"
"To make it visible to Nix, run:\n"
"\n"
"git -C %2% add \"%1%\"",
path.rel(),
repoPath);
else
return RestrictedPathError("path '%s' does not exist in Git repository '%s'", path, url);
return RestrictedPathError("Path '%s' does not exist in Git repository %s.", path.rel(), repoPath);
};
}
@@ -637,7 +643,7 @@ struct GitInputScheme : InputScheme
if (!input.getRev())
setWriteTime(localRefFile, now, now);
} catch (Error & e) {
warn("could not update mtime for file %s: %s", localRefFile, e.info().msg);
warn("could not update mtime for file '%s': %s", localRefFile, e.info().msg);
}
if (!originalRef && !storeCachedHead(repoUrl.to_string(), ref))
warn("could not update cached head '%s' for '%s'", ref, repoInfo.locationToArg());
@@ -747,7 +753,7 @@ struct GitInputScheme : InputScheme
ref<SourceAccessor> accessor =
repo->getAccessor(repoInfo.workdirInfo,
exportIgnore,
makeNotAllowedError(repoInfo.locationToArg()));
makeNotAllowedError(repoPath));
/* If the repo has submodules, return a mounted input accessor
consisting of the accessor for the top-level repo and the
-3
View File
@@ -149,9 +149,6 @@ struct GitArchiveInputScheme : InputScheme
};
if (auto narHash = input.getNarHash())
url.query.insert_or_assign("narHash", narHash->to_string(HashFormat::SRI, true));
auto host = maybeGetStrAttr(input.attrs, "host");
if (host)
url.query.insert_or_assign("host", *host);
return url;
}
+5
View File
@@ -76,10 +76,15 @@ headers = files(
'tarball.hh',
)
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixfetchers',
sources,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
)
+12 -1
View File
@@ -2,6 +2,9 @@
#include "store-api.hh"
#include "archive.hh"
#include "store-path-accessor.hh"
#include "store-path-accessor.hh"
#include "cache.hh"
#include "fetch-to-store.hh"
namespace nix::fetchers {
@@ -125,7 +128,7 @@ struct PathInputScheme : InputScheme
auto absPath = getAbsPath(input);
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying '%s' to the store", absPath));
Activity act(*logger, lvlTalkative, actUnknown, fmt("copying %s to the store", absPath));
// FIXME: check whether access to 'path' is allowed.
auto storePath = store->maybeParseStorePath(absPath.string());
@@ -142,6 +145,14 @@ struct PathInputScheme : InputScheme
storePath = store->addToStoreFromDump(*src, "source");
}
// To avoid copying the path again to the /nix/store, we need to add a cache entry.
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive;
auto fp = getFingerprint(store, input);
if (fp) {
auto cacheKey = makeFetchToStoreCacheKey(input.getName(), *fp, method, "/");
fetchers::getCache()->upsert(cacheKey, *store, {}, *storePath);
}
/* Trust the lastModified value supplied by the user, if
any. It's not a "secure" attribute so we don't care. */
if (!input.getLastModified())
-3
View File
@@ -43,9 +43,6 @@ TEST_F(nix_api_store_test, nix_api_init_global_getFlake_exists)
ASSERT_NE(nullptr, value);
nix_err err = nix_expr_eval_from_string(ctx, state, "builtins.getFlake", ".", value);
nix_state_free(state);
assert_ctx_ok();
ASSERT_EQ(NIX_OK, err);
ASSERT_EQ(NIX_TYPE_FUNCTION, nix_get_type(ctx, value));
+3 -3
View File
@@ -863,11 +863,11 @@ LockedFlake lockFlake(
auto s = chomp(diff);
if (lockFileExists) {
if (s.empty())
warn("updating lock file %s", outputLockFilePath);
warn("updating lock file '%s'", outputLockFilePath);
else
warn("updating lock file %s:\n%s", outputLockFilePath, s);
warn("updating lock file '%s':\n%s", outputLockFilePath, s);
} else
warn("creating lock file %s: \n%s", outputLockFilePath, s);
warn("creating lock file '%s': \n%s", outputLockFilePath, s);
std::optional<std::string> commitMessage = std::nullopt;
+1 -1
View File
@@ -39,7 +39,7 @@ FlakeRef FlakeRef::resolve(
ref<Store> store,
const fetchers::RegistryFilter & filter) const
{
auto [input2, extraAttrs] = lookupInRegistries(store, input);
auto [input2, extraAttrs] = lookupInRegistries(store, input, filter);
return FlakeRef(std::move(input2), fetchers::maybeGetStrAttr(extraAttrs, "dir").value_or(subdir));
}
+5
View File
@@ -58,10 +58,15 @@ headers = files(
'flake/url-name.hh',
)
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixflake',
sources,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
)
+2 -2
View File
@@ -57,7 +57,7 @@ MixCommonArgs::MixCommonArgs(const std::string & programName)
if (hasPrefix(s.first, prefix))
completions.add(s.first, fmt("Set the `%s` setting.", s.first));
}
}
},
});
addFlag({
@@ -75,7 +75,7 @@ MixCommonArgs::MixCommonArgs(const std::string & programName)
.labels = Strings{"jobs"},
.handler = {[=](std::string s) {
settings.set("max-jobs", s);
}}
}},
});
std::string cat = "Options to override configuration settings";
+5
View File
@@ -82,10 +82,15 @@ headers = [config_h] + files(
'shared.hh',
)
subdir('nix-meson-build-support/export-all-symbols')
subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixmain',
sources,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args: linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
)
+1 -1
View File
@@ -228,7 +228,7 @@ LegacyArgs::LegacyArgs(const std::string & programName,
.handler = {[=](std::string s) {
auto n = string2IntWithUnitPrefix<uint64_t>(s);
settings.set(dest, std::to_string(n));
}}
}},
});
};
+27
View File
@@ -0,0 +1,27 @@
#include "nix/tests/outputs-spec.hh"
#include <rapidcheck.h>
namespace rc {
using namespace nix;
Gen<OutputsSpec> Arbitrary<OutputsSpec>::arbitrary()
{
return gen::mapcat(
gen::inRange<uint8_t>(0, std::variant_size_v<OutputsSpec::Raw>), [](uint8_t n) -> Gen<OutputsSpec> {
switch (n) {
case 0:
return gen::just((OutputsSpec) OutputsSpec::All{});
case 1:
return gen::map(
gen::nonEmpty(
gen::container<StringSet>(
gen::map(gen::arbitrary<StorePathName>(), [](StorePathName n) { return n.name; }))),
[](StringSet names) { return (OutputsSpec) OutputsSpec::Names{names}; });
default:
assert(false);
}
});
}
}
+24 -38
View File
@@ -9,63 +9,49 @@ using namespace nix;
Gen<SingleDerivedPath::Opaque> Arbitrary<SingleDerivedPath::Opaque>::arbitrary()
{
return gen::map(gen::arbitrary<StorePath>(), [](StorePath path) {
return DerivedPath::Opaque{
.path = path,
};
return gen::just(DerivedPath::Opaque {
.path = *gen::arbitrary<StorePath>(),
});
}
Gen<SingleDerivedPath::Built> Arbitrary<SingleDerivedPath::Built>::arbitrary()
{
return gen::mapcat(gen::arbitrary<SingleDerivedPath>(), [](SingleDerivedPath drvPath) {
return gen::map(gen::arbitrary<StorePathName>(), [drvPath](StorePathName outputPath) {
return SingleDerivedPath::Built{
.drvPath = make_ref<SingleDerivedPath>(drvPath),
.output = outputPath.name,
};
});
return gen::just(SingleDerivedPath::Built {
.drvPath = make_ref<SingleDerivedPath>(*gen::arbitrary<SingleDerivedPath>()),
.output = (*gen::arbitrary<StorePathName>()).name,
});
}
Gen<DerivedPath::Built> Arbitrary<DerivedPath::Built>::arbitrary()
{
return gen::mapcat(gen::arbitrary<SingleDerivedPath>(), [](SingleDerivedPath drvPath) {
return gen::map(gen::arbitrary<OutputsSpec>(), [drvPath](OutputsSpec outputs) {
return DerivedPath::Built{
.drvPath = make_ref<SingleDerivedPath>(drvPath),
.outputs = outputs,
};
});
return gen::just(DerivedPath::Built {
.drvPath = make_ref<SingleDerivedPath>(*gen::arbitrary<SingleDerivedPath>()),
.outputs = *gen::arbitrary<OutputsSpec>(),
});
}
Gen<SingleDerivedPath> Arbitrary<SingleDerivedPath>::arbitrary()
{
return gen::mapcat(gen::inRange<uint8_t>(0, std::variant_size_v<SingleDerivedPath::Raw>), [](uint8_t n) {
switch (n) {
case 0:
return gen::map(gen::arbitrary<SingleDerivedPath::Opaque>(), [](SingleDerivedPath a) { return a; });
case 1:
return gen::map(gen::arbitrary<SingleDerivedPath::Built>(), [](SingleDerivedPath a) { return a; });
default:
assert(false);
}
});
switch (*gen::inRange<uint8_t>(0, std::variant_size_v<SingleDerivedPath::Raw>)) {
case 0:
return gen::just<SingleDerivedPath>(*gen::arbitrary<SingleDerivedPath::Opaque>());
case 1:
return gen::just<SingleDerivedPath>(*gen::arbitrary<SingleDerivedPath::Built>());
default:
assert(false);
}
}
Gen<DerivedPath> Arbitrary<DerivedPath>::arbitrary()
{
return gen::mapcat(gen::inRange<uint8_t>(0, std::variant_size_v<DerivedPath::Raw>), [](uint8_t n) {
switch (n) {
case 0:
return gen::map(gen::arbitrary<DerivedPath::Opaque>(), [](DerivedPath a) { return a; });
case 1:
return gen::map(gen::arbitrary<DerivedPath::Built>(), [](DerivedPath a) { return a; });
default:
assert(false);
}
});
switch (*gen::inRange<uint8_t>(0, std::variant_size_v<DerivedPath::Raw>)) {
case 0:
return gen::just<DerivedPath>(*gen::arbitrary<DerivedPath::Opaque>());
case 1:
return gen::just<DerivedPath>(*gen::arbitrary<DerivedPath::Built>());
default:
assert(false);
}
}
}
+11 -13
View File
@@ -7,20 +7,18 @@ using namespace nix;
Gen<OutputsSpec> Arbitrary<OutputsSpec>::arbitrary()
{
return gen::mapcat(
gen::inRange<uint8_t>(0, std::variant_size_v<OutputsSpec::Raw>), [](uint8_t n) -> Gen<OutputsSpec> {
switch (n) {
case 0:
return gen::just((OutputsSpec) OutputsSpec::All{});
case 1:
return gen::map(
gen::nonEmpty(gen::container<StringSet>(
gen::map(gen::arbitrary<StorePathName>(), [](StorePathName n) { return n.name; }))),
[](StringSet names) { return (OutputsSpec) OutputsSpec::Names{names}; });
default:
assert(false);
}
switch (*gen::inRange<uint8_t>(0, std::variant_size_v<OutputsSpec::Raw>)) {
case 0:
return gen::just((OutputsSpec) OutputsSpec::All { });
case 1:
return gen::just((OutputsSpec) OutputsSpec::Names {
*gen::nonEmpty(gen::container<StringSet>(gen::map(
gen::arbitrary<StorePathName>(),
[](StorePathName n) { return n.name; }))),
});
default:
assert(false);
}
}
}
+3 -12
View File
@@ -79,19 +79,12 @@ TEST_F(DerivedPathTest, built_built_xp) {
#ifndef COVERAGE
/* TODO: Disabled due to the following error:
path '00000000000000000000000000000000-0^0' is not a valid store path:
name '0^0' contains illegal character '^'
*/
RC_GTEST_FIXTURE_PROP(
DerivedPathTest,
DISABLED_prop_legacy_round_rip,
prop_legacy_round_rip,
(const DerivedPath & o))
{
ExperimentalFeatureSettings xpSettings;
xpSettings.set("experimental-features", "dynamic-derivations");
RC_ASSERT(o == DerivedPath::parseLegacy(*store, o.to_string_legacy(*store), xpSettings));
RC_ASSERT(o == DerivedPath::parseLegacy(*store, o.to_string_legacy(*store)));
}
RC_GTEST_FIXTURE_PROP(
@@ -99,9 +92,7 @@ RC_GTEST_FIXTURE_PROP(
prop_round_rip,
(const DerivedPath & o))
{
ExperimentalFeatureSettings xpSettings;
xpSettings.set("experimental-features", "dynamic-derivations");
RC_ASSERT(o == DerivedPath::parse(*store, o.to_string(*store), xpSettings));
RC_ASSERT(o == DerivedPath::parse(*store, o.to_string(*store)));
}
#endif
+1 -5
View File
@@ -23,11 +23,7 @@ void builtinUnpackChannel(
throw Error("channelName is not allowed to contain filesystem separators, got %1%", channelName);
}
try {
fs::create_directories(out);
} catch (fs::filesystem_error &) {
throw SysError("creating directory '%1%'", out.string());
}
createDirs(out);
unpackTarfile(src, out);
+1
View File
@@ -1025,6 +1025,7 @@ void processConnection(
{
#ifndef _WIN32 // TODO need graceful async exit support on Windows?
auto monitor = !recursive ? std::make_unique<MonitorFdHup>(from.fd) : nullptr;
(void) monitor; // suppress warning
#endif
/* Exchange the greeting. */
+3 -3
View File
@@ -278,21 +278,21 @@ template<> void BaseSetting<SandboxMode>::convertToArg(Args & args, const std::s
.aliases = aliases,
.description = "Enable sandboxing.",
.category = category,
.handler = {[this]() { override(smEnabled); }}
.handler = {[this]() { override(smEnabled); }},
});
args.addFlag({
.longName = "no-" + name,
.aliases = aliases,
.description = "Disable sandboxing.",
.category = category,
.handler = {[this]() { override(smDisabled); }}
.handler = {[this]() { override(smDisabled); }},
});
args.addFlag({
.longName = "relaxed-" + name,
.aliases = aliases,
.description = "Enable sandboxing, but allow builds to disable it.",
.category = category,
.handler = {[this]() { override(smRelaxed); }}
.handler = {[this]() { override(smRelaxed); }},
});
}
+2 -2
View File
@@ -116,7 +116,7 @@ LocalStore::LocalStore(
state->stmts = std::make_unique<State::Stmts>();
/* Create missing state directories if they don't already exist. */
createDirs(realStoreDir);
createDirs(realStoreDir.get());
if (readOnly) {
experimentalFeatureSettings.require(Xp::ReadOnlyLocalStore);
} else {
@@ -247,7 +247,7 @@ LocalStore::LocalStore(
else if (curSchema == 0) { /* new store */
curSchema = nixSchemaVersion;
openDB(*state, true);
writeFile(schemaPath, fmt("%1%", nixSchemaVersion), 0666, true);
writeFile(schemaPath, fmt("%1%", curSchema), 0666, true);
}
else if (curSchema < nixSchemaVersion) {
+2 -1
View File
@@ -112,7 +112,8 @@ deps_public += nlohmann_json
sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19')
deps_private += sqlite
# AWS C++ SDK has bad pkg-config
# AWS C++ SDK has bad pkg-config. See
# https://github.com/aws/aws-sdk-cpp/issues/2673 for details.
aws_s3 = dependency('aws-cpp-sdk-s3', required : false)
configdata.set('ENABLE_S3', aws_s3.found().to_int())
if aws_s3.found()
+5 -10
View File
@@ -21,6 +21,10 @@
version,
embeddedSandboxShell ? stdenv.hostPlatform.isStatic,
withAWS ?
# Default is this way because there have been issues building this dependency
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin),
}:
let
@@ -60,9 +64,7 @@ mkMesonLibrary (finalAttrs: {
++ lib.optional stdenv.hostPlatform.isLinux libseccomp
# There have been issues building these dependencies
++ lib.optional stdenv.hostPlatform.isDarwin darwin.apple_sdk.libs.sandbox
++ lib.optional (
stdenv.hostPlatform == stdenv.buildPlatform && (stdenv.isLinux || stdenv.isDarwin)
) aws-sdk-cpp;
++ lib.optional withAWS aws-sdk-cpp;
propagatedBuildInputs = [
nix-util
@@ -78,13 +80,6 @@ mkMesonLibrary (finalAttrs: {
(lib.mesonOption "sandbox-shell" "${busybox-sandbox-shell}/bin/busybox")
];
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
};
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
};
+1 -1
View File
@@ -28,7 +28,7 @@ RemoteStore::RemoteStore(const Params & params)
: RemoteStoreConfig(params)
, Store(params)
, connections(make_ref<Pool<Connection>>(
std::max(1, maxConnections.get()),
std::max(1, (int) maxConnections),
[this]() {
auto conn = openConnectionWrapper();
try {
@@ -2144,7 +2144,18 @@ void LocalDerivationGoal::runChild()
without file-write* allowed, access() incorrectly returns EPERM
*/
sandboxProfile += "(allow file-read* file-write* process-exec\n";
// We create multiple allow lists, to avoid exceeding a limit in the darwin sandbox interpreter.
// See https://github.com/NixOS/nix/issues/4119
// We split our allow groups approximately at half the actual limit, 1 << 16
const int breakpoint = sandboxProfile.length() + (1 << 14);
for (auto & i : pathsInChroot) {
if (sandboxProfile.length() >= breakpoint) {
debug("Sandbox break: %d %d", sandboxProfile.length(), breakpoint);
sandboxProfile += ")\n(allow file-read* file-write* process-exec\n";
}
if (i.first != i.second.source)
throw Error(
"can't map '%1%' to '%2%': mismatched impure paths not supported on Darwin",
@@ -2909,8 +2920,12 @@ void LocalDerivationGoal::checkOutputs(const std::map<std::string, ValidPathInfo
spec.insert(worker.store.parseStorePath(i));
else if (auto output = get(outputs, i))
spec.insert(output->path);
else
throw BuildError("derivation contains an illegal reference specifier '%s'", i);
else {
std::string outputsListing = concatMapStringsSep(", ", outputs, [](auto & o) { return o.first; });
throw BuildError("derivation '%s' output check for '%s' contains an illegal reference specifier '%s',"
" expected store path or output name (one of [%s])",
worker.store.printStorePath(drvPath), outputName, i, outputsListing);
}
}
auto used = recursive
+18
View File
@@ -0,0 +1,18 @@
#include "util.hh"
#include "monitor-fd.hh"
#include <sys/file.h>
#include <gtest/gtest.h>
namespace nix {
TEST(MonitorFdHup, shouldNotBlock)
{
Pipe p;
p.create();
{
// when monitor gets destroyed it should cancel the
// background thread and do not block
MonitorFdHup monitor(p.readSide.get());
}
}
}
+36
View File
@@ -80,6 +80,42 @@ TEST(concatStringsSep, buildSingleString)
ASSERT_EQ(concatStringsSep(",", strings), "this");
}
TEST(concatMapStringsSep, empty)
{
Strings strings;
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "");
}
TEST(concatMapStringsSep, justOne)
{
Strings strings;
strings.push_back("this");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this");
}
TEST(concatMapStringsSep, two)
{
Strings strings;
strings.push_back("this");
strings.push_back("that");
ASSERT_EQ(concatMapStringsSep(",", strings, [](const std::string & s) { return s; }), "this,that");
}
TEST(concatMapStringsSep, map)
{
std::map<std::string, std::string> strings;
strings["this"] = "that";
strings["1"] = "one";
ASSERT_EQ(
concatMapStringsSep(
", ", strings, [](const std::pair<std::string, std::string> & s) { return s.first + " -> " + s.second; }),
"1 -> one, this -> that");
}
/* ----------------------------------------------------------------------------
* dropEmptyInitThenConcatStringsSep
* --------------------------------------------------------------------------*/
+8
View File
@@ -50,6 +50,14 @@ struct LinesOfCode {
std::optional<std::string> nextLineOfCode;
};
/* NOTE: position.hh recursively depends on source-path.hh -> source-accessor.hh
-> hash.hh -> config.hh -> experimental-features.hh -> error.hh -> Pos.
There are other such cycles.
Thus, Pos has to be an incomplete type in this header. But since ErrorInfo/Trace
have to refer to Pos, they have to use pointer indirection via std::shared_ptr
to break the recursive header dependency.
FIXME: Untangle this mess. Should there be AbstractPos as there used to be before
4feb7d9f71? */
struct Pos;
void printCodeLines(std::ostream & out,
+14 -4
View File
@@ -29,7 +29,17 @@
namespace nix {
namespace fs { using namespace std::filesystem; }
namespace fs {
using namespace std::filesystem;
bool symlink_exists(const std::filesystem::path & path) {
try {
return std::filesystem::exists(std::filesystem::symlink_status(path));
} catch (const std::filesystem::filesystem_error & e) {
throw SysError("cannot check existence of %1%", path);
}
}
}
bool isAbsolute(PathView path)
{
@@ -475,12 +485,12 @@ void createDir(const Path & path, mode_t mode)
throw SysError("creating directory '%1%'", path);
}
void createDirs(const Path & path)
void createDirs(const fs::path & path)
{
try {
fs::create_directories(path);
} catch (fs::filesystem_error & e) {
throw SysError("creating directory '%1%'", path);
throw SysError("creating directory '%1%'", path.string());
}
}
@@ -680,7 +690,7 @@ void setWriteTime(
if (utimes(path.c_str(), times) == -1)
throw SysError("changing modification time of %s (not a symlink)", path);
} else {
throw Error("Cannot modification time of symlink %s", path);
throw Error("Cannot change modification time of symlink %s", path);
}
#endif
#endif
+4 -10
View File
@@ -134,6 +134,7 @@ bool pathExists(const Path & path);
namespace fs {
/**
* TODO: we may actually want to use pathExists instead of this function
* ```
* symlink_exists(p) = std::filesystem::exists(std::filesystem::symlink_status(p))
* ```
@@ -142,9 +143,7 @@ namespace fs {
* std::filesystem::exists(p) = std::filesystem::exists(std::filesystem::status(p))
* ```
*/
inline bool symlink_exists(const std::filesystem::path & path) {
return std::filesystem::exists(std::filesystem::symlink_status(path));
}
bool symlink_exists(const std::filesystem::path & path);
} // namespace fs
@@ -231,14 +230,9 @@ void deletePath(const std::filesystem::path & path, uint64_t & bytesFreed);
/**
* Create a directory and all its parents, if necessary.
*
* In the process of being deprecated for
* `std::filesystem::create_directories`.
* Wrapper around `std::filesystem::create_directories` to handle exceptions.
*/
void createDirs(const Path & path);
inline void createDirs(PathView path)
{
return createDirs(Path(path));
}
void createDirs(const std::filesystem::path & path);
/**
* Create a single directory.
+1 -1
View File
@@ -33,7 +33,7 @@ std::optional<Mode> decodeMode(RawMode m) {
static std::string getStringUntil(Source & source, char byte)
{
std::string s;
char n[1];
char n[1] = { 0 };
source(std::string_view { n, 1 });
while (*n != byte) {
s += *n;
+3
View File
@@ -155,6 +155,7 @@ sources = files(
'memory-source-accessor.cc',
'mounted-source-accessor.cc',
'position.cc',
'pos-table.cc',
'posix-source-accessor.cc',
'references.cc',
'serialise.cc',
@@ -225,6 +226,8 @@ headers = [config_h] + files(
'muxable-pipe.hh',
'os-string.hh',
'pool.hh',
'pos-idx.hh',
'pos-table.hh',
'position.hh',
'posix-source-accessor.hh',
'processes.hh',
-7
View File
@@ -58,13 +58,6 @@ mkMesonLibrary (finalAttrs: {
(lib.mesonEnable "cpuid" stdenv.hostPlatform.isx86_64)
];
env = {
# Needed for Meson to find Boost.
# https://github.com/NixOS/nixpkgs/issues/86131.
BOOST_INCLUDEDIR = "${lib.getDev boost}/include";
BOOST_LIBRARYDIR = "${lib.getLib boost}/lib";
};
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
};
@@ -1,4 +1,5 @@
#pragma once
///@file
#include <cinttypes>
#include <functional>
+37
View File
@@ -0,0 +1,37 @@
#include "pos-table.hh"
#include <algorithm>
namespace nix {
/* Position table. */
Pos PosTable::operator[](PosIdx p) const
{
auto origin = resolve(p);
if (!origin)
return {};
const auto offset = origin->offsetOf(p);
Pos result{0, 0, origin->origin};
auto lines = this->lines.lock();
auto linesForInput = (*lines)[origin->offset];
if (linesForInput.empty()) {
auto source = result.getSource().value_or("");
const char * begin = source.data();
for (Pos::LinesIterator it(source), end; it != end; it++)
linesForInput.push_back(it->data() - begin);
if (linesForInput.empty())
linesForInput.push_back(0);
}
// as above: the first line starts at byte 0 and is always present
auto lineStartOffset = std::prev(std::upper_bound(linesForInput.begin(), linesForInput.end(), offset));
result.line = 1 + (lineStartOffset - linesForInput.begin());
result.column = 1 + (offset - *lineStartOffset);
return result;
}
}
@@ -1,4 +1,5 @@
#pragma once
///@file
#include <cstdint>
#include <vector>
@@ -18,9 +19,12 @@ public:
private:
uint32_t offset;
Origin(Pos::Origin origin, uint32_t offset, size_t size):
offset(offset), origin(origin), size(size)
{}
Origin(Pos::Origin origin, uint32_t offset, size_t size)
: offset(offset)
, origin(origin)
, size(size)
{
}
public:
const Pos::Origin origin;
@@ -72,6 +76,17 @@ public:
return PosIdx(1 + origin.offset + offset);
}
/**
* Convert a byte-offset PosIdx into a Pos with line/column information.
*
* @param p Byte offset into the virtual concatenation of all parsed contents
* @return Position
*
* @warning Very expensive to call, as this has to read the entire source
* into memory each time. Call this only if absolutely necessary. Prefer
* to keep PosIdx around instead of needlessly converting it into Pos by
* using this lookup method.
*/
Pos operator[](PosIdx p) const;
Pos::Origin originOf(PosIdx p) const
+7
View File
@@ -66,6 +66,13 @@ std::optional<std::string> Pos::getSource() const
}, origin);
}
std::optional<SourcePath> Pos::getSourcePath() const
{
if (auto * path = std::get_if<SourcePath>(&origin))
return *path;
return std::nullopt;
}
void Pos::print(std::ostream & out, bool showOrigin) const
{
if (showOrigin) {
+2 -3
View File
@@ -50,6 +50,7 @@ struct Pos
explicit operator bool() const { return line > 0; }
/* TODO: Why std::shared_ptr<Pos> and not std::shared_ptr<const Pos>? */
operator std::shared_ptr<Pos>() const;
/**
@@ -69,9 +70,7 @@ struct Pos
/**
* Get the SourcePath, if the source was loaded from a file.
*/
std::optional<SourcePath> getSourcePath() const {
return *std::get_if<SourcePath>(&origin);
}
std::optional<SourcePath> getSourcePath() const;
struct LinesIterator {
using difference_type = size_t;
+1
View File
@@ -37,6 +37,7 @@ basicSplitString(std::basic_string_view<OsChar> s, std::basic_string_view<OsChar
template std::string concatStringsSep(std::string_view, const std::list<std::string> &);
template std::string concatStringsSep(std::string_view, const std::set<std::string> &);
template std::string concatStringsSep(std::string_view, const std::vector<std::string> &);
template std::string concatStringsSep(std::string_view, const boost::container::small_vector<std::string, 64> &);
typedef std::string_view strings_2[2];
template std::string concatStringsSep(std::string_view, const strings_2 &);
+17
View File
@@ -6,6 +6,8 @@
#include <string>
#include <vector>
#include <boost/container/small_vector.hpp>
namespace nix {
/*
@@ -54,6 +56,21 @@ std::string concatStringsSep(const std::string_view sep, const C & ss);
extern template std::string concatStringsSep(std::string_view, const std::list<std::string> &);
extern template std::string concatStringsSep(std::string_view, const std::set<std::string> &);
extern template std::string concatStringsSep(std::string_view, const std::vector<std::string> &);
extern template std::string concatStringsSep(std::string_view, const boost::container::small_vector<std::string, 64> &);
/**
* Apply a function to the `iterable`'s items and concat them with `separator`.
*/
template<class C, class F>
std::string concatMapStringsSep(std::string_view separator, const C & iterable, F fn)
{
boost::container::small_vector<std::string, 64> strings;
strings.reserve(iterable.size());
for (const auto & elem : iterable) {
strings.push_back(fn(elem));
}
return concatStringsSep(separator, strings);
}
/**
* Ignore any empty strings at the start of the list, and then concatenate the
+2 -2
View File
@@ -166,7 +166,7 @@ void unpackTarfile(Source & source, const fs::path & destDir)
{
auto archive = TarArchive(source);
fs::create_directories(destDir);
createDirs(destDir);
extract_archive(archive, destDir);
}
@@ -174,7 +174,7 @@ void unpackTarfile(const fs::path & tarFile, const fs::path & destDir)
{
auto archive = TarArchive(tarFile);
fs::create_directories(destDir);
createDirs(destDir);
extract_archive(archive, destDir);
}
+40 -4
View File
@@ -5,9 +5,27 @@
#include <fcntl.h>
#include <unistd.h>
#include <poll.h>
namespace nix {
namespace {
// This function is needed to handle non-blocking reads/writes. This is needed in the buildhook, because
// somehow the json logger file descriptor ends up beeing non-blocking and breaks remote-building.
// TODO: get rid of buildhook and remove this function again (https://github.com/NixOS/nix/issues/12688)
void pollFD(int fd, int events)
{
struct pollfd pfd;
pfd.fd = fd;
pfd.events = events;
int ret = poll(&pfd, 1, -1);
if (ret == -1) {
throw SysError("poll on file descriptor failed");
}
}
}
std::string readFile(int fd)
{
struct stat st;
@@ -17,14 +35,18 @@ std::string readFile(int fd)
return drainFD(fd, true, st.st_size);
}
void readFull(int fd, char * buf, size_t count)
{
while (count) {
checkInterrupt();
ssize_t res = read(fd, buf, count);
if (res == -1) {
if (errno == EINTR) continue;
switch (errno) {
case EINTR: continue;
case EAGAIN:
pollFD(fd, POLLIN);
continue;
}
throw SysError("reading from file");
}
if (res == 0) throw EndOfFile("unexpected end-of-file");
@@ -39,8 +61,15 @@ void writeFull(int fd, std::string_view s, bool allowInterrupts)
while (!s.empty()) {
if (allowInterrupts) checkInterrupt();
ssize_t res = write(fd, s.data(), s.size());
if (res == -1 && errno != EINTR)
if (res == -1) {
switch (errno) {
case EINTR: continue;
case EAGAIN:
pollFD(fd, POLLOUT);
continue;
}
throw SysError("writing to file");
}
if (res > 0)
s.remove_prefix(res);
}
@@ -56,8 +85,15 @@ std::string readLine(int fd, bool eofOk)
// FIXME: inefficient
ssize_t rd = read(fd, &ch, 1);
if (rd == -1) {
if (errno != EINTR)
switch (errno) {
case EINTR: continue;
case EAGAIN: {
pollFD(fd, POLLIN);
continue;
}
default:
throw SysError("reading a line");
}
} else if (rd == 0) {
if (eofOk)
return s;
+80 -24
View File
@@ -14,35 +14,74 @@
namespace nix {
class MonitorFdHup
{
private:
std::thread thread;
Pipe notifyPipe;
public:
MonitorFdHup(int fd)
{
thread = std::thread([fd]() {
notifyPipe.create();
thread = std::thread([this, fd]() {
while (true) {
/* Wait indefinitely until a POLLHUP occurs. */
struct pollfd fds[1];
fds[0].fd = fd;
/* Polling for no specific events (i.e. just waiting
for an error/hangup) doesn't work on macOS
anymore. So wait for read events and ignore
them. */
fds[0].events =
#ifdef __APPLE__
POLLRDNORM
#else
// There is a POSIX violation on macOS: you have to listen for
// at least POLLHUP to receive HUP events for a FD. POSIX says
// this is not so, and you should just receive them regardless.
// However, as of our testing on macOS 14.5, the events do not
// get delivered if in the all-bits-unset case, but do get
// delivered if `POLLHUP` is set.
//
// This bug filed as rdar://37537852
// (https://openradar.appspot.com/37537852).
//
// macOS's own man page
// (https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/poll.2.html)
// additionally says that `POLLHUP` is ignored as an input. It
// seems the likely order of events here was
//
// 1. macOS did not follow the POSIX spec
//
// 2. Somebody ninja-fixed this other spec violation to make
// sure `POLLHUP` was not forgotten about, even though they
// "fixed" this issue in a spec-non-compliant way. Whatever,
// we'll use the fix.
//
// Relevant code, current version, which shows the :
// https://github.com/apple-oss-distributions/xnu/blob/94d3b452840153a99b38a3a9659680b2a006908e/bsd/kern/sys_generic.c#L1751-L1758
//
// The `POLLHUP` detection was added in
// https://github.com/apple-oss-distributions/xnu/commit/e13b1fa57645afc8a7b2e7d868fe9845c6b08c40#diff-a5aa0b0e7f4d866ca417f60702689fc797e9cdfe33b601b05ccf43086c35d395R1468
// That means added in 2007 or earlier. Should be good enough
// for us.
short hangup_events =
#ifdef __APPLE__
POLLHUP
#else
0
#endif
#endif
;
auto count = poll(fds, 1, -1);
if (count == -1)
unreachable();
/* Wait indefinitely until a POLLHUP occurs. */
constexpr size_t num_fds = 2;
struct pollfd fds[num_fds] = {
{
.fd = fd,
.events = hangup_events,
},
{
.fd = notifyPipe.readSide.get(),
.events = hangup_events,
},
};
auto count = poll(fds, num_fds, -1);
if (count == -1) {
if (errno == EINTR || errno == EAGAIN)
continue;
throw SysError("failed to poll() in MonitorFdHup");
}
/* This shouldn't happen, but can on macOS due to a bug.
See rdar://37550628.
@@ -50,25 +89,42 @@ public:
coordination with the main thread if spinning proves
too harmful.
*/
if (count == 0) continue;
if (count == 0)
continue;
if (fds[0].revents & POLLHUP) {
unix::triggerInterrupt();
break;
}
/* This will only happen on macOS. We sleep a bit to
avoid waking up too often if the client is sending
input. */
sleep(1);
if (fds[1].revents & POLLHUP) {
break;
}
// On macOS, (jade thinks that) it is possible (although not
// observed on macOS 14.5) that in some limited cases on buggy
// kernel versions, all the non-POLLHUP events for the socket
// get delivered.
//
// We could sleep to avoid pointlessly spinning a thread on
// those, but this opens up a different problem, which is that
// if do sleep, it will be longer before the daemon fork for a
// client exits. Imagine a sequential shell script, running Nix
// commands, each of which talk to the daemon. If the previous
// command registered a temp root, exits, and then the next
// command issues a delete request before the temp root is
// cleaned up, that delete request might fail.
//
// Not sleeping doesn't actually fix the race condition --- we
// would need to block on the old connections' tempt roots being
// cleaned up in in the new connection --- but it does make it
// much less likely.
}
});
};
~MonitorFdHup()
{
pthread_cancel(thread.native_handle());
notifyPipe.writeSide.close();
thread.join();
}
};
}
+1 -1
View File
@@ -55,7 +55,7 @@ struct CmdBuild : InstallablesCommand, MixDryRun, MixJSON, MixProfile
.description = "Use *path* as prefix for the symlinks to the build results. It defaults to `result`.",
.labels = {"path"},
.handler = {&outLink},
.completer = completePath
.completer = completePath,
});
addFlag({
+2 -2
View File
@@ -24,7 +24,7 @@ struct CmdBundle : InstallableValueCommand
.handler = {&bundler},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, getStore(), prefix);
}}
}},
});
addFlag({
@@ -33,7 +33,7 @@ struct CmdBundle : InstallableValueCommand
.description = "Override the name of the symlink to the build result. It defaults to the base name of the app.",
.labels = {"path"},
.handler = {&outLink},
.completer = completePath
.completer = completePath,
});
}
+1 -1
View File
@@ -21,7 +21,7 @@ struct CmdCopy : virtual CopyCommand, virtual BuiltPathsCommand, MixProfile
.description = "Create symlinks prefixed with *path* to the top-level store paths fetched from the source store.",
.labels = {"path"},
.handler = {&outLink},
.completer = completePath
.completer = completePath,
});
addFlag({
+67
View File
@@ -0,0 +1,67 @@
#include "crash-handler.hh"
#include "fmt.hh"
#include "logging.hh"
#include <boost/core/demangle.hpp>
#include <exception>
#include <sstream>
// Darwin and FreeBSD stdenv do not define _GNU_SOURCE but do have _Unwind_Backtrace.
#if __APPLE__ || __FreeBSD__
# define BOOST_STACKTRACE_GNU_SOURCE_NOT_REQUIRED
#endif
#include <boost/stacktrace/stacktrace.hpp>
#ifndef _WIN32
# include <syslog.h>
#endif
namespace nix {
namespace {
void logFatal(std::string const & s)
{
writeToStderr(s + "\n");
// std::string for guaranteed null termination
#ifndef _WIN32
syslog(LOG_CRIT, "%s", s.c_str());
#endif
}
void onTerminate()
{
logFatal(
"Nix crashed. This is a bug. Please report this at https://github.com/NixOS/nix/issues with the following information included:\n");
try {
std::exception_ptr eptr = std::current_exception();
if (eptr) {
std::rethrow_exception(eptr);
} else {
logFatal("std::terminate() called without exception");
}
} catch (const std::exception & ex) {
logFatal(fmt("Exception: %s: %s", boost::core::demangle(typeid(ex).name()), ex.what()));
} catch (...) {
logFatal("Unknown exception!");
}
logFatal("Stack trace:");
std::stringstream ss;
ss << boost::stacktrace::stacktrace();
logFatal(ss.str());
std::abort();
}
}
void registerCrashHandler()
{
// DO NOT use this for signals. Boost stacktrace is very much not
// async-signal-safe, and in a world with ASLR, addr2line is pointless.
//
// If you want signals, set up a minidump system and do it out-of-process.
std::set_terminate(onTerminate);
}
}
+11
View File
@@ -0,0 +1,11 @@
#pragma once
/// @file Crash handler for Nix that prints back traces (hopefully in instances where it is not just going to crash the
/// process itself).
namespace nix {
/** Registers the Nix crash handler for std::terminate (currently; will support more crashes later). See also
* detectStackOverflow(). */
void registerCrashHandler();
}
+1 -1
View File
@@ -21,7 +21,7 @@ struct CmdShowDerivation : InstallablesCommand
.longName = "recursive",
.shortName = 'r',
.description = "Include the dependencies of the specified derivations.",
.handler = {&recursive, true}
.handler = {&recursive, true},
});
}
+2 -2
View File
@@ -334,7 +334,7 @@ struct Common : InstallableCommand, MixProfile
.labels = {"installable", "outputs-dir"},
.handler = {[&](std::string installable, std::string outputsDir) {
redirects.push_back({installable, outputsDir});
}}
}},
});
}
@@ -524,7 +524,7 @@ struct CmdDevelop : Common, MixEnvironment
.handler = {[&](std::vector<std::string> ss) {
if (ss.empty()) throw UsageError("--command requires at least one argument");
command = ss;
}}
}},
});
addFlag({
+11 -10
View File
@@ -38,16 +38,17 @@ struct CmdShell : InstallablesCommand, MixEnvironment
CmdShell()
{
addFlag(
{.longName = "command",
.shortName = 'c',
.description = "Command and arguments to be executed, defaulting to `$SHELL`",
.labels = {"command", "args"},
.handler = {[&](std::vector<std::string> ss) {
if (ss.empty())
throw UsageError("--command requires at least one argument");
command = ss;
}}});
addFlag({
.longName = "command",
.shortName = 'c',
.description = "Command and arguments to be executed, defaulting to `$SHELL`",
.labels = {"command", "args"},
.handler = {[&](std::vector<std::string> ss) {
if (ss.empty())
throw UsageError("--command requires at least one argument");
command = ss;
}},
});
}
std::string description() override
+11 -11
View File
@@ -90,7 +90,7 @@ public:
.handler={&flakeUrl},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, getStore(), prefix);
}}
}},
});
expectArgs({
.label="inputs",
@@ -111,7 +111,7 @@ public:
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeInputAttrPath(completions, getEvalState(), getFlakeRefsForCompletion(), prefix);
}}
}},
});
/* Remove flags that don't make sense. */
@@ -336,12 +336,12 @@ struct CmdFlakeCheck : FlakeCommand
addFlag({
.longName = "no-build",
.description = "Do not build checks.",
.handler = {&build, false}
.handler = {&build, false},
});
addFlag({
.longName = "all-systems",
.description = "Check the outputs for all systems.",
.handler = {&checkAllSystems, true}
.handler = {&checkAllSystems, true},
});
}
@@ -874,7 +874,7 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
defaultTemplateAttrPathsPrefixes,
defaultTemplateAttrPaths,
prefix);
}}
}},
});
}
@@ -905,7 +905,7 @@ struct CmdFlakeInitCommon : virtual Args, EvalCommand
std::function<void(const SourcePath & from, const fs::path & to)> copyDir;
copyDir = [&](const SourcePath & from, const fs::path & to)
{
fs::create_directories(to);
createDirs(to);
for (auto & [name, entry] : from.readDirectory()) {
checkInterrupt();
@@ -1034,7 +1034,7 @@ struct CmdFlakeClone : FlakeCommand
.shortName = 'f',
.description = "Clone the flake to path *dest*.",
.labels = {"path"},
.handler = {&destDir}
.handler = {&destDir},
});
}
@@ -1057,7 +1057,7 @@ struct CmdFlakeArchive : FlakeCommand, MixJSON, MixDryRun
.longName = "to",
.description = "URI of the destination Nix store",
.labels = {"store-uri"},
.handler = {&dstUri}
.handler = {&dstUri},
});
}
@@ -1137,12 +1137,12 @@ struct CmdFlakeShow : FlakeCommand, MixJSON
addFlag({
.longName = "legacy",
.description = "Show the contents of the `legacyPackages` output.",
.handler = {&showLegacy, true}
.handler = {&showLegacy, true},
});
addFlag({
.longName = "all-systems",
.description = "Show the contents of outputs for all systems.",
.handler = {&showAllSystems, true}
.handler = {&showAllSystems, true},
});
}
@@ -1443,7 +1443,7 @@ struct CmdFlakePrefetch : FlakeCommand, MixJSON
.description = "Create symlink named *path* to the resulting store path.",
.labels = {"path"},
.handler = {&outLink},
.completer = completePath
.completer = completePath,
});
}
+3
View File
@@ -20,6 +20,7 @@
#include "flake/flake.hh"
#include "self-exe.hh"
#include "json-utils.hh"
#include "crash-handler.hh"
#include <sys/types.h>
#include <regex>
@@ -354,6 +355,8 @@ void mainWrapped(int argc, char * * argv)
{
savedArgv = argv;
registerCrashHandler();
/* The chroot helper needs to be run before any threads have been
started. */
#ifndef _WIN32
+1
View File
@@ -77,6 +77,7 @@ nix_sources = [config_h] + files(
'config-check.cc',
'config.cc',
'copy.cc',
'crash-handler.cc',
'derivation-add.cc',
'derivation-show.cc',
'derivation.cc',
+2 -2
View File
@@ -274,7 +274,7 @@ struct CmdStorePrefetchFile : StoreCommand, MixJSON
.longName = "name",
.description = "Override the name component of the resulting store path. It defaults to the base name of *url*.",
.labels = {"name"},
.handler = {&name}
.handler = {&name},
});
addFlag({
@@ -283,7 +283,7 @@ struct CmdStorePrefetchFile : StoreCommand, MixJSON
.labels = {"hash"},
.handler = {[&](std::string s) {
expectedHash = Hash::parseAny(s, hashAlgo);
}}
}},
});
addFlag(flag::hashAlgo("hash-type", &hashAlgo));
+1 -1
View File
@@ -104,7 +104,7 @@ struct CmdSign : StorePathsCommand
.description = "File containing the secret signing key.",
.labels = {"file"},
.handler = {&secretKeyFile},
.completer = completePath
.completer = completePath,
});
}
+1 -1
View File
@@ -16,7 +16,7 @@ struct CmdStoreDelete : StorePathsCommand
addFlag({
.longName = "ignore-liveness",
.description = "Do not check whether the paths are reachable from a root.",
.handler = {&options.ignoreLiveness, true}
.handler = {&options.ignoreLiveness, true},
});
}
+1 -1
View File
@@ -17,7 +17,7 @@ struct CmdStoreGC : StoreCommand, MixDryRun
.longName = "max",
.description = "Stop after freeing *n* bytes of disk space.",
.labels = {"n"},
.handler = {&options.maxFreed}
.handler = {&options.maxFreed},
});
}
+2 -2
View File
@@ -546,7 +546,7 @@ static int main_nix_daemon(int argc, char * * argv)
static RegisterLegacyCommand r_nix_daemon("nix-daemon", main_nix_daemon);
struct CmdDaemon : StoreCommand
struct CmdDaemon : Command
{
bool stdio = false;
std::optional<TrustedFlag> isTrustedOpt = std::nullopt;
@@ -615,7 +615,7 @@ struct CmdDaemon : StoreCommand
;
}
void run(ref<Store> store) override
void run() override
{
runDaemon(stdio, isTrustedOpt, processOps);
}
+2 -2
View File
@@ -23,14 +23,14 @@ struct CmdUpgradeNix : MixDryRun, StoreCommand
.shortName = 'p',
.description = "The path to the Nix profile to upgrade.",
.labels = {"profile-dir"},
.handler = {&profileDir}
.handler = {&profileDir},
});
addFlag({
.longName = "nix-store-paths-url",
.description = "The URL of the file that contains the store paths of the latest Nix release.",
.labels = {"url"},
.handler = {&(std::string&) settings.upgradeNixStorePathUrl}
.handler = {&(std::string&) settings.upgradeNixStorePathUrl},
});
}
+2 -2
View File
@@ -37,7 +37,7 @@ struct CmdVerify : StorePathsCommand
.shortName = 's',
.description = "Use signatures from the specified store.",
.labels = {"store-uri"},
.handler = {[&](std::string s) { substituterUris.push_back(s); }}
.handler = {[&](std::string s) { substituterUris.push_back(s); }},
});
addFlag({
@@ -45,7 +45,7 @@ struct CmdVerify : StorePathsCommand
.shortName = 'n',
.description = "Require that each path is signed by at least *n* different keys.",
.labels = {"n"},
.handler = {&sigsNeeded}
.handler = {&sigsNeeded},
});
}
+2 -2
View File
@@ -31,7 +31,7 @@ perl.pkgs.toPerlModule (
./meson.build
./meson.options
]
++ lib.optionals finalAttrs.doCheck [
++ lib.optionals finalAttrs.finalPackage.doCheck [
./.yath.rc.in
./t
]
@@ -70,7 +70,7 @@ perl.pkgs.toPerlModule (
mesonFlags = [
(lib.mesonOption "dbi_path" "${perlPackages.DBI}/${perl.libPrefix}")
(lib.mesonOption "dbd_sqlite_path" "${perlPackages.DBDSQLite}/${perl.libPrefix}")
(lib.mesonEnable "tests" finalAttrs.doCheck)
(lib.mesonEnable "tests" finalAttrs.finalPackage.doCheck)
];
mesonCheckFlags = [
+9
View File
@@ -79,4 +79,13 @@ rec {
buildCommand = ''echo ${dep} > "''${outputs[out]}"'';
};
test12 = makeTest 12 {
builder = builtins.toFile "builder.sh" "mkdir $out $lib";
outputs = [
"out"
"lib"
];
disallowedReferences = [ "dev" ];
};
}
Loaded 100 of 118 files, more files were not shown because too many files have changed in this diff. Show more