Commit Graph
12590 Commits
Author SHA1 Message Date
Robert Hensing 9aa3cc9c8f libexpr: Call destructor on exception_ptr
This uses `gc_cleanup` to call the exception_ptr destructor when the
`Failed` is collected.
I don't know exactly how bad it is to deallocate `std::exception_ptr`
without destruction, but I guess it ranges from small leak to UB and crash.
2025-09-10 12:49:41 +02:00
Eelco Dolstra b13143280c Introduce a "failed" value type
In the multithreaded evaluator, it's possible for multiple threads to
wait on the same thunk. If evaluation of the thunk results in an
exception, the waiting threads shouldn't try to re-force the thunk.
Instead, they should rethrow the same exception, without duplicating
any work.

Therefore, there is now a new value type `tFailed` that stores an
std::exception_ptr. If evaluation of a thunk/app results in an
exception, `forceValue()` overwrites the value with a `tFailed`. If
`forceValue()` encounters a `tFailed`, it rethrows the exception. So
you normally never need to check for failed values (since forcing them
causes a rethrow).
2025-09-10 12:49:41 +02:00
Jörg Thalheim 8c789db05b Merge pull request #13956 from NixOS/drop-unused-addMultipleToStoreLegacy
Drop unused LegacySSHStore::addMultipleToStoreLegacy()
2025-09-10 10:52:05 +02:00
Eelco Dolstra f8b15bfc7f Merge pull request #13951 from NixOS/drop-old-daemon-protocol
Remove support for worker protocol version < 18
2025-09-10 10:43:48 +02:00
Eelco Dolstra 5013b38df4 Drop unused LegacySSHStore::addMultipleToStoreLegacy() 2025-09-10 10:27:07 +02:00
Eelco Dolstra 247d16a530 Merge pull request #13954 from xokdvium/empty-list-elems-fix
libexpr: Fix Value::mkList for empty lists
2025-09-10 10:23:49 +02:00
Jörg Thalheim 7d26bf8cc7 Merge pull request #13906 from obsidiansystems/derivation-builder-simpler
More `DerivationBuilder` simplifications
2025-09-10 09:58:07 +02:00
Jörg Thalheim 9c186c35fa Merge pull request #13932 from NixOS/move-pathInfoCache
Reduce false sharing between pathInfoCache and Store
2025-09-10 09:56:46 +02:00
Sergei Zimmerman 2ed2c79721 libexpr: Fix Value::mkList for empty lists
This code used to save the pointer to a small
list allocated on the stack to the Value, which
is unintended.
2025-09-10 01:37:39 +03:00
Sergei Zimmerman 3c331b7ef3 Merge pull request #13953 from xokdvium/value-alignment
libexpr: Overalign Value to 16 bytes
2025-09-09 20:44:43 +00:00
Sergei Zimmerman 4524235af4 libexpr: Overalign Value to 16 bytes
This is necessary to make use of 128 bit atomics on x86_64 [1],
since MOVAPD, MOVAPS, and MOVDQA need memory operands to be 16-byte
aligned. We are not losing anything here, because Value is already 16-byte
wide and Boehm allocates memory in granules that are 16 bytes by default
on 64 bit systems [2].

[1]: https://patchwork.sourceware.org/project/gcc/patch/YhxkfzGEEQ9KHbBC@tucnak/
[2]: https://github.com/bdwgc/bdwgc/blob/54ac18ccbc5a833dd7edaff94a10ab9b65044d61/include/gc/gc_tiny_fl.h#L31-L33
2025-09-09 22:18:52 +03:00
Eelco Dolstra 86d19956f2 Remove WorkerProto::Op::ImportPaths
This was obsoleted in May 2016 (538a64e8c3).
2025-09-09 15:41:17 +02:00
Eelco Dolstra 4fb61bc5af Remove WorkerProto::Op::ExportPath
This was obsoleted in May 2016 (538a64e8c3).
2025-09-09 15:41:12 +02:00
Eelco Dolstra 137a55122c Remove support for daemon protocol version < 18
Version 18 was introduced in November 2016 (4b8f1b0ec0).
2025-09-09 15:41:01 +02:00
Eelco Dolstra a97c5df47c Merge pull request #13939 from DeterminateSystems/fix-inputs-from-ignoring-dir-param-upstreaming
Pass `dir` in extraAttrs when overriding the registry
2025-09-09 09:36:33 +02:00
Sergei Zimmerman 7cc654afa9 libstore: Reallow unbracketed IPv6 addresses in store references
This implements a special back-compat shim to specifically allow
unbracketed IPv6 addresses in store references. This is something
that is relied upon in the wild and the old parsing logic accepted
both ways (brackets were optional). This patch restores this behavior.
As always, we didn't have any tests for this.

Addresses #13937.
2025-09-09 00:41:03 +03:00
Cole Helbling 38663fb434 Pass dir in extraAttrs when overriding the registry
This is handled similarly in the handler for `--override-flake` in
`MixEvalArgs`.
2025-09-08 09:00:59 +02:00
Eelco Dolstra 12db0726e9 Merge pull request #13934 from DeterminateSystems/fix-flake-registry-ignoring-dir-param-upstreaming
Fix flake registry ignoring `dir` parameter
2025-09-08 07:50:54 +02:00
Eelco Dolstra 525245181a Merge pull request #13933 from NixOS/local-store-state
LocalStore::State: Put behind a ref to reduce false sharing
2025-09-08 06:22:04 +02:00
Eelco Dolstra 9302ec5e0e Add comment 2025-09-08 05:57:02 +02:00
Cole Helbling 9c832a08b0 fixup: cached case
I couldn't come up with a test that failed before this, but my existing
test still passes so 🤷
2025-09-07 19:40:24 +02:00
Cole Helbling bccdb95a86 Fix flake registry ignoring dir parameter
This broke in e3042f10af.
2025-09-07 19:40:24 +02:00
Tom Westerhout dbc235cc62 Generalize recognized git url schemas (#13925)
Use `parseUrlScheme` instead of manually parsing `url.scheme`.
2025-09-07 15:22:20 +02:00
Eelco Dolstra e791ede495 LocalStore::State: Put behind a ref to reduce false sharing 2025-09-07 14:32:24 +02:00
Eelco Dolstra a73cf447ac Reduce false sharing between pathInfoCache and Store
`perf c2c` shows a lot of cacheline conflicts between purely read-only
Store methods (like `parseStorePath()`) and the Sync classes. So
allocate pathInfoCache separately to avoid that.
2025-09-07 14:27:38 +02:00
Eelco Dolstra 9ff427d7ba Merge pull request #13911 from xokdvium/store-uri-daemon-local
libstore: Do not normalize daemon -> unix://, local -> local://
2025-09-07 14:10:45 +02:00
Eelco Dolstra 4dd27a292c Merge pull request #13929 from NixOS/remove-unused
Remove unused function setChildSignalMask()
2025-09-07 13:30:09 +02:00
Jörg Thalheim 5ae1b5f88b Merge pull request #13916 from sinanmohd/fix/develop-interactive-shell
nix/develop: pass down the interactive shell to subshells
2025-09-07 10:14:25 +02:00
Eelco Dolstra f363d958a7 Fix hang in enterChroot() draining userNamespaceSync
Calling `drainFD()` will hang if another process has the write side
open, since then the child won't get an EOF. This can happen if we
have multiple threads doing a build, since in that case another thread
may fork a child process that inherits the write side of the first
thread.

We could set O_CLOEXEC on the write side (using pipe2()) but it won't
help here since we don't always do an exec() in the child, e.g. in the
case of builtin builders. (We need a "close-on-fork", not a
"close-on-exec".)
2025-09-07 01:12:44 +02:00
Eelco Dolstra a44dcbff13 Remove unused function setChildSignalMask() 2025-09-06 23:02:57 +02:00
Sergei Zimmerman bbdabe4973 libexpr: Remove decl for undefined overload of Value::mkPath 2025-09-06 16:36:16 +03:00
Jörg Thalheim 1d62ccdb3d Merge pull request #13767 from ethanavatar/master
libutil, libexpr: #10542 abstract over getrusage for getting cpuTime stat and implement windows version
2025-09-06 09:26:13 +02:00
Jörg Thalheim 533c6d38aa Merge pull request #13901 from Mic92/fix-macos-hup-detection
Fix macOS HUP detection using kqueue instead of poll
2025-09-06 09:19:51 +02:00
Sergei Zimmerman 738924b705 libexpr: Slim down Bindings to 8 bytes (on 64 bit systems)
Since the only construction and push_back() calls
to Bindings happen through the `BindingsBuilder` [1] we don't
need to keep `capacity` around on the heap anymore. This saves 8 bytes
(because of the member alignment padding)
per one Bindings allocation. This isn't that much, but it does
save significant memory.

This also shows that the Bindings don't necessarily have to
be mutable, which opens up opportunities for doing small bindings
optimization and storing a 1-element Bindings directly in Value.

For the following scenario:

nix-env --query --available --out-path --file ../nixpkgs --eval-system x86_64-linux

(nixpkgs revision: ddcddd7b09a417ca9a88899f4bd43a8edb72308d)

This patch results in reduction of `sets.bytes` 13115104016 -> 12653087640,
which amounts to 462 MB less bytes allocated for Bindings.

[1]: Not actually, `getBuiltins` does mutate bindings, but this is pretty
     inconsequential and doesn't lead to problems.
2025-09-06 00:23:54 +03:00
sinanmohd 211cbe4abf nix/develop: pass down the interactive shell to subshells 2025-09-05 20:18:25 +05:30
Sergei Zimmerman 3513ab13dc libstore: Do not normalize daemon -> unix://, local -> local://
This is relied upon (specifically the `local` store) by existing
tooling [1] and we broke this in 3e7879e6df (which
was first released in 2.31).

To lessen the scope of the breakage we should not normalize "auto" references
and explicitly specified references like "local" or "daemon". It also makes
sense to canonicalize local://,daemon:// to be more compatible with prior
behavior.

[1]: https://github.com/maralorn/nix-output-monitor/blob/05e1b3cba2fa328a1781390a4e4515e9c432229e/lib/NOM/Builds.hs#L60-L64
2025-09-05 04:14:36 +03:00
John Ericson 49e9c14e2f Merge pull request #13900 from NixOS/fix-mingw-windows-build
Fix downstream MinGW build by not looking for Boost Regex
2025-09-04 21:05:12 -04:00
John Ericson 25d3c197b8 Merge pull request #13902 from NixOS/ssh-master-deadlock
Fix deadlock in SSHMaster::addCommonSSHOpts()
2025-09-03 21:44:06 -04:00
John Ericson 2acb9559d5 Combine DerivationBuilder::{prepareBuild,startBuilder}
After many other cleanups, it turns out there is no reason for these to
be separate methods. We can combine them to simplify things.
2025-09-03 17:58:50 -04:00
John Ericson 14c206f05a DerivationBuilder no more callback soup for logging
`startBuilder` just returns the descriptor for the pipe now.
2025-09-03 17:34:45 -04:00
John Ericson 7f3314a68c DerivationBuilder::initialOutputs make const
At one point I remember it did mutatate `initialOutputs`, but not
anymore!
2025-09-03 17:34:45 -04:00
John Ericson b69576e2b3 Merge pull request #13905 from obsidiansystems/derivation-building-goal-simplify-0
Derivation building goal simplify -- no `goto`
2025-09-03 17:34:27 -04:00
John Ericson 7b22cd5105 Merge pull request #13839 from Mic92/infra
don't include derivation name in temporary build directories
2025-09-03 17:15:03 -04:00
Jörg Thalheim 1732b4a61b Merge pull request #13885 from netadr/fix-ssh-key-ids
libfetchers: Fix SSH key types for sk type keys
2025-09-03 23:13:48 +02:00
John Ericson 819bf13607 Merge pull request #13880 from Mic92/static-alloc-symbol-ids
libexpr: Convert Symbol comparisons to switch statements
2025-09-03 17:13:12 -04:00
John Ericson a30bf96349 DerivationBuildingGoal::initialOutputs make local variable
Also inline `assertPathValidity` in the process.
2025-09-03 17:04:13 -04:00
John Ericson c0c2a89f05 DerivationBuildingGoal::initialOutputs move initialization down to tryToBuild
Will help us make this a local variable.
2025-09-03 17:04:08 -04:00
John Ericson 450633aa8c Move machineName from DerivationBuildingGoal to HookInstance
Exactly why is is correct is a little subtle, because sometimes the
worker is owned by the worker. But the commit message in
e437b08250 explained the situation well
enough: I made that commit message part of the ABI docs, and now it
should be understandable to the next person.
2025-09-03 17:03:56 -04:00
netadr 671c21db9f libfetchers: Fix SSH key identifiers for sk type keys
libfetchers: Mark ssh-ecdsa-sk key type mapping as a TODO for now
2025-09-03 22:56:33 +02:00
John Ericson 8089102164 Separate internal from non-internal unit tests of the C API
This helps us make sure that the external C API is sufficient for the
tasks that we think it is sufficient for.
2025-09-03 22:50:42 +02:00