Commit Graph
3564 Commits
Author SHA1 Message Date
Sergei Zimmerman 614072adcb Merge pull request #15286 from NixOS/failed-values-v2
Introduce a "failed" value type (v2)
2026-02-21 09:05:15 +00:00
siddhantCodesandAmaan Qureshi 6808bfab92 libutil: return std::filesystem::path from XDG directory helpers
`getCacheDir`, `getConfigDir`, `getDataDir`, `getStateDir`, and related functions now return `std::filesystem::path` and use `getEnvOs` for native OS string handling, letting callsites replace string concatenation with `operator/` and drop the ad-hoc `namespace nix::fs` alias from six CLI files. `expandTilde` is fixed to strip both `~` and `/` before joining with `operator/` (an absolute right-hand operand silently replaces the left-hand side), and `ExecutablePath` gains `parseAppend` for incremental `PATH` construction.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-20 22:35:47 -05:00
Robert Hensing 17f344cdda libexpr: Add recoverable errors
This provides an explicit API for call-fail-retry-succeed evaluation
flows, such as currently used in NixOps4.

An alternative design would simply reset the `Value` to the original
thunk instead of `tFailed` under the condition of catching a
`RecoverableEvalError`.
That is somewhat simpler, but I believe the presence of `tFailed` is
beneficial for possible use in the repl; being able to show the error
sooner, without re-evaluation.

The hasPos method is required in order to avoid an include loop.
2026-02-19 14:23:02 +03:00
Eelco DolstraandRobert Hensing c33d9e31cc Introduce a "failed" value type
In the multithreaded evaluator, it's possible for multiple threads to
wait on the same thunk. If evaluation of the thunk results in an
exception, the waiting threads shouldn't try to re-force the thunk.
Instead, they should rethrow the same exception, without duplicating
any work.

Therefore, there is now a new value type `tFailed` that stores an
std::exception_ptr. If evaluation of a thunk/app results in an
exception, `forceValue()` overwrites the value with a `tFailed`. If
`forceValue()` encounters a `tFailed`, it rethrows the exception. So
you normally never need to check for failed values (since forcing them
causes a rethrow).

Co-authored-by: Robert Hensing <robert@roberthensing.nl>
2026-02-19 14:23:01 +03:00
Sergei Zimmerman 89158eedb5 treewide: Make exceptions cloneable
This is needed to make it possible to store exceptions in failed values
with each new rethrow getting a fresh copy of the exception object.
2026-02-19 14:22:58 +03:00
Sergei Zimmerman 247cc7013a libexpr: Use temporary value in ExprOpUpdate::eval 2026-02-19 02:49:10 +03:00
John Ericson 16b0bb7548 Merge pull request #15270 from NixOS/inline-lookup-var
libexpr: Make sure `EvalState::lookupVar` is inlined
2026-02-17 15:12:00 +00:00
Sergei Zimmerman 3df91bea62 libexpr: Optimise Value::type(), ValueStorage::getInternalType()
Using nix::unreachable() in getInternalType() and type() turns
out to be quite expensive and prevents inlining. Also Value::type
got compiled to a jump table which has a high overhead from indirect
jumps. Using an explicit lookup table turns out to be more efficient.

This does mean that we lose out on nice diagnostics from nix::unreachable
calls, but this code is probably one of the hottests functions in the whole
evaluator, so I think the tradeoff is worth it. The nixUnreachableWhenHardened
boils down to nix::unreachable when UBSan is enabled so we still have good
coverage there.
2026-02-17 16:50:07 +03:00
Sergei Zimmerman aaabe82483 libexpr: Make sure EvalState::lookupVar is inlined
This makes sure that ExprVar::eval inlines lookupVar call. In practice
this seems to reduce instruction count by ~2%, though it doesn't have
a statistically significant impact on the wall time.
2026-02-17 15:32:26 +03:00
Sergei Zimmerman a81f83604b libexpr: Add marker values to InternalType enum
This reduces the churn when changing up the order of
values in a follow-up commit. This should have been done
from the start ideally to improve readability.
2026-02-17 13:32:45 +03:00
John Ericson fefa66880a Remove suppport for not having lchown
Linux, macOS, and all 3 BSDs have it (according to man page google
search), so let's just drop this. Support for not having it was added in
d03f0d4117 in 2006, things have changed in
the last 20 years!
2026-02-16 13:40:29 -05:00
Sergei Zimmerman 6cbf80a0b0 Merge pull request #15219 from obsidiansystems/writeDerivation-lighter-read-only
Get rid of the settings-dependent `writeDerivation` wrapper
2026-02-14 21:27:52 +00:00
John EricsonandAmaan Qureshi 002cbefa9f libstore: extract ProfileDirsOptions from Settings
This commit moves `nixStateDir` and `useXDGBaseDirectories` into a dedicated `ProfileDirsOptions` struct and threads it through the profile directory functions (`profilesDir`, `rootProfilesDir`, `defaultChannelsDir`, `rootChannelsDir`, `getDefaultProfile`) so they no longer read from the global `Settings` object directly. This follows the same pattern as `LocalSettings`, `WorkerSettings`, and `NarInfoDiskCacheSettings`.

Progress on #5638

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-13 12:43:53 -05:00
John Ericson 7926a629e2 Get rid of the settings-dependent writeDerivation wrapper
It was a crude hack that this one low-level function was dependent on
the high-level read-only mode setting --- all the more so because rather
than making derivation writing fail, that setting made it silently
"succeed" why not actually writing the derivation. (Also, for context,
we didn't have an such behavior for any other store-mutating operations,
just for this one function.)

I have gotten rid of the wrapper, and updated the call sites
accordingly.

- For the ones that should remain dependent on this setting, I made this
  explicit, and added a comment.

- For others, surrounding operations assumed writability (e.g. we had
  written something before, or were about to try to read back the
  written derivation after), and so I just made those do the underlying
  `Store::writeDerivation` operation.
2026-02-12 20:26:24 -05:00
Sergei Zimmerman eae7e0151c Merge pull request #15213 from xokdvium/unhardcode-alignas-cache-line-size
Unhardcode alignas cache line size
2026-02-12 20:53:09 +00:00
Sergei Zimmerman 7352205ce9 libexpr: Replace hardcoded cache line size with std::hardware_destructive_interference_size
This expands to __GCC_DESTRUCTIVE_SIZE, which is also 64 (at least in the x86_64 stdenv).
Let the compiler decide what's the appropriate cache line size is. Also, on aarch64-darwin
the cache line size 128 bytes, so the previous fix didn't actually get rid of false sharing
reliably. Clang does this [1] [2], so it overestimates the sizes somewhat, but that's still enough
for avoiding false sharing on darwin.

[1]: https://github.com/llvm/llvm-project/blob/a289341dedf4d73955faefbb0b3c13881e13dd06/clang/lib/Frontend/InitPreprocessor.cpp#L1331-L1339
[2]: https://github.com/llvm/llvm-project/blob/6f51f8e0f93e6ff2baf0ed2f60021308920a557f/clang/lib/Basic/Targets/AArch64.h#L262-L264
2026-02-12 23:04:40 +03:00
Sergei Zimmerman df21c81191 libexpr: Fix some typos in value.hh 2026-02-12 20:51:38 +03:00
Amaan Qureshi f9300514cd globals: use StoreReference types in CLI argument handlers
The CLI flags `--from`, `--to`, `--eval-store`, and substituter URIs now
parse to `StoreReference` at the argument boundary. `fetchClosure` uses
`StoreReference::parse` instead of `parseURL`. This also adds
`operator<=>` to `StoreReference`.
2026-02-10 18:37:34 -05:00
Eelco Dolstra de6b5f60cd Replace fetchToStore cache by sourcePathToHash cache
Caching NAR hashes instead of store paths makes the cache more
general, because we can always compute the store path from the NAR
hash, but not the other way around. This is useful for lazy trees,
where we want to compute the NAR hash of an input with caching.
2026-02-05 16:45:50 +01:00
Sergei Zimmerman 47f261cc19 libutil: Add openNewFileForWrite helper function, wrap callsites
This is purely a fix to use CreateFileW in mingw builds. Also adds some
FIXMEs for suspicious symlink following on truncation that can probably
be tightened down without any problems (other than nix-channel), but for
now this is a no-op change other than consistently using O_CLOEXEC, which
is harmless.
2026-02-04 02:10:12 +03:00
John Ericson 929022c8f8 Merge pull request #15092 from NixOS/improve-error-messages
libexpr/parser: Use readable tokens in error messages instead of inte…
2026-01-27 00:01:21 +00:00
Sergei Zimmerman 68cf0a7f8a libexpr/parser: Use readable tokens in error messages instead of internal token names
Very low-hanging fruit for improving parser error messages.
2026-01-27 02:11:40 +03:00
Sergei Zimmerman d45004f5ec treewide: Apply formatter diffs
Also disable some churny formatters on some specific files.
2026-01-25 22:03:16 +03:00
Amaan Qureshi 751a0f40bc libstore: add SQLite::Settings struct for explicit configuration
Progress on #5638

Replace the SQLite constructor's mode parameter with a Settings struct
that includes both the open mode and useWAL flag. This makes the
dependency on useSQLiteWAL explicit at call sites rather than having
it read from the global settings inside the constructor.

All call sites now explicitly pass settings.useSQLiteWAL, preparing
for downstream work where stores can pass their own settings instead
of relying on the global.
2026-01-22 12:06:26 -05:00
Eelco Dolstra 4f733f736e realisePath(): Move into EvalState
This allows it to be used by primops defined outside of primops.cc.
2026-01-21 14:43:58 +01:00
Sergei Zimmerman 6dd89b5a2a libutil: Add PathFmt wrapped type for formatting fs::path, fix all double-quoting issues
This will once and for all get rid of all double-quoting issues. On windows the quoting
is doubly bad because it escaped all \ to \\, which is very bad for error messages. In
order to prevent future regression std::filesystem::path formatting now must use a special
type PathFmt (like Magenta). In the future we could even change how we render filesystem paths.
2026-01-21 06:06:19 +03:00
Eelco Dolstra a3043d991f Merge pull request #14925 from Zaczero/zaczero/trycat
Remove redundant try/catch
2026-01-06 17:42:51 +00:00
Kamil Monicz 1176d59c8a Remove redundant try/catch 2026-01-06 17:51:14 +01:00
Graham Christensen 05df7d716a Auto-replace actually existing store path hashes 2026-01-06 17:26:20 +01:00
Sergei Zimmerman 28c7e42ab5 Merge pull request #14624 from roberth/deepSeq-stack-overflow
Fix most remaining stack overflows
2026-01-02 14:53:07 +00:00
tomberek fc52891b44 Merge pull request #14892 from roberth/flake-compat-44-builtins-path-hash
Improve builtins.path docs wrt recursive, sha256
2025-12-31 02:20:56 +00:00
Eelco Dolstra 2c55c4aae4 Revert "Add builtins.imap function"
This reverts commit 4db99ea955.
2025-12-30 19:09:59 +01:00
Eelco Dolstra 80c9ad7de4 Revert "Pre-allocate small integers in builtins.{genList,imap}"
This reverts commit 24610d51f4.
2025-12-30 19:09:55 +01:00
Eelco Dolstra 24610d51f4 Pre-allocate small integers in builtins.{genList,imap} 2025-12-30 18:56:34 +01:00
Eelco Dolstra 4db99ea955 Add builtins.imap function
This allows the `imap0` and `imap1` functions (which are called
hundreds of thousands of times during NixOS evaluation) to be done
more efficiently.
2025-12-30 18:56:34 +01:00
Robert Hensing 08bcd70ecb doc: builtins.path: link to content-address docs for hash methods
The sha256 parameter documentation said "file at the path" but it
works with directories too (using NAR hashing). Link to the
content-address documentation instead of duplicating information.
2025-12-29 22:15:45 +01:00
Connor Baker edebabe9bf eval-cache: fix parsing of contexts
The members of the context were serialized with a space as the delimiter, not a semicolon.

Signed-off-by: Connor Baker <ConnorBaker01@gmail.com>
2025-12-28 15:38:31 +00:00
Kamil Monicz 048d0b6781 libexpr: avoid regex engine in getDerivations attr filtering
- getDerivations() filters attribute names with std::regex_match, which runs the regex engine for every attribute visited during nixpkgs scanning.
- BM_GetDerivationsAttrScan/10000_mean: 3.338 ms → 1.506 ms (≈ -54.9%)
2025-12-19 02:10:11 +01:00
Kamil Monicz 0c8751d3f4 libexpr: avoid std::regex copies on RegexCache hits
- RegexCache::get() returned std::regex by value, copying the compiled regex on every cache hit.
- Store the compiled regex behind std::shared_ptr<const std::regex> and return the shared pointer instead, so callers reuse the same compiled object.
- BM_EvalManyBuiltinsMatchSameRegex_mean improved about 8%
2025-12-19 02:10:11 +01:00
Sergei Zimmerman e0830681e2 Merge pull request #14552 from hsjobeki/docs-sort
docs: add explanation to sort primop
2025-12-16 20:31:12 +00:00
Sergei Zimmerman 54d2268d84 treewide: Follow rule of five
Good to explicitly declare things to not accidentally do twice the work by
preventing that kind of misuse.
This is essentially just cppcoreguidelines-special-member-functions lint
in clang-tidy.
2025-12-15 01:35:20 +03:00
Eelco Dolstra 02055c5a48 addPath(): Restore catching InvalidPathError 2025-12-08 19:17:07 +01:00
Eelco Dolstra c080c4ca56 builtins.path: Propagate references from derivation outputs
This restores compatibility with Nix 2.18, which behaved this
way. Note that this doesn't scan for the actually visible references.

Unlike in Nix 2.18, we only do this for paths with context, i.e. it
applies to `builtins.storePath "/nix/store/bla..."` but not
`"/nix/store/bla..."`. We don't want the latter because it shouldn't
matter whether a source file happens to be in the Nix store.
2025-12-08 19:16:44 +01:00
Eelco Dolstra 20fc54c00d Introduce AttrPath type
This is basically an alias for std::vector<Symbol>.
2025-12-05 13:41:59 +01:00
Eelco Dolstra 294e68a3f6 Rename AttrPath -> AttrSelectionPath 2025-12-05 12:57:19 +01:00
Johannes Kirschbauer af6326dfa4 docs: add explanation to sort primop 2025-12-05 10:03:43 +01:00
Robert Hensing b69c565fdb Document scopedImport builtin 2025-12-04 01:40:06 +01:00
Ubuntu e761a9fb6d Use std::filesystem::path instead of Path in libexpr. 2025-11-26 11:18:38 -05:00
John Ericson 31ce0c8169 Merge pull request #14649 from vinayakankugoyal/path
Use std::filesystem::path instead of Path in libexpr.
2025-11-26 01:23:16 +00:00
Ubuntu 697b068756 Use std::filesystem::path instead of Path in libexpr. 2025-11-26 00:22:26 +00:00