Compare commits

...

48 Commits

Author SHA1 Message Date
Eelco Dolstra
490cb842cc Add release note 2026-02-18 22:50:37 +01:00
Eelco Dolstra
6992698ac5 builtins.getFlake: Support path values
This allows doing `builtins.getFlake ./subflake` instead of ugly hacks.
2026-02-18 22:12:09 +01:00
Eelco Dolstra
9868310d6f Add test for builtins.getFlake 2026-02-18 21:58:36 +01:00
John Ericson
08ce8dbfba Merge pull request #15283 from obsidiansystems/filesytem-error-improvements
Filesystem error improvements
2026-02-18 18:29:44 +00:00
John Ericson
bbcf2041e1 File system error improvements
- Make `descriptorToPath` cross-platform (renamed from
  `windows::handleToPath`). Uses `/proc/self/fd` on Linux and
  `F_GETPATH` on macOS. Add `HAVE_F_GETPATH` meson check.

  This is based on 7226a116a0, which was
  removed in 479c356510, but is now
  introduced more judiciously.

- Unix error messages in `readFull`, `writeFull`, `readLine` now include
  file paths via `descriptorToPath`.

- Convert `std::filesystem::filesystem_error` to `SystemError`

  Wrappers like `readLink`, `createDirs`, `DirectoryIterator`, etc. now
  catch `std::filesystem::filesystem_error` and rethrow as `SystemError`
  with the error code preserved. This ensures consistent exception types
  throughout the codebase.

  Call sites that previously caught `filesystem_error` and rethrew with
  `throw;` now throw `SystemError(e.code(), ...)` instead.

  Some call sites can stop catching `filesystem_error` at all,
  because they only call the wrapped functions.

- Rework `SystemError` constructors to auto-append error message

  The public `SystemError(std::error_code, ...)` constructor now
  automatically appends `errorCode.message()` to the error message.
  A protected constructor takes an explicit error message string for
  subclasses.

  `SysError` delegates to the protected constructor with `strerror(errNo)`.
  `WinError` delegates with `renderError(lastError)` (now static).

  This removes the need to manually append `e.code().message()` at call
  sites when converting `filesystem_error` to `SystemError`.

- Use perfect forwarding (`Args &&...` with `std::forward`) consistently
  in `BaseError`, `SystemError`, `SysError`, and `WinError` constructors.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-02-18 12:29:11 -05:00
John Ericson
96bcf5928f Merge pull request #15273 from NixOS/more-robust-ubsan-macro
libutil: More robust check for NIX_UBSAN_ENABLED
2026-02-18 16:15:26 +00:00
Sergei Zimmerman
db853cf4fb libutil: More robust check for NIX_UBSAN_ENABLED
In 3df91bea62 I forgot that the header
might get included out-of-tree with -Wundef. Let's make this a public
config option for libutil as it can affect function bodies in headers.
2026-02-18 17:33:51 +03:00
John Ericson
663db5b48b Merge pull request #15278 from puffnfresh/windows/bar-log-format
Windows: don't use bar log format
2026-02-18 05:14:27 +00:00
Brian McKenna
c486e78235 Windows: don't use bar log format
Relies on terminal features that don't always work on Windows.
2026-02-18 14:35:35 +11:00
John Ericson
4fff871383 Merge pull request #15274 from obsidiansystems/tryToBuild-raii
libstore: refactor `tryToBuild` with coroutine lambdas and RAII
2026-02-17 22:10:42 +00:00
Amaan Qureshi
b9acea908e libstore: refactor tryToBuild with coroutine lambdas and RAII
`tryToBuild` threaded a single `PathLocks outputLocks` by reference
across all build phases and managed a `std::unique_ptr<Activity> actLock`
with explicit `if (!actLock)` guards and `.reset()` calls around the hook
retry loop. This commit introduces coroutine lambdas for the three phases:
`tryHookLoop` owns a `PathLocks` in a scoped block for the first attempt
and per-iteration in the retry loop, `tryBuildLocally` acquires its own
`PathLocks`, and the hook-wait `Activity` is a stack variable scoped to
the postpone block.
2026-02-17 16:23:44 -05:00
John Ericson
c3f0670b4e Merge pull request #15266 from obsidiansystems/fix-maxjobs-error
libstore: structured diagnostics for local build rejection
2026-02-17 18:39:58 +00:00
Amaan Qureshi
7cd4359a8b libstore: structured diagnostics for local build rejection
When `max-jobs = 0` and no remote builders are available, Nix reported
"required system or feature not available" even though the system and
features matched fine. The `canBuildLocally` lambda returned a plain
`bool`, conflating a configuration knob (`max-jobs = 0`) with actual
incompatibility (wrong platform, missing features). It also short-circuited
on the first failing check, so a user with both a platform mismatch and
missing features would only see one of the two.

This commit replaces the bool with a `LocalBuildRejection` struct whose
`WrongLocalStore` variant collects all applicable failures into
`badPlatform`, `missingFeatures`, and an orthogonal `maxJobsZero` flag.
Platform mismatch and missing features now produce separate error
paragraphs, and all applicable reasons appear in a single message.

The local-build capability check also now returns
`std::variant<LocalBuildCapability, LocalBuildRejection>`, bundling
the `LocalStore &` and optional `ExternalBuilder *` together.
2026-02-17 12:54:24 -05:00
John Ericson
6e725093e6 Merge pull request #15143 from obsidiansystems/rootless-daemon-minimal
Support garbage collection in external daemon
2026-02-17 16:53:06 +00:00
Artemis Tosini
96fef69755 libstore: support searching for roots from an external daemon
This comes in two parts: a `nix store roots-daemon` command that
can run as root and list runtime roots,
and client logic to find runtime roots for a `LocalStore` by connecting
to that daemon.

This may be useful with an unprivileged nix daemon, as it would
otherwise be unable to find runtime roots from process open files
and maps.
2026-02-17 10:42:04 -05:00
John Ericson
16b0bb7548 Merge pull request #15270 from NixOS/inline-lookup-var
libexpr: Make sure `EvalState::lookupVar` is inlined
2026-02-17 15:12:00 +00:00
John Ericson
ebcd31e434 Merge pull request #15271 from NixOS/faster-type-internal-type
libexpr: Optimise `Value::type()`, `ValueStorage::getInternalType()`
2026-02-17 15:11:23 +00:00
John Ericson
f940ab5146 Merge pull request #15265 from xokdvium/libgit2-error
libfetchers/git-utils: Add GitError class for deduplicating error…
2026-02-17 15:06:31 +00:00
Sergei Zimmerman
3df91bea62 libexpr: Optimise Value::type(), ValueStorage::getInternalType()
Using nix::unreachable() in getInternalType() and type() turns
out to be quite expensive and prevents inlining. Also Value::type
got compiled to a jump table which has a high overhead from indirect
jumps. Using an explicit lookup table turns out to be more efficient.

This does mean that we lose out on nice diagnostics from nix::unreachable
calls, but this code is probably one of the hottests functions in the whole
evaluator, so I think the tradeoff is worth it. The nixUnreachableWhenHardened
boils down to nix::unreachable when UBSan is enabled so we still have good
coverage there.
2026-02-17 16:50:07 +03:00
Sergei Zimmerman
aaabe82483 libexpr: Make sure EvalState::lookupVar is inlined
This makes sure that ExprVar::eval inlines lookupVar call. In practice
this seems to reduce instruction count by ~2%, though it doesn't have
a statistically significant impact on the wall time.
2026-02-17 15:32:26 +03:00
Sergei Zimmerman
a81f83604b libexpr: Add marker values to InternalType enum
This reduces the churn when changing up the order of
values in a follow-up commit. This should have been done
from the start ideally to improve readability.
2026-02-17 13:32:45 +03:00
Sergei Zimmerman
c1bfa30303 libfetchers/git-utils: Add GitError class for deduplicating error message printing
Consolidates all the error message formatting in one place. It was very weird
and tiring to remember to call git_error_last() in all the places.
2026-02-17 12:18:37 +03:00
John Ericson
509694d5f0 Merge pull request #15267 from obsidiansystems/fix-external-builders-path
tests: quote `PATH` in external-builders test heredoc
2026-02-17 05:53:17 +00:00
Amaan Qureshi
0b7629da08 tests: quote PATH in external-builders test heredoc
The external-builders test expands `$PATH` into a heredoc without quotes,
so any `PATH` entry containing spaces causes bash to parse the line as a
command instead of an assignment, failing the test.
2026-02-16 23:20:10 -05:00
Sergei Zimmerman
e7e5eaaa37 Merge pull request #15255 from obsidiansystems/fix-repl-tab-crash
repl: catch all errors during tab completion
2026-02-16 21:58:22 +00:00
Jörg Thalheim
974545290e Merge pull request #15252 from obsidiansystems/fix-docker-compression
upload-release: disable containerd image store to preserve gzip layer compression
2026-02-16 21:26:31 +00:00
Amaan Qureshi
be6e72f11b repl: prevent exceptions from escaping editline callbacks
The tab completion handler in `completePrefix` only caught `ParseError`,
`EvalError`, `BadURL`, and `FileNotFound`. Other error types like
`JSONParseError` (which derives from `Error`, not `EvalError`) escaped
the catch block and propagated through editline's C code as undefined
behavior, crashing the REPL. This happened when tab-completing
expressions like `(builtins.fromJSON "invalid").` where evaluation
throws a non-`EvalError` exception.

This commit marks `completionCallback` and `listPossibleCallback` as
`noexcept` with function-try-blocks that catch all exceptions at the
C/C++ boundary, preventing any exception from reaching editline.

Fixes #15133.
2026-02-16 16:02:37 -05:00
Sergei Zimmerman
27782fcc42 Merge pull request #15253 from obsidiansystems/fix-url-assertion
libflake: fix assertion crash when malformed URL falls through to path scheme
2026-02-16 20:49:49 +00:00
John Ericson
06d4d5779f Merge pull request #15251 from obsidiansystems/file-system-at
Split `file-system-at.{cc,hh}` from `file-descriptor.{cc,hh}`
2026-02-16 20:10:28 +00:00
Amaan Qureshi
a32cd16f64 libflake: fix assertion crash when malformed URL falls through to path scheme
When a URL like `github:nixos/nixpkgs/nixpkgs.git?ref=<hash>` (using
`ref` instead of `rev`) failed the github input scheme, it fell
through to `parsePathFlakeRefWithFragment` which constructed a `path:`
`ParsedURL` with an empty authority but a relative path. This violated
RFC 3986 section 3.3 (authority present requires path starting with
`/`), causing an assertion failure in `renderAuthorityAndPath` when
`PathInputScheme` tried to format the URL for an error message.

This commit only sets the authority on absolute paths. Relative paths
get `std::nullopt` for authority, which is the correct representation
per the URL spec.

Fixes #15196. Fixes #14830.
2026-02-16 15:10:19 -05:00
Sergei Zimmerman
46a4a554ca Merge pull request #15237 from xokdvium/add-missing-temp-roots
Add missing temproots for cached sources and existing derivations
2026-02-16 19:35:15 +00:00
John Ericson
cc0b489967 Merge pull request #15250 from obsidiansystems/assume-lchown
Remove suppport for not having `lchown`
2026-02-16 19:29:08 +00:00
John Ericson
af7e585009 Split file-system-at.{cc,hh} from file-descriptor.{cc,hh}
`file-descriptor.{cc,hh}` was getting too big, split out
`file-system-at.{cc,hh}` for the FD-based file system stuff,
`file-descriptor.{cc,hh}` will only be for the fundamental primitives
that are file-system agnostic and work on almost all file types.

Review with `git show --color-moved` to see that this is indeed all
moving.
2026-02-16 14:21:52 -05:00
Amaan Qureshi
2ccb8a9a56 upload-release: disable containerd image store to preserve gzip layer compression
Docker 28+ defaults to the containerd image store, which pushes layers
uncompressed instead of gzip. The GHA runner image updated Docker to
29.x (actions/runner-images#13633), causing the `nixos/nix:2.33.3`
image to balloon from 138 MB to 505 MB, with all 70 layers pushed as
`application/vnd.docker.image.rootfs.diff.tar` instead of `.tar.gzip`.
OCI clients that only support gzip (e.g. `go-containerregistry`, used
by Concourse CI) fail with "gzip: invalid header".

This commit disables the containerd snapshotter in the release workflow
before any Docker operations, restoring the classic storage driver that
preserves gzip compression through the `docker load` / `docker push`
pipeline.

Fixes #15246
2026-02-16 14:08:08 -05:00
John Ericson
fefa66880a Remove suppport for not having lchown
Linux, macOS, and all 3 BSDs have it (according to man page google
search), so let's just drop this. Support for not having it was added in
d03f0d4117 in 2006, things have changed in
the last 20 years!
2026-02-16 13:40:29 -05:00
John Ericson
a53391fd0e Merge pull request #15247 from roberth/clarify-ref-upcasting
Better `ref` casting DX
2026-02-16 17:09:16 +00:00
Robert Hensing
771421a34e fix(ref): improve cast exception type and add demangled type names
When ref::cast() fails, the error message was cryptic ("null pointer
cast to ref"). Now it throws a proper bad_ref_cast (a std::bad_cast
subclass) with a clear message showing the actual types involved:

    ref<nix::Base> cannot be cast to ref<nix::Derived>

This also adds a demangle.hh utility.
2026-02-16 17:07:40 +01:00
Robert Hensing
5aaa0cc4a6 refactor(ref): clarify implicit conversion semantics with requires clause
ref<Derived> was already implicitly convertible to ref<Base>, but the
mechanism was unclear and error messages for rejected downcasts were
more cryptic than necessary. This change:

- Adds RefImplicitlyUpcastableTo concept to constrain the conversion
  operator, making the intent explicit and improving error messages
- Documents .cast() and .dynamic_pointer_cast() as alternatives for
  explicit downcasting
- Adds unit tests for covariance behavior
2026-02-16 16:43:08 +01:00
John Ericson
0749ec4e55 Merge pull request #15230 from obsidiansystems/new-wine
flake: Use Wine 11 for running mingw tests
2026-02-15 16:41:52 +00:00
Artemis Tosini
4cc97150df flake: Use Wine 11 for running mingw tests
Set wine_11 as the emulator for Windows.
2026-02-15 10:56:02 -05:00
John Ericson
2bbd1094a2 flake.lock: Update Nixpkgs
Flake lock file updates:

• Updated input 'nixpkgs':
    'https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz?narHash=sha256-Xu%2B7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4%3D' (2026-01-22)
  → 'https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz?narHash=sha256-WoiezqWJQ3OHILah%2Bp6rzNXdJceEAmAhyDFZFZ6pZzY%3D' (2026-02-14)

This will be needed to get Wine 11.
2026-02-15 10:53:15 -05:00
John Ericson
95251a51dd Merge pull request #15241 from obsidiansystems/fix-isindir
libutil: fix `isInDir` rejecting paths starting with dot
2026-02-15 15:52:37 +00:00
John Ericson
02d9f4ecb4 Merge pull request #15239 from xokdvium/fix-warnings-no-intereference-size
meson: Only enable -Wno-interference-size with GCC
2026-02-15 15:06:54 +00:00
John Ericson
3269c71e9d Merge pull request #15240 from xokdvium/fix-mtls-redirect-test
libstore-tests: Fix mTLS test for redirect, correctly propagate tries
2026-02-15 15:04:50 +00:00
Amaan Qureshi
ad0055e67c libutil: fix isInDir rejecting paths starting with dot
The old check rejected any relative path whose first character was a
dot, producing false negatives for valid descendants like `.ssh` or
`.config`. This commit changes the logic such that now it inspects the
first path component via `path::begin()`, only rejects `.` and `..`
rather than anything dot-prefixed. Fixes #15207.
2026-02-15 10:04:08 -05:00
Sergei Zimmerman
d3d63a4b5b libstore-tests: Fix mTLS test for redirect, correctly propagate tries
The fake cacert didn't have subjectAltName for 127.0.0.1, so the test
was failing for a different reason. Also `tries` setting wasn't being respected.
There's no callsite specifying it in the request, so just use the one specified
in the FileTransferSettings and remove the fields from the FileTransferRequest.
2026-02-15 00:08:21 +03:00
Sergei Zimmerman
6a5ee08737 meson: Only enable -Wno-interference-size with GCC
Clang doesn't recognise this option.
2026-02-14 23:42:28 +03:00
Sergei Zimmerman
ac2dd58b6f Add missing temproots for cached sources and existing derivations 2026-02-14 12:09:24 +03:00
78 changed files with 1961 additions and 1138 deletions

View File

@@ -39,6 +39,17 @@ jobs:
role-to-assume: "arn:aws:iam::080433136561:role/nix-release"
role-session-name: nix-release-oidc-${{ github.run_id }}
aws-region: eu-west-1
- name: Disable containerd image store
run: |
# Docker 28+ defaults to the containerd image store, which
# pushes layers uncompressed instead of gzip. OCI clients
# that only support gzip (e.g. go-containerregistry) fail
# with "gzip: invalid header". Disabling the containerd
# snapshotter restores the classic storage driver, which
# preserves gzip-compressed layers through the
# `docker load` / `docker push` pipeline.
echo '{"features":{"containerd-snapshotter":false}}' | sudo tee /etc/docker/daemon.json > /dev/null
sudo systemctl restart docker
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:

View File

@@ -0,0 +1,6 @@
---
synopsis: "`builtins.getFlake` now supports path values"
prs: [15290]
---
`builtins.getFlake` now accepts path values in addition to flakerefs, allowing you to write `builtins.getFlake ./subflake` instead of having to use ugly workarounds to construct a pure flakeref.

View File

@@ -0,0 +1,11 @@
---
synopsis: New command `nix store roots-daemon` for serving GC roots
prs: [15143]
---
New command [`nix store roots-daemon`](@docroot@/command-ref/new-cli/nix3-store-roots-daemon.md) runs a daemon that serves garbage collector roots over a Unix domain socket.
It enables the garbage collector to discover runtime roots when the main Nix daemon doesn't have `CAP_SYS_PTRACE` capability and therefore cannot scan `/proc`.
The garbage collector can be configured to use this daemon via the [`use-roots-daemon`](@docroot@/store/types/local-store.md#store-experimental-option-use-roots-daemon) store setting.
This feature requires the [`local-overlay-store` experimental feature](@docroot@/development/experimental-features.md#xp-feature-local-overlay-store).

8
flake.lock generated
View File

@@ -63,11 +63,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1769089682,
"narHash": "sha256-Xu+7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4=",
"rev": "078d69f03934859a181e81ba987c2bb033eebfc5",
"lastModified": 1771043024,
"narHash": "sha256-WoiezqWJQ3OHILah+p6rzNXdJceEAmAhyDFZFZ6pZzY=",
"rev": "3aadb7ca9eac2891d52a9dec199d9580a6e2bf44",
"type": "tarball",
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz"
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz"
},
"original": {
"type": "tarball",

View File

@@ -115,6 +115,9 @@
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
}
// lib.optionalAttrs (crossSystem == "x86_64-w64-mingw32") {
emulator = pkgs: "${pkgs.buildPackages.wineWow64Packages.stable_11}/bin/wine";
};
overlays = [
(overlayFor (pkgs: pkgs.${stdenv}))

View File

@@ -27,13 +27,19 @@ add_project_arguments(
'-Wignored-qualifiers',
'-Wimplicit-fallthrough',
'-Wno-deprecated-declarations',
'-Wno-interference-size', # Used for C++ ABI only. We don't provide any guarantees about different march tunings.
language : 'cpp',
)
# GCC doesn't benefit much from precompiled headers.
do_pch = cxx.get_id() == 'clang'
if cxx.get_id() == 'gcc'
add_project_arguments(
'-Wno-interference-size', # Used for C++ ABI only. We don't provide any guarantees about different march tunings.
language : 'cpp',
)
endif
# This is a clang-only option for improving build times.
# It forces the instantiation of templates in the PCH itself and
# not every translation unit it's included in.

View File

@@ -40,8 +40,8 @@ void sigintHandler(int signo)
static detail::ReplCompleterMixin * curRepl; // ugly
#if !USE_READLINE
static char * completionCallback(char * s, int * match)
{
static char * completionCallback(char * s, int * match) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
@@ -73,10 +73,12 @@ static char * completionCallback(char * s, int * match)
*match = 0;
return nullptr;
} catch (...) {
return nullptr;
}
static int listPossibleCallback(char * s, char *** avp)
{
static int listPossibleCallback(char * s, char *** avp) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() > (std::numeric_limits<int>::max() / sizeof(char *)))
@@ -105,6 +107,9 @@ static int listPossibleCallback(char * s, char *** avp)
*avp = vp;
return ac;
} catch (...) {
*avp = nullptr;
return 0;
}
#endif

View File

@@ -710,7 +710,7 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
try {
cwd = std::filesystem::current_path();
} catch (std::filesystem::filesystem_error & e) {
throw SysError("cannot determine current working directory");
throw SystemError(e.code(), "cannot determine current working directory");
}
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, cwd.string(), true);

View File

@@ -13,8 +13,7 @@ TEST_F(ValueTest, unsetValue)
{
Value unsetValue;
ASSERT_EQ(false, unsetValue.isValid());
ASSERT_EQ(nThunk, unsetValue.type(true));
ASSERT_DEATH(unsetValue.type(), "");
ASSERT_EQ(nThunk, unsetValue.type</*invalidIsThunk=*/true>());
}
TEST_F(ValueTest, vInt)

View File

@@ -466,7 +466,7 @@ void EvalState::addConstant(const std::string & name, Value * v, Constant info)
We might know the type of a thunk in advance, so be allowed
to just write it down in that case. */
if (auto gotType = v->type(true); gotType != nThunk)
if (auto gotType = v->type</*invalidIsThunk=*/true>(); gotType != nThunk)
assert(info.type == gotType);
/* Install value the base environment. */
@@ -885,7 +885,7 @@ void Value::mkPath(const SourcePath & path, EvalMemory & mem)
mkPath(&*path.accessor, StringData::make(mem, path.path.abs()));
}
inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
[[gnu::always_inline]] inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
{
for (auto l = var.level; l; --l, env = env->up)
;
@@ -903,11 +903,11 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
while (1) {
forceAttrs(*env->values[0], fromWith->pos, "while evaluating the first subexpression of a with expression");
if (auto j = env->values[0]->attrs()->get(var.name)) {
if (countCalls)
if (countCalls) [[unlikely]]
attrSelects[j->pos]++;
return j->value;
}
if (!fromWith->parentWith)
if (!fromWith->parentWith) [[unlikely]]
error<UndefinedVarError>("undefined variable '%1%'", symbols[var.name])
.atPos(var.pos)
.withFrame(*env, var)

View File

@@ -35,7 +35,7 @@ class BindingsBuilder;
* about how this is mapped into the alignment bits to save significant memory.
* This also restricts the number of internal types represented with distinct memory layouts.
*/
typedef enum {
enum InternalType {
tUninitialized = 0,
/* layout: Single/zero field payload */
tInt = 1,
@@ -46,16 +46,20 @@ typedef enum {
tPrimOp,
tAttrs,
/* layout: Pair of pointers payload */
tListSmall,
tFirstPairOfPointers,
tListSmall = tFirstPairOfPointers,
tPrimOpApp,
tApp,
tThunk,
tLambda,
tLastPairOfPointers = tLambda,
/* layout: Single untaggable field */
tListN,
tFirstSingleUntaggable,
tListN = tFirstSingleUntaggable,
tString,
tPath,
} InternalType;
tNumberOfInternalTypes, // Must be last
};
/**
* This type abstracts over all actual value types in the language,
@@ -633,7 +637,7 @@ class alignas(16)
template<InternalType type, typename T, typename U>
void setPairOfPointersPayload(T * firstPtrField, U * secondPtrField) noexcept
{
static_assert(type >= tListSmall && type <= tLambda);
static_assert(type >= tFirstPairOfPointers && type <= tLastPairOfPointers);
{
auto firstFieldPayload = std::bit_cast<PackedPointer>(firstPtrField);
assertAligned(firstFieldPayload);
@@ -642,7 +646,7 @@ class alignas(16)
{
auto secondFieldPayload = std::bit_cast<PackedPointer>(secondPtrField);
assertAligned(secondFieldPayload);
payload[1] = (type - tListSmall) | secondFieldPayload;
payload[1] = (type - tFirstPairOfPointers) | secondFieldPayload;
}
}
@@ -670,11 +674,11 @@ protected:
case pdListN:
case pdString:
case pdPath:
return static_cast<InternalType>(tListN + (pd - pdListN));
return static_cast<InternalType>(tFirstSingleUntaggable + (pd - pdListN));
case pdPairOfPointers:
return static_cast<InternalType>(tListSmall + (payload[1] & discriminatorMask));
return static_cast<InternalType>(tFirstPairOfPointers + (payload[1] & discriminatorMask));
[[unlikely]] default:
unreachable();
nixUnreachableWhenHardened();
}
}
@@ -1027,7 +1031,7 @@ private:
T getStorage() const noexcept
{
if (getInternalType() != detail::payloadTypeToInternalType<T>) [[unlikely]]
unreachable();
nixUnreachableWhenHardened();
T out;
ValueStorage::getStorage(out);
return out;
@@ -1079,45 +1083,44 @@ public:
* Returns the normal type of a Value. This only returns nThunk if
* the Value hasn't been forceValue'd
*
* @param invalidIsThunk Instead of aborting an an invalid (probably
* @param invalidIsThunk Instead of UB an an invalid (probably
* 0, so uninitialized) internal type, return `nThunk`.
*/
inline ValueType type(bool invalidIsThunk = false) const
template<bool invalidIsThunk = false>
inline ValueType type() const
{
switch (getInternalType()) {
case tUninitialized:
break;
case tInt:
return nInt;
case tBool:
return nBool;
case tString:
return nString;
case tPath:
return nPath;
case tNull:
return nNull;
case tAttrs:
return nAttrs;
case tListSmall:
case tListN:
return nList;
case tLambda:
case tPrimOp:
case tPrimOpApp:
return nFunction;
case tExternal:
return nExternal;
case tFloat:
return nFloat;
case tThunk:
case tApp:
return nThunk;
/* Explicit lookup table. switch() might compile down (and it does at least with GCC 14)
to a jump table. Let's help the compiler a bit here. */
static constexpr auto table = [] {
std::array<ValueType, tNumberOfInternalTypes> t{};
t[tUninitialized] = nThunk;
t[tInt] = nInt;
t[tBool] = nBool;
t[tNull] = nNull;
t[tFloat] = nFloat;
t[tExternal] = nExternal;
t[tAttrs] = nAttrs;
t[tPrimOp] = nFunction;
t[tLambda] = nFunction;
t[tPrimOpApp] = nFunction;
t[tApp] = nThunk;
t[tThunk] = nThunk;
t[tListSmall] = nList;
t[tListN] = nList;
t[tString] = nString;
t[tPath] = nPath;
return t;
}();
auto it = getInternalType();
if (it == tUninitialized || it >= tNumberOfInternalTypes) [[unlikely]] {
if constexpr (invalidIsThunk)
return nThunk;
else
nixUnreachableWhenHardened();
}
if (invalidIsThunk)
return nThunk;
else
unreachable();
return table[it];
}
/**

View File

@@ -28,7 +28,7 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/big-objs')
# Check for each of these functions, and create a define like `#define HAVE_LCHOWN 1`.
# Check for each of these functions, and create a define like `#define HAVE_SYSCONF 1`.
check_funcs = [
'sysconf',
]

View File

@@ -47,6 +47,12 @@ std::pair<StorePath, Hash> fetchToStore2(
auto hash = Hash::parseSRI(fetchers::getStrAttr(*res, "hash"));
auto storePath =
store.makeFixedOutputPathFromCA(name, ContentAddressWithReferences::fromParts(method, hash, {}));
/* Add a temproot before the call to isValidPath to prevent accidental GC in case the
input is cached. Note that this must be done before to avoid races. */
if (mode != FetchMode::DryRun)
store.addTempRoot(storePath);
if (mode == FetchMode::DryRun || store.isValidPath(storePath)) {
debug(
"source path '%s' cache hit in '%s' (hash '%s')",

View File

@@ -74,6 +74,29 @@ namespace nix {
struct GitSourceAccessor;
struct GitError : public Error
{
template<typename... Ts>
GitError(const git_error & error, Ts &&... args)
: Error("")
{
auto hf = HintFmt(std::forward<Ts>(args)...);
err.msg = HintFmt("%1%: %2% (libgit2 error code = %3%)", Uncolored(hf.str()), error.message, error.klass);
}
template<typename... Ts>
GitError(Ts &&... args)
: GitError(
[]() -> const git_error & {
const git_error * p = git_error_last();
assert(p && "git_error_last() is unexpectedly null");
return *p;
}(),
std::forward<Ts>(args)...)
{
}
};
typedef std::unique_ptr<git_repository, Deleter<git_repository_free>> Repository;
typedef std::unique_ptr<git_tree_entry, Deleter<git_tree_entry_free>> TreeEntry;
typedef std::unique_ptr<git_tree, Deleter<git_tree_free>> Tree;
@@ -106,7 +129,7 @@ static void initLibGit2()
static std::once_flag initialized;
std::call_once(initialized, []() {
if (git_libgit2_init() < 0)
throw Error("initialising libgit2: %s", git_error_last()->message);
throw GitError("initialising libgit2");
});
}
@@ -114,7 +137,7 @@ static git_oid hashToOID(const Hash & hash)
{
git_oid oid;
if (git_oid_fromstr(&oid, hash.gitRev().c_str()))
throw Error("cannot convert '%s' to a Git OID", hash.gitRev());
throw GitError("cannot convert '%s' to a Git OID", hash.gitRev());
return oid;
}
@@ -122,8 +145,7 @@ static Object lookupObject(git_repository * repo, const git_oid & oid, git_objec
{
Object obj;
if (git_object_lookup(Setter(obj), repo, &oid, type)) {
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return obj;
}
@@ -133,8 +155,7 @@ static T peelObject(git_object * obj, git_object_t type)
{
T obj2;
if (git_object_peel((git_object **) (typename T::pointer *) Setter(obj2), obj, type)) {
auto err = git_error_last();
throw Error("peeling Git object '%s': %s", *git_object_id(obj), err->message);
throw Error("peeling Git object '%s'", *git_object_id(obj));
}
return obj2;
}
@@ -144,7 +165,7 @@ static T dupObject(typename T::pointer obj)
{
T obj2;
if (git_object_dup((git_object **) (typename T::pointer *) Setter(obj2), (git_object *) obj))
throw Error("duplicating object '%s': %s", *git_object_id((git_object *) obj), git_error_last()->message);
throw GitError("duplicating object '%s'", *git_object_id((git_object *) obj));
return obj2;
}
@@ -216,7 +237,7 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
Repository tmpRepo;
if (git_repository_init(Setter(tmpRepo), tmpDir.string().c_str(), options.bare))
throw Error("creating Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("creating Git repository %s", PathFmt(path));
try {
std::filesystem::rename(tmpDir, path);
} catch (std::filesystem::filesystem_error & e) {
@@ -226,7 +247,8 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
|| e.code() == std::errc::directory_not_empty) {
return;
} else
throw SysError("moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
throw SystemError(
e.code(), "moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
}
// we successfully moved the repository, so the temporary directory no longer exists.
delTmpDir.cancel();
@@ -266,7 +288,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
initRepoAtomically(path, options);
if (git_repository_open(Setter(repo), path.string().c_str()))
throw Error("opening Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("opening Git repository %s", PathFmt(path));
ObjectDb odb;
if (options.packfilesOnly) {
@@ -279,28 +301,28 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
*/
if (git_odb_new(Setter(odb)))
throw Error("creating Git object database: %s", git_error_last()->message);
throw GitError("creating Git object database");
if (git_odb_backend_pack(&packBackend, (path / "objects").string().c_str()))
throw Error("creating pack backend: %s", git_error_last()->message);
throw GitError("creating pack backend");
if (git_odb_add_backend(odb.get(), packBackend, 1))
throw Error("adding pack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding pack backend to Git object database");
} else {
if (git_repository_odb(Setter(odb), repo.get()))
throw Error("getting Git object database: %s", git_error_last()->message);
throw GitError("getting Git object database");
}
// mempack_backend will be owned by the repository, so we are not expected to free it ourselves.
if (git_mempack_new(&mempackBackend))
throw Error("creating mempack backend: %s", git_error_last()->message);
throw GitError("creating mempack backend");
if (git_odb_add_backend(odb.get(), mempackBackend, 999))
throw Error("adding mempack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding mempack backend to Git object database");
if (options.packfilesOnly) {
if (git_repository_set_odb(repo.get(), odb.get()))
throw Error("setting Git object database: %s", git_error_last()->message);
throw GitError("setting Git object database");
}
}
@@ -340,7 +362,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Indexer indexer;
git_indexer_progress stats;
if (git_indexer_new(Setter(indexer), pack_dir_path.c_str(), 0, nullptr, nullptr))
throw Error("creating git packfile indexer: %s", git_error_last()->message);
throw GitError("creating git packfile indexer");
// TODO: provide index callback for checkInterrupt() termination
// though this is about an order of magnitude faster than the packbuilder
@@ -348,15 +370,15 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
constexpr size_t chunkSize = 128 * 1024;
for (size_t offset = 0; offset < buf.size; offset += chunkSize) {
if (git_indexer_append(indexer.get(), buf.ptr + offset, std::min(chunkSize, buf.size - offset), &stats))
throw Error("appending to git packfile index: %s", git_error_last()->message);
throw GitError("appending to git packfile index");
checkInterrupt();
}
if (git_indexer_commit(indexer.get(), &stats))
throw Error("committing git packfile index: %s", git_error_last()->message);
throw GitError("committing git packfile index");
if (git_mempack_reset(mempackBackend))
throw Error("resetting git mempack backend: %s", git_error_last()->message);
throw GitError("resetting git mempack backend");
checkInterrupt();
}
@@ -449,7 +471,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
void setRemote(const std::string & name, const std::string & url) override
{
if (git_remote_set_url(*this, name.c_str(), url.c_str()))
throw Error("setting remote '%s' URL to '%s': %s", name, url, git_error_last()->message);
throw GitError("setting remote '%s' URL to '%s'", name, url);
}
Hash resolveRef(std::string ref) override
@@ -462,7 +484,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
// an object_id.
std::string peeledRef = ref + "^{commit}";
if (git_revparse_single(Setter(object), *this, peeledRef.c_str()))
throw Error("resolving Git reference '%s': %s", ref, git_error_last()->message);
throw GitError("resolving Git reference '%s'", ref);
auto oid = git_object_id(object.get());
return toHash(*oid);
}
@@ -471,11 +493,11 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
{
GitConfig config;
if (git_config_open_ondisk(Setter(config), configFile.string().c_str()))
throw Error("parsing .gitmodules file: %s", git_error_last()->message);
throw GitError("parsing .gitmodules file");
ConfigIterator it;
if (git_config_iterator_glob_new(Setter(it), config.get(), "^submodule\\..*\\.(path|url|branch)$"))
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
StringMap entries;
@@ -484,7 +506,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto err = git_config_next(&entry, it.get())) {
if (err == GIT_ITEROVER)
break;
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
}
entries.emplace(entry->name + 10, entry->value);
}
@@ -521,7 +543,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
git_oid headRev;
if (auto err = git_reference_name_to_id(&headRev, *this, "HEAD")) {
if (err != GIT_ENOTFOUND)
throw Error("resolving HEAD: %s", git_error_last()->message);
throw GitError("resolving HEAD");
} else
info.headRev = toHash(headRev);
@@ -544,7 +566,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
options.flags |= GIT_STATUS_OPT_INCLUDE_UNMODIFIED;
options.flags |= GIT_STATUS_OPT_EXCLUDE_SUBMODULES;
if (git_status_foreach_ext(*this, &options, &statusCallbackTrampoline, &statusCallback))
throw Error("getting working directory status: %s", git_error_last()->message);
throw GitError("getting working directory status");
/* Get submodule info. */
auto modulesFile = path / ".gitmodules";
@@ -587,8 +609,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto errCode = git_object_lookup(Setter(obj), *this, &oid, GIT_OBJECT_ANY)) {
if (errCode == GIT_ENOTFOUND)
return false;
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return true;
@@ -918,7 +939,7 @@ struct GitSourceAccessor : SourceAccessor
TreeEntry copy;
if (git_tree_entry_dup(Setter(copy), entry))
throw Error("dupping tree entry: %s", git_error_last()->message);
throw GitError("dupping tree entry");
auto entryName = std::string_view(git_tree_entry_name(entry));
@@ -948,7 +969,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -983,7 +1004,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -1016,7 +1037,7 @@ struct GitSourceAccessor : SourceAccessor
Blob blob;
if (git_tree_entry_to_object((git_object **) (git_blob **) Setter(blob), *state.repo, entry))
throw Error("looking up file '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up file '%s'", showPath(path));
return blob;
}
@@ -1068,7 +1089,7 @@ struct GitExportIgnoreSourceAccessor : CachingFilteringSourceAccessor
if (git_error_last()->klass == GIT_ENOTFOUND)
return false;
else
throw Error("looking up '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up '%s'", showPath(path));
} else {
// Official git will silently reject export-ignore lines that have
// values. We do the same.
@@ -1224,17 +1245,17 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
repo.emplace(parent.repoPool.get());
if (git_blob_create_from_stream(Setter(stream), **repo, nullptr))
throw Error("creating a blob stream object: %s", git_error_last()->message);
throw GitError("creating a blob stream object");
if (stream->write(stream.get(), contents.data(), contents.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
parent.totalBufSize -= contents.size();
contents.clear();
}
} else {
if (stream->write(stream.get(), data.data(), data.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
}
}
@@ -1256,7 +1277,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
acquires ownership and frees the stream. */
git_oid oid;
if (git_blob_create_from_stream_commit(&oid, crf->stream.release()))
throw Error("creating a blob object for '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for '%s'", path);
addNode(
*_state.lock(),
crf->path,
@@ -1270,8 +1291,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, crf->contents.data(), crf->contents.size()))
throw Error(
"creating a blob object for '%s' from in-memory buffer: %s", crf->path, git_error_last()->message);
throw GitError("creating a blob object for '%s' from in-memory buffer", crf->path);
addNode(
*_state.lock(),
@@ -1295,8 +1315,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, target.c_str(), target.size()))
throw Error(
"creating a blob object for tarball symlink member '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for tarball symlink member '%s'", path);
auto state(_state.lock());
addNode(*state, path, Child{GIT_FILEMODE_LINK, oid});
@@ -1357,19 +1376,19 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
// Write this directory.
git_treebuilder * b;
if (git_treebuilder_new(&b, *repo, nullptr))
throw Error("creating a tree builder: %s", git_error_last()->message);
throw GitError("creating a tree builder");
TreeBuilder builder(b);
for (auto & [name, child] : node.children) {
auto oid_p = std::get_if<git_oid>(&child.file);
auto oid = oid_p ? *oid_p : std::get<Directory>(child.file).oid.value();
if (git_treebuilder_insert(nullptr, builder.get(), name.c_str(), &oid, child.mode))
throw Error("adding a file to a tree builder: %s", git_error_last()->message);
throw GitError("adding a file to a tree builder");
}
git_oid oid;
if (git_treebuilder_write(&oid, builder.get()))
throw Error("creating a tree object: %s", git_error_last()->message);
throw GitError("creating a tree object");
node.oid = oid;
}(_state.lock()->root);

View File

@@ -283,4 +283,18 @@ TEST(to_string, doesntReencodeUrl)
ASSERT_EQ(unparsed, expected);
}
TEST(parseFlakeRef, malformedGithubUrlDoesNotCrash)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
fetchers::Settings fetchSettings;
// Using ref= instead of rev= with a github: URL should produce an
// error, not an assertion failure in renderAuthorityAndPath
// (https://github.com/NixOS/nix/issues/15196).
EXPECT_THROW(
parseFlakeRef(fetchSettings, "github:nixos/nixpkgs/nixpkgs.git?ref=aead170c1a49253ebfa5027010dfd89a77b73ca4"),
Error);
}
} // namespace nix

View File

@@ -35,35 +35,39 @@ namespace nix::flake::primops {
PrimOp getFlake(const Settings & settings)
{
auto prim_getFlake = [&settings](EvalState & state, const PosIdx pos, Value ** args, Value & v) {
std::string flakeRefS(
state.forceStringNoCtx(*args[0], pos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = nix::parseFlakeRef(state.fetchSettings, flakeRefS, {}, true);
if (state.settings.pureEval && !flakeRef.input.isLocked(state.fetchSettings))
throw Error(
"cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)",
flakeRefS,
state.positions[pos]);
state.forceValue(*args[0], pos);
callFlake(
state,
lockFlake(
settings,
state,
flakeRef,
LockFlags{
.updateLockFile = false,
.writeLockFile = false,
.useRegistries = !state.settings.pureEval && settings.useRegistries,
.allowUnlocked = !state.settings.pureEval,
}),
v);
LockFlags lockFlags{
.updateLockFile = false,
.writeLockFile = false,
.useRegistries = !state.settings.pureEval && settings.useRegistries,
.allowUnlocked = !state.settings.pureEval,
};
if (args[0]->type() == nPath) {
auto path = state.realisePath(pos, *args[0]);
callFlake(state, lockFlake(settings, state, path, lockFlags), v);
} else {
NixStringContext context;
std::string flakeRefS(
state.forceStringNoCtx(*args[0], pos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = nix::parseFlakeRef(state.fetchSettings, flakeRefS, {}, true);
if (state.settings.pureEval && !flakeRef.input.isLocked(state.fetchSettings))
throw Error(
"cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)",
flakeRefS,
state.positions[pos]);
callFlake(state, lockFlake(settings, state, flakeRef, lockFlags), v);
}
};
return PrimOp{
.name = "__getFlake",
.args = {"args"},
.doc = R"(
Fetch a flake from a flake reference, and return its output attributes and some metadata. For example:
Fetch a flake from a flake reference or a path, and return its output attributes and some metadata. For example:
```nix
(builtins.getFlake "nix/55bc52401966fbffa525c574c14f67b00bc4fb3a").packages.x86_64-linux.nix

View File

@@ -416,10 +416,8 @@ static LockFile readLockFile(const fetchers::Settings & fetchSettings, const Sou
: LockFile();
}
/* Compute an in-memory lock file for the specified top-level flake,
and optionally write it to file, if the flake is writable. */
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags)
LockedFlake lockFlake(
const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags, Flake flake)
{
experimentalFeatureSettings.require(Xp::Flakes);
@@ -427,8 +425,6 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
auto useRegistriesTop = useRegistries ? fetchers::UseRegistries::All : fetchers::UseRegistries::No;
auto useRegistriesInputs = useRegistries ? fetchers::UseRegistries::Limited : fetchers::UseRegistries::No;
auto flake = getFlake(state, topRef, useRegistriesTop, {});
if (lockFlags.applyNixConfig) {
flake.config.apply(settings);
state.store->setOptions();
@@ -908,6 +904,22 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
}
}
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags)
{
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto useRegistriesTop = useRegistries ? fetchers::UseRegistries::All : fetchers::UseRegistries::No;
return lockFlake(settings, state, topRef, lockFlags, getFlake(state, topRef, useRegistriesTop, {}));
}
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const SourcePath & flakeDir, const LockFlags & lockFlags)
{
/* We need a fake flakeref to put in the `Flake` struct, but it's not used for anything. */
auto fakeRef = parseFlakeRef(state.fetchSettings, "flake:get-flake");
return lockFlake(settings, state, fakeRef, lockFlags, readFlake(state, fakeRef, fakeRef, fakeRef, flakeDir, {}));
}
static ref<SourceAccessor> makeInternalFS()
{
auto internalFS = make_ref<MemorySourceAccessor>(MemorySourceAccessor{});

View File

@@ -205,7 +205,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
fetchSettings,
{
.scheme = "path",
.authority = ParsedURL::Authority{},
.authority = isAbsolute(path) ? std::optional{ParsedURL::Authority{}} : std::nullopt,
.path = splitString<std::vector<std::string>>(path, "/"),
.query = query,
.fragment = fragment,

View File

@@ -214,9 +214,16 @@ struct LockFlags
std::set<NonEmptyInputAttrPath> inputUpdates;
};
/*
* Compute an in-memory lock file for the specified top-level flake, and optionally write it to file, if the flake is
* writable.
*/
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & flakeRef, const LockFlags & lockFlags);
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const SourcePath & flakeDir, const LockFlags & lockFlags);
void callFlake(EvalState & state, const LockedFlake & lockedFlake, Value & v);
/**

View File

@@ -50,9 +50,11 @@ void HttpsBinaryCacheStoreTest::SetUp()
// clang-format off
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", caKey.string()});
openssl({"req", "-new", "-x509", "-days", "1", "-key", caKey.string(), "-out", caCert.string(), "-subj", "/CN=TestCA"});
auto serverExtFile = tmpDir / "server.ext";
writeFile(serverExtFile, "subjectAltName=DNS:localhost,IP:127.0.0.1");
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", serverKey.string()});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1"});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost,IP:127.0.0.1"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1", "-extfile", serverExtFile.string()});
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", clientKey.string()});
openssl({"req", "-new", "-key", clientKey.string(), "-out", (tmpDir / "client.csr").string(), "-subj", "/CN=TestClient"});
openssl({"x509", "-req", "-in", (tmpDir / "client.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", clientCert.string(), "-days", "1"});

View File

@@ -15,6 +15,7 @@
#include "nix/store/local-store.hh" // TODO remove, along with remaining downcasts
#include "nix/store/globals.hh"
#include <algorithm>
#include <fstream>
#include <sys/types.h>
#include <fcntl.h>
@@ -184,6 +185,82 @@ struct LogFile
~LogFile();
};
struct LocalBuildRejection
{
bool maxJobsZero = false;
struct NoLocalStore
{};
/**
* We have a local store, but we don't have an external derivation builder (which is fine), if we did, it'd be
* fine because we would not care about platforms and features then. Since we don't, we either have the wrong
* platform, or we are missing some system features.
*/
struct WrongLocalStore
{
template<typename T>
struct Pair
{
T derivation;
T localStore;
};
std::optional<Pair<std::string>> badPlatform;
std::optional<Pair<StringSet>> missingFeatures;
};
std::variant<NoLocalStore, WrongLocalStore> rejection;
};
static BuildError reject(const LocalBuildRejection & rejection, std::string_view thingCannotBuild)
{
if (std::get_if<LocalBuildRejection::NoLocalStore>(&rejection.rejection))
return BuildError(
BuildResult::Failure::InputRejected,
"Unable to build with a primary store that isn't a local store; "
"either pass a different '--store' or enable remote builds.\n\n"
"For more information check 'man nix.conf' and search for '/machines'.");
auto & wrongStore = std::get<LocalBuildRejection::WrongLocalStore>(rejection.rejection);
std::string msg = fmt("Cannot build '%s'.", Magenta(thingCannotBuild));
if (rejection.maxJobsZero)
msg += "\nReason: " ANSI_RED "local builds are disabled" ANSI_NORMAL
" (max-jobs = 0)"
"\nHint: set 'max-jobs' to a non-zero value to enable local builds, "
"or configure remote builders via 'builders'";
if (wrongStore.badPlatform)
msg +=
fmt("\nReason: " ANSI_RED "platform mismatch" ANSI_NORMAL
"\nRequired system: '%s'"
"\nCurrent system: '%s'",
Magenta(wrongStore.badPlatform->derivation),
Magenta(wrongStore.badPlatform->localStore));
if (wrongStore.missingFeatures)
msg +=
fmt("\nReason: " ANSI_RED "missing system features" ANSI_NORMAL
"\nRequired features: {%s}"
"\nAvailable features: {%s}",
concatStringsSep(", ", wrongStore.missingFeatures->derivation),
concatStringsSep<StringSet>(", ", wrongStore.missingFeatures->localStore));
if (wrongStore.badPlatform || wrongStore.missingFeatures) {
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their
// hardware - we should tell them to run the command to install Rosetta
if (wrongStore.badPlatform && wrongStore.badPlatform->derivation == "x86_64-darwin"
&& wrongStore.badPlatform->localStore == "aarch64-darwin")
msg +=
fmt("\nNote: run `%s` to run programs for x86_64-darwin",
Magenta("/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
}
return BuildError(BuildResult::Failure::InputRejected, std::move(msg));
}
Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
{
auto drvOptions = [&] {
@@ -245,29 +322,57 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
}
checkPathValidity(initialOutputs);
/**
* Activity that denotes waiting for a lock.
*/
std::unique_ptr<Activity> actLock;
auto localBuildResult = [&]() -> std::variant<LocalBuildCapability, LocalBuildRejection> {
bool maxJobsZero = worker.settings.maxBuildJobs.get() == 0;
/**
* Locks on (fixed) output paths.
*/
PathLocks outputLocks;
auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store);
if (!localStoreP)
return LocalBuildRejection{.maxJobsZero = maxJobsZero, .rejection = LocalBuildRejection::NoLocalStore{}};
bool useHook;
/**
* Now that we've decided we can't / won't do a remote build, check
* that we can in fact build locally. First see if there is an
* external builder for a "semi-local build". If there is, prefer to
* use that. If there is not, then check if we can do a "true" local
* build.
*/
auto * ext = settings.getLocalSettings().findExternalDerivationBuilderIfSupported(*drv);
const ExternalBuilder * externalBuilder = nullptr;
if (ext)
return LocalBuildCapability{*localStoreP, ext};
while (true) {
using WrongLocalStore = LocalBuildRejection::WrongLocalStore;
WrongLocalStore wrongStore;
if (drv->platform != settings.thisSystem.get() && !settings.extraPlatforms.get().count(drv->platform)
&& !drv->isBuiltin())
wrongStore.badPlatform = WrongLocalStore::Pair<std::string>{drv->platform, settings.thisSystem.get()};
{
auto required = drvOptions.getRequiredSystemFeatures(*drv);
auto & available = worker.store.config.systemFeatures.get();
if (std::ranges::any_of(required, [&](const std::string & f) { return !available.count(f); }))
wrongStore.missingFeatures = WrongLocalStore::Pair<StringSet>{required, available};
}
if (maxJobsZero || wrongStore.badPlatform || wrongStore.missingFeatures)
return LocalBuildRejection{.maxJobsZero = maxJobsZero, .rejection = std::move(wrongStore)};
return LocalBuildCapability{*localStoreP, ext};
}();
auto acquireResources = [&](bool & done, PathLocks & outputLocks) -> Goal::Co {
trace("trying to build");
/* Obtain locks on all output paths, if the paths are known a priori.
The locks are automatically released when we exit this function or Nix
crashes. If we can't acquire the lock, then continue; hopefully some
other goal can start a build, and if not, the main loop will sleep a few
seconds and then retry this goal. */
/**
* Output paths to acquire locks on, if known a priori.
*
* The locks are automatically released when the caller's `PathLocks` goes
* out of scope, including on exception unwinding. If we can't acquire the lock, then
* continue; hopefully some other goal can start a build, and if not, the
* main loop will sleep a few seconds and then retry this goal.
*/
std::set<std::filesystem::path> lockFiles;
/* FIXME: Should lock something like the drv itself so we don't build same
CA drv concurrently */
@@ -311,7 +416,8 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
debug("skipping build of derivation '%s', someone beat us to it", worker.store.printStorePath(drvPath));
outputLocks.setDeletion(true);
outputLocks.unlock();
co_return doneSuccess(BuildResult::Success::AlreadyValid, std::move(validOutputs));
done = true;
co_return Return{};
}
/* If any of the outputs already exist but are not valid, delete
@@ -326,112 +432,133 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
}
}
bool canBuildLocally = [&] {
if (drv->platform != settings.thisSystem.get() && !settings.extraPlatforms.get().count(drv->platform)
&& !drv->isBuiltin())
return false;
co_return Return{};
};
if (worker.settings.maxBuildJobs.get() == 0 && !drv->isBuiltin())
return false;
auto tryHookLoop = [&](bool & valid) -> Goal::Co {
{
PathLocks outputLocks;
co_await acquireResources(valid, outputLocks);
if (valid)
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
for (auto & feature : drvOptions.getRequiredSystemFeatures(*drv))
if (!worker.store.config.systemFeatures.get().count(feature))
return false;
return true;
}();
/* Don't do a remote build if the derivation has the attribute
`preferLocalBuild' set. Also, check and repair modes are only
supported for local builds. */
bool buildLocally = (buildMode != bmNormal || (drvOptions.preferLocalBuild && canBuildLocally))
&& worker.settings.maxBuildJobs.get() != 0;
if (buildLocally) {
useHook = false;
} else {
switch (tryBuildHook(drvOptions)) {
case rpAccept:
/* Yes, it has started doing so. Wait until we get
EOF from the hook. */
useHook = true;
break;
valid = true;
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
case rpDecline:
// We should do it ourselves.
co_return Return{};
case rpPostpone:
/* Not now; wait until at least one child finishes or
the wake-up timeout expires. */
if (!actLock)
actLock = std::make_unique<Activity>(
*logger,
lvlWarn,
actBuildWaiting,
fmt("waiting for a machine to build '%s'", Magenta(worker.store.printStorePath(drvPath))));
outputLocks.unlock();
co_await waitForAWhile();
continue;
case rpDecline:
/* We should do it ourselves.
Now that we've decided we can't / won't do a remote build, check
that we can in fact build locally. First see if there is an
external builder for a "semi-local build". If there is, prefer to
use that. If there is not, then check if we can do a "true" local
build. */
externalBuilder = settings.findExternalDerivationBuilderIfSupported(*drv);
if (!externalBuilder && !canBuildLocally) {
auto msg =
fmt("Cannot build '%s'.\n"
"Reason: " ANSI_RED "required system or feature not available" ANSI_NORMAL
"\n"
"Required system: '%s' with features {%s}\n"
"Current system: '%s' with features {%s}",
Magenta(worker.store.printStorePath(drvPath)),
Magenta(drv->platform),
concatStringsSep(", ", drvOptions.getRequiredSystemFeatures(*drv)),
Magenta(settings.thisSystem),
concatStringsSep<StringSet>(", ", worker.store.Store::config.systemFeatures));
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their hardware -
// we should tell them to run the command to install Darwin 2
if (drv->platform == "x86_64-darwin" && settings.thisSystem == "aarch64-darwin")
msg += fmt(
"\nNote: run `%s` to run programs for x86_64-darwin",
Magenta(
"/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
outputLocks.unlock();
co_return doneFailure({BuildResult::Failure::InputRejected, std::move(msg)});
}
useHook = false;
break;
}
}
break;
}
actLock.reset();
PathLocks outputLocks;
{
// First attempt was postponed. Retry in a loop with an activity
// that lives until accept or decline.
Activity act(
*logger,
lvlWarn,
actBuildWaiting,
fmt("waiting for a machine to build '%s'", Magenta(worker.store.printStorePath(drvPath))));
if (useHook) {
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
} else if (auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store)) {
co_return buildLocally(
*localStoreP,
std::move(inputPaths),
std::move(initialOutputs),
std::move(drvOptions),
std::move(outputLocks),
externalBuilder);
while (true) {
co_await waitForAWhile();
co_await acquireResources(valid, outputLocks);
if (valid)
break;
switch (tryBuildHook(drvOptions)) {
case rpAccept:
/* Yes, it has started doing so. Wait until we get
EOF from the hook. */
break;
case rpPostpone:
/* Not now; wait until at least one child finishes or
the wake-up timeout expires. */
outputLocks.unlock();
continue;
case rpDecline:
// We should do it ourselves.
co_return Return{};
}
break;
}
}
if (valid) {
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
} else {
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
}
};
auto tryBuildLocally = [&](bool & valid) -> Goal::Co {
if (auto * cap = std::get_if<LocalBuildCapability>(&localBuildResult)) {
PathLocks outputLocks;
co_await acquireResources(valid, outputLocks);
if (valid)
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
valid = true;
co_return buildLocally(
*cap, std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
}
co_return Return{};
};
if (buildMode != bmNormal) {
// Check and repair modes operate on the state of this store specifically,
// so they must always build locally.
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
} else if (drvOptions.preferLocalBuild) {
// Local is preferred, so try it first. If it's not available, fall back to the hook.
{
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
}
{
bool valid = false;
co_await tryHookLoop(valid);
if (valid)
co_return Return{};
}
} else {
throw Error(
R"(
Unable to build with a primary store that isn't a local store;
either pass a different '--store' or enable remote builds.
For more information check 'man nix.conf' and search for '/machines'.
)");
// Default preference is a remote build: they tend to be faster and preserve local
// resources for other tasks. Fall back to local if no remote is available.
{
bool valid = false;
co_await tryHookLoop(valid);
if (valid)
co_return Return{};
}
{
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
}
}
std::string storePath = worker.store.printStorePath(drvPath);
auto * rejection = std::get_if<LocalBuildRejection>(&localBuildResult);
assert(rejection);
co_return doneFailure(reject(*rejection, storePath));
}
Goal::Co DerivationBuildingGoal::buildWithHook(
@@ -639,12 +766,11 @@ Goal::Co DerivationBuildingGoal::buildWithHook(
}
Goal::Co DerivationBuildingGoal::buildLocally(
LocalStore & localStore,
LocalBuildCapability localBuildCap,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks,
const ExternalBuilder * externalBuilder)
PathLocks outputLocks)
{
co_await yield();
@@ -728,7 +854,7 @@ Goal::Co DerivationBuildingGoal::buildLocally(
};
decltype(DerivationBuilderParams::defaultPathsInChroot) defaultPathsInChroot =
localStore.config->getLocalSettings().sandboxPaths.get();
localBuildCap.localStore.config->getLocalSettings().sandboxPaths.get();
DesugaredEnv desugaredEnv;
/* Add the closure of store paths to the chroot. */
@@ -770,14 +896,14 @@ Goal::Co DerivationBuildingGoal::buildLocally(
/* If we have to wait and retry (see below), then `builder` will
already be created, so we don't need to create it again. */
builder = externalBuilder
builder = localBuildCap.externalBuilder
? makeExternalDerivationBuilder(
localStore,
localBuildCap.localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params),
*externalBuilder)
*localBuildCap.externalBuilder)
: makeDerivationBuilder(
localStore,
localBuildCap.localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params));
}

View File

@@ -36,8 +36,8 @@ static void builtinUnpackChannel(const BuiltinBuilderContext & ctx)
auto target = out / channelName;
try {
std::filesystem::rename(fileName, target);
} catch (std::filesystem::filesystem_error &) {
throw SysError("failed to rename %1% to %2%", fileName, target.string());
} catch (std::filesystem::filesystem_error & e) {
throw SystemError(e.code(), "failed to rename %1% to %2%", fileName, target.string());
}
}

View File

@@ -136,6 +136,12 @@ StorePath Store::writeDerivation(const Derivation & drv, RepairFlag repair)
{
auto [suffix, contents, references, path] = infoForDerivation(*this, drv);
/* In case the derivation is already valid, we bail out early since that's
faster. But we need to make sure that the derivation has a corresponding
temproot. It is added by the remote in addToStoreFromDump, but we'd like
to avoid sending a lot of drv contents to the daemon. */
addTempRoot(path);
if (isValidPath(path) && !repair)
return path;

View File

@@ -737,7 +737,7 @@ struct curlFileTransfer : public FileTransfer
download after a while. If we're writing to a
sink, we can only retry if the server supports
ranged requests. */
if (err == Transient && attempt < request.tries
if (err == Transient && attempt < fileTransfer.settings.tries
&& (!this->request.dataCallback || writtenToSink == 0 || (acceptRanges && encoding.empty()))) {
int ms = retryTimeMs
* std::pow(

View File

@@ -3,6 +3,7 @@
#include "nix/store/local-gc.hh"
#include "nix/store/local-store.hh"
#include "nix/store/path.hh"
#include "nix/util/configuration.hh"
#include "nix/util/finally.hh"
#include "nix/util/unix-domain-socket.hh"
#include "nix/util/signals.hh"
@@ -271,7 +272,7 @@ void LocalStore::findRoots(const Path & path, std::filesystem::file_type type, R
|| e.code() == std::errc::not_a_directory)
printInfo("cannot read potential root '%1%'", path);
else
throw;
throw SystemError(e.code(), "finding GC roots in '%1%'", path);
}
catch (SystemError & e) {
@@ -306,9 +307,33 @@ Roots LocalStore::findRoots(bool censor)
return roots;
}
static Roots requestRuntimeRoots(const LocalStoreConfig & config, const std::filesystem::path & socketPath)
{
Roots roots;
auto socket = connect(socketPath);
auto socketSource = FdSource(socket.get());
while (1) {
auto line = socketSource.readLine(true, '\0');
if (line == "")
break;
roots[config.parseStorePath(line)].insert(censored);
};
return roots;
}
void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
{
auto unchecked = findRuntimeRootsUnchecked(*config);
Roots unchecked;
if (config->useRootsDaemon) {
experimentalFeatureSettings.require(Xp::LocalOverlayStore);
unchecked = requestRuntimeRoots(*config, config->getRootsSocketPath());
} else {
unchecked = findRuntimeRootsUnchecked(*config);
}
for (auto & [path, links] : unchecked) {
if (!isValidPath(path))

View File

@@ -279,7 +279,7 @@ bool Settings::isWSL1()
#endif
}
const ExternalBuilder * Settings::findExternalDerivationBuilderIfSupported(const Derivation & drv)
const ExternalBuilder * LocalSettings::findExternalDerivationBuilderIfSupported(const Derivation & drv)
{
if (auto it = std::ranges::find_if(
externalBuilders.get(), [&](const auto & handler) { return handler.systems.contains(drv.platform); });
@@ -429,17 +429,17 @@ unsigned int MaxBuildJobsSetting::parse(const std::string & str) const
}
template<>
Settings::ExternalBuilders BaseSetting<Settings::ExternalBuilders>::parse(const std::string & str) const
LocalSettings::ExternalBuilders BaseSetting<LocalSettings::ExternalBuilders>::parse(const std::string & str) const
{
try {
return nlohmann::json::parse(str).template get<Settings::ExternalBuilders>();
return nlohmann::json::parse(str).template get<LocalSettings::ExternalBuilders>();
} catch (std::exception & e) {
throw UsageError("parsing setting '%s': %s", name, e.what());
}
}
template<>
std::string BaseSetting<Settings::ExternalBuilders>::to_string() const
std::string BaseSetting<LocalSettings::ExternalBuilders>::to_string() const
{
return nlohmann::json(value).dump();
}

View File

@@ -64,6 +64,12 @@ private:
std::string key() override;
struct LocalBuildCapability
{
LocalStore & localStore;
const ExternalBuilder * externalBuilder;
};
/**
* The states.
*/
@@ -75,12 +81,11 @@ private:
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks);
Co buildLocally(
LocalStore & localStore,
LocalBuildCapability localBuildCap,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks,
const ExternalBuilder * externalBuilder);
PathLocks outputLocks);
/**
* Is the build hook willing to perform the build?

View File

@@ -182,7 +182,6 @@ struct FileTransferRequest
Headers headers;
std::string expectedETag;
HttpMethod method = HttpMethod::Get;
size_t tries = fileTransferSettings.tries;
unsigned int baseRetryTimeMs = RETRY_TIME_MS_DEFAULT;
ActivityId parentAct;
bool decompress = true;

View File

@@ -104,8 +104,6 @@ public:
Settings();
using ExternalBuilders = std::vector<ExternalBuilder>;
/**
* Get the local store settings.
*/
@@ -410,12 +408,6 @@ public:
Set it to 1 to warn on all paths.
)"};
/**
* Finds the first external derivation builder that supports this
* derivation, or else returns a null pointer.
*/
const ExternalBuilder * findExternalDerivationBuilderIfSupported(const Derivation & drv);
/**
* Get the options needed for profile directory functions.
*/

View File

@@ -707,6 +707,12 @@ public:
// Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}
// Xp::ExternalBuilders
};
/**
* Finds the first external derivation builder that supports this
* derivation, or else returns a null pointer.
*/
const ExternalBuilder * findExternalDerivationBuilderIfSupported(const Derivation & drv);
};
} // namespace nix

View File

@@ -133,6 +133,27 @@ public:
Xp::LocalOverlayStore,
};
Setting<bool> useRootsDaemon{
this,
false,
"use-roots-daemon",
R"(
Whether to request garbage collector roots from an external daemon.
When enabled, the garbage collector connects to a Unix domain socket
at [`<state-dir>`](@docroot@/store/types/local-store.md#store-option-state)`/gc-roots-socket/socket` to discover additional roots
that should not be collected. This is useful when the Nix daemon runs
without root privileges and cannot scan `/proc` for runtime roots.
The daemon can be started with [`nix store roots-daemon`](@docroot@/command-ref/new-cli/nix3-store-roots-daemon.md).
)",
{},
true,
Xp::LocalOverlayStore,
};
std::filesystem::path getRootsSocketPath() const;
static const std::string name()
{
return "Local Store";

View File

@@ -35,7 +35,7 @@ static void readProcLink(const std::filesystem::path & file, UncheckedRoots & ro
if (e.code() == std::errc::no_such_file_or_directory || e.code() == std::errc::permission_denied
|| e.code() == std::errc::no_such_process)
return;
throw;
throw SystemError(e.code(), "reading symlink '%s'", PathFmt(file));
}
if (buf.is_absolute())
roots[buf.string()].emplace(file.string());

View File

@@ -457,6 +457,11 @@ LocalStore::~LocalStore()
}
}
std::filesystem::path LocalStoreConfig::getRootsSocketPath() const
{
return std::filesystem::path(stateDir.get()) / "gc-roots-socket" / "socket";
}
StoreReference LocalStoreConfig::getReference() const
{
auto params = getQueryParams();

View File

@@ -111,8 +111,8 @@ static std::vector<std::string> expandBuilderLines(const std::string & builders)
std::string text;
try {
text = readFile(path);
} catch (const SysError & e) {
if (e.errNo != ENOENT)
} catch (const SystemError & e) {
if (!e.is(std::errc::no_such_file_or_directory))
throw;
debug("cannot find machines file '%s'", path);
continue;

View File

@@ -68,10 +68,8 @@ endif
summary('can hardlink to symlink', can_link_symlink, bool_yn : true)
configdata_priv.set('CAN_LINK_SYMLINK', can_link_symlink.to_int())
# Check for each of these functions, and create a define like `#define HAVE_LCHOWN 1`.
# Check for each of these functions, and create a define like `#define HAVE_POSIX_FALLOCATE 1`.
check_funcs = [
# Optionally used for canonicalising files from the build
'lchown',
'posix_fallocate',
'statvfs',
]

View File

@@ -202,14 +202,13 @@ void LocalStore::optimisePath_(
full. When that happens, it's fine to ignore it: we
just effectively disable deduplication of this
file.
TODO: Get rid of errno, use error code.
*/
printInfo("cannot link %s to '%s': %s", PathFmt(linkPath), path, strerror(errno));
printInfo("cannot link %s to '%s': %s", PathFmt(linkPath), path, e.code().message());
return;
}
else
throw;
throw SystemError(e.code(), "creating hard link from %1% to %2%", PathFmt(linkPath), path);
}
}
@@ -250,7 +249,7 @@ void LocalStore::optimisePath_(
printInfo("%1% has maximum number of links", PathFmt(linkPath));
return;
}
throw;
throw SystemError(e.code(), "creating hard link from %1% to %2%", PathFmt(linkPath), PathFmt(tempLink));
}
/* Atomically replace the old file with the new hard link. */
@@ -271,7 +270,7 @@ void LocalStore::optimisePath_(
debug("%s has reached maximum number of links", PathFmt(linkPath));
return;
}
throw;
throw SystemError(e.code(), "renaming %1% to %2%", PathFmt(tempLink), path);
}
stats.filesLinked++;

View File

@@ -107,19 +107,9 @@ canonicalisePathMetaData_(const Path & path, CanonicalizePathMetadataOptions opt
canonicaliseTimestampAndPermissions(path, st);
#ifndef _WIN32
/* Change ownership to the current uid. If it's a symlink, use
lchown if available, otherwise don't bother. Wrong ownership
of a symlink doesn't matter, since the owning user can't change
the symlink and can't delete it because the directory is not
writable. The only exception is top-level paths in the Nix
store (since that directory is group-writable for the Nix build
users group); we check for this case below. */
/* Change ownership to the current uid. */
if (st.st_uid != geteuid()) {
# if HAVE_LCHOWN
if (lchown(path.c_str(), geteuid(), getegid()) == -1)
# else
if (!S_ISLNK(st.st_mode) && chown(path.c_str(), geteuid(), getegid()) == -1)
# endif
throw SysError("changing owner of '%1%' to %2%", path, geteuid());
}
#endif
@@ -135,17 +125,6 @@ canonicalisePathMetaData_(const Path & path, CanonicalizePathMetadataOptions opt
void canonicalisePathMetaData(const Path & path, CanonicalizePathMetadataOptions options, InodesSeen & inodesSeen)
{
canonicalisePathMetaData_(path, options, inodesSeen);
#ifndef _WIN32
/* On platforms that don't have lchown(), the top-level path can't
be a symlink, since we can't change its ownership. */
auto st = lstat(path);
if (st.st_uid != geteuid()) {
assert(S_ISLNK(st.st_mode));
throw Error("wrong ownership of top-level store path '%1%'", path);
}
#endif
}
void canonicalisePathMetaData(const Path & path, CanonicalizePathMetadataOptions options)

View File

@@ -103,7 +103,7 @@ static void removeFile(const std::filesystem::path & path)
try {
std::filesystem::remove(path);
} catch (std::filesystem::filesystem_error & e) {
throw SysError("removing file %1%", PathFmt(path));
throw SystemError(e.code(), "removing file %1%", PathFmt(path));
}
}
@@ -324,8 +324,6 @@ std::filesystem::path getDefaultProfile(ProfileDirsOptions settings)
return absPath(readLink(profileLink), &linkDir);
} catch (Error &) {
return profileLink;
} catch (std::filesystem::filesystem_error &) {
return profileLink;
}
}

View File

@@ -3,8 +3,6 @@
#include "nix/util/file-descriptor.hh"
#include "nix/util/serialise.hh"
#include "nix/util/file-system.hh"
#include "nix/util/fs-sink.hh"
#include <cstring>
@@ -245,166 +243,4 @@ TEST(BufferedSourceReadLine, BufferExhaustedThenEof)
EXPECT_EQ(source.readLine(/*eofOk=*/true), "");
}
/* ----------------------------------------------------------------------------
* readLinkAt
* --------------------------------------------------------------------------*/
TEST(readLinkAt, works)
{
std::filesystem::path tmpDir = nix::createTempDir();
nix::AutoDelete delTmpDir(tmpDir, /*recursive=*/true);
constexpr size_t maxPathLength =
#ifdef _WIN32
260
#else
PATH_MAX
#endif
;
std::string mediumTarget(maxPathLength / 2, 'x');
std::string longTarget(maxPathLength - 1, 'y');
{
RestoreSink sink(/*startFsync=*/false);
sink.dstPath = tmpDir;
sink.dirFd = openDirectory(tmpDir);
sink.createSymlink(CanonPath("link"), "target");
sink.createSymlink(CanonPath("relative"), "../relative/path");
sink.createSymlink(CanonPath("absolute"), "/absolute/path");
sink.createSymlink(CanonPath("medium"), mediumTarget);
sink.createSymlink(CanonPath("long"), longTarget);
sink.createDirectory(CanonPath("a"));
sink.createDirectory(CanonPath("a/b"));
sink.createSymlink(CanonPath("a/b/link"), "nested_target");
sink.createRegularFile(CanonPath("regular"), [](CreateRegularFileSink &) {});
sink.createDirectory(CanonPath("dir"));
}
AutoCloseFD dirFd = openDirectory(tmpDir);
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("link")), OS_STR("target"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("relative")), OS_STR("../relative/path"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("absolute")), OS_STR("/absolute/path"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("medium")), string_to_os_string(mediumTarget));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("long")), string_to_os_string(longTarget));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("a/b/link")), OS_STR("nested_target"));
AutoCloseFD subDirFd = openDirectory(tmpDir / "a");
EXPECT_EQ(readLinkAt(subDirFd.get(), CanonPath("b/link")), OS_STR("nested_target"));
// Test error cases - expect SystemError on both platforms
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("regular")), SystemError);
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("dir")), SystemError);
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("nonexistent")), SystemError);
}
/* ----------------------------------------------------------------------------
* openFileEnsureBeneathNoSymlinks
* --------------------------------------------------------------------------*/
TEST(openFileEnsureBeneathNoSymlinks, works)
{
std::filesystem::path tmpDir = nix::createTempDir();
nix::AutoDelete delTmpDir(tmpDir, /*recursive=*/true);
{
RestoreSink sink(/*startFsync=*/false);
sink.dstPath = tmpDir;
sink.dirFd = openDirectory(tmpDir);
sink.createDirectory(CanonPath("a"));
sink.createDirectory(CanonPath("c"));
sink.createDirectory(CanonPath("c/d"));
sink.createRegularFile(CanonPath("c/d/regular"), [](CreateRegularFileSink & crf) { crf("some contents"); });
sink.createSymlink(CanonPath("a/absolute_symlink"), tmpDir.string());
sink.createSymlink(CanonPath("a/relative_symlink"), "../.");
sink.createSymlink(CanonPath("a/broken_symlink"), "./nonexistent");
sink.createDirectory(CanonPath("a/b"), [](FileSystemObjectSink & dirSink, const CanonPath & relPath) {
dirSink.createDirectory(CanonPath("d"));
dirSink.createSymlink(CanonPath("c"), "./d");
});
// FIXME: This still follows symlinks on Unix (incorrectly succeeds)
sink.createDirectory(CanonPath("a/b/c/e"));
// Test that symlinks in intermediate path are detected during nested operations
ASSERT_THROW(
sink.createDirectory(
CanonPath("a/b/c/f"), [](FileSystemObjectSink & dirSink, const CanonPath & relPath) {}),
SymlinkNotAllowed);
ASSERT_THROW(
sink.createRegularFile(
CanonPath("a/b/c/regular"), [](CreateRegularFileSink & crf) { crf("some contents"); }),
SymlinkNotAllowed);
}
AutoCloseFD dirFd = openDirectory(tmpDir);
// Helper to open files with platform-specific arguments
auto openRead = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_READ_DATA | FILE_READ_ATTRIBUTES | SYNCHRONIZE,
0
#else
O_RDONLY,
0
#endif
);
};
auto openReadDir = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_READ_ATTRIBUTES | SYNCHRONIZE,
FILE_DIRECTORY_FILE
#else
O_RDONLY | O_DIRECTORY,
0
#endif
);
};
auto openCreateExclusive = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_WRITE_DATA | SYNCHRONIZE,
0,
FILE_CREATE // Create new file, fail if exists (equivalent to O_CREAT | O_EXCL)
#else
O_CREAT | O_WRONLY | O_EXCL,
0666
#endif
);
};
// Test that symlinks are detected and rejected
EXPECT_THROW(openRead("a/absolute_symlink"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/relative_symlink"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/absolute_symlink/a"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/absolute_symlink/c/d"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/relative_symlink/c"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/b/c/d"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/broken_symlink"), SymlinkNotAllowed);
#if !defined(_WIN32) && !defined(__CYGWIN__)
// This returns ELOOP on cygwin when O_NOFOLLOW is used
EXPECT_EQ(openCreateExclusive("a/broken_symlink"), INVALID_DESCRIPTOR);
/* Sanity check, no symlink shenanigans and behaves the same as regular openat with O_EXCL | O_CREAT. */
EXPECT_EQ(errno, EEXIST);
#endif
EXPECT_THROW(openCreateExclusive("a/absolute_symlink/broken_symlink"), SymlinkNotAllowed);
// Test invalid paths
EXPECT_EQ(openRead("c/d/regular/a"), INVALID_DESCRIPTOR);
EXPECT_EQ(openReadDir("c/d/regular"), INVALID_DESCRIPTOR);
// Test valid paths work
EXPECT_TRUE(AutoCloseFD{openRead("c/d/regular")});
EXPECT_TRUE(AutoCloseFD{openCreateExclusive("a/regular")});
}
} // namespace nix

View File

@@ -0,0 +1,172 @@
#include <gtest/gtest.h>
#include <gmock/gmock.h>
#include "nix/util/file-system-at.hh"
#include "nix/util/file-system.hh"
#include "nix/util/fs-sink.hh"
namespace nix {
/* ----------------------------------------------------------------------------
* readLinkAt
* --------------------------------------------------------------------------*/
TEST(readLinkAt, works)
{
std::filesystem::path tmpDir = nix::createTempDir();
nix::AutoDelete delTmpDir(tmpDir, /*recursive=*/true);
constexpr size_t maxPathLength =
#ifdef _WIN32
260
#else
PATH_MAX
#endif
;
std::string mediumTarget(maxPathLength / 2, 'x');
std::string longTarget(maxPathLength - 1, 'y');
{
RestoreSink sink(/*startFsync=*/false);
sink.dstPath = tmpDir;
sink.dirFd = openDirectory(tmpDir);
sink.createSymlink(CanonPath("link"), "target");
sink.createSymlink(CanonPath("relative"), "../relative/path");
sink.createSymlink(CanonPath("absolute"), "/absolute/path");
sink.createSymlink(CanonPath("medium"), mediumTarget);
sink.createSymlink(CanonPath("long"), longTarget);
sink.createDirectory(CanonPath("a"));
sink.createDirectory(CanonPath("a/b"));
sink.createSymlink(CanonPath("a/b/link"), "nested_target");
sink.createRegularFile(CanonPath("regular"), [](CreateRegularFileSink &) {});
sink.createDirectory(CanonPath("dir"));
}
AutoCloseFD dirFd = openDirectory(tmpDir);
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("link")), OS_STR("target"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("relative")), OS_STR("../relative/path"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("absolute")), OS_STR("/absolute/path"));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("medium")), string_to_os_string(mediumTarget));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("long")), string_to_os_string(longTarget));
EXPECT_EQ(readLinkAt(dirFd.get(), CanonPath("a/b/link")), OS_STR("nested_target"));
AutoCloseFD subDirFd = openDirectory(tmpDir / "a");
EXPECT_EQ(readLinkAt(subDirFd.get(), CanonPath("b/link")), OS_STR("nested_target"));
// Test error cases - expect SystemError on both platforms
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("regular")), SystemError);
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("dir")), SystemError);
EXPECT_THROW(readLinkAt(dirFd.get(), CanonPath("nonexistent")), SystemError);
}
/* ----------------------------------------------------------------------------
* openFileEnsureBeneathNoSymlinks
* --------------------------------------------------------------------------*/
TEST(openFileEnsureBeneathNoSymlinks, works)
{
std::filesystem::path tmpDir = nix::createTempDir();
nix::AutoDelete delTmpDir(tmpDir, /*recursive=*/true);
{
RestoreSink sink(/*startFsync=*/false);
sink.dstPath = tmpDir;
sink.dirFd = openDirectory(tmpDir);
sink.createDirectory(CanonPath("a"));
sink.createDirectory(CanonPath("c"));
sink.createDirectory(CanonPath("c/d"));
sink.createRegularFile(CanonPath("c/d/regular"), [](CreateRegularFileSink & crf) { crf("some contents"); });
sink.createSymlink(CanonPath("a/absolute_symlink"), tmpDir.string());
sink.createSymlink(CanonPath("a/relative_symlink"), "../.");
sink.createSymlink(CanonPath("a/broken_symlink"), "./nonexistent");
sink.createDirectory(CanonPath("a/b"), [](FileSystemObjectSink & dirSink, const CanonPath & relPath) {
dirSink.createDirectory(CanonPath("d"));
dirSink.createSymlink(CanonPath("c"), "./d");
});
// FIXME: This still follows symlinks on Unix (incorrectly succeeds)
sink.createDirectory(CanonPath("a/b/c/e"));
// Test that symlinks in intermediate path are detected during nested operations
ASSERT_THROW(
sink.createDirectory(
CanonPath("a/b/c/f"), [](FileSystemObjectSink & dirSink, const CanonPath & relPath) {}),
SymlinkNotAllowed);
ASSERT_THROW(
sink.createRegularFile(
CanonPath("a/b/c/regular"), [](CreateRegularFileSink & crf) { crf("some contents"); }),
SymlinkNotAllowed);
}
AutoCloseFD dirFd = openDirectory(tmpDir);
// Helper to open files with platform-specific arguments
auto openRead = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_READ_DATA | FILE_READ_ATTRIBUTES | SYNCHRONIZE,
0
#else
O_RDONLY,
0
#endif
);
};
auto openReadDir = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_READ_ATTRIBUTES | SYNCHRONIZE,
FILE_DIRECTORY_FILE
#else
O_RDONLY | O_DIRECTORY,
0
#endif
);
};
auto openCreateExclusive = [&](std::string_view path) -> Descriptor {
return openFileEnsureBeneathNoSymlinks(
dirFd.get(),
CanonPath(path),
#ifdef _WIN32
FILE_WRITE_DATA | SYNCHRONIZE,
0,
FILE_CREATE // Create new file, fail if exists (equivalent to O_CREAT | O_EXCL)
#else
O_CREAT | O_WRONLY | O_EXCL,
0666
#endif
);
};
// Test that symlinks are detected and rejected
EXPECT_THROW(openRead("a/absolute_symlink"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/relative_symlink"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/absolute_symlink/a"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/absolute_symlink/c/d"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/relative_symlink/c"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/b/c/d"), SymlinkNotAllowed);
EXPECT_THROW(openRead("a/broken_symlink"), SymlinkNotAllowed);
#if !defined(_WIN32) && !defined(__CYGWIN__)
// This returns ELOOP on cygwin when O_NOFOLLOW is used
EXPECT_EQ(openCreateExclusive("a/broken_symlink"), INVALID_DESCRIPTOR);
/* Sanity check, no symlink shenanigans and behaves the same as regular openat with O_EXCL | O_CREAT. */
EXPECT_EQ(errno, EEXIST);
#endif
EXPECT_THROW(openCreateExclusive("a/absolute_symlink/broken_symlink"), SymlinkNotAllowed);
// Test invalid paths
EXPECT_EQ(openRead("c/d/regular/a"), INVALID_DESCRIPTOR);
EXPECT_EQ(openReadDir("c/d/regular"), INVALID_DESCRIPTOR);
// Test valid paths work
EXPECT_TRUE(AutoCloseFD{openRead("c/d/regular")});
EXPECT_TRUE(AutoCloseFD{openCreateExclusive("a/regular")});
}
} // namespace nix

View File

@@ -196,20 +196,42 @@ TEST(baseNameOf, absoluteNothingSlashNothing)
TEST(isInDir, trivialCase)
{
auto p1 = isInDir("/foo/bar", "/foo");
ASSERT_EQ(p1, true);
EXPECT_TRUE(isInDir("/foo/bar", "/foo"));
}
TEST(isInDir, notInDir)
{
auto p1 = isInDir("/zes/foo/bar", "/foo");
ASSERT_EQ(p1, false);
EXPECT_FALSE(isInDir("/zes/foo/bar", "/foo"));
}
TEST(isInDir, emptyDir)
{
auto p1 = isInDir("/zes/foo/bar", "");
ASSERT_EQ(p1, false);
EXPECT_FALSE(isInDir("/zes/foo/bar", ""));
}
TEST(isInDir, hiddenSubdirectory)
{
EXPECT_TRUE(isInDir("/foo/.ssh", "/foo"));
}
TEST(isInDir, ellipsisEntry)
{
EXPECT_TRUE(isInDir("/foo/...", "/foo"));
}
TEST(isInDir, sameDir)
{
EXPECT_FALSE(isInDir("/foo", "/foo"));
}
TEST(isInDir, sameDirDot)
{
EXPECT_FALSE(isInDir("/foo/.", "/foo"));
}
TEST(isInDir, dotDotPrefix)
{
EXPECT_FALSE(isInDir("/foo/../bar", "/foo"));
}
/* ----------------------------------------------------------------------------
@@ -297,7 +319,7 @@ TEST(chmodIfNeeded, works)
TEST(chmodIfNeeded, nonexistent)
{
ASSERT_THROW(chmodIfNeeded("/schnitzel/darmstadt/pommes", 0755), SysError);
ASSERT_THROW(chmodIfNeeded("/schnitzel/darmstadt/pommes", 0755), SystemError);
}
/* ----------------------------------------------------------------------------
@@ -318,7 +340,7 @@ TEST(DirectoryIterator, works)
TEST(DirectoryIterator, nonexistent)
{
ASSERT_THROW(DirectoryIterator("/schnitzel/darmstadt/pommes"), SysError);
ASSERT_THROW(DirectoryIterator("/schnitzel/darmstadt/pommes"), SystemError);
}
/* ----------------------------------------------------------------------------

View File

@@ -60,6 +60,7 @@ sources = files(
'executable-path.cc',
'file-content-address.cc',
'file-descriptor.cc',
'file-system-at.cc',
'file-system.cc',
'git.cc',
'hash.cc',
@@ -75,6 +76,7 @@ sources = files(
'pool.cc',
'position.cc',
'processes.cc',
'ref.cc',
'sort.cc',
'source-accessor.cc',
'spawn.cc',

88
src/libutil-tests/ref.cc Normal file
View File

@@ -0,0 +1,88 @@
#include <gtest/gtest.h>
#include <type_traits>
#include "nix/util/demangle.hh"
#include "nix/util/ref.hh"
namespace nix {
// Test hierarchy for ref covariance tests
struct Base
{
virtual ~Base() = default;
};
struct Derived : Base
{};
TEST(ref, upcast_is_implicit)
{
// ref<Derived> should be implicitly convertible to ref<Base>
static_assert(std::is_convertible_v<ref<Derived>, ref<Base>>);
// Runtime test
auto derived = make_ref<Derived>();
ref<Base> base = derived; // implicit upcast
EXPECT_NE(&*base, nullptr);
}
TEST(ref, downcast_is_rejected)
{
// ref<Base> should NOT be implicitly convertible to ref<Derived>
static_assert(!std::is_convertible_v<ref<Base>, ref<Derived>>);
// Uncomment to see error message:
// auto base = make_ref<Base>();
// ref<Derived> d = base;
}
TEST(ref, same_type_conversion)
{
// ref<T> should be convertible to ref<T>
static_assert(std::is_convertible_v<ref<Base>, ref<Base>>);
static_assert(std::is_convertible_v<ref<Derived>, ref<Derived>>);
}
TEST(ref, explicit_downcast_with_cast)
{
// .cast() should work for valid downcasts at runtime
auto derived = make_ref<Derived>();
ref<Base> base = derived;
// Downcast back to Derived using .cast()
ref<Derived> backToDerived = base.cast<Derived>();
EXPECT_NE(&*backToDerived, nullptr);
}
TEST(ref, invalid_cast_throws)
{
// .cast() throws bad_ref_cast (a std::bad_cast subclass) with type info on invalid downcast
// (unlike .dynamic_pointer_cast() which returns nullptr)
auto base = make_ref<Base>();
try {
base.cast<Derived>();
FAIL() << "Expected bad_ref_cast";
} catch (const bad_ref_cast & e) {
std::string expected = "ref<" + demangle(typeid(Base).name()) + "> cannot be cast to ref<"
+ demangle(typeid(Derived).name()) + ">";
EXPECT_EQ(e.what(), expected);
}
}
TEST(ref, explicit_downcast_with_dynamic_pointer_cast)
{
// .dynamic_pointer_cast() returns nullptr for invalid casts
auto base = make_ref<Base>();
// Invalid downcast returns nullptr
auto invalidCast = base.dynamic_pointer_cast<Derived>();
EXPECT_EQ(invalidCast, nullptr);
// Valid downcast returns non-null
auto derived = make_ref<Derived>();
ref<Base> baseFromDerived = derived;
auto validCast = baseFromDerived.dynamic_pointer_cast<Derived>();
EXPECT_NE(validCast, nullptr);
}
} // namespace nix

View File

@@ -1,6 +1,6 @@
#include <gtest/gtest.h>
#include "nix/util/file-descriptor.hh"
#include "nix/util/file-system-at.hh"
#include "nix/util/file-system.hh"
#include "nix/util/fs-sink.hh"
#include "nix/util/processes.hh"

View File

@@ -1,3 +1,3 @@
sources += files(
'file-descriptor.cc',
'file-system-at.cc',
)

View File

@@ -40,9 +40,9 @@ DirectoryIterator::DirectoryIterator(const std::filesystem::path & p)
// **Attempt to create the underlying directory_iterator**
it_ = std::filesystem::directory_iterator(p);
} catch (const std::filesystem::filesystem_error & e) {
// **Catch filesystem_error and throw SysError**
// Adapt the error message as needed for SysError
throw SysError("cannot read directory %s", PathFmt(p));
// **Catch filesystem_error and throw SystemError**
// Adapt the error message as needed for SystemError
throw SystemError(e.code(), "cannot read directory %s", PathFmt(p));
}
}
@@ -183,9 +183,11 @@ bool isInDir(const std::filesystem::path & path, const std::filesystem::path & d
/* Note that while the standard doesn't guarantee this, the
`lexically_*` functions should do no IO and not throw. */
auto rel = path.lexically_relative(dir);
/* Method from
https://stackoverflow.com/questions/62503197/check-if-path-contains-another-in-c++ */
return !rel.empty() && rel.native()[0] != OS_STR('.');
if (rel.empty())
return false;
auto first = *rel.begin();
return first != "." && first != "..";
}
bool isDirOrInDir(const std::filesystem::path & path, const std::filesystem::path & dir)
@@ -278,7 +280,11 @@ bool pathAccessible(const std::filesystem::path & path)
std::filesystem::path readLink(const std::filesystem::path & path)
{
checkInterrupt();
return std::filesystem::read_symlink(path);
try {
return std::filesystem::read_symlink(path);
} catch (std::filesystem::filesystem_error & e) {
throw SystemError(e.code(), "reading symbolic link '%s'", PathFmt(path));
}
}
Path readLink(const Path & path)
@@ -461,7 +467,7 @@ void createDirs(const std::filesystem::path & path)
try {
std::filesystem::create_directories(path);
} catch (std::filesystem::filesystem_error & e) {
throw SysError("creating directory '%1%'", path.string());
throw SystemError(e.code(), "creating directory '%1%'", path.string());
}
}
@@ -662,7 +668,7 @@ void replaceSymlink(const std::filesystem::path & target, const std::filesystem:
} catch (std::filesystem::filesystem_error & e) {
if (e.code() == std::errc::file_exists)
continue;
throw SysError("creating symlink %1% -> %2%", PathFmt(tmp), PathFmt(target));
throw SystemError(e.code(), "creating symlink %1% -> %2%", PathFmt(tmp), PathFmt(target));
}
try {
@@ -670,7 +676,7 @@ void replaceSymlink(const std::filesystem::path & target, const std::filesystem:
} catch (std::filesystem::filesystem_error & e) {
if (e.code() == std::errc::file_exists)
continue;
throw SysError("renaming %1% to %2%", PathFmt(tmp), PathFmt(link));
throw SystemError(e.code(), "renaming %1% to %2%", PathFmt(tmp), PathFmt(link));
}
break;
@@ -771,7 +777,7 @@ std::filesystem::path makeParentCanonical(const std::filesystem::path & rawPath)
}
return std::filesystem::canonical(parent) / path.filename();
} catch (std::filesystem::filesystem_error & e) {
throw SysError("canonicalising parent path of %1%", PathFmt(path));
throw SystemError(e.code(), "canonicalising parent path of %1%", PathFmt(path));
}
}

View File

@@ -2,6 +2,7 @@
#include "nix/util/error.hh"
#include "nix/util/config-global.hh"
#include "nix/util/file-system-at.hh"
#include "nix/util/fs-sink.hh"
#ifdef _WIN32

View File

@@ -0,0 +1,26 @@
#pragma once
///@file
#include <cstdlib>
#include <cxxabi.h>
#include <string>
namespace nix {
/**
* Demangle a C++ type name.
* Returns the demangled name, or the original if demangling fails.
*/
inline std::string demangle(const char * name)
{
int status;
char * demangled = abi::__cxa_demangle(name, nullptr, nullptr, &status);
if (demangled) {
std::string result(demangled);
std::free(demangled);
return result;
}
return name;
}
} // namespace nix

View File

@@ -17,6 +17,7 @@
#include "nix/util/suggestions.hh"
#include "nix/util/fmt.hh"
#include "nix/util/config.hh"
#include <cstring>
#include <list>
@@ -126,20 +127,20 @@ public:
BaseError & operator=(BaseError &&) = default;
template<typename... Args>
BaseError(unsigned int status, const Args &... args)
: err{.level = lvlError, .msg = HintFmt(args...), .pos = {}, .status = status}
BaseError(unsigned int status, Args &&... args)
: err{.level = lvlError, .msg = HintFmt(std::forward<Args>(args)...), .pos = {}, .status = status}
{
}
template<typename... Args>
explicit BaseError(const std::string & fs, const Args &... args)
: err{.level = lvlError, .msg = HintFmt(fs, args...), .pos = {}}
explicit BaseError(const std::string & fs, Args &&... args)
: err{.level = lvlError, .msg = HintFmt(fs, std::forward<Args>(args)...), .pos = {}}
{
}
template<typename... Args>
BaseError(const Suggestions & sug, const Args &... args)
: err{.level = lvlError, .msg = HintFmt(args...), .pos = {}, .suggestions = sug}
BaseError(const Suggestions & sug, Args &&... args)
: err{.level = lvlError, .msg = HintFmt(std::forward<Args>(args)...), .pos = {}, .suggestions = sug}
{
}
@@ -203,9 +204,9 @@ public:
* @param args... Format string arguments.
*/
template<typename... Args>
void addTrace(std::shared_ptr<const Pos> && pos, std::string_view fs, const Args &... args)
void addTrace(std::shared_ptr<const Pos> && pos, std::string_view fs, Args &&... args)
{
addTrace(std::move(pos), HintFmt(std::string(fs), args...));
addTrace(std::move(pos), HintFmt(std::string(fs), std::forward<Args>(args)...));
}
/**
@@ -247,19 +248,39 @@ MakeError(UnimplementedError, Error);
class SystemError : public Error
{
std::error_code errorCode;
std::string errorDetails;
public:
protected:
/**
* Just here to allow derived classes to use the right constructor
* (the protected one).
*/
struct Disambig
{};
/**
* Protected constructor for subclasses that provide their own error message.
* The error message is appended to the formatted hint.
*/
template<typename... Args>
SystemError(std::errc posixErrNo, Args &&... args)
: Error(std::forward<Args>(args)...)
, errorCode(std::make_error_code(posixErrNo))
SystemError(Disambig, std::error_code errorCode, std::string_view errorDetails, Args &&... args)
: Error("")
, errorCode(errorCode)
, errorDetails(errorDetails)
{
auto hf = HintFmt(std::forward<Args>(args)...);
err.msg = HintFmt("%s: %s", Uncolored(hf.str()), errorDetails);
}
public:
/**
* Construct with an error code. The error code's message is automatically
* appended to the error message.
*/
template<typename... Args>
SystemError(std::error_code errorCode, Args &&... args)
: Error(std::forward<Args>(args)...)
, errorCode(errorCode)
: SystemError(Disambig{}, errorCode, errorCode.message(), std::forward<Args>(args)...)
{
}
@@ -300,12 +321,14 @@ public:
* will be used to try to add additional information to the message.
*/
template<typename... Args>
SysError(int errNo, const Args &... args)
: SystemError(static_cast<std::errc>(errNo), "")
SysError(int errNo, Args &&... args)
: SystemError(
Disambig{},
std::make_error_code(static_cast<std::errc>(errNo)),
strerror(errNo),
std::forward<Args>(args)...)
, errNo(errNo)
{
auto hf = HintFmt(args...);
err.msg = HintFmt("%1%: %2%", Uncolored(hf.str()), strerror(errNo));
}
/**
@@ -315,8 +338,8 @@ public:
* calling this constructor!
*/
template<typename... Args>
SysError(const Args &... args)
: SysError(errno, args...)
SysError(Args &&... args)
: SysError(errno, std::forward<Args>(args)...)
{
}
};
@@ -350,6 +373,15 @@ int handleExceptions(const std::string & programName, std::function<void()> fun)
*/
[[gnu::noinline, gnu::cold, noreturn]] void unreachable(std::source_location loc = std::source_location::current());
#if NIX_UBSAN_ENABLED
/* When building with sanitizers, also enable expensive unreachable checks. In
optimised builds this explicitly invokes UB with std::unreachable for better
optimisations. */
# define nixUnreachableWhenHardened ::nix::unreachable
#else
# define nixUnreachableWhenHardened std::unreachable
#endif
#ifdef _WIN32
namespace windows {
@@ -371,12 +403,14 @@ public:
* information to the message.
*/
template<typename... Args>
WinError(DWORD lastError, const Args &... args)
: SystemError(std::error_code(lastError, std::system_category()), "")
WinError(DWORD lastError, Args &&... args)
: SystemError(
Disambig{},
std::error_code(lastError, std::system_category()),
renderError(lastError),
std::forward<Args>(args)...)
, lastError(lastError)
{
auto hf = HintFmt(args...);
err.msg = HintFmt("%1%: %2%", Uncolored(hf.str()), renderError(lastError));
}
/**
@@ -386,14 +420,14 @@ public:
* before calling this constructor!
*/
template<typename... Args>
WinError(const Args &... args)
: WinError(GetLastError(), args...)
WinError(Args &&... args)
: WinError(GetLastError(), std::forward<Args>(args)...)
{
}
private:
std::string renderError(DWORD lastError);
static std::string renderError(DWORD lastError);
};
} // namespace windows

View File

@@ -1,5 +1,13 @@
#pragma once
///@file
/**
* @file
*
* @brief File descriptor operations for almost arbitrary file
* descriptors.
*
* More specialized file-system-specific operations are in
* @ref file-system-at.hh.
*/
#include "nix/util/canon-path.hh"
#include "nix/util/error.hh"
@@ -184,14 +192,6 @@ std::string drainFD(Descriptor fd, DrainFdOpts opts = {});
*/
void drainFD(Descriptor fd, Sink & sink, DrainFdSinkOpts opts = {});
/**
* Read a symlink relative to a directory file descriptor.
*
* @throws SystemError on any I/O errors.
* @throws Interrupted if interrupted.
*/
OsString readLinkAt(Descriptor dirFd, const CanonPath & path);
/**
* Get [Standard Input](https://en.wikipedia.org/wiki/Standard_streams#Standard_input_(stdin))
*/
@@ -288,77 +288,6 @@ void closeOnExec(Descriptor fd);
} // namespace unix
#endif
#ifdef __linux__
namespace linux {
/**
* Wrapper around Linux's openat2 syscall introduced in Linux 5.6.
*
* @see https://man7.org/linux/man-pages/man2/openat2.2.html
* @see https://man7.org/linux/man-pages/man2/open_how.2type.html
v*
* @param flags O_* flags
* @param mode Mode for O_{CREAT,TMPFILE}
* @param resolve RESOLVE_* flags
*
* @return nullopt if openat2 is not supported by the kernel.
*/
std::optional<Descriptor> openat2(Descriptor dirFd, const char * path, uint64_t flags, uint64_t mode, uint64_t resolve);
} // namespace linux
#endif
/**
* Safe(r) function to open a file relative to dirFd, while
* disallowing escaping from a directory and any symlinks in the process.
*
* @note On Windows, implemented via NtCreateFile single path component traversal
* with FILE_OPEN_REPARSE_POINT. On Unix, uses RESOLVE_BENEATH with openat2 when
* available, or falls back to openat single path component traversal.
*
* @param dirFd Directory handle to open relative to
* @param path Relative path (no .. or . components)
* @param desiredAccess (Windows) Windows ACCESS_MASK (e.g., GENERIC_READ, FILE_WRITE_DATA)
* @param createOptions (Windows) Windows create options (e.g., FILE_NON_DIRECTORY_FILE)
* @param createDisposition (Windows) FILE_OPEN, FILE_CREATE, etc.
* @param flags (Unix) O_* flags
* @param mode (Unix) Mode for O_{CREAT,TMPFILE}
*
* @pre path.isRoot() is false
*
* @throws SymlinkNotAllowed if any path components are symlinks
* @throws SystemError on other errors
*/
Descriptor openFileEnsureBeneathNoSymlinks(
Descriptor dirFd,
const CanonPath & path,
#ifdef _WIN32
ACCESS_MASK desiredAccess,
ULONG createOptions,
ULONG createDisposition = FILE_OPEN
#else
int flags,
mode_t mode = 0
#endif
);
#ifndef _WIN32
namespace unix {
/**
* Try to change the mode of file named by \ref path relative to the parent directory denoted by \ref dirFd.
*
* @note When on linux without fchmodat2 support and without procfs mounted falls back to fchmodat without
* AT_SYMLINK_NOFOLLOW, since it's the best we can do without failing.
*
* @pre path.isRoot() is false
* @throws SysError if any operation fails
*/
void fchmodatTryNoFollow(Descriptor dirFd, const CanonPath & path, mode_t mode);
} // namespace unix
#endif
MakeError(EndOfFile, Error);
#ifdef _WIN32

View File

@@ -0,0 +1,107 @@
#pragma once
/**
* @file
*
* @brief File system operations relative to directory file descriptors.
*
* This header provides cross-platform wrappers for POSIX `*at` functions
* (e.g., `symlinkat`, `mkdirat`, `readlinkat`) that operate relative to
* a directory file descriptor.
*
* Prefer this to @ref file-system.hh because file descriptor-based file
* system operations are necessary to avoid
* [TOCTOU](https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use)
* issues.
*/
#include "nix/util/file-descriptor.hh"
#include <optional>
#ifdef _WIN32
# define WIN32_LEAN_AND_MEAN
# include <windows.h>
#endif
namespace nix {
/**
* Read a symlink relative to a directory file descriptor.
*
* @throws SystemError on any I/O errors.
* @throws Interrupted if interrupted.
*/
OsString readLinkAt(Descriptor dirFd, const CanonPath & path);
/**
* Safe(r) function to open a file relative to dirFd, while
* disallowing escaping from a directory and any symlinks in the process.
*
* @note On Windows, implemented via NtCreateFile single path component traversal
* with FILE_OPEN_REPARSE_POINT. On Unix, uses RESOLVE_BENEATH with openat2 when
* available, or falls back to openat single path component traversal.
*
* @param dirFd Directory handle to open relative to
* @param path Relative path (no .. or . components)
* @param desiredAccess (Windows) Windows ACCESS_MASK (e.g., GENERIC_READ, FILE_WRITE_DATA)
* @param createOptions (Windows) Windows create options (e.g., FILE_NON_DIRECTORY_FILE)
* @param createDisposition (Windows) FILE_OPEN, FILE_CREATE, etc.
* @param flags (Unix) O_* flags
* @param mode (Unix) Mode for O_{CREAT,TMPFILE}
*
* @pre path.isRoot() is false
*
* @throws SymlinkNotAllowed if any path components are symlinks
* @throws SystemError on other errors
*/
Descriptor openFileEnsureBeneathNoSymlinks(
Descriptor dirFd,
const CanonPath & path,
#ifdef _WIN32
ACCESS_MASK desiredAccess,
ULONG createOptions,
ULONG createDisposition = FILE_OPEN
#else
int flags,
mode_t mode = 0
#endif
);
#ifdef __linux__
namespace linux {
/**
* Wrapper around Linux's openat2 syscall introduced in Linux 5.6.
*
* @see https://man7.org/linux/man-pages/man2/openat2.2.html
* @see https://man7.org/linux/man-pages/man2/open_how.2type.html
*
* @param flags O_* flags
* @param mode Mode for O_{CREAT,TMPFILE}
* @param resolve RESOLVE_* flags
*
* @return nullopt if openat2 is not supported by the kernel.
*/
std::optional<Descriptor> openat2(Descriptor dirFd, const char * path, uint64_t flags, uint64_t mode, uint64_t resolve);
} // namespace linux
#endif
#ifndef _WIN32
namespace unix {
/**
* Try to change the mode of file named by \ref path relative to the parent directory denoted by \ref dirFd.
*
* @note When on linux without fchmodat2 support and without procfs mounted falls back to fchmodat without
* AT_SYMLINK_NOFOLLOW, since it's the best we can do without failing.
*
* @pre path.isRoot() is false
* @throws SysError if any operation fails
*/
void fchmodatTryNoFollow(Descriptor dirFd, const CanonPath & path, mode_t mode);
} // namespace unix
#endif
} // namespace nix

View File

@@ -2,7 +2,10 @@
/**
* @file
*
* Utilities for working with the file system and file paths.
* @brief Utilities for working with the file system and file paths.
*
* Please try to use @ref file-system-at.hh instead of this where
* possible, for the reasons given in the documentation of that header.
*/
#include "nix/util/types.hh"
@@ -186,21 +189,19 @@ Path readLink(const Path & path);
*/
std::filesystem::path readLink(const std::filesystem::path & path);
#ifdef _WIN32
namespace windows {
/**
* Get the path associated with a file handle.
* Get the path associated with a file descriptor.
*
* @note One MUST only use this for error handling, because it creates
* TOCTOU issues. We don't mind if error messages point to out of date
* paths (that is a rather trivial TOCTOU --- the error message is best
* effort) but for anything else we do.
*
* @note this function will clobber `errno` (Unix) / "last error"
* (Windows), so care must be used to get those error codes, then call
* this, then build a `SysError` / `WinError` with the saved error code.
*/
std::filesystem::path handleToPath(Descriptor handle);
} // namespace windows
#endif
std::filesystem::path descriptorToPath(Descriptor fd);
/**
* Open a `Descriptor` with read-only access to the given directory.

View File

@@ -2,7 +2,12 @@
include_dirs = [ include_directories('../..') ]
headers = files(
config_pub_h = configure_file(
configuration : configdata_pub,
output : 'config.hh',
)
headers = [ config_pub_h ] + files(
'abstract-setting-to-json.hh',
'alignment.hh',
'ansicolor.hh',
@@ -26,6 +31,7 @@ headers = files(
'config-impl.hh',
'configuration.hh',
'current-process.hh',
'demangle.hh',
'english.hh',
'environment-variables.hh',
'error.hh',
@@ -37,6 +43,7 @@ headers = files(
'file-descriptor.hh',
'file-path-impl.hh',
'file-path.hh',
'file-system-at.hh',
'file-system.hh',
'finally.hh',
'fmt.hh',

View File

@@ -3,9 +3,46 @@
#include <memory>
#include <stdexcept>
#include <string>
#include <typeinfo>
#include "nix/util/demangle.hh"
namespace nix {
/**
* Exception thrown by ref::cast() when dynamic_pointer_cast fails.
* Inherits from std::bad_cast for semantic correctness, but carries a message with type info.
*/
class bad_ref_cast : public std::bad_cast
{
std::string msg;
public:
bad_ref_cast(std::string msg)
: msg(std::move(msg))
{
}
const char * what() const noexcept override
{
return msg.c_str();
}
};
/**
* Concept for implicit ref covariance: From* must be implicitly convertible to To*.
*
* This allows implicit upcasts (Derived -> Base) but rejects downcasts.
*/
// Design note: This named concept is technically redundant but provides a readable hint
// in error messages. Alternative: static_assert can have custom messages, but doesn't
// participate in SFINAE, so std::is_convertible_v<ref<Base>, ref<Derived>> would
// incorrectly return true (the conversion would exist but fail at instantiation
// rather than being excluded).
template<typename From, typename To>
concept RefImplicitlyUpcastableTo = std::is_convertible_v<From *, To *>;
/**
* A simple non-nullable reference-counted pointer. Actually a wrapper
* around std::shared_ptr that prevents null constructions.
@@ -76,7 +113,11 @@ public:
template<typename T2>
ref<T2> cast() const
{
return ref<T2>(std::dynamic_pointer_cast<T2>(p));
auto casted = std::dynamic_pointer_cast<T2>(p);
if (!casted)
throw bad_ref_cast(
"ref<" + demangle(typeid(T).name()) + "> cannot be cast to ref<" + demangle(typeid(T2).name()) + ">");
return ref<T2>(std::move(casted));
}
template<typename T2>
@@ -85,10 +126,15 @@ public:
return std::dynamic_pointer_cast<T2>(p);
}
/**
* Implicit conversion to ref of base type (covariance).
* Downcasts are rejected; use .cast() (throws bad_ref_cast) or .dynamic_pointer_cast() (returns nullptr) instead.
*/
template<typename T2>
requires RefImplicitlyUpcastableTo<T, T2>
operator ref<T2>() const
{
return ref<T2>((std::shared_ptr<T2>) p);
return ref<T2>(p);
}
bool operator==(const ref<T> & other) const

View File

@@ -16,7 +16,8 @@ cxx = meson.get_compiler('cpp')
subdir('nix-meson-build-support/deps-lists')
configdata = configuration_data()
configdata_pub = configuration_data()
configdata_priv = configuration_data()
deps_private_maybe_subproject = []
deps_public_maybe_subproject = []
@@ -34,9 +35,15 @@ check_funcs = [
foreach funcspec : check_funcs
define_name = 'HAVE_' + funcspec[0].underscorify().to_upper()
define_value = cxx.has_function(funcspec[0]).to_int()
configdata.set(define_name, define_value, description : funcspec[1])
configdata_priv.set(define_name, define_value, description : funcspec[1])
endforeach
configdata_pub.set(
'NIX_UBSAN_ENABLED',
('undefined' in get_option('b_sanitize')).to_int(),
description : 'Whether nix has been built with UBSan enabled',
)
subdir('nix-meson-build-support/libatomic')
if host_machine.system() == 'windows'
@@ -104,7 +111,7 @@ cpuid = dependency(
version : '>= 0.7.0',
required : cpuid_required,
)
configdata.set('HAVE_LIBCPUID', cpuid.found().to_int())
configdata_priv.set('HAVE_LIBCPUID', cpuid.found().to_int())
deps_private += cpuid
nlohmann_json = dependency('nlohmann_json', version : '>= 3.9')
@@ -113,7 +120,7 @@ deps_public += nlohmann_json
cxx = meson.get_compiler('cpp')
config_priv_h = configure_file(
configuration : configdata,
configuration : configdata_priv,
output : 'util-config-private.hh',
)

View File

@@ -170,7 +170,7 @@ SourceAccessor::DirEntries PosixSourceAccessor::readDirectory(const CanonPath &
if (e.code() == std::errc::permission_denied || e.code() == std::errc::operation_not_permitted)
return std::nullopt;
else
throw;
throw SystemError(e.code(), "getting status of '%s'", PathFmt(entry.path()));
}
}();
res.emplace(entry.path().filename().string(), type);

View File

@@ -1,32 +1,13 @@
#include "nix/util/canon-path.hh"
#include "nix/util/file-system.hh"
#include "nix/util/signals.hh"
#include "nix/util/finally.hh"
#include "nix/util/serialise.hh"
#include "nix/util/source-accessor.hh"
#include <fcntl.h>
#include <unistd.h>
#include <poll.h>
#include <span>
#if defined(__linux__)
# include <sys/syscall.h> /* pull __NR_* definitions */
#endif
#if defined(__linux__) && defined(__NR_openat2)
# define HAVE_OPENAT2 1
# include <linux/openat2.h>
#else
# define HAVE_OPENAT2 0
#endif
#if defined(__linux__) && defined(__NR_fchmodat2)
# define HAVE_FCHMODAT2 1
#else
# define HAVE_FCHMODAT2 0
#endif
#include "util-config-private.hh"
#include "util-unix-config-private.hh"
@@ -68,7 +49,8 @@ void readFull(int fd, char * buf, size_t count)
pollFD(fd, POLLIN);
continue;
}
throw SysError("reading from file");
auto savedErrno = errno;
throw SysError(savedErrno, "reading from file %s", PathFmt(descriptorToPath(fd)));
}
if (res == 0)
throw EndOfFile("unexpected end-of-file");
@@ -91,7 +73,8 @@ void writeFull(int fd, std::string_view s, bool allowInterrupts)
pollFD(fd, POLLOUT);
continue;
}
throw SysError("writing to file");
auto savedErrno = errno;
throw SysError(savedErrno, "writing to file %s", PathFmt(descriptorToPath(fd)));
}
if (res > 0)
s.remove_prefix(res);
@@ -114,8 +97,10 @@ std::string readLine(int fd, bool eofOk, char terminator)
pollFD(fd, POLLIN);
continue;
}
default:
throw SysError("reading a line");
default: {
auto savedErrno = errno;
throw SysError(savedErrno, "reading a line from %s", PathFmt(descriptorToPath(fd)));
}
}
} else if (rd == 0) {
if (eofOk)
@@ -222,192 +207,4 @@ void unix::closeOnExec(int fd)
throw SysError("setting close-on-exec flag");
}
#ifdef __linux__
namespace linux {
std::optional<Descriptor> openat2(Descriptor dirFd, const char * path, uint64_t flags, uint64_t mode, uint64_t resolve)
{
# if HAVE_OPENAT2
/* Cache the result of whether openat2 is not supported. */
static std::atomic_flag unsupported{};
if (!unsupported.test()) {
/* No glibc wrapper yet, but there's a patch:
* https://patchwork.sourceware.org/project/glibc/patch/20251029200519.3203914-1-adhemerval.zanella@linaro.org/
*/
auto how = ::open_how{.flags = flags, .mode = mode, .resolve = resolve};
auto res = ::syscall(__NR_openat2, dirFd, path, &how, sizeof(how));
/* Cache that the syscall is not supported. */
if (res < 0 && errno == ENOSYS) {
unsupported.test_and_set();
return std::nullopt;
}
return res;
}
# endif
return std::nullopt;
}
} // namespace linux
#endif
void unix::fchmodatTryNoFollow(Descriptor dirFd, const CanonPath & path, mode_t mode)
{
assert(!path.isRoot());
#if HAVE_FCHMODAT2
/* Cache whether fchmodat2 is not supported. */
static std::atomic_flag fchmodat2Unsupported{};
if (!fchmodat2Unsupported.test()) {
/* Try with fchmodat2 first. */
auto res = ::syscall(__NR_fchmodat2, dirFd, path.rel_c_str(), mode, AT_SYMLINK_NOFOLLOW);
/* Cache that the syscall is not supported. */
if (res < 0) {
if (errno == ENOSYS)
fchmodat2Unsupported.test_and_set();
else
throw SysError("fchmodat2 '%s' relative to parent directory", path.rel());
} else
return;
}
#endif
#ifdef __linux__
AutoCloseFD pathFd = ::openat(dirFd, path.rel_c_str(), O_PATH | O_NOFOLLOW | O_CLOEXEC);
if (!pathFd)
throw SysError(
"opening '%s' relative to parent directory to get an O_PATH file descriptor (fchmodat2 is unsupported)",
path.rel());
struct ::stat st;
/* Possible since https://github.com/torvalds/linux/commit/55815f70147dcfa3ead5738fd56d3574e2e3c1c2 (3.6) */
if (::fstat(pathFd.get(), &st) == -1)
throw SysError("statting '%s' relative to parent directory via O_PATH file descriptor", path.rel());
if (S_ISLNK(st.st_mode))
throw SysError(EOPNOTSUPP, "can't change mode of symlink '%s' relative to parent directory", path.rel());
static std::atomic_flag dontHaveProc{};
if (!dontHaveProc.test()) {
static const CanonPath selfProcFd = CanonPath("/proc/self/fd");
auto selfProcFdPath = selfProcFd / std::to_string(pathFd.get());
if (int res = ::chmod(selfProcFdPath.c_str(), mode); res == -1) {
if (errno == ENOENT)
dontHaveProc.test_and_set();
else
throw SysError("chmod '%s' ('%s' relative to parent directory)", selfProcFdPath, path);
} else
return;
}
static std::atomic<bool> warned = false;
warnOnce(warned, "kernel doesn't support fchmodat2 and procfs isn't mounted, falling back to fchmodat");
#endif
int res = ::fchmodat(
dirFd,
path.rel_c_str(),
mode,
#if defined(__APPLE__) || defined(__FreeBSD__)
AT_SYMLINK_NOFOLLOW
#else
0
#endif
);
if (res == -1)
throw SysError("fchmodat '%s' relative to parent directory", path.rel());
}
static Descriptor
openFileEnsureBeneathNoSymlinksIterative(Descriptor dirFd, const CanonPath & path, int flags, mode_t mode)
{
AutoCloseFD parentFd;
auto nrComponents = std::ranges::distance(path);
assert(nrComponents >= 1);
auto components = std::views::take(path, nrComponents - 1); /* Everything but last component */
auto getParentFd = [&]() { return parentFd ? parentFd.get() : dirFd; };
/* This rather convoluted loop is necessary to avoid TOCTOU when validating that
no inner path component is a symlink. */
for (auto it = components.begin(); it != components.end(); ++it) {
auto component = std::string(*it); /* Copy into a string to make NUL terminated. */
assert(component != ".." && !component.starts_with('/')); /* In case invariant is broken somehow.. */
AutoCloseFD parentFd2 = ::openat(
getParentFd(), /* First iteration uses dirFd. */
component.c_str(),
O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC
#ifdef __linux__
| O_PATH /* Linux-specific optimization. Files are open only for path resolution purposes. */
#endif
#ifdef __FreeBSD__
| O_RESOLVE_BENEATH /* Further guard against any possible SNAFUs. */
#endif
);
if (!parentFd2) {
/* Construct the CanonPath for error message. */
auto path2 = std::ranges::fold_left(components.begin(), ++it, CanonPath::root, [](auto lhs, auto rhs) {
lhs.push(rhs);
return lhs;
});
if (errno == ENOTDIR) /* Path component might be a symlink. */ {
struct ::stat st;
if (::fstatat(getParentFd(), component.c_str(), &st, AT_SYMLINK_NOFOLLOW) == 0 && S_ISLNK(st.st_mode))
throw SymlinkNotAllowed(path2);
errno = ENOTDIR; /* Restore the errno. */
} else if (errno == ELOOP) {
throw SymlinkNotAllowed(path2);
}
return INVALID_DESCRIPTOR;
}
parentFd = std::move(parentFd2);
}
auto res = ::openat(getParentFd(), std::string(path.baseName().value()).c_str(), flags | O_NOFOLLOW, mode);
if (res < 0 && errno == ELOOP)
throw SymlinkNotAllowed(path);
return res;
}
Descriptor openFileEnsureBeneathNoSymlinks(Descriptor dirFd, const CanonPath & path, int flags, mode_t mode)
{
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
assert(!path.isRoot());
#if HAVE_OPENAT2
auto maybeFd = linux::openat2(
dirFd, path.rel_c_str(), flags, static_cast<uint64_t>(mode), RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS);
if (maybeFd) {
if (*maybeFd < 0 && errno == ELOOP)
throw SymlinkNotAllowed(path);
return *maybeFd;
}
#endif
return openFileEnsureBeneathNoSymlinksIterative(dirFd, path, flags, mode);
}
OsString readLinkAt(Descriptor dirFd, const CanonPath & path)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
std::vector<char> buf;
for (ssize_t bufSize = PATH_MAX / 4; true; bufSize += bufSize / 2) {
checkInterrupt();
buf.resize(bufSize);
ssize_t rlSize = ::readlinkat(dirFd, path.rel_c_str(), buf.data(), bufSize);
if (rlSize == -1)
throw SysError("reading symbolic link '%1%' relative to parent directory", path.rel());
else if (rlSize < bufSize)
return {buf.data(), static_cast<std::size_t>(rlSize)};
}
}
} // namespace nix

View File

@@ -0,0 +1,227 @@
#include "nix/util/file-system-at.hh"
#include "nix/util/file-system.hh"
#include "nix/util/signals.hh"
#include "nix/util/source-accessor.hh"
#include <fcntl.h>
#include <unistd.h>
#if defined(__linux__)
# include <sys/syscall.h> /* pull __NR_* definitions */
#endif
#if defined(__linux__) && defined(__NR_openat2)
# define HAVE_OPENAT2 1
# include <linux/openat2.h>
#else
# define HAVE_OPENAT2 0
#endif
#if defined(__linux__) && defined(__NR_fchmodat2)
# define HAVE_FCHMODAT2 1
#else
# define HAVE_FCHMODAT2 0
#endif
namespace nix {
#ifdef __linux__
namespace linux {
std::optional<Descriptor> openat2(Descriptor dirFd, const char * path, uint64_t flags, uint64_t mode, uint64_t resolve)
{
# if HAVE_OPENAT2
/* Cache the result of whether openat2 is not supported. */
static std::atomic_flag unsupported{};
if (!unsupported.test()) {
/* No glibc wrapper yet, but there's a patch:
* https://patchwork.sourceware.org/project/glibc/patch/20251029200519.3203914-1-adhemerval.zanella@linaro.org/
*/
auto how = ::open_how{.flags = flags, .mode = mode, .resolve = resolve};
auto res = ::syscall(__NR_openat2, dirFd, path, &how, sizeof(how));
/* Cache that the syscall is not supported. */
if (res < 0 && errno == ENOSYS) {
unsupported.test_and_set();
return std::nullopt;
}
return res;
}
# endif
return std::nullopt;
}
} // namespace linux
#endif
void unix::fchmodatTryNoFollow(Descriptor dirFd, const CanonPath & path, mode_t mode)
{
assert(!path.isRoot());
#if HAVE_FCHMODAT2
/* Cache whether fchmodat2 is not supported. */
static std::atomic_flag fchmodat2Unsupported{};
if (!fchmodat2Unsupported.test()) {
/* Try with fchmodat2 first. */
auto res = ::syscall(__NR_fchmodat2, dirFd, path.rel_c_str(), mode, AT_SYMLINK_NOFOLLOW);
/* Cache that the syscall is not supported. */
if (res < 0) {
if (errno == ENOSYS)
fchmodat2Unsupported.test_and_set();
else {
auto savedErrno = errno;
throw SysError(savedErrno, "fchmodat2 %s", PathFmt(descriptorToPath(dirFd) / path.rel()));
}
} else
return;
}
#endif
#ifdef __linux__
AutoCloseFD pathFd = ::openat(dirFd, path.rel_c_str(), O_PATH | O_NOFOLLOW | O_CLOEXEC);
if (!pathFd) {
auto savedErrno = errno;
throw SysError(
savedErrno,
"opening %s to get an O_PATH file descriptor (fchmodat2 is unsupported)",
PathFmt(descriptorToPath(dirFd) / path.rel()));
}
struct ::stat st;
/* Possible since https://github.com/torvalds/linux/commit/55815f70147dcfa3ead5738fd56d3574e2e3c1c2 (3.6) */
if (::fstat(pathFd.get(), &st) == -1)
throw SysError("statting '%s' relative to parent directory via O_PATH file descriptor", path.rel());
if (S_ISLNK(st.st_mode))
throw SysError(EOPNOTSUPP, "can't change mode of symlink %s", PathFmt(descriptorToPath(dirFd) / path.rel()));
static std::atomic_flag dontHaveProc{};
if (!dontHaveProc.test()) {
static const CanonPath selfProcFd = CanonPath("/proc/self/fd");
auto selfProcFdPath = selfProcFd / std::to_string(pathFd.get());
if (int res = ::chmod(selfProcFdPath.c_str(), mode); res == -1) {
if (errno == ENOENT)
dontHaveProc.test_and_set();
else {
auto savedErrno = errno;
throw SysError(
savedErrno, "chmod %s (%s)", selfProcFdPath, PathFmt(descriptorToPath(dirFd) / path.rel()));
}
} else
return;
}
static std::atomic<bool> warned = false;
warnOnce(warned, "kernel doesn't support fchmodat2 and procfs isn't mounted, falling back to fchmodat");
#endif
int res = ::fchmodat(
dirFd,
path.rel_c_str(),
mode,
#if defined(__APPLE__) || defined(__FreeBSD__)
AT_SYMLINK_NOFOLLOW
#else
0
#endif
);
if (res == -1) {
auto savedErrno = errno;
throw SysError(savedErrno, "fchmodat %s", PathFmt(descriptorToPath(dirFd) / path.rel()));
}
}
static Descriptor
openFileEnsureBeneathNoSymlinksIterative(Descriptor dirFd, const CanonPath & path, int flags, mode_t mode)
{
AutoCloseFD parentFd;
auto nrComponents = std::ranges::distance(path);
assert(nrComponents >= 1);
auto components = std::views::take(path, nrComponents - 1); /* Everything but last component */
auto getParentFd = [&]() { return parentFd ? parentFd.get() : dirFd; };
/* This rather convoluted loop is necessary to avoid TOCTOU when validating that
no inner path component is a symlink. */
for (auto it = components.begin(); it != components.end(); ++it) {
auto component = std::string(*it); /* Copy into a string to make NUL terminated. */
assert(component != ".." && !component.starts_with('/')); /* In case invariant is broken somehow.. */
AutoCloseFD parentFd2 = ::openat(
getParentFd(), /* First iteration uses dirFd. */
component.c_str(),
O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC
#ifdef __linux__
| O_PATH /* Linux-specific optimization. Files are open only for path resolution purposes. */
#endif
#ifdef __FreeBSD__
| O_RESOLVE_BENEATH /* Further guard against any possible SNAFUs. */
#endif
);
if (!parentFd2) {
/* Construct the CanonPath for error message. */
auto path2 = std::ranges::fold_left(components.begin(), ++it, CanonPath::root, [](auto lhs, auto rhs) {
lhs.push(rhs);
return lhs;
});
if (errno == ENOTDIR) /* Path component might be a symlink. */ {
struct ::stat st;
if (::fstatat(getParentFd(), component.c_str(), &st, AT_SYMLINK_NOFOLLOW) == 0 && S_ISLNK(st.st_mode))
throw SymlinkNotAllowed(path2);
errno = ENOTDIR; /* Restore the errno. */
} else if (errno == ELOOP) {
throw SymlinkNotAllowed(path2);
}
return INVALID_DESCRIPTOR;
}
parentFd = std::move(parentFd2);
}
auto res = ::openat(getParentFd(), std::string(path.baseName().value()).c_str(), flags | O_NOFOLLOW, mode);
if (res < 0 && errno == ELOOP)
throw SymlinkNotAllowed(path);
return res;
}
Descriptor openFileEnsureBeneathNoSymlinks(Descriptor dirFd, const CanonPath & path, int flags, mode_t mode)
{
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
assert(!path.isRoot());
#if HAVE_OPENAT2
auto maybeFd = linux::openat2(
dirFd, path.rel_c_str(), flags, static_cast<uint64_t>(mode), RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS);
if (maybeFd) {
if (*maybeFd < 0 && errno == ELOOP)
throw SymlinkNotAllowed(path);
return *maybeFd;
}
#endif
return openFileEnsureBeneathNoSymlinksIterative(dirFd, path, flags, mode);
}
OsString readLinkAt(Descriptor dirFd, const CanonPath & path)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
std::vector<char> buf;
for (ssize_t bufSize = PATH_MAX / 4; true; bufSize += bufSize / 2) {
checkInterrupt();
buf.resize(bufSize);
ssize_t rlSize = ::readlinkat(dirFd, path.rel_c_str(), buf.data(), bufSize);
if (rlSize == -1) {
auto savedErrno = errno;
throw SysError(savedErrno, "reading symbolic link %1%", PathFmt(descriptorToPath(dirFd) / path.rel()));
} else if (rlSize < bufSize)
return {buf.data(), static_cast<std::size_t>(rlSize)};
}
}
} // namespace nix

View File

@@ -14,6 +14,7 @@
#endif
#include "nix/util/file-system.hh"
#include "nix/util/file-system-at.hh"
#include "nix/util/environment-variables.hh"
#include "nix/util/signals.hh"
#include "nix/util/util.hh"
@@ -45,6 +46,31 @@ Descriptor openNewFileForWrite(const std::filesystem::path & path, mode_t mode,
return open(path.c_str(), flags, mode);
}
std::filesystem::path descriptorToPath(Descriptor fd)
{
if (fd == STDIN_FILENO)
return "<stdin>";
if (fd == STDOUT_FILENO)
return "<stdout>";
if (fd == STDERR_FILENO)
return "<stderr>";
#if defined(__linux__)
try {
return readLink("/proc/self/fd/" + std::to_string(fd));
} catch (SystemError &) {
}
#elif HAVE_F_GETPATH
/* F_GETPATH requires PATH_MAX buffer per POSIX */
char buf[PATH_MAX];
if (fcntl(fd, F_GETPATH, buf) != -1)
return buf;
#endif
/* Fallback for unknown fd or unsupported platform */
return "<fd " + std::to_string(fd) + ">";
}
std::filesystem::path defaultTempDir()
{
return getEnvNonEmpty("TMPDIR").value_or("/tmp");

View File

@@ -8,6 +8,12 @@ configdata_unix.set(
description : 'Optionally used for changing the files and symlinks.',
)
configdata_unix.set(
'HAVE_F_GETPATH',
cxx.has_header_symbol('fcntl.h', 'F_GETPATH').to_int(),
description : 'Optionally used for getting the path of a file descriptor (macOS).',
)
# Check for each of these functions, and create a define like `#define
# HAVE_CLOSE_RANGE 1`.
check_funcs_unix = [
@@ -53,6 +59,7 @@ sources += files(
'environment-variables.cc',
'file-descriptor.cc',
'file-path.cc',
'file-system-at.cc',
'file-system.cc',
'muxable-pipe.cc',
'os-string.cc',

View File

@@ -2,8 +2,6 @@
#include "nix/util/signals.hh"
#include "nix/util/finally.hh"
#include "nix/util/serialise.hh"
#include "nix/util/file-path.hh"
#include "nix/util/source-accessor.hh"
#include <span>
@@ -13,8 +11,6 @@
#include <namedpipeapi.h>
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winioctl.h>
#include <winternl.h>
namespace nix {
@@ -49,9 +45,9 @@ void writeFull(HANDLE handle, std::string_view s, bool allowInterrupts)
checkInterrupt();
DWORD res;
if (!WriteFile(handle, s.data(), s.size(), &res, NULL)) {
// Do this because `handleToPath` will overwrite the last error.
// Do this because `descriptorToPath` will overwrite the last error.
auto lastError = GetLastError();
auto path = handleToPath(handle);
auto path = descriptorToPath(handle);
throw WinError(lastError, "writing to file %d:%s", handle, PathFmt(path));
}
if (res > 0)
@@ -152,296 +148,4 @@ off_t lseek(HANDLE h, off_t offset, int whence)
return newPos.QuadPart;
}
namespace windows {
namespace {
/**
* Open a file/directory relative to a directory handle using NtCreateFile.
*
* @param dirFd Directory handle to open relative to
* @param pathComponent Single path component (not a full path)
* @param desiredAccess Access rights requested
* @param createOptions NT create options flags
* @param createDisposition FILE_OPEN, FILE_CREATE, etc.
* @return Handle to the opened file/directory (caller must close)
*/
HANDLE ntOpenAt(
Descriptor dirFd,
std::wstring_view pathComponent,
ACCESS_MASK desiredAccess,
ULONG createOptions,
ULONG createDisposition = FILE_OPEN)
{
/* Set up UNICODE_STRING for the relative path */
UNICODE_STRING pathStr;
pathStr.Buffer = const_cast<PWSTR>(pathComponent.data());
pathStr.Length = static_cast<USHORT>(pathComponent.size() * sizeof(wchar_t));
pathStr.MaximumLength = pathStr.Length;
/* Set up OBJECT_ATTRIBUTES to open relative to dirFd */
OBJECT_ATTRIBUTES objAttrs;
InitializeObjectAttributes(
&objAttrs,
&pathStr,
0, // No special flags
dirFd, // RootDirectory
nullptr // No security descriptor
);
/* Open using NT API */
IO_STATUS_BLOCK ioStatus;
HANDLE h;
NTSTATUS status = NtCreateFile(
&h,
desiredAccess,
&objAttrs,
&ioStatus,
nullptr, // No allocation size
FILE_ATTRIBUTE_NORMAL,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
createDisposition,
createOptions | FILE_SYNCHRONOUS_IO_NONALERT,
nullptr, // No EA buffer
0 // No EA length
);
if (status != 0)
throw WinError(
RtlNtStatusToDosError(status), "opening %s relative to directory handle", PathFmt(pathComponent));
return h;
}
/**
* Open a symlink relative to a directory handle without following it.
*
* @param dirFd Directory handle to open relative to
* @param path Relative path to the symlink
* @return Handle to the symlink (caller must close)
*/
HANDLE openSymlinkAt(Descriptor dirFd, const CanonPath & path)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
std::wstring wpath = string_to_os_string(path.rel());
return ntOpenAt(dirFd, wpath, FILE_READ_ATTRIBUTES | SYNCHRONIZE, FILE_OPEN_REPARSE_POINT);
}
/**
* This struct isn't defined in the normal Windows SDK, but only in the Windows Driver Kit.
*
* I (@Ericson2314) would not normally do something like this, but LLVM
* has decided that this is in fact stable, per
* https://github.com/llvm/llvm-project/blob/main/libcxx/src/filesystem/posix_compat.h,
* so I guess that is good enough for us. GCC doesn't support symlinks
* at all on windows so we have to put it here, not grab it from private
* c++ standard library headers anyways.
*/
struct ReparseDataBuffer
{
unsigned long ReparseTag;
unsigned short ReparseDataLength;
unsigned short Reserved;
union
{
struct
{
unsigned short SubstituteNameOffset;
unsigned short SubstituteNameLength;
unsigned short PrintNameOffset;
unsigned short PrintNameLength;
unsigned long Flags;
wchar_t PathBuffer[1];
} SymbolicLinkReparseBuffer;
struct
{
unsigned short SubstituteNameOffset;
unsigned short SubstituteNameLength;
unsigned short PrintNameOffset;
unsigned short PrintNameLength;
wchar_t PathBuffer[1];
} MountPointReparseBuffer;
struct
{
unsigned char DataBuffer[1];
} GenericReparseBuffer;
};
};
/**
* Read the target of a symlink from an open handle.
*
* @param linkHandle Handle to a symlink (must have been opened with FILE_OPEN_REPARSE_POINT)
* @return The symlink target as a wide string
*/
OsString readSymlinkTarget(HANDLE linkHandle)
{
uint8_t buf[MAXIMUM_REPARSE_DATA_BUFFER_SIZE];
DWORD out;
checkInterrupt();
if (!DeviceIoControl(linkHandle, FSCTL_GET_REPARSE_POINT, nullptr, 0, buf, sizeof(buf), &out, nullptr))
throw WinError("reading reparse point for handle %d", linkHandle);
const auto * reparse = reinterpret_cast<const ReparseDataBuffer *>(buf);
size_t path_buf_offset = offsetof(ReparseDataBuffer, SymbolicLinkReparseBuffer.PathBuffer[0]);
if (out < path_buf_offset) {
auto fullPath = handleToPath(linkHandle);
throw WinError(
DWORD{ERROR_REPARSE_TAG_INVALID}, "invalid reparse data for %d:%s", linkHandle, PathFmt(fullPath));
}
if (reparse->ReparseTag != IO_REPARSE_TAG_SYMLINK) {
auto fullPath = handleToPath(linkHandle);
throw WinError(DWORD{ERROR_REPARSE_TAG_INVALID}, "not a symlink: %d:%s", linkHandle, PathFmt(fullPath));
}
const auto & symlink = reparse->SymbolicLinkReparseBuffer;
unsigned short name_offset, name_length;
/* Prefer PrintName over SubstituteName if available */
if (symlink.PrintNameLength == 0) {
name_offset = symlink.SubstituteNameOffset;
name_length = symlink.SubstituteNameLength;
} else {
name_offset = symlink.PrintNameOffset;
name_length = symlink.PrintNameLength;
}
if (path_buf_offset + name_offset + name_length > out) {
auto fullPath = handleToPath(linkHandle);
throw WinError(
DWORD{ERROR_REPARSE_TAG_INVALID}, "invalid symlink data for %d:%s", linkHandle, PathFmt(fullPath));
}
/* Extract the target path */
const wchar_t * target_start = &symlink.PathBuffer[name_offset / sizeof(wchar_t)];
size_t target_len = name_length / sizeof(wchar_t);
return {target_start, target_len};
}
/**
* Check if a handle refers to a reparse point (e.g., symlink).
*
* @param handle Open file/directory handle
* @return true if the handle refers to a reparse point
*/
bool isReparsePoint(HANDLE handle)
{
FILE_BASIC_INFO basicInfo;
if (!GetFileInformationByHandleEx(handle, FileBasicInfo, &basicInfo, sizeof(basicInfo)))
throw WinError("GetFileInformationByHandleEx");
return (basicInfo.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0;
}
} // anonymous namespace
} // namespace windows
Descriptor openFileEnsureBeneathNoSymlinks(
Descriptor dirFd, const CanonPath & path, ACCESS_MASK desiredAccess, ULONG createOptions, ULONG createDisposition)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
AutoCloseFD parentFd;
auto nrComponents = std::ranges::distance(path);
assert(nrComponents >= 1);
auto components = std::views::take(path, nrComponents - 1); /* Everything but last component */
auto getParentFd = [&]() { return parentFd ? parentFd.get() : dirFd; };
/* Helper to construct CanonPath from components up to (and including) the given iterator */
auto pathUpTo = [&](auto it) {
return std::ranges::fold_left(components.begin(), it, CanonPath::root, [](auto lhs, auto rhs) {
lhs.push(rhs);
return lhs;
});
};
/* Helper to check if a component is a symlink and throw SymlinkNotAllowed if so */
auto throwIfSymlink = [&](std::wstring_view component, const CanonPath & pathForError) {
try {
auto testFd =
ntOpenAt(getParentFd(), component, FILE_READ_ATTRIBUTES | SYNCHRONIZE, FILE_OPEN_REPARSE_POINT);
AutoCloseFD testHandle(testFd);
if (isReparsePoint(testHandle.get()))
throw SymlinkNotAllowed(pathForError);
} catch (SymlinkNotAllowed &) {
throw;
} catch (...) {
/* If we can't determine, ignore and let caller handle original error */
}
};
/* Iterate through each path component to ensure no symlinks in intermediate directories.
* This prevents TOCTOU issues by opening each component relative to the parent. */
for (auto it = components.begin(); it != components.end(); ++it) {
std::wstring wcomponent = string_to_os_string(std::string(*it));
/* Open directory without following symlinks */
AutoCloseFD parentFd2;
try {
parentFd2 = ntOpenAt(
getParentFd(),
wcomponent,
FILE_TRAVERSE | SYNCHRONIZE, // Just need traversal rights
FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT // Open directory, don't follow symlinks
);
} catch (WinError & e) {
/* Check if this is because it's a symlink */
if (e.lastError == ERROR_CANT_ACCESS_FILE || e.lastError == ERROR_ACCESS_DENIED) {
throwIfSymlink(wcomponent, pathUpTo(std::next(it)));
}
throw;
}
/* Check if what we opened is actually a symlink */
if (isReparsePoint(parentFd2.get())) {
throw SymlinkNotAllowed(pathUpTo(std::next(it)));
}
parentFd = std::move(parentFd2);
}
/* Now open the final component with requested flags */
std::wstring finalComponent = string_to_os_string(std::string(path.baseName().value()));
HANDLE finalHandle;
try {
finalHandle = ntOpenAt(
getParentFd(),
finalComponent,
desiredAccess,
createOptions | FILE_OPEN_REPARSE_POINT, // Don't follow symlinks on final component either
createDisposition);
} catch (WinError & e) {
/* Check if final component is a symlink when we requested to not follow it */
if (e.lastError == ERROR_CANT_ACCESS_FILE) {
throwIfSymlink(finalComponent, path);
}
throw;
}
/* Final check: did we accidentally open a symlink? */
if (isReparsePoint(finalHandle))
throw SymlinkNotAllowed(path);
return finalHandle;
}
OsString readLinkAt(Descriptor dirFd, const CanonPath & path)
{
AutoCloseFD linkHandle(windows::openSymlinkAt(dirFd, path));
return windows::readSymlinkTarget(linkHandle.get());
}
} // namespace nix

View File

@@ -0,0 +1,310 @@
#include "nix/util/file-system-at.hh"
#include "nix/util/file-system.hh"
#include "nix/util/signals.hh"
#include "nix/util/file-path.hh"
#include "nix/util/source-accessor.hh"
#include <fileapi.h>
#include <error.h>
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <winioctl.h>
#include <winternl.h>
namespace nix {
using namespace nix::windows;
namespace windows {
namespace {
/**
* Open a file/directory relative to a directory handle using NtCreateFile.
*
* @param dirFd Directory handle to open relative to
* @param pathComponent Single path component (not a full path)
* @param desiredAccess Access rights requested
* @param createOptions NT create options flags
* @param createDisposition FILE_OPEN, FILE_CREATE, etc.
* @return Handle to the opened file/directory (caller must close)
*/
HANDLE ntOpenAt(
Descriptor dirFd,
std::wstring_view pathComponent,
ACCESS_MASK desiredAccess,
ULONG createOptions,
ULONG createDisposition = FILE_OPEN)
{
/* Set up UNICODE_STRING for the relative path */
UNICODE_STRING pathStr;
pathStr.Buffer = const_cast<PWSTR>(pathComponent.data());
pathStr.Length = static_cast<USHORT>(pathComponent.size() * sizeof(wchar_t));
pathStr.MaximumLength = pathStr.Length;
/* Set up OBJECT_ATTRIBUTES to open relative to dirFd */
OBJECT_ATTRIBUTES objAttrs;
InitializeObjectAttributes(
&objAttrs,
&pathStr,
0, // No special flags
dirFd, // RootDirectory
nullptr // No security descriptor
);
/* Open using NT API */
IO_STATUS_BLOCK ioStatus;
HANDLE h;
NTSTATUS status = NtCreateFile(
&h,
desiredAccess,
&objAttrs,
&ioStatus,
nullptr, // No allocation size
FILE_ATTRIBUTE_NORMAL,
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
createDisposition,
createOptions | FILE_SYNCHRONOUS_IO_NONALERT,
nullptr, // No EA buffer
0 // No EA length
);
if (status != 0)
throw WinError(
RtlNtStatusToDosError(status), "opening %s relative to directory handle", PathFmt(pathComponent));
return h;
}
/**
* Open a symlink relative to a directory handle without following it.
*
* @param dirFd Directory handle to open relative to
* @param path Relative path to the symlink
* @return Handle to the symlink (caller must close)
*/
HANDLE openSymlinkAt(Descriptor dirFd, const CanonPath & path)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
std::wstring wpath = string_to_os_string(path.rel());
return ntOpenAt(dirFd, wpath, FILE_READ_ATTRIBUTES | SYNCHRONIZE, FILE_OPEN_REPARSE_POINT);
}
/**
* This struct isn't defined in the normal Windows SDK, but only in the Windows Driver Kit.
*
* I (@Ericson2314) would not normally do something like this, but LLVM
* has decided that this is in fact stable, per
* https://github.com/llvm/llvm-project/blob/main/libcxx/src/filesystem/posix_compat.h,
* so I guess that is good enough for us. GCC doesn't support symlinks
* at all on windows so we have to put it here, not grab it from private
* c++ standard library headers anyways.
*/
struct ReparseDataBuffer
{
unsigned long ReparseTag;
unsigned short ReparseDataLength;
unsigned short Reserved;
union
{
struct
{
unsigned short SubstituteNameOffset;
unsigned short SubstituteNameLength;
unsigned short PrintNameOffset;
unsigned short PrintNameLength;
unsigned long Flags;
wchar_t PathBuffer[1];
} SymbolicLinkReparseBuffer;
struct
{
unsigned short SubstituteNameOffset;
unsigned short SubstituteNameLength;
unsigned short PrintNameOffset;
unsigned short PrintNameLength;
wchar_t PathBuffer[1];
} MountPointReparseBuffer;
struct
{
unsigned char DataBuffer[1];
} GenericReparseBuffer;
};
};
/**
* Read the target of a symlink from an open handle.
*
* @param linkHandle Handle to a symlink (must have been opened with FILE_OPEN_REPARSE_POINT)
* @return The symlink target as a wide string
*/
OsString readSymlinkTarget(HANDLE linkHandle)
{
uint8_t buf[MAXIMUM_REPARSE_DATA_BUFFER_SIZE];
DWORD out;
checkInterrupt();
if (!DeviceIoControl(linkHandle, FSCTL_GET_REPARSE_POINT, nullptr, 0, buf, sizeof(buf), &out, nullptr))
throw WinError("reading reparse point for handle %d", linkHandle);
const auto * reparse = reinterpret_cast<const ReparseDataBuffer *>(buf);
size_t path_buf_offset = offsetof(ReparseDataBuffer, SymbolicLinkReparseBuffer.PathBuffer[0]);
if (out < path_buf_offset) {
auto fullPath = descriptorToPath(linkHandle);
throw WinError(
DWORD{ERROR_REPARSE_TAG_INVALID}, "invalid reparse data for %d:%s", linkHandle, PathFmt(fullPath));
}
if (reparse->ReparseTag != IO_REPARSE_TAG_SYMLINK) {
auto fullPath = descriptorToPath(linkHandle);
throw WinError(DWORD{ERROR_REPARSE_TAG_INVALID}, "not a symlink: %d:%s", linkHandle, PathFmt(fullPath));
}
const auto & symlink = reparse->SymbolicLinkReparseBuffer;
unsigned short name_offset, name_length;
/* Prefer PrintName over SubstituteName if available */
if (symlink.PrintNameLength == 0) {
name_offset = symlink.SubstituteNameOffset;
name_length = symlink.SubstituteNameLength;
} else {
name_offset = symlink.PrintNameOffset;
name_length = symlink.PrintNameLength;
}
if (path_buf_offset + name_offset + name_length > out) {
auto fullPath = descriptorToPath(linkHandle);
throw WinError(
DWORD{ERROR_REPARSE_TAG_INVALID}, "invalid symlink data for %d:%s", linkHandle, PathFmt(fullPath));
}
/* Extract the target path */
const wchar_t * target_start = &symlink.PathBuffer[name_offset / sizeof(wchar_t)];
size_t target_len = name_length / sizeof(wchar_t);
return {target_start, target_len};
}
/**
* Check if a handle refers to a reparse point (e.g., symlink).
*
* @param handle Open file/directory handle
* @return true if the handle refers to a reparse point
*/
bool isReparsePoint(HANDLE handle)
{
FILE_BASIC_INFO basicInfo;
if (!GetFileInformationByHandleEx(handle, FileBasicInfo, &basicInfo, sizeof(basicInfo)))
throw WinError("GetFileInformationByHandleEx");
return (basicInfo.FileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0;
}
} // anonymous namespace
} // namespace windows
Descriptor openFileEnsureBeneathNoSymlinks(
Descriptor dirFd, const CanonPath & path, ACCESS_MASK desiredAccess, ULONG createOptions, ULONG createDisposition)
{
assert(!path.isRoot());
assert(!path.rel().starts_with('/')); /* Just in case the invariant is somehow broken. */
AutoCloseFD parentFd;
auto nrComponents = std::ranges::distance(path);
assert(nrComponents >= 1);
auto components = std::views::take(path, nrComponents - 1); /* Everything but last component */
auto getParentFd = [&]() { return parentFd ? parentFd.get() : dirFd; };
/* Helper to construct CanonPath from components up to (and including) the given iterator */
auto pathUpTo = [&](auto it) {
return std::ranges::fold_left(components.begin(), it, CanonPath::root, [](auto lhs, auto rhs) {
lhs.push(rhs);
return lhs;
});
};
/* Helper to check if a component is a symlink and throw SymlinkNotAllowed if so */
auto throwIfSymlink = [&](std::wstring_view component, const CanonPath & pathForError) {
try {
auto testFd =
ntOpenAt(getParentFd(), component, FILE_READ_ATTRIBUTES | SYNCHRONIZE, FILE_OPEN_REPARSE_POINT);
AutoCloseFD testHandle(testFd);
if (isReparsePoint(testHandle.get()))
throw SymlinkNotAllowed(pathForError);
} catch (SymlinkNotAllowed &) {
throw;
} catch (...) {
/* If we can't determine, ignore and let caller handle original error */
}
};
/* Iterate through each path component to ensure no symlinks in intermediate directories.
* This prevents TOCTOU issues by opening each component relative to the parent. */
for (auto it = components.begin(); it != components.end(); ++it) {
std::wstring wcomponent = string_to_os_string(std::string(*it));
/* Open directory without following symlinks */
AutoCloseFD parentFd2;
try {
parentFd2 = ntOpenAt(
getParentFd(),
wcomponent,
FILE_TRAVERSE | SYNCHRONIZE, // Just need traversal rights
FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT // Open directory, don't follow symlinks
);
} catch (WinError & e) {
/* Check if this is because it's a symlink */
if (e.lastError == ERROR_CANT_ACCESS_FILE || e.lastError == ERROR_ACCESS_DENIED) {
throwIfSymlink(wcomponent, pathUpTo(std::next(it)));
}
throw;
}
/* Check if what we opened is actually a symlink */
if (isReparsePoint(parentFd2.get())) {
throw SymlinkNotAllowed(pathUpTo(std::next(it)));
}
parentFd = std::move(parentFd2);
}
/* Now open the final component with requested flags */
std::wstring finalComponent = string_to_os_string(std::string(path.baseName().value()));
HANDLE finalHandle;
try {
finalHandle = ntOpenAt(
getParentFd(),
finalComponent,
desiredAccess,
createOptions | FILE_OPEN_REPARSE_POINT, // Don't follow symlinks on final component either
createDisposition);
} catch (WinError & e) {
/* Check if final component is a symlink when we requested to not follow it */
if (e.lastError == ERROR_CANT_ACCESS_FILE) {
throwIfSymlink(finalComponent, path);
}
throw;
}
/* Final check: did we accidentally open a symlink? */
if (isReparsePoint(finalHandle))
throw SymlinkNotAllowed(path);
return finalHandle;
}
OsString readLinkAt(Descriptor dirFd, const CanonPath & path)
{
AutoCloseFD linkHandle(windows::openSymlinkAt(dirFd, path));
return windows::readSymlinkTarget(linkHandle.get());
}
} // namespace nix

View File

@@ -83,7 +83,7 @@ void deletePath(const std::filesystem::path & path, uint64_t & bytesFreed)
deletePath(path);
}
std::filesystem::path windows::handleToPath(HANDLE handle)
std::filesystem::path descriptorToPath(Descriptor handle)
{
std::vector<wchar_t> buf(0x100);
DWORD dw = GetFinalPathNameByHandleW(handle, buf.data(), buf.size(), FILE_NAME_OPENED);

View File

@@ -3,6 +3,7 @@ sources += files(
'environment-variables.cc',
'file-descriptor.cc',
'file-path.cc',
'file-system-at.cc',
'file-system.cc',
'known-folders.cc',
'muxable-pipe.cc',

View File

@@ -427,7 +427,9 @@ void mainWrapped(int argc, char ** argv)
evalSettings.pureEval = true;
#ifndef _WIN32
setLogFormat("bar");
#endif
settings.verboseBuild = false;
// If on a terminal, progress will be displayed via progress bars etc. (thus verbosity=notice)

View File

@@ -113,6 +113,7 @@ nix_sources = [ config_priv_h ] + files(
if host_machine.system() != 'windows'
nix_sources += files(
'unix/daemon.cc',
'unix/store-roots-daemon.cc',
)
endif

View File

@@ -43,8 +43,8 @@ void removeOldGenerations(std::filesystem::path dir)
std::string link;
try {
link = readLink(path);
} catch (std::filesystem::filesystem_error & e) {
if (e.code() == std::errc::no_such_file_or_directory)
} catch (SystemError & e) {
if (e.is(std::errc::no_such_file_or_directory))
continue;
throw;
}

View File

@@ -1419,7 +1419,6 @@ static int main_nix_env(int argc, char ** argv)
replaceSymlink(defaultChannelsDir(profilesDirOpts), nixExprPath / "channels");
if (!isRootUser())
replaceSymlink(rootChannelsDir(profilesDirOpts), nixExprPath / "channels_root");
} catch (std::filesystem::filesystem_error &) {
} catch (Error &) {
}
}

View File

@@ -52,8 +52,8 @@ sockets:
```
[Socket]
ListenStream=/nix/var/nix/unix/daemon-socket/socket
ListenStream=/nix/var/nix/unix/daemon-socket/socket-2
ListenStream=/nix/var/nix/daemon-socket/socket
ListenStream=/nix/var/nix/daemon-socket/socket-2
```
)""

View File

@@ -0,0 +1,66 @@
#include "nix/cmd/command.hh"
#include "nix/cmd/unix-socket-server.hh"
#include "nix/store/local-store.hh"
#include "nix/store/store-api.hh"
#include "nix/store/local-gc.hh"
#include "nix/util/file-descriptor.hh"
#include <thread>
using namespace nix;
struct CmdRootsDaemon : StoreConfigCommand
{
CmdRootsDaemon() {}
std::string description() override
{
return "run a daemon that returns garbage collector roots on request";
}
std::string doc() override
{
return
#include "store-roots-daemon.md"
;
}
std::optional<ExperimentalFeature> experimentalFeature() override
{
return Xp::LocalOverlayStore;
}
void run(ref<StoreConfig> storeConfig) override
{
auto localStoreConfig = dynamic_cast<LocalStoreConfig *>(&*storeConfig);
if (!localStoreConfig) {
throw UsageError(
"Roots daemon only functions with a local store, not '%s'", storeConfig->getHumanReadableURI());
}
auto gcSocketPath = localStoreConfig->getRootsSocketPath();
unix::serveUnixSocket(
{
.socketPath = gcSocketPath,
.socketMode = 0666,
},
[&](AutoCloseFD remote, std::function<void()> closeListeners) {
std::thread([&, remote = std::move(remote)]() mutable {
auto roots = findRuntimeRootsUnchecked(*localStoreConfig);
FdSink sink(remote.get());
for (auto & [key, _] : roots) {
sink(localStoreConfig->printStorePath(key));
sink(std::string_view("\0", 1));
}
sink.flush();
remote.close();
}).detach();
});
}
};
static auto rCmdStoreRootsDaemon = registerCommand2<CmdRootsDaemon>({"store", "roots-daemon"});

View File

@@ -0,0 +1,46 @@
R""(
# Examples
* Run the daemon:
```console
# nix store roots-daemon
```
# Description
This command runs a daemon that serves garbage collector roots from a Unix domain socket.
It is not required in all Nix installations, but is useful when the main Nix daemon
is not running as root and therefore cannot find runtime roots by scanning `/proc`.
When the garbage collector runs with [`use-roots-daemon`](@docroot@/store/types/local-store.md#store-experimental-option-use-roots-daemon)
enabled, it connects to this daemon to discover additional roots that should not be collected.
The daemon listens on [`<state-dir>`](@docroot@/store/types/local-store.md#store-option-state)`/gc-roots-socket/socket` (typically `/nix/var/nix/gc-roots-socket/socket`).
# Protocol
The protocol is simple.
For each client-initiated Unix socket connection, the server:
1. Sends zero or more [store paths](@docroot@/store/store-path.md) as NUL-terminated (`\0`) strings.
2. Closes the connection.
Example (with `\0` shown as newlines for clarity):
```
/nix/store/s66mzxpvicwk07gjbjfw9izjfa797vsw-hello-2.12.1
/nix/store/fvpr7x8l3illdnziggvkhdpf6vikg65w-git-2.44.0
```
# Security
No information is provided as to which processes are opening which store paths.
While only the main Nix daemon needs to use this daemon, any user able to talk to the main Nix daemon can already obtain the same information with [`nix-store --gc --print-roots`](@docroot@/command-ref/nix-store/gc.md).
Therefore, restricting this daemon to only accept the Nix daemon as a client is, while recommended for defense-in-depth reasons, strictly speaking not reducing what information can be extracted versus merely restricting this daemon to accept connections from any [allowed user](@docroot@/command-ref/conf-file.md#conf-allowed-users).
# Systemd socket activation
`nix store roots-daemon` supports systemd socket-based activation, [just like `nix-daemon`](@docroot@/command-ref/nix-daemon.md#systemd-socket-activation).
)""

View File

@@ -282,3 +282,14 @@ if isDaemonNewer "2.34pre" && canUseSandbox; then
# Error messages should not be empty (end with just "failed:")
<<<"$out" grepQuietInverse -E "^error:.*failed: *$"
fi
# https://github.com/NixOS/nix/issues/14883
# When max-jobs=0 and no remote builders, the error should say
# "local builds are disabled" instead of the misleading
# "required system or feature not available".
if isDaemonNewer "2.34pre"; then
expectStderr 1 nix build --impure --max-jobs 0 --expr \
'derivation { name = "test-maxjobs"; builder = "/bin/sh"; args = ["-c" "exit 0"]; system = builtins.currentSystem; }' \
--no-link \
| grepQuiet "local builds are disabled"
fi

View File

@@ -23,7 +23,7 @@ external_builder="$TEST_ROOT/external-builder.sh"
cat > "$external_builder" <<EOF
#! $SHELL -e
PATH=$PATH
PATH=${PATH@Q}
[[ "\$1" = bla ]]

View File

@@ -32,6 +32,8 @@ writeSimpleFlake() {
baseName = builtins.baseNameOf ./.;
root = ./.;
number = 123;
};
}
EOF

View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
source ./common.sh
createFlake1
mkdir -p "$flake1Dir/subflake"
cat > "$flake1Dir/subflake/flake.nix" <<EOF
{
outputs = { self }:
let
# Bad, legacy way of getting a flake from an input.
parentFlake = builtins.getFlake (builtins.flakeRefToString { type = "path"; path = self.sourceInfo.outPath; narHash = self.narHash; });
# Better way using a path value.
parentFlake2 = builtins.getFlake ./..;
in {
x = parentFlake.number;
y = parentFlake2.number;
};
}
EOF
git -C "$flake1Dir" add subflake/flake.nix
[[ $(nix eval "$flake1Dir/subflake#x") = 123 ]]
[[ $(nix eval "$flake1Dir/subflake#y") = 123 ]]

View File

@@ -34,6 +34,7 @@ suites += {
'source-paths.sh',
'old-lockfiles.sh',
'trace-ifd.sh',
'get-flake.sh',
],
'workdir' : meson.current_source_dir(),
}

View File

@@ -64,7 +64,7 @@ in
environment = {
CURL_CA_BUNDLE = config.security.pki.caBundle;
NIX_REMOTE = "local?ignore-gc-delete-failure=true";
NIX_REMOTE = "local?ignore-gc-delete-failure=true&use-roots-daemon=true";
NIX_CONFIG = ''
experimental-features = local-overlay-store auto-allocate-uids
build-users-group =
@@ -85,6 +85,32 @@ in
};
};
systemd.services.nix-roots-daemon = {
environment = {
# `use-roots-daemon` is not needed because it is only relevant
# for the *client* of this daemon (i.e. the nix daemon opening
# the local store in this case). The Nix roots daemon *itself*
# doesn't care about this setting --- there's no problem if
# someone else opens the local store and directly scans for
# roots instead of using this daemon, for example.
NIX_REMOTE = "local";
NIX_CONFIG = ''
extra-experimental-features = local-overlay-store
'';
};
serviceConfig.ExecStart = "${config.nix.package.out}/bin/nix --extra-experimental-features nix-command store roots-daemon";
};
systemd.sockets.nix-roots-daemon = {
wantedBy = [
"nix-daemon.service"
];
listenStreams = [ "/nix/var/nix/gc-roots-socket/socket" ];
unitConfig = {
ConditionPathIsReadWrite = "/nix/var/nix/gc-roots-socket";
RequiresMountsFor = "/nix/store";
};
};
systemd.sockets.nix-daemon.wantedBy = [ "sockets.target" ];
systemd.tmpfiles.rules = [
@@ -97,6 +123,7 @@ in
"d /nix/var/nix/daemon-socket 0755 nix-daemon nix-daemon - -"
"d /nix/var/nix/gcroots 0755 nix-daemon nix-daemon - -"
"L+ /nix/var/nix/gcroots/booted-system 0755 nix-daemon nix-daemon - /run/booted-system"
"d /nix/var/nix/gc-roots-socket 0755 nix-daemon nix-daemon - -"
"d /var/empty/.cache/nix 0755 nix-daemon nix-daemon - -"
];
};

View File

@@ -29,6 +29,25 @@ runCommand "repl-completion"
exit 1
}
}
# Regression https://github.com/NixOS/nix/issues/15133
# Tab-completing an expression that throws a non-EvalError (e.g.
# JSONParseError from fromJSON) should not crash the REPL.
send "err1 = builtins.fromJSON \"nixnix\"\n"
expect "nix-repl>"
send "err1.\t"
sleep 0.5
# Send Ctrl-C to cancel the current line and get a fresh prompt,
# since tab with no completions leaves the cursor on the same line.
send "\x03"
expect {
"nix-repl>" {
puts "Got another prompt after fromJSON error."
}
eof {
puts "REPL crashed after fromJSON tab-complete."
exit 1
}
}
exit 0
'';
passAsFile = [ "expectScript" ];