Compare commits

..

48 Commits

Author SHA1 Message Date
Eelco Dolstra
490cb842cc Add release note 2026-02-18 22:50:37 +01:00
Eelco Dolstra
6992698ac5 builtins.getFlake: Support path values
This allows doing `builtins.getFlake ./subflake` instead of ugly hacks.
2026-02-18 22:12:09 +01:00
Eelco Dolstra
9868310d6f Add test for builtins.getFlake 2026-02-18 21:58:36 +01:00
John Ericson
08ce8dbfba Merge pull request #15283 from obsidiansystems/filesytem-error-improvements
Filesystem error improvements
2026-02-18 18:29:44 +00:00
John Ericson
bbcf2041e1 File system error improvements
- Make `descriptorToPath` cross-platform (renamed from
  `windows::handleToPath`). Uses `/proc/self/fd` on Linux and
  `F_GETPATH` on macOS. Add `HAVE_F_GETPATH` meson check.

  This is based on 7226a116a0, which was
  removed in 479c356510, but is now
  introduced more judiciously.

- Unix error messages in `readFull`, `writeFull`, `readLine` now include
  file paths via `descriptorToPath`.

- Convert `std::filesystem::filesystem_error` to `SystemError`

  Wrappers like `readLink`, `createDirs`, `DirectoryIterator`, etc. now
  catch `std::filesystem::filesystem_error` and rethrow as `SystemError`
  with the error code preserved. This ensures consistent exception types
  throughout the codebase.

  Call sites that previously caught `filesystem_error` and rethrew with
  `throw;` now throw `SystemError(e.code(), ...)` instead.

  Some call sites can stop catching `filesystem_error` at all,
  because they only call the wrapped functions.

- Rework `SystemError` constructors to auto-append error message

  The public `SystemError(std::error_code, ...)` constructor now
  automatically appends `errorCode.message()` to the error message.
  A protected constructor takes an explicit error message string for
  subclasses.

  `SysError` delegates to the protected constructor with `strerror(errNo)`.
  `WinError` delegates with `renderError(lastError)` (now static).

  This removes the need to manually append `e.code().message()` at call
  sites when converting `filesystem_error` to `SystemError`.

- Use perfect forwarding (`Args &&...` with `std::forward`) consistently
  in `BaseError`, `SystemError`, `SysError`, and `WinError` constructors.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-02-18 12:29:11 -05:00
John Ericson
96bcf5928f Merge pull request #15273 from NixOS/more-robust-ubsan-macro
libutil: More robust check for NIX_UBSAN_ENABLED
2026-02-18 16:15:26 +00:00
Sergei Zimmerman
db853cf4fb libutil: More robust check for NIX_UBSAN_ENABLED
In 3df91bea62 I forgot that the header
might get included out-of-tree with -Wundef. Let's make this a public
config option for libutil as it can affect function bodies in headers.
2026-02-18 17:33:51 +03:00
John Ericson
663db5b48b Merge pull request #15278 from puffnfresh/windows/bar-log-format
Windows: don't use bar log format
2026-02-18 05:14:27 +00:00
Brian McKenna
c486e78235 Windows: don't use bar log format
Relies on terminal features that don't always work on Windows.
2026-02-18 14:35:35 +11:00
John Ericson
4fff871383 Merge pull request #15274 from obsidiansystems/tryToBuild-raii
libstore: refactor `tryToBuild` with coroutine lambdas and RAII
2026-02-17 22:10:42 +00:00
Amaan Qureshi
b9acea908e libstore: refactor tryToBuild with coroutine lambdas and RAII
`tryToBuild` threaded a single `PathLocks outputLocks` by reference
across all build phases and managed a `std::unique_ptr<Activity> actLock`
with explicit `if (!actLock)` guards and `.reset()` calls around the hook
retry loop. This commit introduces coroutine lambdas for the three phases:
`tryHookLoop` owns a `PathLocks` in a scoped block for the first attempt
and per-iteration in the retry loop, `tryBuildLocally` acquires its own
`PathLocks`, and the hook-wait `Activity` is a stack variable scoped to
the postpone block.
2026-02-17 16:23:44 -05:00
John Ericson
c3f0670b4e Merge pull request #15266 from obsidiansystems/fix-maxjobs-error
libstore: structured diagnostics for local build rejection
2026-02-17 18:39:58 +00:00
Amaan Qureshi
7cd4359a8b libstore: structured diagnostics for local build rejection
When `max-jobs = 0` and no remote builders are available, Nix reported
"required system or feature not available" even though the system and
features matched fine. The `canBuildLocally` lambda returned a plain
`bool`, conflating a configuration knob (`max-jobs = 0`) with actual
incompatibility (wrong platform, missing features). It also short-circuited
on the first failing check, so a user with both a platform mismatch and
missing features would only see one of the two.

This commit replaces the bool with a `LocalBuildRejection` struct whose
`WrongLocalStore` variant collects all applicable failures into
`badPlatform`, `missingFeatures`, and an orthogonal `maxJobsZero` flag.
Platform mismatch and missing features now produce separate error
paragraphs, and all applicable reasons appear in a single message.

The local-build capability check also now returns
`std::variant<LocalBuildCapability, LocalBuildRejection>`, bundling
the `LocalStore &` and optional `ExternalBuilder *` together.
2026-02-17 12:54:24 -05:00
John Ericson
6e725093e6 Merge pull request #15143 from obsidiansystems/rootless-daemon-minimal
Support garbage collection in external daemon
2026-02-17 16:53:06 +00:00
Artemis Tosini
96fef69755 libstore: support searching for roots from an external daemon
This comes in two parts: a `nix store roots-daemon` command that
can run as root and list runtime roots,
and client logic to find runtime roots for a `LocalStore` by connecting
to that daemon.

This may be useful with an unprivileged nix daemon, as it would
otherwise be unable to find runtime roots from process open files
and maps.
2026-02-17 10:42:04 -05:00
John Ericson
16b0bb7548 Merge pull request #15270 from NixOS/inline-lookup-var
libexpr: Make sure `EvalState::lookupVar` is inlined
2026-02-17 15:12:00 +00:00
John Ericson
ebcd31e434 Merge pull request #15271 from NixOS/faster-type-internal-type
libexpr: Optimise `Value::type()`, `ValueStorage::getInternalType()`
2026-02-17 15:11:23 +00:00
John Ericson
f940ab5146 Merge pull request #15265 from xokdvium/libgit2-error
libfetchers/git-utils: Add GitError class for deduplicating error…
2026-02-17 15:06:31 +00:00
Sergei Zimmerman
3df91bea62 libexpr: Optimise Value::type(), ValueStorage::getInternalType()
Using nix::unreachable() in getInternalType() and type() turns
out to be quite expensive and prevents inlining. Also Value::type
got compiled to a jump table which has a high overhead from indirect
jumps. Using an explicit lookup table turns out to be more efficient.

This does mean that we lose out on nice diagnostics from nix::unreachable
calls, but this code is probably one of the hottests functions in the whole
evaluator, so I think the tradeoff is worth it. The nixUnreachableWhenHardened
boils down to nix::unreachable when UBSan is enabled so we still have good
coverage there.
2026-02-17 16:50:07 +03:00
Sergei Zimmerman
aaabe82483 libexpr: Make sure EvalState::lookupVar is inlined
This makes sure that ExprVar::eval inlines lookupVar call. In practice
this seems to reduce instruction count by ~2%, though it doesn't have
a statistically significant impact on the wall time.
2026-02-17 15:32:26 +03:00
Sergei Zimmerman
a81f83604b libexpr: Add marker values to InternalType enum
This reduces the churn when changing up the order of
values in a follow-up commit. This should have been done
from the start ideally to improve readability.
2026-02-17 13:32:45 +03:00
Sergei Zimmerman
c1bfa30303 libfetchers/git-utils: Add GitError class for deduplicating error message printing
Consolidates all the error message formatting in one place. It was very weird
and tiring to remember to call git_error_last() in all the places.
2026-02-17 12:18:37 +03:00
John Ericson
509694d5f0 Merge pull request #15267 from obsidiansystems/fix-external-builders-path
tests: quote `PATH` in external-builders test heredoc
2026-02-17 05:53:17 +00:00
Amaan Qureshi
0b7629da08 tests: quote PATH in external-builders test heredoc
The external-builders test expands `$PATH` into a heredoc without quotes,
so any `PATH` entry containing spaces causes bash to parse the line as a
command instead of an assignment, failing the test.
2026-02-16 23:20:10 -05:00
Sergei Zimmerman
e7e5eaaa37 Merge pull request #15255 from obsidiansystems/fix-repl-tab-crash
repl: catch all errors during tab completion
2026-02-16 21:58:22 +00:00
Jörg Thalheim
974545290e Merge pull request #15252 from obsidiansystems/fix-docker-compression
upload-release: disable containerd image store to preserve gzip layer compression
2026-02-16 21:26:31 +00:00
Amaan Qureshi
be6e72f11b repl: prevent exceptions from escaping editline callbacks
The tab completion handler in `completePrefix` only caught `ParseError`,
`EvalError`, `BadURL`, and `FileNotFound`. Other error types like
`JSONParseError` (which derives from `Error`, not `EvalError`) escaped
the catch block and propagated through editline's C code as undefined
behavior, crashing the REPL. This happened when tab-completing
expressions like `(builtins.fromJSON "invalid").` where evaluation
throws a non-`EvalError` exception.

This commit marks `completionCallback` and `listPossibleCallback` as
`noexcept` with function-try-blocks that catch all exceptions at the
C/C++ boundary, preventing any exception from reaching editline.

Fixes #15133.
2026-02-16 16:02:37 -05:00
Sergei Zimmerman
27782fcc42 Merge pull request #15253 from obsidiansystems/fix-url-assertion
libflake: fix assertion crash when malformed URL falls through to path scheme
2026-02-16 20:49:49 +00:00
John Ericson
06d4d5779f Merge pull request #15251 from obsidiansystems/file-system-at
Split `file-system-at.{cc,hh}` from `file-descriptor.{cc,hh}`
2026-02-16 20:10:28 +00:00
Amaan Qureshi
a32cd16f64 libflake: fix assertion crash when malformed URL falls through to path scheme
When a URL like `github:nixos/nixpkgs/nixpkgs.git?ref=<hash>` (using
`ref` instead of `rev`) failed the github input scheme, it fell
through to `parsePathFlakeRefWithFragment` which constructed a `path:`
`ParsedURL` with an empty authority but a relative path. This violated
RFC 3986 section 3.3 (authority present requires path starting with
`/`), causing an assertion failure in `renderAuthorityAndPath` when
`PathInputScheme` tried to format the URL for an error message.

This commit only sets the authority on absolute paths. Relative paths
get `std::nullopt` for authority, which is the correct representation
per the URL spec.

Fixes #15196. Fixes #14830.
2026-02-16 15:10:19 -05:00
Sergei Zimmerman
46a4a554ca Merge pull request #15237 from xokdvium/add-missing-temp-roots
Add missing temproots for cached sources and existing derivations
2026-02-16 19:35:15 +00:00
John Ericson
cc0b489967 Merge pull request #15250 from obsidiansystems/assume-lchown
Remove suppport for not having `lchown`
2026-02-16 19:29:08 +00:00
John Ericson
af7e585009 Split file-system-at.{cc,hh} from file-descriptor.{cc,hh}
`file-descriptor.{cc,hh}` was getting too big, split out
`file-system-at.{cc,hh}` for the FD-based file system stuff,
`file-descriptor.{cc,hh}` will only be for the fundamental primitives
that are file-system agnostic and work on almost all file types.

Review with `git show --color-moved` to see that this is indeed all
moving.
2026-02-16 14:21:52 -05:00
Amaan Qureshi
2ccb8a9a56 upload-release: disable containerd image store to preserve gzip layer compression
Docker 28+ defaults to the containerd image store, which pushes layers
uncompressed instead of gzip. The GHA runner image updated Docker to
29.x (actions/runner-images#13633), causing the `nixos/nix:2.33.3`
image to balloon from 138 MB to 505 MB, with all 70 layers pushed as
`application/vnd.docker.image.rootfs.diff.tar` instead of `.tar.gzip`.
OCI clients that only support gzip (e.g. `go-containerregistry`, used
by Concourse CI) fail with "gzip: invalid header".

This commit disables the containerd snapshotter in the release workflow
before any Docker operations, restoring the classic storage driver that
preserves gzip compression through the `docker load` / `docker push`
pipeline.

Fixes #15246
2026-02-16 14:08:08 -05:00
John Ericson
fefa66880a Remove suppport for not having lchown
Linux, macOS, and all 3 BSDs have it (according to man page google
search), so let's just drop this. Support for not having it was added in
d03f0d4117 in 2006, things have changed in
the last 20 years!
2026-02-16 13:40:29 -05:00
John Ericson
a53391fd0e Merge pull request #15247 from roberth/clarify-ref-upcasting
Better `ref` casting DX
2026-02-16 17:09:16 +00:00
Robert Hensing
771421a34e fix(ref): improve cast exception type and add demangled type names
When ref::cast() fails, the error message was cryptic ("null pointer
cast to ref"). Now it throws a proper bad_ref_cast (a std::bad_cast
subclass) with a clear message showing the actual types involved:

    ref<nix::Base> cannot be cast to ref<nix::Derived>

This also adds a demangle.hh utility.
2026-02-16 17:07:40 +01:00
Robert Hensing
5aaa0cc4a6 refactor(ref): clarify implicit conversion semantics with requires clause
ref<Derived> was already implicitly convertible to ref<Base>, but the
mechanism was unclear and error messages for rejected downcasts were
more cryptic than necessary. This change:

- Adds RefImplicitlyUpcastableTo concept to constrain the conversion
  operator, making the intent explicit and improving error messages
- Documents .cast() and .dynamic_pointer_cast() as alternatives for
  explicit downcasting
- Adds unit tests for covariance behavior
2026-02-16 16:43:08 +01:00
John Ericson
0749ec4e55 Merge pull request #15230 from obsidiansystems/new-wine
flake: Use Wine 11 for running mingw tests
2026-02-15 16:41:52 +00:00
Artemis Tosini
4cc97150df flake: Use Wine 11 for running mingw tests
Set wine_11 as the emulator for Windows.
2026-02-15 10:56:02 -05:00
John Ericson
2bbd1094a2 flake.lock: Update Nixpkgs
Flake lock file updates:

• Updated input 'nixpkgs':
    'https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz?narHash=sha256-Xu%2B7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4%3D' (2026-01-22)
  → 'https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz?narHash=sha256-WoiezqWJQ3OHILah%2Bp6rzNXdJceEAmAhyDFZFZ6pZzY%3D' (2026-02-14)

This will be needed to get Wine 11.
2026-02-15 10:53:15 -05:00
John Ericson
95251a51dd Merge pull request #15241 from obsidiansystems/fix-isindir
libutil: fix `isInDir` rejecting paths starting with dot
2026-02-15 15:52:37 +00:00
John Ericson
02d9f4ecb4 Merge pull request #15239 from xokdvium/fix-warnings-no-intereference-size
meson: Only enable -Wno-interference-size with GCC
2026-02-15 15:06:54 +00:00
John Ericson
3269c71e9d Merge pull request #15240 from xokdvium/fix-mtls-redirect-test
libstore-tests: Fix mTLS test for redirect, correctly propagate tries
2026-02-15 15:04:50 +00:00
Amaan Qureshi
ad0055e67c libutil: fix isInDir rejecting paths starting with dot
The old check rejected any relative path whose first character was a
dot, producing false negatives for valid descendants like `.ssh` or
`.config`. This commit changes the logic such that now it inspects the
first path component via `path::begin()`, only rejects `.` and `..`
rather than anything dot-prefixed. Fixes #15207.
2026-02-15 10:04:08 -05:00
Sergei Zimmerman
d3d63a4b5b libstore-tests: Fix mTLS test for redirect, correctly propagate tries
The fake cacert didn't have subjectAltName for 127.0.0.1, so the test
was failing for a different reason. Also `tries` setting wasn't being respected.
There's no callsite specifying it in the request, so just use the one specified
in the FileTransferSettings and remove the fields from the FileTransferRequest.
2026-02-15 00:08:21 +03:00
Sergei Zimmerman
6a5ee08737 meson: Only enable -Wno-interference-size with GCC
Clang doesn't recognise this option.
2026-02-14 23:42:28 +03:00
Sergei Zimmerman
ac2dd58b6f Add missing temproots for cached sources and existing derivations 2026-02-14 12:09:24 +03:00
185 changed files with 2379 additions and 1801 deletions

View File

@@ -39,6 +39,17 @@ jobs:
role-to-assume: "arn:aws:iam::080433136561:role/nix-release"
role-session-name: nix-release-oidc-${{ github.run_id }}
aws-region: eu-west-1
- name: Disable containerd image store
run: |
# Docker 28+ defaults to the containerd image store, which
# pushes layers uncompressed instead of gzip. OCI clients
# that only support gzip (e.g. go-containerregistry) fail
# with "gzip: invalid header". Disabling the containerd
# snapshotter restores the classic storage driver, which
# preserves gzip-compressed layers through the
# `docker load` / `docker push` pipeline.
echo '{"features":{"containerd-snapshotter":false}}' | sudo tee /etc/docker/daemon.json > /dev/null
sudo systemctl restart docker
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:

View File

@@ -0,0 +1,6 @@
---
synopsis: "`builtins.getFlake` now supports path values"
prs: [15290]
---
`builtins.getFlake` now accepts path values in addition to flakerefs, allowing you to write `builtins.getFlake ./subflake` instead of having to use ugly workarounds to construct a pure flakeref.

View File

@@ -0,0 +1,11 @@
---
synopsis: New command `nix store roots-daemon` for serving GC roots
prs: [15143]
---
New command [`nix store roots-daemon`](@docroot@/command-ref/new-cli/nix3-store-roots-daemon.md) runs a daemon that serves garbage collector roots over a Unix domain socket.
It enables the garbage collector to discover runtime roots when the main Nix daemon doesn't have `CAP_SYS_PTRACE` capability and therefore cannot scan `/proc`.
The garbage collector can be configured to use this daemon via the [`use-roots-daemon`](@docroot@/store/types/local-store.md#store-experimental-option-use-roots-daemon) store setting.
This feature requires the [`local-overlay-store` experimental feature](@docroot@/development/experimental-features.md#xp-feature-local-overlay-store).

8
flake.lock generated
View File

@@ -63,11 +63,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1769089682,
"narHash": "sha256-Xu+7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4=",
"rev": "078d69f03934859a181e81ba987c2bb033eebfc5",
"lastModified": 1771043024,
"narHash": "sha256-WoiezqWJQ3OHILah+p6rzNXdJceEAmAhyDFZFZ6pZzY=",
"rev": "3aadb7ca9eac2891d52a9dec199d9580a6e2bf44",
"type": "tarball",
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz"
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.5960.3aadb7ca9eac/nixexprs.tar.xz"
},
"original": {
"type": "tarball",

View File

@@ -115,6 +115,9 @@
}
// lib.optionalAttrs (crossSystem == "x86_64-unknown-freebsd13") {
useLLVM = true;
}
// lib.optionalAttrs (crossSystem == "x86_64-w64-mingw32") {
emulator = pkgs: "${pkgs.buildPackages.wineWow64Packages.stable_11}/bin/wine";
};
overlays = [
(overlayFor (pkgs: pkgs.${stdenv}))

View File

@@ -27,13 +27,19 @@ add_project_arguments(
'-Wignored-qualifiers',
'-Wimplicit-fallthrough',
'-Wno-deprecated-declarations',
'-Wno-interference-size', # Used for C++ ABI only. We don't provide any guarantees about different march tunings.
language : 'cpp',
)
# GCC doesn't benefit much from precompiled headers.
do_pch = cxx.get_id() == 'clang'
if cxx.get_id() == 'gcc'
add_project_arguments(
'-Wno-interference-size', # Used for C++ ABI only. We don't provide any guarantees about different march tunings.
language : 'cpp',
)
endif
# This is a clang-only option for improving build times.
# It forces the instantiation of templates in the PCH itself and
# not every translation unit it's included in.

View File

@@ -4,7 +4,6 @@
#include "nix/cmd/command.hh"
#include "nix/cmd/legacy.hh"
#include "nix/cmd/markdown.hh"
#include "nix/main/shared.hh"
#include "nix/store/globals.hh"
#include "nix/store/store-open.hh"
#include "nix/store/local-fs-store.hh"
@@ -76,7 +75,7 @@ ref<StoreConfig> StoreConfigCommand::getStoreConfig()
ref<StoreConfig> StoreConfigCommand::createStoreConfig()
{
return resolveStoreConfig(settings, StoreReference{settings.storeUri.get()});
return resolveStoreConfig(StoreReference{settings.storeUri.get()});
}
void StoreConfigCommand::run()
@@ -130,7 +129,7 @@ CopyCommand::CopyCommand()
ref<StoreConfig> CopyCommand::createStoreConfig()
{
return !srcUri ? StoreCommand::createStoreConfig() : resolveStoreConfig(settings, StoreReference{*srcUri});
return !srcUri ? StoreCommand::createStoreConfig() : resolveStoreConfig(StoreReference{*srcUri});
}
ref<Store> CopyCommand::getDstStore()
@@ -138,7 +137,7 @@ ref<Store> CopyCommand::getDstStore()
if (!srcUri && !dstUri)
throw UsageError("you must pass '--from' and/or '--to'");
return !dstUri ? openStore(settings) : openStore(settings, StoreReference{*dstUri});
return !dstUri ? openStore() : openStore(StoreReference{*dstUri});
}
EvalCommand::EvalCommand()
@@ -160,7 +159,7 @@ EvalCommand::~EvalCommand()
ref<Store> EvalCommand::getEvalStore()
{
if (!evalStore)
evalStore = evalStoreUrl ? openStore(settings, StoreReference{*evalStoreUrl}) : getStore();
evalStore = evalStoreUrl ? openStore(StoreReference{*evalStoreUrl}) : getStore();
return ref<Store>(evalStore);
}

View File

@@ -19,12 +19,12 @@
namespace nix {
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
static GlobalConfig::Register rFetchSettings(&fetchSettings);
EvalSettings evalSettings{
settings,
settings.readOnlyMode,
{
{
"flake",
@@ -135,7 +135,7 @@ MixEvalArgs::MixEvalArgs()
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
}},
.completer = {[&](AddCompletions & completions, size_t, std::string_view prefix) {
completeFlakeRef(completions, openStore(settings), prefix);
completeFlakeRef(completions, openStore(), prefix);
}},
});

View File

@@ -4,9 +4,9 @@
namespace nix {
std::string fetchBuildLog(Settings & settings, ref<Store> store, const StorePath & path, std::string_view what)
std::string fetchBuildLog(ref<Store> store, const StorePath & path, std::string_view what)
{
auto subs = getDefaultSubstituters(settings);
auto subs = getDefaultSubstituters();
subs.push_front(store);

View File

@@ -8,8 +8,6 @@
namespace nix {
class Settings;
/**
* Fetch the build log for a store path, searching the store and its
* substituters.
@@ -20,6 +18,6 @@ class Settings;
* @return The build log content.
* @throws Error if the build log is not available.
*/
std::string fetchBuildLog(Settings & settings, ref<Store> store, const StorePath & path, std::string_view what);
std::string fetchBuildLog(ref<Store> store, const StorePath & path, std::string_view what);
} // namespace nix

View File

@@ -1,6 +1,7 @@
#pragma once
///@file
#include "nix/store/globals.hh"
#include "nix/cmd/installable-value.hh"
#include "nix/store/outputs-spec.hh"
#include "nix/cmd/command.hh"

View File

@@ -40,8 +40,8 @@ void sigintHandler(int signo)
static detail::ReplCompleterMixin * curRepl; // ugly
#if !USE_READLINE
static char * completionCallback(char * s, int * match)
{
static char * completionCallback(char * s, int * match) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() == 1) {
*match = 1;
@@ -73,10 +73,12 @@ static char * completionCallback(char * s, int * match)
*match = 0;
return nullptr;
} catch (...) {
return nullptr;
}
static int listPossibleCallback(char * s, char *** avp)
{
static int listPossibleCallback(char * s, char *** avp) noexcept
try {
auto possible = curRepl->completePrefix(s);
if (possible.size() > (std::numeric_limits<int>::max() / sizeof(char *)))
@@ -105,6 +107,9 @@ static int listPossibleCallback(char * s, char *** avp)
*avp = vp;
return ac;
} catch (...) {
*avp = nullptr;
return 0;
}
#endif

View File

@@ -564,7 +564,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
settings.readOnlyMode = true;
Finally roModeReset([&]() { settings.readOnlyMode = false; });
RunPager pager;
auto log = fetchBuildLog(settings, state->store, drvPath, drvPathRaw);
auto log = fetchBuildLog(state->store, drvPath, drvPathRaw);
logger->writeToStdout(log);
} else {
runNix("nix-shell", {drvPathRaw});
@@ -710,7 +710,7 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
try {
cwd = std::filesystem::current_path();
} catch (std::filesystem::filesystem_error & e) {
throw SysError("cannot determine current working directory");
throw SystemError(e.code(), "cannot determine current working directory");
}
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, cwd.string(), true);

View File

@@ -132,8 +132,9 @@ nix_eval_state_builder * nix_eval_state_builder_new(nix_c_context * context, Sto
return unsafe_new_with_self<nix_eval_state_builder>([&](auto * self) {
return nix_eval_state_builder{
.store = nix::ref<nix::Store>(store->ptr),
.settings = nix::EvalSettings{cStoreSettings},
.fetchSettings = nix::fetchers::Settings{cStoreSettings},
.settings = nix::EvalSettings{/* &bool */ self->readOnlyMode},
.fetchSettings = nix::fetchers::Settings{},
.readOnlyMode = true,
};
});
}
@@ -152,7 +153,8 @@ nix_err nix_eval_state_builder_load(nix_c_context * context, nix_eval_state_buil
if (context)
context->last_err_code = NIX_OK;
try {
loadConfFile(cStoreSettings);
// TODO: load in one go?
builder->settings.readOnlyMode = nix::settings.readOnlyMode;
loadConfFile(builder->settings);
loadConfFile(builder->fetchSettings);
}

View File

@@ -16,6 +16,8 @@ struct nix_eval_state_builder
nix::EvalSettings settings;
nix::fetchers::Settings fetchSettings;
nix::LookupPath lookupPath;
// TODO: make an EvalSettings setting own this instead?
bool readOnlyMode;
};
struct EvalState

View File

@@ -26,18 +26,18 @@ public:
}
protected:
LibExprTest(auto && makeEvalSettings, auto &&... args)
: LibStoreTest(args...)
, evalSettings(makeEvalSettings(settings))
LibExprTest(ref<Store> store, auto && makeEvalSettings)
: LibStoreTest()
, evalSettings(makeEvalSettings(readOnlyMode))
, state({}, store, fetchSettings, evalSettings, nullptr)
{
}
LibExprTest()
: LibExprTest([](Settings & settings) {
EvalSettings evalSettings{settings};
evalSettings.nixPath = {};
return evalSettings;
: LibExprTest(openStore("dummy://"), [](bool & readOnlyMode) {
EvalSettings settings{readOnlyMode};
settings.nixPath = {};
return settings;
})
{
}
@@ -66,8 +66,8 @@ protected:
}
bool readOnlyMode = true;
fetchers::Settings fetchSettings{settings};
EvalSettings evalSettings{settings};
fetchers::Settings fetchSettings{};
EvalSettings evalSettings{readOnlyMode};
EvalState state;
};

View File

@@ -179,15 +179,12 @@ class PureEvalTest : public LibExprTest
{
public:
PureEvalTest()
: LibExprTest{
[](auto & settings) {
EvalSettings evalSettings{settings};
evalSettings.pureEval = true;
evalSettings.restrictEval = true;
return evalSettings;
},
[](auto & settings) { return openStore(settings, "dummy://", {{"read-only", "false"}}); },
}
: LibExprTest(openStore("dummy://", {{"read-only", "false"}}), [](bool & readOnlyMode) {
EvalSettings settings{readOnlyMode};
settings.pureEval = true;
settings.restrictEval = true;
return settings;
})
{
}
};

View File

@@ -13,8 +13,7 @@ TEST_F(ValueTest, unsetValue)
{
Value unsetValue;
ASSERT_EQ(false, unsetValue.isValid());
ASSERT_EQ(nThunk, unsetValue.type(true));
ASSERT_DEATH(unsetValue.type(), "");
ASSERT_EQ(nThunk, unsetValue.type</*invalidIsThunk=*/true>());
}
TEST_F(ValueTest, vInt)

View File

@@ -63,7 +63,7 @@ struct AttrDb
SymbolTable & symbols;
AttrDb(bool useSQLiteWAL, const StoreDirConfig & cfg, const Hash & fingerprint, SymbolTable & symbols)
AttrDb(const StoreDirConfig & cfg, const Hash & fingerprint, SymbolTable & symbols)
: cfg(cfg)
, _state(std::make_unique<Sync<State>>())
, symbols(symbols)
@@ -75,7 +75,7 @@ struct AttrDb
auto dbPath = cacheDir / (fingerprint.to_string(HashFormat::Base16, false) + ".sqlite");
state->db = SQLite(dbPath, {.useWAL = useSQLiteWAL});
state->db = SQLite(dbPath, {.useWAL = settings.useSQLiteWAL});
state->db.isCache();
state->db.exec(schema);
@@ -287,11 +287,10 @@ struct AttrDb
}
};
static std::shared_ptr<AttrDb>
makeAttrDb(bool useSQLiteWAL, const StoreDirConfig & cfg, const Hash & fingerprint, SymbolTable & symbols)
static std::shared_ptr<AttrDb> makeAttrDb(const StoreDirConfig & cfg, const Hash & fingerprint, SymbolTable & symbols)
{
try {
return std::make_shared<AttrDb>(useSQLiteWAL, cfg, fingerprint, symbols);
return std::make_shared<AttrDb>(cfg, fingerprint, symbols);
} catch (SQLiteError &) {
ignoreExceptionExceptInterrupt();
return nullptr;
@@ -300,8 +299,7 @@ makeAttrDb(bool useSQLiteWAL, const StoreDirConfig & cfg, const Hash & fingerpri
EvalCache::EvalCache(
std::optional<std::reference_wrapper<const Hash>> useCache, EvalState & state, RootLoader rootLoader)
: db(useCache ? makeAttrDb(state.store->config.settings.useSQLiteWAL, *state.store, *useCache, state.symbols)
: nullptr)
: db(useCache ? makeAttrDb(*state.store, *useCache, state.symbols) : nullptr)
, state(state)
, rootLoader(rootLoader)
{
@@ -709,7 +707,7 @@ StorePath AttrCursor::forceDerivation()
auto aDrvPath = getAttr(root->state.s.drvPath);
auto drvPath = root->state.store->parseStorePath(aDrvPath->getString());
drvPath.requireDerivation();
if (!root->state.store->isValidPath(drvPath) && !root->state.store->config.settings.readOnlyMode) {
if (!root->state.store->isValidPath(drvPath) && !settings.readOnlyMode) {
/* The eval cache contains 'drvPath', but the actual path has
been garbage-collected. So force it to be regenerated. */
aDrvPath->forceValue();

View File

@@ -48,8 +48,8 @@ Strings EvalSettings::parseNixPath(const std::string & s)
return res;
}
EvalSettings::EvalSettings(nix::Settings & storeSettings, EvalSettings::LookupPathHooks lookupPathHooks)
: storeSettings{storeSettings}
EvalSettings::EvalSettings(bool & readOnlyMode, EvalSettings::LookupPathHooks lookupPathHooks)
: readOnlyMode{readOnlyMode}
, lookupPathHooks{lookupPathHooks}
{
auto var = getEnv("NIX_ABORT_ON_WARN");
@@ -57,7 +57,7 @@ EvalSettings::EvalSettings(nix::Settings & storeSettings, EvalSettings::LookupPa
builtinsAbortOnWarn = true;
}
Strings EvalSettings::getDefaultNixPath(nix::Settings & settings)
Strings EvalSettings::getDefaultNixPath()
{
Strings res;
auto add = [&](const std::filesystem::path & p, const std::string & s = std::string()) {
@@ -70,7 +70,7 @@ Strings EvalSettings::getDefaultNixPath(nix::Settings & settings)
}
};
add(std::filesystem::path{getNixDefExpr(settings)} / "channels");
add(std::filesystem::path{getNixDefExpr()} / "channels");
auto profilesDirOpts = settings.getProfileDirsOptions();
add(rootChannelsDir(profilesDirOpts) / "nixpkgs", "nixpkgs");
add(rootChannelsDir(profilesDirOpts));
@@ -101,10 +101,10 @@ std::string EvalSettings::resolvePseudoUrl(std::string_view url)
const std::string & EvalSettings::getCurrentSystem() const
{
const auto & evalSystem = currentSystem.get();
return evalSystem != "" ? evalSystem : storeSettings.thisSystem.get();
return evalSystem != "" ? evalSystem : settings.thisSystem.get();
}
std::filesystem::path getNixDefExpr(const Settings & settings)
std::filesystem::path getNixDefExpr()
{
return settings.useXDGBaseDirectories ? getStateDir() / "defexpr" : getHome() / ".nix-defexpr";
}

View File

@@ -15,7 +15,6 @@
#include "nix/store/filetransfer.hh"
#include "nix/expr/function-trace.hh"
#include "nix/store/profiles.hh"
#include "nix/store/globals.hh"
#include "nix/expr/print.hh"
#include "nix/fetchers/filtering-source-accessor.hh"
#include "nix/util/memory-source-accessor.hh"
@@ -331,7 +330,7 @@ EvalState::EvalState(
lookupPath.elements.emplace_back(LookupPath::Elem::parse(i));
}
if (!settings.restrictEval) {
for (auto & i : EvalSettings::getDefaultNixPath(store->config.settings)) {
for (auto & i : EvalSettings::getDefaultNixPath()) {
lookupPath.elements.emplace_back(LookupPath::Elem::parse(i));
}
}
@@ -467,7 +466,7 @@ void EvalState::addConstant(const std::string & name, Value * v, Constant info)
We might know the type of a thunk in advance, so be allowed
to just write it down in that case. */
if (auto gotType = v->type(true); gotType != nThunk)
if (auto gotType = v->type</*invalidIsThunk=*/true>(); gotType != nThunk)
assert(info.type == gotType);
/* Install value the base environment. */
@@ -886,7 +885,7 @@ void Value::mkPath(const SourcePath & path, EvalMemory & mem)
mkPath(&*path.accessor, StringData::make(mem, path.path.abs()));
}
inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
[[gnu::always_inline]] inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
{
for (auto l = var.level; l; --l, env = env->up)
;
@@ -904,11 +903,11 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
while (1) {
forceAttrs(*env->values[0], fromWith->pos, "while evaluating the first subexpression of a with expression");
if (auto j = env->values[0]->attrs()->get(var.name)) {
if (countCalls)
if (countCalls) [[unlikely]]
attrSelects[j->pos]++;
return j->value;
}
if (!fromWith->parentWith)
if (!fromWith->parentWith) [[unlikely]]
error<UndefinedVarError>("undefined variable '%1%'", symbols[var.name])
.atPos(var.pos)
.withFrame(*env, var)
@@ -2509,7 +2508,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
fetchSettings,
*store,
path.resolveSymlinks(SymlinkResolution::Ancestors),
settings.storeSettings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
path.baseName(),
ContentAddressMethod::Raw::NixArchive,
nullptr,

View File

@@ -9,15 +9,9 @@ namespace nix {
class EvalState;
struct PrimOp;
class Settings;
struct EvalSettings : Config
{
/**
* Reference to the "parent" store-layer settings.
*/
nix::Settings & storeSettings;
/**
* Function used to interpret look path entries of a given scheme.
*
@@ -44,9 +38,11 @@ struct EvalSettings : Config
*/
using LookupPathHooks = std::map<std::string, std::function<LookupPathHook>>;
EvalSettings(nix::Settings & settings, LookupPathHooks lookupPathHooks = {});
EvalSettings(bool & readOnlyMode, LookupPathHooks lookupPathHooks = {});
static Strings getDefaultNixPath(nix::Settings & settings);
bool & readOnlyMode;
static Strings getDefaultNixPath();
static bool isPseudoUrl(std::string_view s);
@@ -370,6 +366,6 @@ struct EvalSettings : Config
/**
* Conventionally part of the default nix path in impure mode.
*/
std::filesystem::path getNixDefExpr(const Settings & settings);
std::filesystem::path getNixDefExpr();
} // namespace nix

View File

@@ -35,7 +35,7 @@ class BindingsBuilder;
* about how this is mapped into the alignment bits to save significant memory.
* This also restricts the number of internal types represented with distinct memory layouts.
*/
typedef enum {
enum InternalType {
tUninitialized = 0,
/* layout: Single/zero field payload */
tInt = 1,
@@ -46,16 +46,20 @@ typedef enum {
tPrimOp,
tAttrs,
/* layout: Pair of pointers payload */
tListSmall,
tFirstPairOfPointers,
tListSmall = tFirstPairOfPointers,
tPrimOpApp,
tApp,
tThunk,
tLambda,
tLastPairOfPointers = tLambda,
/* layout: Single untaggable field */
tListN,
tFirstSingleUntaggable,
tListN = tFirstSingleUntaggable,
tString,
tPath,
} InternalType;
tNumberOfInternalTypes, // Must be last
};
/**
* This type abstracts over all actual value types in the language,
@@ -633,7 +637,7 @@ class alignas(16)
template<InternalType type, typename T, typename U>
void setPairOfPointersPayload(T * firstPtrField, U * secondPtrField) noexcept
{
static_assert(type >= tListSmall && type <= tLambda);
static_assert(type >= tFirstPairOfPointers && type <= tLastPairOfPointers);
{
auto firstFieldPayload = std::bit_cast<PackedPointer>(firstPtrField);
assertAligned(firstFieldPayload);
@@ -642,7 +646,7 @@ class alignas(16)
{
auto secondFieldPayload = std::bit_cast<PackedPointer>(secondPtrField);
assertAligned(secondFieldPayload);
payload[1] = (type - tListSmall) | secondFieldPayload;
payload[1] = (type - tFirstPairOfPointers) | secondFieldPayload;
}
}
@@ -670,11 +674,11 @@ protected:
case pdListN:
case pdString:
case pdPath:
return static_cast<InternalType>(tListN + (pd - pdListN));
return static_cast<InternalType>(tFirstSingleUntaggable + (pd - pdListN));
case pdPairOfPointers:
return static_cast<InternalType>(tListSmall + (payload[1] & discriminatorMask));
return static_cast<InternalType>(tFirstPairOfPointers + (payload[1] & discriminatorMask));
[[unlikely]] default:
unreachable();
nixUnreachableWhenHardened();
}
}
@@ -1027,7 +1031,7 @@ private:
T getStorage() const noexcept
{
if (getInternalType() != detail::payloadTypeToInternalType<T>) [[unlikely]]
unreachable();
nixUnreachableWhenHardened();
T out;
ValueStorage::getStorage(out);
return out;
@@ -1079,45 +1083,44 @@ public:
* Returns the normal type of a Value. This only returns nThunk if
* the Value hasn't been forceValue'd
*
* @param invalidIsThunk Instead of aborting an an invalid (probably
* @param invalidIsThunk Instead of UB an an invalid (probably
* 0, so uninitialized) internal type, return `nThunk`.
*/
inline ValueType type(bool invalidIsThunk = false) const
template<bool invalidIsThunk = false>
inline ValueType type() const
{
switch (getInternalType()) {
case tUninitialized:
break;
case tInt:
return nInt;
case tBool:
return nBool;
case tString:
return nString;
case tPath:
return nPath;
case tNull:
return nNull;
case tAttrs:
return nAttrs;
case tListSmall:
case tListN:
return nList;
case tLambda:
case tPrimOp:
case tPrimOpApp:
return nFunction;
case tExternal:
return nExternal;
case tFloat:
return nFloat;
case tThunk:
case tApp:
return nThunk;
/* Explicit lookup table. switch() might compile down (and it does at least with GCC 14)
to a jump table. Let's help the compiler a bit here. */
static constexpr auto table = [] {
std::array<ValueType, tNumberOfInternalTypes> t{};
t[tUninitialized] = nThunk;
t[tInt] = nInt;
t[tBool] = nBool;
t[tNull] = nNull;
t[tFloat] = nFloat;
t[tExternal] = nExternal;
t[tAttrs] = nAttrs;
t[tPrimOp] = nFunction;
t[tLambda] = nFunction;
t[tPrimOpApp] = nFunction;
t[tApp] = nThunk;
t[tThunk] = nThunk;
t[tListSmall] = nList;
t[tListN] = nList;
t[tString] = nString;
t[tPath] = nPath;
return t;
}();
auto it = getInternalType();
if (it == tUninitialized || it >= tNumberOfInternalTypes) [[unlikely]] {
if constexpr (invalidIsThunk)
return nThunk;
else
nixUnreachableWhenHardened();
}
if (invalidIsThunk)
return nThunk;
else
unreachable();
return table[it];
}
/**

View File

@@ -28,7 +28,7 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/big-objs')
# Check for each of these functions, and create a define like `#define HAVE_LCHOWN 1`.
# Check for each of these functions, and create a define like `#define HAVE_SYSCONF 1`.
check_funcs = [
'sysconf',
]

View File

@@ -1819,8 +1819,8 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
Unless we are in read-only mode, that is, in which case we do not
write anything. Users commonly do this to speed up evaluation in
contexts where they don't actually want to build anything. */
auto drvPath = state.settings.storeSettings.readOnlyMode ? computeStorePath(*state.store, drv)
: state.store->writeDerivation(drv, state.repair);
auto drvPath =
settings.readOnlyMode ? computeStorePath(*state.store, drv) : state.store->writeDerivation(drv, state.repair);
auto drvPathS = state.store->printStorePath(drvPath);
printMsg(lvlChatty, "instantiated '%1%' -> '%2%'", drvName, drvPathS);
@@ -1935,7 +1935,7 @@ static void prim_storePath(EvalState & state, const PosIdx pos, Value ** args, V
if (!state.store->isInStore(path.abs()))
state.error<EvalError>("path '%1%' is not in the Nix store", path).atPos(pos).debugThrow();
auto path2 = state.store->toStorePath(path.abs()).first;
if (!state.settings.storeSettings.readOnlyMode)
if (!settings.readOnlyMode)
state.store->ensurePath(path2);
context.insert(NixStringContextElem::Opaque{.path = path2});
v.mkString(path.abs(), context, state.mem);
@@ -2676,24 +2676,23 @@ static void prim_toFile(EvalState & state, const PosIdx pos, Value ** args, Valu
.debugThrow();
}
auto storePath = state.settings.storeSettings.readOnlyMode
? state.store->makeFixedOutputPathFromCA(
name,
TextInfo{
.hash = hashString(HashAlgorithm::SHA256, contents),
.references = std::move(refs),
})
: ({
StringSource s{contents};
state.store->addToStoreFromDump(
s,
name,
FileSerialisationMethod::Flat,
ContentAddressMethod::Raw::Text,
HashAlgorithm::SHA256,
refs,
state.repair);
});
auto storePath = settings.readOnlyMode ? state.store->makeFixedOutputPathFromCA(
name,
TextInfo{
.hash = hashString(HashAlgorithm::SHA256, contents),
.references = std::move(refs),
})
: ({
StringSource s{contents};
state.store->addToStoreFromDump(
s,
name,
FileSerialisationMethod::Flat,
ContentAddressMethod::Raw::Text,
HashAlgorithm::SHA256,
refs,
state.repair);
});
/* Note: we don't need to add `context' to the context of the
result, since `storePath' itself has references to the paths
@@ -2837,24 +2836,23 @@ static void addPath(
if (!expectedHash || !state.store->isValidPath(*expectedStorePath)) {
// FIXME: support refs in fetchToStore()?
auto dstPath = refs.empty()
? fetchToStore(
state.fetchSettings,
*state.store,
path.resolveSymlinks(),
state.settings.storeSettings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
name,
method,
filter.get(),
state.repair)
: state.store->addToStore(
name,
path.resolveSymlinks(),
method,
HashAlgorithm::SHA256,
refs,
filter ? *filter.get() : defaultPathFilter,
state.repair);
auto dstPath = refs.empty() ? fetchToStore(
state.fetchSettings,
*state.store,
path.resolveSymlinks(),
settings.readOnlyMode ? FetchMode::DryRun : FetchMode::Copy,
name,
method,
filter.get(),
state.repair)
: state.store->addToStore(
name,
path.resolveSymlinks(),
method,
HashAlgorithm::SHA256,
refs,
filter ? *filter.get() : defaultPathFilter,
state.repair);
if (expectedHash && expectedStorePath != dstPath)
state.error<EvalError>("store path mismatch in (possibly filtered) path added from '%s'", path)
.atPos(pos)

View File

@@ -271,7 +271,7 @@ static void prim_appendContext(EvalState & state, const PosIdx pos, Value ** arg
if (!state.store->isStorePath(name))
state.error<EvalError>("context key '%s' is not a store path", name).atPos(i.pos).debugThrow();
auto namePath = state.store->parseStorePath(name);
if (!state.settings.storeSettings.readOnlyMode)
if (!settings.readOnlyMode)
state.store->ensurePath(namePath);
state.forceAttrs(*i.value, i.pos, "while evaluating the value of a string context");

View File

@@ -209,7 +209,7 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value ** args
{.msg = HintFmt("'fetchClosure' does not support URL query parameters (in '%s')", *fromStoreUrl),
.pos = state.positions[pos]});
auto fromStore = openStore(state.settings.storeSettings, std::move(storeRef));
auto fromStore = openStore(std::move(storeRef));
if (toPath)
runFetchClosureWithRewrite(state, pos, *fromStore, *fromPath, *toPath, v);

View File

@@ -1,14 +1,13 @@
#include "nix_api_fetchers.h"
#include "nix_api_fetchers_internal.hh"
#include "nix_api_util_internal.h"
#include "nix_api_store_internal.h"
extern "C" {
nix_fetchers_settings * nix_fetchers_settings_new(nix_c_context * context)
{
try {
auto fetchersSettings = nix::make_ref<nix::fetchers::Settings>(nix::fetchers::Settings{cStoreSettings});
auto fetchersSettings = nix::make_ref<nix::fetchers::Settings>(nix::fetchers::Settings{});
return new nix_fetchers_settings{
.settings = fetchersSettings,
};

View File

@@ -1,12 +1,9 @@
#include <nlohmann/json.hpp>
#include <gtest/gtest.h>
#include "nix/util/json-utils.hh"
#include "nix/store/globals.hh"
#include "nix/fetchers/fetchers.hh"
#include "nix/fetchers/fetch-settings.hh"
#include "nix/store/tests/test-main.hh"
#include "nix/util/json-utils.hh"
#include "nix/util/tests/characterization.hh"
namespace nix::fetchers {
@@ -28,8 +25,7 @@ public:
TEST_F(AccessKeysTest, singleOrgGitHub)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"github.com/a", "token"});
auto i = Input::fromURL(fetchSettings, "github:a/b");
@@ -39,8 +35,7 @@ TEST_F(AccessKeysTest, singleOrgGitHub)
TEST_F(AccessKeysTest, nonMatches)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"github.com", "token"});
auto i = Input::fromURL(fetchSettings, "gitlab:github.com/evil");
@@ -50,8 +45,7 @@ TEST_F(AccessKeysTest, nonMatches)
TEST_F(AccessKeysTest, noPartialMatches)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"github.com/partial", "token"});
auto i = Input::fromURL(fetchSettings, "github:partial-match/repo");
@@ -61,8 +55,7 @@ TEST_F(AccessKeysTest, noPartialMatches)
TEST_F(AccessKeysTest, repoGitHub)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"github.com", "token"});
fetchSettings.accessTokens.get().insert({"github.com/a/b", "another_token"});
fetchSettings.accessTokens.get().insert({"github.com/a/c", "yet_another_token"});
@@ -80,8 +73,7 @@ TEST_F(AccessKeysTest, repoGitHub)
TEST_F(AccessKeysTest, multipleGitLab)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"gitlab.com", "token"});
fetchSettings.accessTokens.get().insert({"gitlab.com/a/b", "another_token"});
auto i = Input::fromURL(fetchSettings, "gitlab:a/b");
@@ -95,8 +87,7 @@ TEST_F(AccessKeysTest, multipleGitLab)
TEST_F(AccessKeysTest, multipleSourceHut)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings = fetchers::Settings{settings};
fetchers::Settings fetchSettings = fetchers::Settings{};
fetchSettings.accessTokens.get().insert({"git.sr.ht", "token"});
fetchSettings.accessTokens.get().insert({"git.sr.ht/~a/b", "another_token"});
auto i = Input::fromURL(fetchSettings, "sourcehut:a/b");

View File

@@ -5,8 +5,6 @@
#include "nix/fetchers/fetchers.hh"
#include "nix/fetchers/git-utils.hh"
#include "nix/store/tests/test-main.hh"
#include <git2.h>
#include <gtest/gtest.h>
@@ -182,15 +180,13 @@ TEST_F(GitTest, submodulePeriodSupport)
// 6) Commit the addition in super
commitAll(super.get(), "Add submodule with branch='.'");
auto settings = getTestSettings();
auto store = [&] {
auto cfg = make_ref<DummyStoreConfig>(settings, StoreReference::Params{});
auto store = [] {
auto cfg = make_ref<DummyStoreConfig>(StoreReference::Params{});
cfg->readOnly = false;
return cfg->openStore();
}();
auto fetchSettings = fetchers::Settings{settings};
auto settings = fetchers::Settings{};
auto input = fetchers::Input::fromAttrs(
settings,
{
@@ -200,7 +196,7 @@ TEST_F(GitTest, submodulePeriodSupport)
{"ref", "main"},
});
auto [accessor, i] = input.getAccessor(fetchSettings, *store);
auto [accessor, i] = input.getAccessor(settings, *store);
ASSERT_EQ(accessor->readFile(CanonPath("deps/sub/lib.txt")), "hello from submodule\n");
}

View File

@@ -1,11 +1,7 @@
#include "nix/store/globals.hh"
#include "nix/fetchers/fetch-settings.hh"
#include "nix/fetchers/attrs.hh"
#include "nix/fetchers/fetchers.hh"
#include "nix/store/tests/test-main.hh"
#include <gtest/gtest.h>
#include <string>
@@ -27,8 +23,7 @@ class InputFromAttrsTest : public ::testing::WithParamInterface<InputFromAttrsTe
TEST_P(InputFromAttrsTest, attrsAreCorrectAndRoundTrips)
{
auto settings = getTestSettings();
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
const auto & testCase = GetParam();

View File

@@ -47,7 +47,7 @@ struct CacheImpl : Cache
auto dbPath = (getCacheDir() / "fetcher-cache-v4.sqlite").string();
createDirs(dirOf(dbPath));
state->db = SQLite(dbPath, {.useWAL = settings.storeSettings.useSQLiteWAL});
state->db = SQLite(dbPath, {.useWAL = nix::settings.useSQLiteWAL});
state->db.isCache();
state->db.exec(schema);

View File

@@ -2,9 +2,6 @@
namespace nix::fetchers {
Settings::Settings(const nix::Settings & storeSettings)
: storeSettings(storeSettings)
{
}
Settings::Settings() {}
} // namespace nix::fetchers

View File

@@ -47,6 +47,12 @@ std::pair<StorePath, Hash> fetchToStore2(
auto hash = Hash::parseSRI(fetchers::getStrAttr(*res, "hash"));
auto storePath =
store.makeFixedOutputPathFromCA(name, ContentAddressWithReferences::fromParts(method, hash, {}));
/* Add a temproot before the call to isValidPath to prevent accidental GC in case the
input is cached. Note that this must be done before to avoid races. */
if (mode != FetchMode::DryRun)
store.addTempRoot(storePath);
if (mode == FetchMode::DryRun || store.isValidPath(storePath)) {
debug(
"source path '%s' cache hit in '%s' (hash '%s')",
@@ -76,8 +82,8 @@ std::pair<StorePath, Hash> fetchToStore2(
auto [storePath, hash] =
mode == FetchMode::DryRun
? [&]() {
auto [storePath, hash] = store.computeStorePath(
store.config.settings, name, path, method, HashAlgorithm::SHA256, {}, filter2);
auto [storePath, hash] =
store.computeStorePath(name, path, method, HashAlgorithm::SHA256, {}, filter2);
debug(
"hashed '%s' to '%s' (hash '%s')",
path,

View File

@@ -74,6 +74,29 @@ namespace nix {
struct GitSourceAccessor;
struct GitError : public Error
{
template<typename... Ts>
GitError(const git_error & error, Ts &&... args)
: Error("")
{
auto hf = HintFmt(std::forward<Ts>(args)...);
err.msg = HintFmt("%1%: %2% (libgit2 error code = %3%)", Uncolored(hf.str()), error.message, error.klass);
}
template<typename... Ts>
GitError(Ts &&... args)
: GitError(
[]() -> const git_error & {
const git_error * p = git_error_last();
assert(p && "git_error_last() is unexpectedly null");
return *p;
}(),
std::forward<Ts>(args)...)
{
}
};
typedef std::unique_ptr<git_repository, Deleter<git_repository_free>> Repository;
typedef std::unique_ptr<git_tree_entry, Deleter<git_tree_entry_free>> TreeEntry;
typedef std::unique_ptr<git_tree, Deleter<git_tree_free>> Tree;
@@ -106,7 +129,7 @@ static void initLibGit2()
static std::once_flag initialized;
std::call_once(initialized, []() {
if (git_libgit2_init() < 0)
throw Error("initialising libgit2: %s", git_error_last()->message);
throw GitError("initialising libgit2");
});
}
@@ -114,7 +137,7 @@ static git_oid hashToOID(const Hash & hash)
{
git_oid oid;
if (git_oid_fromstr(&oid, hash.gitRev().c_str()))
throw Error("cannot convert '%s' to a Git OID", hash.gitRev());
throw GitError("cannot convert '%s' to a Git OID", hash.gitRev());
return oid;
}
@@ -122,8 +145,7 @@ static Object lookupObject(git_repository * repo, const git_oid & oid, git_objec
{
Object obj;
if (git_object_lookup(Setter(obj), repo, &oid, type)) {
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return obj;
}
@@ -133,8 +155,7 @@ static T peelObject(git_object * obj, git_object_t type)
{
T obj2;
if (git_object_peel((git_object **) (typename T::pointer *) Setter(obj2), obj, type)) {
auto err = git_error_last();
throw Error("peeling Git object '%s': %s", *git_object_id(obj), err->message);
throw Error("peeling Git object '%s'", *git_object_id(obj));
}
return obj2;
}
@@ -144,7 +165,7 @@ static T dupObject(typename T::pointer obj)
{
T obj2;
if (git_object_dup((git_object **) (typename T::pointer *) Setter(obj2), (git_object *) obj))
throw Error("duplicating object '%s': %s", *git_object_id((git_object *) obj), git_error_last()->message);
throw GitError("duplicating object '%s'", *git_object_id((git_object *) obj));
return obj2;
}
@@ -216,7 +237,7 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
Repository tmpRepo;
if (git_repository_init(Setter(tmpRepo), tmpDir.string().c_str(), options.bare))
throw Error("creating Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("creating Git repository %s", PathFmt(path));
try {
std::filesystem::rename(tmpDir, path);
} catch (std::filesystem::filesystem_error & e) {
@@ -226,7 +247,8 @@ static void initRepoAtomically(std::filesystem::path & path, GitRepo::Options op
|| e.code() == std::errc::directory_not_empty) {
return;
} else
throw SysError("moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
throw SystemError(
e.code(), "moving temporary git repository from %s to %s", PathFmt(tmpDir), PathFmt(path));
}
// we successfully moved the repository, so the temporary directory no longer exists.
delTmpDir.cancel();
@@ -266,7 +288,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
initRepoAtomically(path, options);
if (git_repository_open(Setter(repo), path.string().c_str()))
throw Error("opening Git repository %s: %s", PathFmt(path), git_error_last()->message);
throw GitError("opening Git repository %s", PathFmt(path));
ObjectDb odb;
if (options.packfilesOnly) {
@@ -279,28 +301,28 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
*/
if (git_odb_new(Setter(odb)))
throw Error("creating Git object database: %s", git_error_last()->message);
throw GitError("creating Git object database");
if (git_odb_backend_pack(&packBackend, (path / "objects").string().c_str()))
throw Error("creating pack backend: %s", git_error_last()->message);
throw GitError("creating pack backend");
if (git_odb_add_backend(odb.get(), packBackend, 1))
throw Error("adding pack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding pack backend to Git object database");
} else {
if (git_repository_odb(Setter(odb), repo.get()))
throw Error("getting Git object database: %s", git_error_last()->message);
throw GitError("getting Git object database");
}
// mempack_backend will be owned by the repository, so we are not expected to free it ourselves.
if (git_mempack_new(&mempackBackend))
throw Error("creating mempack backend: %s", git_error_last()->message);
throw GitError("creating mempack backend");
if (git_odb_add_backend(odb.get(), mempackBackend, 999))
throw Error("adding mempack backend to Git object database: %s", git_error_last()->message);
throw GitError("adding mempack backend to Git object database");
if (options.packfilesOnly) {
if (git_repository_set_odb(repo.get(), odb.get()))
throw Error("setting Git object database: %s", git_error_last()->message);
throw GitError("setting Git object database");
}
}
@@ -340,7 +362,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
Indexer indexer;
git_indexer_progress stats;
if (git_indexer_new(Setter(indexer), pack_dir_path.c_str(), 0, nullptr, nullptr))
throw Error("creating git packfile indexer: %s", git_error_last()->message);
throw GitError("creating git packfile indexer");
// TODO: provide index callback for checkInterrupt() termination
// though this is about an order of magnitude faster than the packbuilder
@@ -348,15 +370,15 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
constexpr size_t chunkSize = 128 * 1024;
for (size_t offset = 0; offset < buf.size; offset += chunkSize) {
if (git_indexer_append(indexer.get(), buf.ptr + offset, std::min(chunkSize, buf.size - offset), &stats))
throw Error("appending to git packfile index: %s", git_error_last()->message);
throw GitError("appending to git packfile index");
checkInterrupt();
}
if (git_indexer_commit(indexer.get(), &stats))
throw Error("committing git packfile index: %s", git_error_last()->message);
throw GitError("committing git packfile index");
if (git_mempack_reset(mempackBackend))
throw Error("resetting git mempack backend: %s", git_error_last()->message);
throw GitError("resetting git mempack backend");
checkInterrupt();
}
@@ -449,7 +471,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
void setRemote(const std::string & name, const std::string & url) override
{
if (git_remote_set_url(*this, name.c_str(), url.c_str()))
throw Error("setting remote '%s' URL to '%s': %s", name, url, git_error_last()->message);
throw GitError("setting remote '%s' URL to '%s'", name, url);
}
Hash resolveRef(std::string ref) override
@@ -462,7 +484,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
// an object_id.
std::string peeledRef = ref + "^{commit}";
if (git_revparse_single(Setter(object), *this, peeledRef.c_str()))
throw Error("resolving Git reference '%s': %s", ref, git_error_last()->message);
throw GitError("resolving Git reference '%s'", ref);
auto oid = git_object_id(object.get());
return toHash(*oid);
}
@@ -471,11 +493,11 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
{
GitConfig config;
if (git_config_open_ondisk(Setter(config), configFile.string().c_str()))
throw Error("parsing .gitmodules file: %s", git_error_last()->message);
throw GitError("parsing .gitmodules file");
ConfigIterator it;
if (git_config_iterator_glob_new(Setter(it), config.get(), "^submodule\\..*\\.(path|url|branch)$"))
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
StringMap entries;
@@ -484,7 +506,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto err = git_config_next(&entry, it.get())) {
if (err == GIT_ITEROVER)
break;
throw Error("iterating over .gitmodules: %s", git_error_last()->message);
throw GitError("iterating over .gitmodules");
}
entries.emplace(entry->name + 10, entry->value);
}
@@ -521,7 +543,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
git_oid headRev;
if (auto err = git_reference_name_to_id(&headRev, *this, "HEAD")) {
if (err != GIT_ENOTFOUND)
throw Error("resolving HEAD: %s", git_error_last()->message);
throw GitError("resolving HEAD");
} else
info.headRev = toHash(headRev);
@@ -544,7 +566,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
options.flags |= GIT_STATUS_OPT_INCLUDE_UNMODIFIED;
options.flags |= GIT_STATUS_OPT_EXCLUDE_SUBMODULES;
if (git_status_foreach_ext(*this, &options, &statusCallbackTrampoline, &statusCallback))
throw Error("getting working directory status: %s", git_error_last()->message);
throw GitError("getting working directory status");
/* Get submodule info. */
auto modulesFile = path / ".gitmodules";
@@ -587,8 +609,7 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (auto errCode = git_object_lookup(Setter(obj), *this, &oid, GIT_OBJECT_ANY)) {
if (errCode == GIT_ENOTFOUND)
return false;
auto err = git_error_last();
throw Error("getting Git object '%s': %s", oid, err->message);
throw GitError("getting Git object '%s'", oid);
}
return true;
@@ -918,7 +939,7 @@ struct GitSourceAccessor : SourceAccessor
TreeEntry copy;
if (git_tree_entry_dup(Setter(copy), entry))
throw Error("dupping tree entry: %s", git_error_last()->message);
throw GitError("dupping tree entry");
auto entryName = std::string_view(git_tree_entry_name(entry));
@@ -948,7 +969,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -983,7 +1004,7 @@ struct GitSourceAccessor : SourceAccessor
Tree tree;
if (git_tree_entry_to_object((git_object **) (git_tree **) Setter(tree), *state.repo, entry))
throw Error("looking up directory '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up directory '%s'", showPath(path));
return tree;
}
@@ -1016,7 +1037,7 @@ struct GitSourceAccessor : SourceAccessor
Blob blob;
if (git_tree_entry_to_object((git_object **) (git_blob **) Setter(blob), *state.repo, entry))
throw Error("looking up file '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up file '%s'", showPath(path));
return blob;
}
@@ -1068,7 +1089,7 @@ struct GitExportIgnoreSourceAccessor : CachingFilteringSourceAccessor
if (git_error_last()->klass == GIT_ENOTFOUND)
return false;
else
throw Error("looking up '%s': %s", showPath(path), git_error_last()->message);
throw GitError("looking up '%s'", showPath(path));
} else {
// Official git will silently reject export-ignore lines that have
// values. We do the same.
@@ -1224,17 +1245,17 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
repo.emplace(parent.repoPool.get());
if (git_blob_create_from_stream(Setter(stream), **repo, nullptr))
throw Error("creating a blob stream object: %s", git_error_last()->message);
throw GitError("creating a blob stream object");
if (stream->write(stream.get(), contents.data(), contents.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
parent.totalBufSize -= contents.size();
contents.clear();
}
} else {
if (stream->write(stream.get(), data.data(), data.size()))
throw Error("writing a blob for tarball member '%s': %s", path, git_error_last()->message);
throw GitError("writing a blob for tarball member '%s'", path);
}
}
@@ -1256,7 +1277,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
acquires ownership and frees the stream. */
git_oid oid;
if (git_blob_create_from_stream_commit(&oid, crf->stream.release()))
throw Error("creating a blob object for '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for '%s'", path);
addNode(
*_state.lock(),
crf->path,
@@ -1270,8 +1291,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, crf->contents.data(), crf->contents.size()))
throw Error(
"creating a blob object for '%s' from in-memory buffer: %s", crf->path, git_error_last()->message);
throw GitError("creating a blob object for '%s' from in-memory buffer", crf->path);
addNode(
*_state.lock(),
@@ -1295,8 +1315,7 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_oid oid;
if (git_blob_create_from_buffer(&oid, *repo, target.c_str(), target.size()))
throw Error(
"creating a blob object for tarball symlink member '%s': %s", path, git_error_last()->message);
throw GitError("creating a blob object for tarball symlink member '%s'", path);
auto state(_state.lock());
addNode(*state, path, Child{GIT_FILEMODE_LINK, oid});
@@ -1357,19 +1376,19 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
// Write this directory.
git_treebuilder * b;
if (git_treebuilder_new(&b, *repo, nullptr))
throw Error("creating a tree builder: %s", git_error_last()->message);
throw GitError("creating a tree builder");
TreeBuilder builder(b);
for (auto & [name, child] : node.children) {
auto oid_p = std::get_if<git_oid>(&child.file);
auto oid = oid_p ? *oid_p : std::get<Directory>(child.file).oid.value();
if (git_treebuilder_insert(nullptr, builder.get(), name.c_str(), &oid, child.mode))
throw Error("adding a file to a tree builder: %s", git_error_last()->message);
throw GitError("adding a file to a tree builder");
}
git_oid oid;
if (git_treebuilder_write(&oid, builder.get()))
throw Error("creating a tree object: %s", git_error_last()->message);
throw GitError("creating a tree object");
node.oid = oid;
}(_state.lock()->root);

View File

@@ -13,10 +13,9 @@
namespace nix {
class Settings;
struct GitRepo;
} // namespace nix
}
namespace nix::fetchers {
@@ -24,12 +23,7 @@ struct Cache;
struct Settings : public Config
{
/**
* Reference to the "parent" store-layer settings.
*/
const nix::Settings & storeSettings;
Settings(const nix::Settings & storeSettings);
Settings();
Setting<StringMap> accessTokens{
this,

View File

@@ -3,7 +3,6 @@
#include <string>
#include <utility>
#include "nix/store/globals.hh"
#include "nix/fetchers/fetch-settings.hh"
#include "nix/flake/flakeref.hh"
#include "nix/fetchers/attrs.hh"
@@ -20,8 +19,7 @@ TEST(parseFlakeRef, path)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
Settings settings;
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
{
auto s = "/foo/bar";
@@ -71,8 +69,7 @@ TEST(parseFlakeRef, GitArchiveInput)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
Settings settings;
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
{
auto s = "github:foo/bar/branch%23"; // branch name with `#`
@@ -115,8 +112,7 @@ class InputFromURLTest : public ::testing::WithParamInterface<InputFromURLTestCa
TEST_P(InputFromURLTest, attrsAreCorrectAndRoundTrips)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
Settings settings;
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
const auto & testCase = GetParam();
@@ -278,8 +274,7 @@ INSTANTIATE_TEST_SUITE_P(
TEST(to_string, doesntReencodeUrl)
{
Settings settings;
fetchers::Settings fetchSettings{settings};
fetchers::Settings fetchSettings;
auto s = "http://localhost:8181/test/+3d.tar.gz";
auto flakeref = parseFlakeRef(fetchSettings, s);
auto unparsed = flakeref.to_string();
@@ -288,4 +283,18 @@ TEST(to_string, doesntReencodeUrl)
ASSERT_EQ(unparsed, expected);
}
TEST(parseFlakeRef, malformedGithubUrlDoesNotCrash)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
fetchers::Settings fetchSettings;
// Using ref= instead of rev= with a github: URL should produce an
// error, not an assertion failure in renderAuthorityAndPath
// (https://github.com/NixOS/nix/issues/15196).
EXPECT_THROW(
parseFlakeRef(fetchSettings, "github:nixos/nixpkgs/nixpkgs.git?ref=aead170c1a49253ebfa5027010dfd89a77b73ca4"),
Error);
}
} // namespace nix

View File

@@ -35,35 +35,39 @@ namespace nix::flake::primops {
PrimOp getFlake(const Settings & settings)
{
auto prim_getFlake = [&settings](EvalState & state, const PosIdx pos, Value ** args, Value & v) {
std::string flakeRefS(
state.forceStringNoCtx(*args[0], pos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = nix::parseFlakeRef(state.fetchSettings, flakeRefS, {}, true);
if (state.settings.pureEval && !flakeRef.input.isLocked(state.fetchSettings))
throw Error(
"cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)",
flakeRefS,
state.positions[pos]);
state.forceValue(*args[0], pos);
callFlake(
state,
lockFlake(
settings,
state,
flakeRef,
LockFlags{
.updateLockFile = false,
.writeLockFile = false,
.useRegistries = !state.settings.pureEval && settings.useRegistries,
.allowUnlocked = !state.settings.pureEval,
}),
v);
LockFlags lockFlags{
.updateLockFile = false,
.writeLockFile = false,
.useRegistries = !state.settings.pureEval && settings.useRegistries,
.allowUnlocked = !state.settings.pureEval,
};
if (args[0]->type() == nPath) {
auto path = state.realisePath(pos, *args[0]);
callFlake(state, lockFlake(settings, state, path, lockFlags), v);
} else {
NixStringContext context;
std::string flakeRefS(
state.forceStringNoCtx(*args[0], pos, "while evaluating the argument passed to builtins.getFlake"));
auto flakeRef = nix::parseFlakeRef(state.fetchSettings, flakeRefS, {}, true);
if (state.settings.pureEval && !flakeRef.input.isLocked(state.fetchSettings))
throw Error(
"cannot call 'getFlake' on unlocked flake reference '%s', at %s (use --impure to override)",
flakeRefS,
state.positions[pos]);
callFlake(state, lockFlake(settings, state, flakeRef, lockFlags), v);
}
};
return PrimOp{
.name = "__getFlake",
.args = {"args"},
.doc = R"(
Fetch a flake from a flake reference, and return its output attributes and some metadata. For example:
Fetch a flake from a flake reference or a path, and return its output attributes and some metadata. For example:
```nix
(builtins.getFlake "nix/55bc52401966fbffa525c574c14f67b00bc4fb3a").packages.x86_64-linux.nix

View File

@@ -416,10 +416,8 @@ static LockFile readLockFile(const fetchers::Settings & fetchSettings, const Sou
: LockFile();
}
/* Compute an in-memory lock file for the specified top-level flake,
and optionally write it to file, if the flake is writable. */
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags)
LockedFlake lockFlake(
const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags, Flake flake)
{
experimentalFeatureSettings.require(Xp::Flakes);
@@ -427,8 +425,6 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
auto useRegistriesTop = useRegistries ? fetchers::UseRegistries::All : fetchers::UseRegistries::No;
auto useRegistriesInputs = useRegistries ? fetchers::UseRegistries::Limited : fetchers::UseRegistries::No;
auto flake = getFlake(state, topRef, useRegistriesTop, {});
if (lockFlags.applyNixConfig) {
flake.config.apply(settings);
state.store->setOptions();
@@ -908,6 +904,22 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef,
}
}
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & topRef, const LockFlags & lockFlags)
{
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto useRegistriesTop = useRegistries ? fetchers::UseRegistries::All : fetchers::UseRegistries::No;
return lockFlake(settings, state, topRef, lockFlags, getFlake(state, topRef, useRegistriesTop, {}));
}
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const SourcePath & flakeDir, const LockFlags & lockFlags)
{
/* We need a fake flakeref to put in the `Flake` struct, but it's not used for anything. */
auto fakeRef = parseFlakeRef(state.fetchSettings, "flake:get-flake");
return lockFlake(settings, state, fakeRef, lockFlags, readFlake(state, fakeRef, fakeRef, fakeRef, flakeDir, {}));
}
static ref<SourceAccessor> makeInternalFS()
{
auto internalFS = make_ref<MemorySourceAccessor>(MemorySourceAccessor{});

View File

@@ -205,7 +205,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
fetchSettings,
{
.scheme = "path",
.authority = ParsedURL::Authority{},
.authority = isAbsolute(path) ? std::optional{ParsedURL::Authority{}} : std::nullopt,
.path = splitString<std::vector<std::string>>(path, "/"),
.query = query,
.fragment = fragment,

View File

@@ -214,9 +214,16 @@ struct LockFlags
std::set<NonEmptyInputAttrPath> inputUpdates;
};
/*
* Compute an in-memory lock file for the specified top-level flake, and optionally write it to file, if the flake is
* writable.
*/
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const FlakeRef & flakeRef, const LockFlags & lockFlags);
LockedFlake
lockFlake(const Settings & settings, EvalState & state, const SourcePath & flakeDir, const LockFlags & lockFlags);
void callFlake(EvalState & state, const LockedFlake & lockedFlake, Value & v);
/**

View File

@@ -8,7 +8,6 @@
#include "nix/main/loggers.hh"
#include "nix/util/util.hh"
#include "nix/main/plugin.hh"
#include "nix/main/shared.hh"
namespace nix {

View File

@@ -13,13 +13,6 @@
namespace nix {
class Settings;
// FIXME: don't use a global variable.
extern Settings settings;
int handleExceptions(const std::string & programName, std::function<void()> fun);
/**
* Don't forget to call initPlugins() after settings are initialized!
* @param loadConfig Whether to load configuration from `nix.conf`, `NIX_CONFIG`, etc. May be disabled for unit tests.

View File

@@ -31,17 +31,12 @@
#include "nix/util/exit.hh"
#include "nix/util/strings.hh"
#include "nix/util/config-global.hh"
#include "main-config-private.hh"
#include "nix/expr/config.hh"
namespace nix {
Settings settings;
static GlobalConfig::Register rSettings(&settings);
char ** savedArgv;
static bool gcWarning = true;
@@ -340,8 +335,8 @@ void printVersion(const std::string & programName)
std::cout << "System configuration file: " << nixConfFile() << "\n";
std::cout << "User configuration files: "
<< os_string_to_string(ExecutablePath{.directories = nixUserConfFiles()}.render()) << "\n";
std::cout << "Store directory: "
<< resolveStoreConfig(settings, StoreReference{settings.storeUri.get()})->storeDir << "\n";
std::cout << "Store directory: " << resolveStoreConfig(StoreReference{settings.storeUri.get()})->storeDir
<< "\n";
std::cout << "State directory: " << settings.nixStateDir << "\n";
}
throw Exit();

View File

@@ -18,9 +18,6 @@
extern "C" {
// FIXME: This is going to conflict with the one in libmain.
nix::Settings cStoreSettings;
nix_err nix_libstore_init(nix_c_context * context)
{
if (context)
@@ -49,7 +46,7 @@ Store * nix_store_open(nix_c_context * context, const char * uri, const char ***
std::string uri_str = uri ? uri : "";
if (uri_str.empty())
return new Store{nix::openStore(cStoreSettings)};
return new Store{nix::openStore()};
auto storeRef = nix::StoreReference::parse(uri_str);
@@ -58,7 +55,7 @@ Store * nix_store_open(nix_c_context * context, const char * uri, const char ***
storeRef.params[params[i][0]] = params[i][1];
}
}
return new Store{nix::openStore(cStoreSettings, std::move(storeRef))};
return new Store{nix::openStore(std::move(storeRef))};
}
NIXC_CATCH_ERRS_NULL
}
@@ -315,8 +312,8 @@ StorePath * nix_add_derivation(nix_c_context * context, Store * store, nix_deriv
without actually writing the derivation if this setting is
set, but it was that way already, so we are doing this for
back-compat for now. */
auto ret = cStoreSettings.readOnlyMode ? nix::computeStorePath(*store->ptr, derivation->drv)
: store->ptr->writeDerivation(derivation->drv, nix::NoRepair);
auto ret = nix::settings.readOnlyMode ? nix::computeStorePath(*store->ptr, derivation->drv)
: store->ptr->writeDerivation(derivation->drv, nix::NoRepair);
return new StorePath{ret};
}

View File

@@ -5,8 +5,6 @@
extern "C" {
extern nix::Settings cStoreSettings;
struct Store
{
nix::ref<nix::Store> ptr;

View File

@@ -1,5 +1,4 @@
#include "nix/store/tests/https-store.hh"
#include "nix/store/tests/test-main.hh"
#include <thread>
@@ -37,9 +36,9 @@ void HttpsBinaryCacheStoreTest::SetUp()
cacheDir = tmpDir / "cache";
delTmpDir = std::make_unique<AutoDelete>(tmpDir);
localCacheStore = make_ref<LocalBinaryCacheStoreConfig>(
settings, "file", cacheDir.string(), LocalBinaryCacheStoreConfig::Params{})
->openStore();
localCacheStore =
make_ref<LocalBinaryCacheStoreConfig>("file", cacheDir.string(), LocalBinaryCacheStoreConfig::Params{})
->openStore();
caCert = tmpDir / "ca.crt";
caKey = tmpDir / "ca.key";
@@ -51,9 +50,11 @@ void HttpsBinaryCacheStoreTest::SetUp()
// clang-format off
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", caKey.string()});
openssl({"req", "-new", "-x509", "-days", "1", "-key", caKey.string(), "-out", caCert.string(), "-subj", "/CN=TestCA"});
auto serverExtFile = tmpDir / "server.ext";
writeFile(serverExtFile, "subjectAltName=DNS:localhost,IP:127.0.0.1");
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", serverKey.string()});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1"});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost,IP:127.0.0.1"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1", "-extfile", serverExtFile.string()});
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", clientKey.string()});
openssl({"req", "-new", "-key", clientKey.string(), "-out", (tmpDir / "client.csr").string(), "-subj", "/CN=TestClient"});
openssl({"x509", "-req", "-in", (tmpDir / "client.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", clientCert.string(), "-days", "1"});
@@ -122,7 +123,6 @@ std::vector<std::string> HttpsBinaryCacheStoreMtlsTest::serverArgs()
ref<TestHttpBinaryCacheStoreConfig> HttpsBinaryCacheStoreTest::makeConfig()
{
auto res = make_ref<TestHttpBinaryCacheStoreConfig>(
settings,
ParsedURL{
.scheme = "https",
.authority =

View File

@@ -12,8 +12,6 @@
#include "nix/util/file-system.hh"
#include "nix/util/processes.hh"
#include "nix/store/tests/test-main.hh"
namespace nix::testing {
class TestHttpBinaryCacheStoreConfig;
@@ -44,9 +42,9 @@ public:
class TestHttpBinaryCacheStoreConfig : public HttpBinaryCacheStoreConfig
{
public:
TestHttpBinaryCacheStoreConfig(nix::Settings & settings, ParsedURL url, const Store::Config::Params & params)
: StoreConfig(settings, params)
, HttpBinaryCacheStoreConfig(settings, url, params)
TestHttpBinaryCacheStoreConfig(ParsedURL url, const Store::Config::Params & params)
: StoreConfig(params)
, HttpBinaryCacheStoreConfig(url, params)
{
}
@@ -58,8 +56,6 @@ class HttpsBinaryCacheStoreTest : public virtual LibStoreNetworkTest
std::unique_ptr<AutoDelete> delTmpDir;
public:
Settings settings = getTestSettings();
static void SetUpTestSuite()
{
initLibStore(/*loadConfig=*/false);

View File

@@ -7,7 +7,6 @@
#include "nix/store/store-api.hh"
#include "nix/store/store-open.hh"
#include "nix/store/globals.hh"
#include "nix/store/tests/test-main.hh"
namespace nix {
@@ -19,16 +18,14 @@ public:
initLibStore(false);
}
Settings settings = getTestSettings();
protected:
LibStoreTest(auto && makeStore)
: store(makeStore(settings))
LibStoreTest(ref<Store> store)
: store(std::move(store))
{
}
LibStoreTest()
: LibStoreTest([](auto & settings) { return openStore(settings, "dummy://"); })
: LibStoreTest(openStore("dummy://"))
{
}

View File

@@ -4,13 +4,6 @@
namespace nix {
class Settings;
/**
* Get a Settings object configured appropriately for unit testing.
*/
Settings getTestSettings();
/**
* Call this for a GTest test suite that will including performing Nix
* builds, before running tests.

View File

@@ -7,9 +7,12 @@
namespace nix {
Settings getTestSettings()
int testMainForBuidingPre(int argc, char ** argv)
{
Settings settings;
if (argc > 1 && std::string_view(argv[1]) == "__build-remote") {
printError("test-build-remote: not supported in libexpr unit tests");
return EXIT_FAILURE;
}
// Disable build hook. We won't be testing remote builds in these unit tests. If we do, fix the above build hook.
settings.getWorkerSettings().buildHook = {};
@@ -38,16 +41,6 @@ Settings getTestSettings()
setEnv("_NIX_TEST_NO_SANDBOX", "1");
#endif
return settings;
}
int testMainForBuidingPre(int argc, char ** argv)
{
if (argc > 1 && std::string_view(argv[1]) == "__build-remote") {
printError("test-build-remote: not supported in libexpr unit tests");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}

View File

@@ -193,7 +193,7 @@ TYPED_TEST(DerivationAdvancedAttrsBothTest, advancedAttributes_defaults)
EXPECT_EQ(options, advancedAttributes_defaults);
EXPECT_EQ(options.substitutesAllowed(this->settings.getWorkerSettings()), true);
EXPECT_EQ(options.substitutesAllowed(settings.getWorkerSettings()), true);
EXPECT_EQ(options.useUidRange(got), false);
});
};
@@ -241,7 +241,7 @@ TYPED_TEST(DerivationAdvancedAttrsBothTest, advancedAttributes)
EXPECT_EQ(options, expected);
EXPECT_EQ(options.substitutesAllowed(this->settings.getWorkerSettings()), false);
EXPECT_EQ(options.substitutesAllowed(settings.getWorkerSettings()), false);
EXPECT_EQ(options.useUidRange(got), true);
});
};
@@ -333,7 +333,7 @@ TYPED_TEST(DerivationAdvancedAttrsBothTest, advancedAttributes_structuredAttrs_d
EXPECT_EQ(options, advancedAttributes_structuredAttrs_defaults);
EXPECT_EQ(options.substitutesAllowed(this->settings.getWorkerSettings()), true);
EXPECT_EQ(options.substitutesAllowed(settings.getWorkerSettings()), true);
EXPECT_EQ(options.useUidRange(got), false);
});
};
@@ -398,7 +398,7 @@ TYPED_TEST(DerivationAdvancedAttrsBothTest, advancedAttributes_structuredAttrs)
EXPECT_EQ(options, expected);
EXPECT_EQ(options.substitutesAllowed(this->settings.getWorkerSettings()), false);
EXPECT_EQ(options.substitutesAllowed(settings.getWorkerSettings()), false);
EXPECT_EQ(options.useUidRange(got), true);
});
};

View File

@@ -16,11 +16,11 @@ class FillInOutputPathsTest : public LibStoreTest, public JsonCharacterizationTe
protected:
FillInOutputPathsTest()
: LibStoreTest([](auto & settings) {
auto config = make_ref<DummyStoreConfig>(settings, DummyStoreConfig::Params{});
: LibStoreTest([]() {
auto config = make_ref<DummyStoreConfig>(DummyStoreConfig::Params{});
config->readOnly = false;
return config->openDummyStore();
})
}())
{
}

View File

@@ -7,7 +7,6 @@
#include "nix/store/realisation.hh"
#include "nix/util/tests/json-characterization.hh"
#include "nix/store/tests/test-main.hh"
namespace nix {
@@ -32,10 +31,8 @@ TEST(DummyStore, realisation_read)
{
initLibStore(/*loadConfig=*/false);
auto settings = getTestSettings();
auto store = [&] {
auto cfg = make_ref<DummyStoreConfig>(settings, StoreReference::Params{});
auto store = [] {
auto cfg = make_ref<DummyStoreConfig>(StoreReference::Params{});
cfg->readOnly = false;
return cfg->openDummyStore();
}();
@@ -76,10 +73,9 @@ TEST_P(DummyStoreJsonTest, from_json)
using namespace nlohmann;
/* Cannot use `readJsonTest` because need to dereference the stores
for equality. */
auto settings = getTestSettings();
readTest(Path{name} + ".json", [&](const auto & encodedRaw) {
auto encoded = json::parse(encodedRaw);
ref<DummyStore> decoded = adl_serializer<ref<DummyStore>>::from_json(settings, encoded);
ref<DummyStore> decoded = adl_serializer<ref<DummyStore>>::from_json(encoded);
ASSERT_EQ(*decoded, *expected);
});
}
@@ -92,13 +88,12 @@ TEST_P(DummyStoreJsonTest, to_json)
INSTANTIATE_TEST_SUITE_P(DummyStoreJSON, DummyStoreJsonTest, [] {
initLibStore(false);
auto settings = getTestSettings();
auto writeCfg = make_ref<DummyStore::Config>(settings, DummyStore::Config::Params{});
auto writeCfg = make_ref<DummyStore::Config>(DummyStore::Config::Params{});
writeCfg->readOnly = false;
return ::testing::Values(
std::pair{
"empty",
make_ref<DummyStore::Config>(settings, DummyStore::Config::Params{})->openDummyStore(),
make_ref<DummyStore::Config>(DummyStore::Config::Params{})->openDummyStore(),
},
std::pair{
"one-flat-file",

View File

@@ -1,44 +1,38 @@
#include <gtest/gtest.h>
#include <regex>
#include "nix/store/globals.hh"
#include "nix/store/http-binary-cache-store.hh"
#include "nix/store/tests/https-store.hh"
#include "nix/util/fs-sink.hh"
#include "nix/store/tests/test-main.hh"
namespace nix {
TEST(HttpBinaryCacheStore, constructConfig)
{
auto settings = getTestSettings();
HttpBinaryCacheStoreConfig config{settings, "http", "foo.bar.baz", {}};
HttpBinaryCacheStoreConfig config{"http", "foo.bar.baz", {}};
EXPECT_EQ(config.cacheUri.to_string(), "http://foo.bar.baz");
}
TEST(HttpBinaryCacheStore, constructConfigNoTrailingSlash)
{
auto settings = getTestSettings();
HttpBinaryCacheStoreConfig config{settings, "https", "foo.bar.baz/a/b/", {}};
HttpBinaryCacheStoreConfig config{"https", "foo.bar.baz/a/b/", {}};
EXPECT_EQ(config.cacheUri.to_string(), "https://foo.bar.baz/a/b");
}
TEST(HttpBinaryCacheStore, constructConfigWithParams)
{
auto settings = getTestSettings();
StoreConfig::Params params{{"compression", "xz"}};
HttpBinaryCacheStoreConfig config{settings, "https", "foo.bar.baz/a/b/", params};
HttpBinaryCacheStoreConfig config{"https", "foo.bar.baz/a/b/", params};
EXPECT_EQ(config.cacheUri.to_string(), "https://foo.bar.baz/a/b");
EXPECT_EQ(config.getReference().params, params);
}
TEST(HttpBinaryCacheStore, constructConfigWithParamsAndUrlWithParams)
{
auto settings = getTestSettings();
StoreConfig::Params params{{"compression", "xz"}};
HttpBinaryCacheStoreConfig config{settings, "https", "foo.bar.baz/a/b?some-param=some-value", params};
HttpBinaryCacheStoreConfig config{"https", "foo.bar.baz/a/b?some-param=some-value", params};
EXPECT_EQ(config.cacheUri.to_string(), "https://foo.bar.baz/a/b?some-param=some-value");
EXPECT_EQ(config.getReference().params, params);
}

View File

@@ -1,16 +1,12 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/legacy-ssh-store.hh"
#include "nix/store/tests/test-main.hh"
namespace nix {
TEST(LegacySSHStore, constructConfig)
{
auto settings = getTestSettings();
LegacySSHStoreConfig config(
settings,
"ssh",
"me@localhost:2222",
StoreConfig::Params{

View File

@@ -1,14 +1,12 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/local-binary-cache-store.hh"
namespace nix {
TEST(LocalBinaryCacheStore, constructConfig)
{
Settings settings;
LocalBinaryCacheStoreConfig config{settings, "local", "/foo/bar/baz", {}};
LocalBinaryCacheStoreConfig config{"local", "/foo/bar/baz", {}};
EXPECT_EQ(config.binaryCacheDir, "/foo/bar/baz");
}

View File

@@ -1,16 +1,12 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/local-overlay-store.hh"
#include "nix/store/tests/test-main.hh"
namespace nix {
TEST(LocalOverlayStore, constructConfig_rootQueryParam)
{
auto settings = getTestSettings();
LocalOverlayStoreConfig config{
settings,
"local-overlay",
"",
{
@@ -26,8 +22,7 @@ TEST(LocalOverlayStore, constructConfig_rootQueryParam)
TEST(LocalOverlayStore, constructConfig_rootPath)
{
auto settings = getTestSettings();
LocalOverlayStoreConfig config{settings, "local-overlay", "/foo/bar", {}};
LocalOverlayStoreConfig config{"local-overlay", "/foo/bar", {}};
EXPECT_EQ(config.rootDir.get(), std::optional{"/foo/bar"});
}

View File

@@ -1,16 +1,18 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/local-store.hh"
#include "nix/store/tests/test-main.hh"
// Needed for template specialisations. This is not good! When we
// overhaul how store configs work, this should be fixed.
#include "nix/util/args.hh"
#include "nix/util/config-impl.hh"
#include "nix/util/abstract-setting-to-json.hh"
namespace nix {
TEST(LocalStore, constructConfig_rootQueryParam)
{
auto settings = getTestSettings();
LocalStoreConfig config{
settings,
"local",
"",
{
@@ -26,16 +28,14 @@ TEST(LocalStore, constructConfig_rootQueryParam)
TEST(LocalStore, constructConfig_rootPath)
{
auto settings = getTestSettings();
LocalStoreConfig config{settings, "local", "/foo/bar", {}};
LocalStoreConfig config{"local", "/foo/bar", {}};
EXPECT_EQ(config.rootDir.get(), std::optional{"/foo/bar"});
}
TEST(LocalStore, constructConfig_to_string)
{
auto settings = getTestSettings();
LocalStoreConfig config{settings, "local", "", {}};
LocalStoreConfig config{"local", "", {}};
EXPECT_EQ(config.getReference().to_string(), "local");
}

View File

@@ -1,13 +1,11 @@
#include "nix/store/nar-info-disk-cache.hh"
#include <gtest/gtest.h>
#include <rapidcheck/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/sqlite.hh"
#include <sqlite3.h>
#include "nix/store/globals.hh"
#include "nix/store/sqlite.hh"
#include "nix/store/nar-info-disk-cache.hh"
namespace nix {
TEST(NarInfoDiskCacheImpl, create_and_read)
@@ -26,8 +24,6 @@ TEST(NarInfoDiskCacheImpl, create_and_read)
SQLite db;
SQLiteStmt getIds;
Settings settings;
{
auto cache = NarInfoDiskCache::getTest(
settings.getNarInfoDiskCacheSettings(), {.useWAL = settings.useSQLiteWAL}, dbPath.string());

View File

@@ -299,7 +299,7 @@ public:
nix_api_store_test_base::SetUp();
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
store = open_local_store();
@@ -309,7 +309,7 @@ public:
buffer << t.rdbuf();
// Replace the hardcoded system with the current system
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", cStoreSettings.thisSystem.get());
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", nix::settings.thisSystem.get());
drv = nix_derivation_from_json(ctx, store, jsonStr.c_str());
assert_ctx_ok();
@@ -354,7 +354,7 @@ TEST_F(nix_api_store_test_base, build_from_json)
{
// FIXME get rid of these
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
auto * store = open_local_store();
@@ -365,7 +365,7 @@ TEST_F(nix_api_store_test_base, build_from_json)
buffer << t.rdbuf();
// Replace the hardcoded system with the current system
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", cStoreSettings.thisSystem.get());
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", nix::settings.thisSystem.get());
auto * drv = nix_derivation_from_json(ctx, store, jsonStr.c_str());
assert_ctx_ok();
@@ -401,7 +401,7 @@ TEST_F(nix_api_store_test_base, nix_store_realise_invalid_system)
{
// Test that nix_store_realise properly reports errors when the system is invalid
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
auto * store = open_local_store();
@@ -446,7 +446,7 @@ TEST_F(nix_api_store_test_base, nix_store_realise_builder_fails)
{
// Test that nix_store_realise properly reports errors when the builder fails
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
auto * store = open_local_store();
@@ -456,7 +456,7 @@ TEST_F(nix_api_store_test_base, nix_store_realise_builder_fails)
buffer << t.rdbuf();
// Replace with current system and make builder command fail
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", cStoreSettings.thisSystem.get());
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", nix::settings.thisSystem.get());
jsonStr = nix::replaceStrings(jsonStr, "echo $name foo > $out", "exit 1");
auto * drv = nix_derivation_from_json(ctx, store, jsonStr.c_str());
@@ -491,7 +491,7 @@ TEST_F(nix_api_store_test_base, nix_store_realise_builder_no_output)
{
// Test that nix_store_realise properly reports errors when builder succeeds but produces no output
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
auto * store = open_local_store();
@@ -501,7 +501,7 @@ TEST_F(nix_api_store_test_base, nix_store_realise_builder_no_output)
buffer << t.rdbuf();
// Replace with current system and make builder succeed but not produce output
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", cStoreSettings.thisSystem.get());
std::string jsonStr = nix::replaceStrings(buffer.str(), "x86_64-linux", nix::settings.thisSystem.get());
jsonStr = nix::replaceStrings(jsonStr, "echo $name foo > $out", "true");
auto * drv = nix_derivation_from_json(ctx, store, jsonStr.c_str());
@@ -687,7 +687,7 @@ TEST_F(NixApiStoreTestWithRealisedPath, nix_store_realise_output_ordering)
// This test uses a CA derivation with 10 outputs in randomized input order
// to verify that the callback order is deterministic and alphabetical.
nix::experimentalFeatureSettings.set("extra-experimental-features", "ca-derivations");
cStoreSettings.getWorkerSettings().substituters = {};
nix::settings.getWorkerSettings().substituters = {};
auto * store = open_local_store();
@@ -706,14 +706,14 @@ TEST_F(NixApiStoreTestWithRealisedPath, nix_store_realise_output_ordering)
std::string drvJson = R"({
"version": 4,
"name": "multi-output-test",
"system": ")" + cStoreSettings.thisSystem.get()
"system": ")" + nix::settings.thisSystem.get()
+ R"(",
"builder": "/bin/sh",
"args": ["-c", "echo a > $outa; echo b > $outb; echo c > $outc; echo d > $outd; echo e > $oute; echo f > $outf; echo g > $outg; echo h > $outh; echo i > $outi; echo j > $outj"],
"env": {
"builder": "/bin/sh",
"name": "multi-output-test",
"system": ")" + cStoreSettings.thisSystem.get()
"system": ")" + nix::settings.thisSystem.get()
+ R"(",
"outf": ")" + outf_ph
+ R"(",

View File

@@ -1,9 +1,7 @@
#include "nix/store/globals.hh"
#include "nix/store/s3-binary-cache-store.hh"
#include "nix/store/http-binary-cache-store.hh"
#include "nix/store/filetransfer.hh"
#include "nix/store/s3-url.hh"
#include "nix/store/tests/test-main.hh"
#include <gtest/gtest.h>
@@ -11,8 +9,7 @@ namespace nix {
TEST(S3BinaryCacheStore, constructConfig)
{
auto settings = getTestSettings();
S3BinaryCacheStoreConfig config{settings, "s3", "foobar", {}};
S3BinaryCacheStoreConfig config{"s3", "foobar", {}};
// The bucket name is stored as the host part of the authority in cacheUri
EXPECT_EQ(
@@ -25,9 +22,8 @@ TEST(S3BinaryCacheStore, constructConfig)
TEST(S3BinaryCacheStore, constructConfigWithRegion)
{
auto settings = getTestSettings();
Store::Config::Params params{{"region", "eu-west-1"}};
S3BinaryCacheStoreConfig config{settings, "s3", "my-bucket", params};
S3BinaryCacheStoreConfig config{"s3", "my-bucket", params};
EXPECT_EQ(
config.cacheUri,
@@ -41,8 +37,7 @@ TEST(S3BinaryCacheStore, constructConfigWithRegion)
TEST(S3BinaryCacheStore, defaultSettings)
{
auto settings = getTestSettings();
S3BinaryCacheStoreConfig config{settings, "s3", "test-bucket", {}};
S3BinaryCacheStoreConfig config{"s3", "test-bucket", {}};
EXPECT_EQ(
config.cacheUri,
@@ -63,13 +58,11 @@ TEST(S3BinaryCacheStore, defaultSettings)
*/
TEST(S3BinaryCacheStore, s3StoreConfigPreservesParameters)
{
auto settings = getTestSettings();
StringMap params;
params["region"] = "eu-west-1";
params["endpoint"] = "custom.s3.com";
S3BinaryCacheStoreConfig config(settings, "s3", "test-bucket", params);
S3BinaryCacheStoreConfig config("s3", "test-bucket", params);
// The config should preserve S3-specific parameters
EXPECT_EQ(
@@ -100,15 +93,13 @@ TEST(S3BinaryCacheStore, s3SchemeRegistration)
*/
TEST(S3BinaryCacheStore, parameterFiltering)
{
auto settings = getTestSettings();
StringMap params;
params["region"] = "eu-west-1";
params["endpoint"] = "minio.local";
params["want-mass-query"] = "true"; // Non-S3 store parameter
params["priority"] = "10"; // Non-S3 store parameter
S3BinaryCacheStoreConfig config(settings, "s3", "test-bucket", params);
S3BinaryCacheStoreConfig config("s3", "test-bucket", params);
// Only S3-specific params should be in cacheUri.query
EXPECT_EQ(
@@ -136,19 +127,16 @@ TEST(S3BinaryCacheStore, parameterFiltering)
*/
TEST(S3BinaryCacheStore, storageClassDefault)
{
auto settings = getTestSettings();
S3BinaryCacheStoreConfig config{settings, "s3", "test-bucket", {}};
S3BinaryCacheStoreConfig config{"s3", "test-bucket", {}};
EXPECT_EQ(config.storageClass.get(), std::nullopt);
}
TEST(S3BinaryCacheStore, storageClassConfiguration)
{
auto settings = getTestSettings();
StringMap params;
params["storage-class"] = "GLACIER";
S3BinaryCacheStoreConfig config(settings, "s3", "test-bucket", params);
S3BinaryCacheStoreConfig config("s3", "test-bucket", params);
EXPECT_EQ(config.storageClass.get(), std::optional<std::string>("GLACIER"));
}

View File

@@ -1,16 +1,14 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/ssh-store.hh"
#include "nix/store/tests/test-main.hh"
#include "nix/util/config-impl.hh"
#include "nix/util/abstract-setting-to-json.hh"
namespace nix {
TEST(SSHStore, constructConfig)
{
auto settings = getTestSettings();
SSHStoreConfig config{
settings,
"ssh-ng",
"me@localhost:2222",
StoreConfig::Params{
@@ -36,9 +34,7 @@ TEST(SSHStore, constructConfig)
TEST(MountedSSHStore, constructConfig)
{
auto settings = getTestSettings();
MountedSSHStoreConfig config{
settings,
"mounted-ssh",
"localhost",
StoreConfig::Params{

View File

@@ -1,38 +1,31 @@
#include <gtest/gtest.h>
#include "nix/store/globals.hh"
#include "nix/store/uds-remote-store.hh"
namespace nix {
TEST(UDSRemoteStore, constructConfig)
{
Settings settings;
UDSRemoteStoreConfig config{settings, "unix", "/tmp/socket", {}};
UDSRemoteStoreConfig config{"unix", "/tmp/socket", {}};
EXPECT_EQ(config.path, "/tmp/socket");
}
TEST(UDSRemoteStore, constructConfigWrongScheme)
{
Settings settings;
EXPECT_THROW(UDSRemoteStoreConfig(settings, "http", "/tmp/socket", {}), UsageError);
EXPECT_THROW(UDSRemoteStoreConfig("http", "/tmp/socket", {}), UsageError);
}
TEST(UDSRemoteStore, constructConfig_to_string)
{
Settings settings;
UDSRemoteStoreConfig config{settings, "unix", "", {}};
UDSRemoteStoreConfig config{"unix", "", {}};
EXPECT_EQ(config.getReference().to_string(), "daemon");
}
TEST(UDSRemoteStore, constructConfigWithParams)
{
Settings settings;
StoreConfig::Params params{{"max-connections", "1"}};
UDSRemoteStoreConfig config{settings, "unix", "/tmp/socket", params};
UDSRemoteStoreConfig config{"unix", "/tmp/socket", params};
auto storeReference = config.getReference();
EXPECT_EQ(storeReference.to_string(), "unix:///tmp/socket?max-connections=1");
EXPECT_EQ(storeReference.render(/*withParams=*/false), "unix:///tmp/socket");
@@ -41,11 +34,8 @@ TEST(UDSRemoteStore, constructConfigWithParams)
TEST(UDSRemoteStore, constructConfigWithParamsNoPath)
{
Settings settings;
StoreConfig::Params params{{"max-connections", "1"}};
UDSRemoteStoreConfig config{settings, "unix", "", params};
UDSRemoteStoreConfig config{"unix", "", params};
auto storeReference = config.getReference();
EXPECT_EQ(storeReference.to_string(), "daemon?max-connections=1");
EXPECT_EQ(storeReference.render(/*withParams=*/false), "daemon");

View File

@@ -21,14 +21,14 @@ protected:
ref<DummyStore> substituter;
WorkerSubstitutionTest()
: LibStoreTest([](auto & settings) {
auto config = make_ref<DummyStoreConfig>(settings, DummyStoreConfig::Params{});
: LibStoreTest([] {
auto config = make_ref<DummyStoreConfig>(DummyStoreConfig::Params{});
config->readOnly = false;
return config->openDummyStore();
})
}())
, dummyStore(store.dynamic_pointer_cast<DummyStore>())
, substituter([this] {
auto config = make_ref<DummyStoreConfig>(settings, DummyStoreConfig::Params{});
, substituter([] {
auto config = make_ref<DummyStoreConfig>(DummyStoreConfig::Params{});
config->readOnly = false;
config->isTrusted = true;
return config->openDummyStore();
@@ -67,10 +67,10 @@ TEST_F(WorkerSubstitutionTest, singleStoreObject)
HashAlgorithm::SHA256);
// Snapshot the substituter (has one store object)
checkpointJson("single/substituter", substituter, settings);
checkpointJson("single/substituter", substituter);
// Snapshot the destination store before (should be empty)
checkpointJson("../dummy-store/empty", dummyStore, settings);
checkpointJson("../dummy-store/empty", dummyStore);
// The path should not exist in the destination store yet
ASSERT_FALSE(dummyStore->isValidPath(pathInSubstituter));
@@ -92,7 +92,7 @@ TEST_F(WorkerSubstitutionTest, singleStoreObject)
worker.run(goals);
// Snapshot the destination store after (should match the substituter)
checkpointJson("single/substituter", dummyStore, settings);
checkpointJson("single/substituter", dummyStore);
// The path should now exist in the destination store
ASSERT_TRUE(dummyStore->isValidPath(pathInSubstituter));
@@ -138,10 +138,10 @@ TEST_F(WorkerSubstitutionTest, singleRootStoreObjectWithSingleDepStoreObject)
StorePathSet{dependencyPath});
// Snapshot the substituter (has two store objects)
checkpointJson("with-dep/substituter", substituter, settings);
checkpointJson("with-dep/substituter", substituter);
// Snapshot the destination store before (should be empty)
checkpointJson("../dummy-store/empty", dummyStore, settings);
checkpointJson("../dummy-store/empty", dummyStore);
// Neither path should exist in the destination store yet
ASSERT_FALSE(dummyStore->isValidPath(dependencyPath));
@@ -164,7 +164,7 @@ TEST_F(WorkerSubstitutionTest, singleRootStoreObjectWithSingleDepStoreObject)
worker.run(goals);
// Snapshot the destination store after (should match the substituter)
checkpointJson("with-dep/substituter", dummyStore, settings);
checkpointJson("with-dep/substituter", dummyStore);
// Both paths should now exist in the destination store
ASSERT_TRUE(dummyStore->isValidPath(dependencyPath));
@@ -196,7 +196,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutput)
auto drvPath = dummyStore->writeDerivation(drv);
// Snapshot the destination store before
checkpointJson("ca-drv/store-before", dummyStore, settings);
checkpointJson("ca-drv/store-before", dummyStore);
// Compute the hash modulo of the derivation
// For CA floating derivations, the kind is Deferred since outputs aren't known until build
@@ -233,7 +233,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutput)
});
// Snapshot the substituter
checkpointJson("ca-drv/substituter", substituter, settings);
checkpointJson("ca-drv/substituter", substituter);
// The realisation should not exist in the destination store yet
DrvOutput drvOutput{drvHash, "out"};
@@ -256,7 +256,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutput)
worker.run(goals);
// Snapshot the destination store after
checkpointJson("ca-drv/store-after", dummyStore, settings);
checkpointJson("ca-drv/store-after", dummyStore);
// The output path should now exist in the destination store
ASSERT_TRUE(dummyStore->isValidPath(outputPath));
@@ -351,7 +351,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutputWithDepDrv)
auto rootDrvPath = dummyStore->writeDerivation(rootDrv);
// Snapshot the destination store before
checkpointJson("issue-11928/store-before", dummyStore, settings);
checkpointJson("issue-11928/store-before", dummyStore);
// Compute the hash modulo for the root derivation
auto rootHashModulo = hashDerivationModulo(*dummyStore, rootDrv, true);
@@ -395,7 +395,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutputWithDepDrv)
// Snapshot the substituter
// Note: it has realisations for both drvs, but only the root's output store object
checkpointJson("issue-11928/substituter", substituter, settings);
checkpointJson("issue-11928/substituter", substituter);
// The realisations should not exist in the destination store yet
ASSERT_FALSE(dummyStore->queryRealisation(depDrvOutput));
@@ -418,7 +418,7 @@ TEST_F(WorkerSubstitutionTest, floatingDerivationOutputWithDepDrv)
worker.run(goals);
// Snapshot the destination store after
checkpointJson("issue-11928/store-after", dummyStore, settings);
checkpointJson("issue-11928/store-after", dummyStore);
// The root output path should now exist in the destination store
ASSERT_TRUE(dummyStore->isValidPath(rootOutputPath));

View File

@@ -12,22 +12,19 @@ namespace {
class WriteDerivationTest : public LibStoreTest
{
protected:
WriteDerivationTest(auto && makeConfig)
: LibStoreTest([&](auto & settings) {
this->config = makeConfig(settings);
return config->openDummyStore();
})
WriteDerivationTest(ref<DummyStoreConfig> config_)
: LibStoreTest(config_->openDummyStore())
, config(std::move(config_))
{
config->readOnly = false;
}
WriteDerivationTest()
: WriteDerivationTest(
[](auto & settings) { return make_ref<DummyStoreConfig>(settings, DummyStoreConfig::Params{}); })
: WriteDerivationTest(make_ref<DummyStoreConfig>(DummyStoreConfig::Params{}))
{
}
std::shared_ptr<DummyStoreConfig> config;
ref<DummyStoreConfig> config;
};
static Derivation makeSimpleDrv()

View File

@@ -15,6 +15,7 @@
#include "nix/store/local-store.hh" // TODO remove, along with remaining downcasts
#include "nix/store/globals.hh"
#include <algorithm>
#include <fstream>
#include <sys/types.h>
#include <fcntl.h>
@@ -184,6 +185,82 @@ struct LogFile
~LogFile();
};
struct LocalBuildRejection
{
bool maxJobsZero = false;
struct NoLocalStore
{};
/**
* We have a local store, but we don't have an external derivation builder (which is fine), if we did, it'd be
* fine because we would not care about platforms and features then. Since we don't, we either have the wrong
* platform, or we are missing some system features.
*/
struct WrongLocalStore
{
template<typename T>
struct Pair
{
T derivation;
T localStore;
};
std::optional<Pair<std::string>> badPlatform;
std::optional<Pair<StringSet>> missingFeatures;
};
std::variant<NoLocalStore, WrongLocalStore> rejection;
};
static BuildError reject(const LocalBuildRejection & rejection, std::string_view thingCannotBuild)
{
if (std::get_if<LocalBuildRejection::NoLocalStore>(&rejection.rejection))
return BuildError(
BuildResult::Failure::InputRejected,
"Unable to build with a primary store that isn't a local store; "
"either pass a different '--store' or enable remote builds.\n\n"
"For more information check 'man nix.conf' and search for '/machines'.");
auto & wrongStore = std::get<LocalBuildRejection::WrongLocalStore>(rejection.rejection);
std::string msg = fmt("Cannot build '%s'.", Magenta(thingCannotBuild));
if (rejection.maxJobsZero)
msg += "\nReason: " ANSI_RED "local builds are disabled" ANSI_NORMAL
" (max-jobs = 0)"
"\nHint: set 'max-jobs' to a non-zero value to enable local builds, "
"or configure remote builders via 'builders'";
if (wrongStore.badPlatform)
msg +=
fmt("\nReason: " ANSI_RED "platform mismatch" ANSI_NORMAL
"\nRequired system: '%s'"
"\nCurrent system: '%s'",
Magenta(wrongStore.badPlatform->derivation),
Magenta(wrongStore.badPlatform->localStore));
if (wrongStore.missingFeatures)
msg +=
fmt("\nReason: " ANSI_RED "missing system features" ANSI_NORMAL
"\nRequired features: {%s}"
"\nAvailable features: {%s}",
concatStringsSep(", ", wrongStore.missingFeatures->derivation),
concatStringsSep<StringSet>(", ", wrongStore.missingFeatures->localStore));
if (wrongStore.badPlatform || wrongStore.missingFeatures) {
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their
// hardware - we should tell them to run the command to install Rosetta
if (wrongStore.badPlatform && wrongStore.badPlatform->derivation == "x86_64-darwin"
&& wrongStore.badPlatform->localStore == "aarch64-darwin")
msg +=
fmt("\nNote: run `%s` to run programs for x86_64-darwin",
Magenta("/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
}
return BuildError(BuildResult::Failure::InputRejected, std::move(msg));
}
Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
{
auto drvOptions = [&] {
@@ -245,29 +322,57 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
}
checkPathValidity(initialOutputs);
/**
* Activity that denotes waiting for a lock.
*/
std::unique_ptr<Activity> actLock;
auto localBuildResult = [&]() -> std::variant<LocalBuildCapability, LocalBuildRejection> {
bool maxJobsZero = worker.settings.maxBuildJobs.get() == 0;
/**
* Locks on (fixed) output paths.
*/
PathLocks outputLocks;
auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store);
if (!localStoreP)
return LocalBuildRejection{.maxJobsZero = maxJobsZero, .rejection = LocalBuildRejection::NoLocalStore{}};
bool useHook;
/**
* Now that we've decided we can't / won't do a remote build, check
* that we can in fact build locally. First see if there is an
* external builder for a "semi-local build". If there is, prefer to
* use that. If there is not, then check if we can do a "true" local
* build.
*/
auto * ext = settings.getLocalSettings().findExternalDerivationBuilderIfSupported(*drv);
const ExternalBuilder * externalBuilder = nullptr;
if (ext)
return LocalBuildCapability{*localStoreP, ext};
while (true) {
using WrongLocalStore = LocalBuildRejection::WrongLocalStore;
WrongLocalStore wrongStore;
if (drv->platform != settings.thisSystem.get() && !settings.extraPlatforms.get().count(drv->platform)
&& !drv->isBuiltin())
wrongStore.badPlatform = WrongLocalStore::Pair<std::string>{drv->platform, settings.thisSystem.get()};
{
auto required = drvOptions.getRequiredSystemFeatures(*drv);
auto & available = worker.store.config.systemFeatures.get();
if (std::ranges::any_of(required, [&](const std::string & f) { return !available.count(f); }))
wrongStore.missingFeatures = WrongLocalStore::Pair<StringSet>{required, available};
}
if (maxJobsZero || wrongStore.badPlatform || wrongStore.missingFeatures)
return LocalBuildRejection{.maxJobsZero = maxJobsZero, .rejection = std::move(wrongStore)};
return LocalBuildCapability{*localStoreP, ext};
}();
auto acquireResources = [&](bool & done, PathLocks & outputLocks) -> Goal::Co {
trace("trying to build");
/* Obtain locks on all output paths, if the paths are known a priori.
The locks are automatically released when we exit this function or Nix
crashes. If we can't acquire the lock, then continue; hopefully some
other goal can start a build, and if not, the main loop will sleep a few
seconds and then retry this goal. */
/**
* Output paths to acquire locks on, if known a priori.
*
* The locks are automatically released when the caller's `PathLocks` goes
* out of scope, including on exception unwinding. If we can't acquire the lock, then
* continue; hopefully some other goal can start a build, and if not, the
* main loop will sleep a few seconds and then retry this goal.
*/
std::set<std::filesystem::path> lockFiles;
/* FIXME: Should lock something like the drv itself so we don't build same
CA drv concurrently */
@@ -311,7 +416,8 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
debug("skipping build of derivation '%s', someone beat us to it", worker.store.printStorePath(drvPath));
outputLocks.setDeletion(true);
outputLocks.unlock();
co_return doneSuccess(BuildResult::Success::AlreadyValid, std::move(validOutputs));
done = true;
co_return Return{};
}
/* If any of the outputs already exist but are not valid, delete
@@ -326,112 +432,133 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
}
}
bool canBuildLocally = [&] {
if (drv->platform != worker.store.config.settings.thisSystem.get()
&& !worker.store.config.settings.extraPlatforms.get().count(drv->platform) && !drv->isBuiltin())
return false;
co_return Return{};
};
if (worker.settings.maxBuildJobs.get() == 0 && !drv->isBuiltin())
return false;
auto tryHookLoop = [&](bool & valid) -> Goal::Co {
{
PathLocks outputLocks;
co_await acquireResources(valid, outputLocks);
if (valid)
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
for (auto & feature : drvOptions.getRequiredSystemFeatures(*drv))
if (!worker.store.config.systemFeatures.get().count(feature))
return false;
return true;
}();
/* Don't do a remote build if the derivation has the attribute
`preferLocalBuild' set. Also, check and repair modes are only
supported for local builds. */
bool buildLocally = (buildMode != bmNormal || (drvOptions.preferLocalBuild && canBuildLocally))
&& worker.settings.maxBuildJobs.get() != 0;
if (buildLocally) {
useHook = false;
} else {
switch (tryBuildHook(drvOptions)) {
case rpAccept:
/* Yes, it has started doing so. Wait until we get
EOF from the hook. */
useHook = true;
break;
valid = true;
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
case rpDecline:
// We should do it ourselves.
co_return Return{};
case rpPostpone:
/* Not now; wait until at least one child finishes or
the wake-up timeout expires. */
if (!actLock)
actLock = std::make_unique<Activity>(
*logger,
lvlWarn,
actBuildWaiting,
fmt("waiting for a machine to build '%s'", Magenta(worker.store.printStorePath(drvPath))));
outputLocks.unlock();
co_await waitForAWhile();
continue;
case rpDecline:
/* We should do it ourselves.
Now that we've decided we can't / won't do a remote build, check
that we can in fact build locally. First see if there is an
external builder for a "semi-local build". If there is, prefer to
use that. If there is not, then check if we can do a "true" local
build. */
externalBuilder = worker.store.config.settings.findExternalDerivationBuilderIfSupported(*drv);
if (!externalBuilder && !canBuildLocally) {
auto msg =
fmt("Cannot build '%s'.\n"
"Reason: " ANSI_RED "required system or feature not available" ANSI_NORMAL
"\n"
"Required system: '%s' with features {%s}\n"
"Current system: '%s' with features {%s}",
Magenta(worker.store.printStorePath(drvPath)),
Magenta(drv->platform),
concatStringsSep(", ", drvOptions.getRequiredSystemFeatures(*drv)),
Magenta(worker.store.config.settings.thisSystem),
concatStringsSep<StringSet>(", ", worker.store.Store::config.systemFeatures));
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their hardware -
// we should tell them to run the command to install Darwin 2
if (drv->platform == "x86_64-darwin" && worker.store.config.settings.thisSystem == "aarch64-darwin")
msg += fmt(
"\nNote: run `%s` to run programs for x86_64-darwin",
Magenta(
"/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
outputLocks.unlock();
co_return doneFailure({BuildResult::Failure::InputRejected, std::move(msg)});
}
useHook = false;
break;
}
}
break;
}
actLock.reset();
PathLocks outputLocks;
{
// First attempt was postponed. Retry in a loop with an activity
// that lives until accept or decline.
Activity act(
*logger,
lvlWarn,
actBuildWaiting,
fmt("waiting for a machine to build '%s'", Magenta(worker.store.printStorePath(drvPath))));
if (useHook) {
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
} else if (auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store)) {
co_return buildLocally(
*localStoreP,
std::move(inputPaths),
std::move(initialOutputs),
std::move(drvOptions),
std::move(outputLocks),
externalBuilder);
while (true) {
co_await waitForAWhile();
co_await acquireResources(valid, outputLocks);
if (valid)
break;
switch (tryBuildHook(drvOptions)) {
case rpAccept:
/* Yes, it has started doing so. Wait until we get
EOF from the hook. */
break;
case rpPostpone:
/* Not now; wait until at least one child finishes or
the wake-up timeout expires. */
outputLocks.unlock();
continue;
case rpDecline:
// We should do it ourselves.
co_return Return{};
}
break;
}
}
if (valid) {
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
} else {
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
}
};
auto tryBuildLocally = [&](bool & valid) -> Goal::Co {
if (auto * cap = std::get_if<LocalBuildCapability>(&localBuildResult)) {
PathLocks outputLocks;
co_await acquireResources(valid, outputLocks);
if (valid)
co_return doneSuccess(BuildResult::Success::AlreadyValid, checkPathValidity(initialOutputs).second);
valid = true;
co_return buildLocally(
*cap, std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
}
co_return Return{};
};
if (buildMode != bmNormal) {
// Check and repair modes operate on the state of this store specifically,
// so they must always build locally.
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
} else if (drvOptions.preferLocalBuild) {
// Local is preferred, so try it first. If it's not available, fall back to the hook.
{
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
}
{
bool valid = false;
co_await tryHookLoop(valid);
if (valid)
co_return Return{};
}
} else {
throw Error(
R"(
Unable to build with a primary store that isn't a local store;
either pass a different '--store' or enable remote builds.
For more information check 'man nix.conf' and search for '/machines'.
)");
// Default preference is a remote build: they tend to be faster and preserve local
// resources for other tasks. Fall back to local if no remote is available.
{
bool valid = false;
co_await tryHookLoop(valid);
if (valid)
co_return Return{};
}
{
bool valid = false;
co_await tryBuildLocally(valid);
if (valid)
co_return Return{};
}
}
std::string storePath = worker.store.printStorePath(drvPath);
auto * rejection = std::get_if<LocalBuildRejection>(&localBuildResult);
assert(rejection);
co_return doneFailure(reject(*rejection, storePath));
}
Goal::Co DerivationBuildingGoal::buildWithHook(
@@ -479,8 +606,7 @@ Goal::Co DerivationBuildingGoal::buildWithHook(
hook->toHook.writeSide.close();
/* Create the log file and pipe. */
std::unique_ptr<LogFile> logFile =
std::make_unique<LogFile>(worker.store, drvPath, worker.store.config.settings.getLogFileSettings());
std::unique_ptr<LogFile> logFile = std::make_unique<LogFile>(worker.store, drvPath, settings.getLogFileSettings());
std::set<MuxablePipePollState::CommChannel> fds;
fds.insert(hook->fromHook.readSide.get());
@@ -640,12 +766,11 @@ Goal::Co DerivationBuildingGoal::buildWithHook(
}
Goal::Co DerivationBuildingGoal::buildLocally(
LocalStore & localStore,
LocalBuildCapability localBuildCap,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks,
const ExternalBuilder * externalBuilder)
PathLocks outputLocks)
{
co_await yield();
@@ -656,7 +781,7 @@ Goal::Co DerivationBuildingGoal::buildLocally(
std::unique_ptr<LogFile> logFile;
auto openLogFile = [&]() {
logFile = std::make_unique<LogFile>(worker.store, drvPath, worker.store.config.settings.getLogFileSettings());
logFile = std::make_unique<LogFile>(worker.store, drvPath, settings.getLogFileSettings());
};
auto closeLogFile = [&]() { logFile.reset(); };
@@ -729,7 +854,7 @@ Goal::Co DerivationBuildingGoal::buildLocally(
};
decltype(DerivationBuilderParams::defaultPathsInChroot) defaultPathsInChroot =
localStore.config->getLocalSettings().sandboxPaths.get();
localBuildCap.localStore.config->getLocalSettings().sandboxPaths.get();
DesugaredEnv desugaredEnv;
/* Add the closure of store paths to the chroot. */
@@ -757,7 +882,6 @@ Goal::Co DerivationBuildingGoal::buildLocally(
}
DerivationBuilderParams params{
.settings = worker.store.config.settings,
.drvPath = drvPath,
.buildResult = buildResult,
.drv = *drv,
@@ -772,14 +896,14 @@ Goal::Co DerivationBuildingGoal::buildLocally(
/* If we have to wait and retry (see below), then `builder` will
already be created, so we don't need to create it again. */
builder = externalBuilder
builder = localBuildCap.externalBuilder
? makeExternalDerivationBuilder(
localStore,
localBuildCap.localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params),
*externalBuilder)
*localBuildCap.externalBuilder)
: makeDerivationBuilder(
localStore,
localBuildCap.localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params));
}

View File

@@ -22,9 +22,9 @@ Worker::Worker(Store & store, Store & evalStore)
, actSubstitutions(*logger, actCopyPaths)
, store(store)
, evalStore(evalStore)
, settings(store.config.settings.getWorkerSettings())
, getSubstituters{[&] {
return settings.useSubstitutes ? getDefaultSubstituters(store.config.settings) : std::list<ref<Store>>{};
, settings(nix::settings.getWorkerSettings())
, getSubstituters{[] {
return nix::settings.getWorkerSettings().useSubstitutes ? getDefaultSubstituters() : std::list<ref<Store>>{};
}}
{
nrLocalBuilds = 0;
@@ -367,9 +367,7 @@ void Worker::run(const Goals & _topGoals)
if (!children.empty() || !waitingForAWhile.empty())
waitForInput();
else if (awake.empty() && 0U == settings.maxBuildJobs) {
if (Machine::parseConfig(
{store.config.settings.thisSystem}, store.config.settings.getWorkerSettings().builders)
.empty())
if (Machine::parseConfig({nix::settings.thisSystem}, nix::settings.getWorkerSettings().builders).empty())
throw Error(
"Unable to start any build; either increase '--max-jobs' or enable remote builds.\n"
"\n"

View File

@@ -36,8 +36,8 @@ static void builtinUnpackChannel(const BuiltinBuilderContext & ctx)
auto target = out / channelName;
try {
std::filesystem::rename(fileName, target);
} catch (std::filesystem::filesystem_error &) {
throw SysError("failed to rename %1% to %2%", fileName, target.string());
} catch (std::filesystem::filesystem_error & e) {
throw SystemError(e.code(), "failed to rename %1% to %2%", fileName, target.string());
}
}

View File

@@ -5,15 +5,14 @@
namespace nix {
CommonSSHStoreConfig::CommonSSHStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params)
: CommonSSHStoreConfig(settings, scheme, ParsedURL::Authority::parse(authority), params)
CommonSSHStoreConfig::CommonSSHStoreConfig(std::string_view scheme, std::string_view authority, const Params & params)
: CommonSSHStoreConfig(scheme, ParsedURL::Authority::parse(authority), params)
{
}
CommonSSHStoreConfig::CommonSSHStoreConfig(
nix::Settings & settings, std::string_view scheme, const ParsedURL::Authority & authority, const Params & params)
: StoreConfig(settings, params)
std::string_view scheme, const ParsedURL::Authority & authority, const Params & params)
: StoreConfig(params)
, authority(authority)
{
}

View File

@@ -230,7 +230,7 @@ struct ClientSettings
bool useSubstitutes;
StringMap overrides;
void apply(Settings & settings, TrustedFlag trusted)
void apply(TrustedFlag trusted)
{
settings.keepFailed = keepFailed;
settings.getWorkerSettings().keepGoing = keepGoing;
@@ -783,7 +783,7 @@ static void performOp(
// FIXME: use some setting in recursive mode. Will need to use
// non-global variables.
if (!recursive)
clientSettings.apply(store->config.settings, trusted);
clientSettings.apply(trusted);
logger->stopWork();
break;

View File

@@ -136,6 +136,12 @@ StorePath Store::writeDerivation(const Derivation & drv, RepairFlag repair)
{
auto [suffix, contents, references, path] = infoForDerivation(*this, drv);
/* In case the derivation is already valid, we bail out early since that's
faster. But we need to make sure that the derivation has a corresponding
temproot. It is added by the remote in addToStoreFromDump, but we'd like
to avoid sending a lot of drv contents to the daemon. */
addTempRoot(path);
if (isValidPath(path) && !repair)
return path;

View File

@@ -408,10 +408,10 @@ void adl_serializer<DummyStore::PathInfoAndContents>::to_json(json & json, const
};
}
ref<DummyStoreConfig> adl_serializer<ref<DummyStore::Config>>::from_json(Settings & settings, const json & json)
ref<DummyStoreConfig> adl_serializer<ref<DummyStore::Config>>::from_json(const json & json)
{
auto & obj = getObject(json);
auto cfg = make_ref<DummyStore::Config>(settings, DummyStore::Config::Params{});
auto cfg = make_ref<DummyStore::Config>(DummyStore::Config::Params{});
const_cast<PathSetting &>(cfg->storeDir_).set(getString(valueAt(obj, "store")));
cfg->readOnly = true;
return cfg;
@@ -424,11 +424,10 @@ void adl_serializer<DummyStoreConfig>::to_json(json & json, const DummyStoreConf
};
}
ref<DummyStore> adl_serializer<ref<DummyStore>>::from_json(Settings & settings, const json & json)
ref<DummyStore> adl_serializer<ref<DummyStore>>::from_json(const json & json)
{
auto & obj = getObject(json);
ref<DummyStore> res =
adl_serializer<ref<DummyStoreConfig>>::from_json(settings, valueAt(obj, "config"))->openDummyStore();
ref<DummyStore> res = adl_serializer<ref<DummyStoreConfig>>::from_json(valueAt(obj, "config"))->openDummyStore();
for (auto & [k, v] : getObject(valueAt(obj, "contents")))
res->contents.insert({StorePath{k}, v});
for (auto & [k, v] : getObject(valueAt(obj, "derivations")))

View File

@@ -737,7 +737,7 @@ struct curlFileTransfer : public FileTransfer
download after a while. If we're writing to a
sink, we can only retry if the server supports
ranged requests. */
if (err == Transient && attempt < request.tries
if (err == Transient && attempt < fileTransfer.settings.tries
&& (!this->request.dataCallback || writtenToSink == 0 || (acceptRanges && encoding.empty()))) {
int ms = retryTimeMs
* std::pow(

View File

@@ -3,6 +3,7 @@
#include "nix/store/local-gc.hh"
#include "nix/store/local-store.hh"
#include "nix/store/path.hh"
#include "nix/util/configuration.hh"
#include "nix/util/finally.hh"
#include "nix/util/unix-domain-socket.hh"
#include "nix/util/signals.hh"
@@ -271,7 +272,7 @@ void LocalStore::findRoots(const Path & path, std::filesystem::file_type type, R
|| e.code() == std::errc::not_a_directory)
printInfo("cannot read potential root '%1%'", path);
else
throw;
throw SystemError(e.code(), "finding GC roots in '%1%'", path);
}
catch (SystemError & e) {
@@ -306,9 +307,33 @@ Roots LocalStore::findRoots(bool censor)
return roots;
}
static Roots requestRuntimeRoots(const LocalStoreConfig & config, const std::filesystem::path & socketPath)
{
Roots roots;
auto socket = connect(socketPath);
auto socketSource = FdSource(socket.get());
while (1) {
auto line = socketSource.readLine(true, '\0');
if (line == "")
break;
roots[config.parseStorePath(line)].insert(censored);
};
return roots;
}
void LocalStore::findRuntimeRoots(Roots & roots, bool censor)
{
auto unchecked = findRuntimeRootsUnchecked(*config);
Roots unchecked;
if (config->useRootsDaemon) {
experimentalFeatureSettings.require(Xp::LocalOverlayStore);
unchecked = requestRuntimeRoots(*config, config->getRootsSocketPath());
} else {
unchecked = findRuntimeRootsUnchecked(*config);
}
for (auto & [path, links] : unchecked) {
if (!isValidPath(path))

View File

@@ -79,6 +79,10 @@ LogFileSettings::LogFileSettings()
{
}
Settings settings;
static GlobalConfig::Register rSettings(&settings);
Settings::Settings()
: nixStateDir(canonPath(getEnvNonEmpty("NIX_STATE_DIR").value_or(NIX_STATE_DIR)))
, nixDaemonSocketFile(canonPath(getEnvOsNonEmpty(OS_STR("NIX_DAEMON_SOCKET_PATH"))
@@ -275,7 +279,7 @@ bool Settings::isWSL1()
#endif
}
const ExternalBuilder * Settings::findExternalDerivationBuilderIfSupported(const Derivation & drv)
const ExternalBuilder * LocalSettings::findExternalDerivationBuilderIfSupported(const Derivation & drv)
{
if (auto it = std::ranges::find_if(
externalBuilders.get(), [&](const auto & handler) { return handler.systems.contains(drv.platform); });
@@ -425,17 +429,17 @@ unsigned int MaxBuildJobsSetting::parse(const std::string & str) const
}
template<>
Settings::ExternalBuilders BaseSetting<Settings::ExternalBuilders>::parse(const std::string & str) const
LocalSettings::ExternalBuilders BaseSetting<LocalSettings::ExternalBuilders>::parse(const std::string & str) const
{
try {
return nlohmann::json::parse(str).template get<Settings::ExternalBuilders>();
return nlohmann::json::parse(str).template get<LocalSettings::ExternalBuilders>();
} catch (std::exception & e) {
throw UsageError("parsing setting '%s': %s", name, e.what());
}
}
template<>
std::string BaseSetting<Settings::ExternalBuilders>::to_string() const
std::string BaseSetting<LocalSettings::ExternalBuilders>::to_string() const
{
return nlohmann::json(value).dump();
}

View File

@@ -5,6 +5,7 @@
#include "nix/store/sqlite.hh"
#include "nix/util/callback.hh"
#include "nix/store/store-registration.hh"
#include "nix/store/globals.hh"
namespace nix {
@@ -20,9 +21,8 @@ StringSet HttpBinaryCacheStoreConfig::uriSchemes()
}
HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view _cacheUri, const Params & params)
std::string_view scheme, std::string_view _cacheUri, const Params & params)
: HttpBinaryCacheStoreConfig(
settings,
parseURL(
std::string{scheme} + "://"
+ (!_cacheUri.empty()
@@ -32,10 +32,9 @@ HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig(
{
}
HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig(
nix::Settings & settings, ParsedURL _cacheUri, const Params & params)
: StoreConfig(settings, params)
, BinaryCacheStoreConfig(settings, params)
HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig(ParsedURL _cacheUri, const Params & params)
: StoreConfig(params)
, BinaryCacheStoreConfig(params)
, cacheUri(std::move(_cacheUri))
{
while (!cacheUri.path.empty() && cacheUri.path.back() == "")
@@ -67,8 +66,7 @@ HttpBinaryCacheStore::HttpBinaryCacheStore(ref<Config> config, ref<FileTransfer>
, fileTransfer{fileTransfer}
, config{config}
{
diskCache = NarInfoDiskCache::get(
config->settings.getNarInfoDiskCacheSettings(), {.useWAL = config->settings.useSQLiteWAL});
diskCache = NarInfoDiskCache::get(settings.getNarInfoDiskCacheSettings(), {.useWAL = settings.useSQLiteWAL});
}
void HttpBinaryCacheStore::init()
@@ -106,7 +104,7 @@ std::optional<CompressionAlgo> HttpBinaryCacheStore::getCompressionMethod(const
void HttpBinaryCacheStore::maybeDisable()
{
auto state(_state.lock());
if (state->enabled && config->settings.getWorkerSettings().tryFallback) {
if (state->enabled && settings.getWorkerSettings().tryFallback) {
int t = 60;
printError("disabling binary cache '%s' for %s seconds", config->getHumanReadableURI(), t);
state->enabled = false;

View File

@@ -16,8 +16,6 @@
namespace nix {
class Settings;
/**
* Denotes a build failure that stemmed from the builder exiting with a
* failing exist status.
@@ -57,8 +55,6 @@ typedef std::map<std::filesystem::path, ChrootPath> PathsInChroot; // maps targe
*/
struct DerivationBuilderParams
{
Settings & settings;
/** The path of the derivation. */
const StorePath & drvPath;

View File

@@ -64,6 +64,12 @@ private:
std::string key() override;
struct LocalBuildCapability
{
LocalStore & localStore;
const ExternalBuilder * externalBuilder;
};
/**
* The states.
*/
@@ -75,12 +81,11 @@ private:
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks);
Co buildLocally(
LocalStore & localStore,
LocalBuildCapability localBuildCap,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks,
const ExternalBuilder * externalBuilder);
PathLocks outputLocks);
/**
* Is the build hook willing to perform the build?

View File

@@ -12,13 +12,8 @@ struct CommonSSHStoreConfig : virtual StoreConfig
{
using StoreConfig::StoreConfig;
CommonSSHStoreConfig(
nix::Settings & settings,
std::string_view scheme,
const ParsedURL::Authority & authority,
const Params & params);
CommonSSHStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params);
CommonSSHStoreConfig(std::string_view scheme, const ParsedURL::Authority & authority, const Params & params);
CommonSSHStoreConfig(std::string_view scheme, std::string_view authority, const Params & params);
Setting<Path> sshKey{
this, "", "ssh-key", "Path to the SSH private key used to authenticate to the remote machine."};

View File

@@ -12,16 +12,15 @@ struct DummyStore;
struct DummyStoreConfig : public std::enable_shared_from_this<DummyStoreConfig>, virtual StoreConfig
{
DummyStoreConfig(nix::Settings & settings, const Params & params)
: StoreConfig(settings, params)
DummyStoreConfig(const Params & params)
: StoreConfig(params)
{
// Disable caching since this a temporary in-memory store.
pathInfoCacheSize = 0;
}
DummyStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params)
: DummyStoreConfig(settings, params)
DummyStoreConfig(std::string_view scheme, std::string_view authority, const Params & params)
: DummyStoreConfig(params)
{
if (!authority.empty())
throw UsageError("`%s` store URIs must not contain an authority part %s", scheme, authority);
@@ -91,20 +90,11 @@ namespace nlohmann {
template<>
JSON_IMPL_INNER_TO(nix::DummyStoreConfig);
template<>
struct adl_serializer<nix::ref<nix::DummyStoreConfig>>
{
static nix::ref<nix::DummyStoreConfig> from_json(nix::Settings & settings, const json & json);
};
JSON_IMPL_INNER_FROM(nix::ref<nix::DummyStoreConfig>);
template<>
JSON_IMPL_INNER_TO(nix::DummyStore);
template<>
struct adl_serializer<nix::ref<nix::DummyStore>>
{
static nix::ref<nix::DummyStore> from_json(nix::Settings & settings, const json & json);
};
JSON_IMPL_INNER_FROM(nix::ref<nix::DummyStore>);
} // namespace nlohmann

View File

@@ -182,7 +182,6 @@ struct FileTransferRequest
Headers headers;
std::string expectedETag;
HttpMethod method = HttpMethod::Get;
size_t tries = fileTransferSettings.tries;
unsigned int baseRetryTimeMs = RETRY_TIME_MS_DEFAULT;
ActivityId parentAct;
bool decompress = true;

View File

@@ -104,8 +104,6 @@ public:
Settings();
using ExternalBuilders = std::vector<ExternalBuilder>;
/**
* Get the local store settings.
*/
@@ -410,18 +408,15 @@ public:
Set it to 1 to warn on all paths.
)"};
/**
* Finds the first external derivation builder that supports this
* derivation, or else returns a null pointer.
*/
const ExternalBuilder * findExternalDerivationBuilderIfSupported(const Derivation & drv);
/**
* Get the options needed for profile directory functions.
*/
ProfileDirsOptions getProfileDirsOptions() const;
};
// FIXME: don't use a global variable.
extern nix::Settings settings;
/**
* Load the configuration (from `nix.conf`, `NIX_CONFIG`, etc.) into the
* given configuration object.

View File

@@ -17,12 +17,9 @@ struct HttpBinaryCacheStoreConfig : std::enable_shared_from_this<HttpBinaryCache
using BinaryCacheStoreConfig::BinaryCacheStoreConfig;
HttpBinaryCacheStoreConfig(
nix::Settings & settings,
std::string_view scheme,
std::string_view cacheUri,
const Store::Config::Params & params);
std::string_view scheme, std::string_view cacheUri, const Store::Config::Params & params);
HttpBinaryCacheStoreConfig(nix::Settings & settings, ParsedURL cacheUri, const Store::Config::Params & params);
HttpBinaryCacheStoreConfig(ParsedURL cacheUri, const Store::Config::Params & params);
ParsedURL cacheUri;

View File

@@ -5,12 +5,6 @@
namespace nix {
class Settings;
PublicKeys getDefaultPublicKeys();
/**
* @todo use more narrow settings, or rethink whether this is a good
* idea at all, vs always associating keys with stores.
*/
PublicKeys getDefaultPublicKeys(const Settings & settings);
} // namespace nix
}

View File

@@ -14,8 +14,7 @@ struct LegacySSHStoreConfig : std::enable_shared_from_this<LegacySSHStoreConfig>
{
using CommonSSHStoreConfig::CommonSSHStoreConfig;
LegacySSHStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params);
LegacySSHStoreConfig(std::string_view scheme, std::string_view authority, const Params & params);
#ifndef _WIN32
// Hack for getting remote build log output.

View File

@@ -12,8 +12,7 @@ struct LocalBinaryCacheStoreConfig : std::enable_shared_from_this<LocalBinaryCac
* @param binaryCacheDir `file://` is a short-hand for `file:///`
* for now.
*/
LocalBinaryCacheStoreConfig(
nix::Settings & settings, std::string_view scheme, PathView binaryCacheDir, const Params & params);
LocalBinaryCacheStoreConfig(std::string_view scheme, PathView binaryCacheDir, const Params & params);
Path binaryCacheDir;

View File

@@ -30,7 +30,7 @@ public:
*
* @todo Make this less error-prone with new store settings system.
*/
LocalFSStoreConfig(nix::Settings & settings, PathView path, const Params & params);
LocalFSStoreConfig(PathView path, const Params & params);
OptionalPathSetting rootDir = makeRootDirSetting(*this, std::nullopt);
@@ -40,25 +40,25 @@ private:
* An indirection so that we don't need to refer to global settings
* in headers.
*/
static Path getDefaultStateDir(const nix::Settings & settings);
static Path getDefaultStateDir();
/**
* An indirection so that we don't need to refer to global settings
* in headers.
*/
static Path getDefaultLogDir(const nix::Settings & settings);
static Path getDefaultLogDir();
public:
PathSetting stateDir{
this,
rootDir.get() ? *rootDir.get() + "/nix/var/nix" : getDefaultStateDir(settings),
rootDir.get() ? *rootDir.get() + "/nix/var/nix" : getDefaultStateDir(),
"state",
"Directory where Nix stores state."};
PathSetting logDir{
this,
rootDir.get() ? *rootDir.get() + "/nix/var/log/nix" : getDefaultLogDir(settings),
rootDir.get() ? *rootDir.get() + "/nix/var/log/nix" : getDefaultLogDir(),
"log",
"directory where Nix stores log files."};

View File

@@ -7,15 +7,15 @@ namespace nix {
*/
struct LocalOverlayStoreConfig : virtual LocalStoreConfig
{
LocalOverlayStoreConfig(nix::Settings & settings, const StringMap & params)
: LocalOverlayStoreConfig(settings, "local-overlay", "", params)
LocalOverlayStoreConfig(const StringMap & params)
: LocalOverlayStoreConfig("local-overlay", "", params)
{
}
LocalOverlayStoreConfig(nix::Settings & settings, std::string_view scheme, PathView path, const Params & params)
: StoreConfig(settings, params)
, LocalFSStoreConfig(settings, path, params)
, LocalStoreConfig(settings, scheme, path, params)
LocalOverlayStoreConfig(std::string_view scheme, PathView path, const Params & params)
: StoreConfig(params)
, LocalFSStoreConfig(path, params)
, LocalStoreConfig(scheme, path, params)
{
}

View File

@@ -707,6 +707,12 @@ public:
// Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}
// Xp::ExternalBuilders
};
/**
* Finds the first external derivation builder that supports this
* derivation, or else returns a null pointer.
*/
const ExternalBuilder * findExternalDerivationBuilderIfSupported(const Derivation & drv);
};
} // namespace nix

View File

@@ -82,8 +82,7 @@ struct LocalStoreConfig : std::enable_shared_from_this<LocalStoreConfig>,
{
using LocalFSStoreConfig::LocalFSStoreConfig;
LocalStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params);
LocalStoreConfig(std::string_view scheme, std::string_view authority, const Params & params);
private:
@@ -134,6 +133,27 @@ public:
Xp::LocalOverlayStore,
};
Setting<bool> useRootsDaemon{
this,
false,
"use-roots-daemon",
R"(
Whether to request garbage collector roots from an external daemon.
When enabled, the garbage collector connects to a Unix domain socket
at [`<state-dir>`](@docroot@/store/types/local-store.md#store-option-state)`/gc-roots-socket/socket` to discover additional roots
that should not be collected. This is useful when the Nix daemon runs
without root privileges and cannot scan `/proc` for runtime roots.
The daemon can be started with [`nix store roots-daemon`](@docroot@/command-ref/new-cli/nix3-store-roots-daemon.md).
)",
{},
true,
Xp::LocalOverlayStore,
};
std::filesystem::path getRootsSocketPath() const;
static const std::string name()
{
return "Local Store";

View File

@@ -6,7 +6,6 @@
namespace nix {
class Settings;
class Store;
struct Machine;
@@ -68,7 +67,7 @@ struct Machine
* nix::openStore(completeStoreReference())
* ```
*/
ref<Store> openStore(Settings & settings) const;
ref<Store> openStore() const;
/**
* Parse a machine configuration.

View File

@@ -16,7 +16,6 @@
namespace nix {
class Settings;
class StorePath;
/**
@@ -217,7 +216,7 @@ struct ProfileDirsOptions
};
/**
* Create and return the path to a directory suitable for storing the user's
* Create and return the path to a directory suitable for storing the users
* profiles.
*/
std::filesystem::path profilesDir(ProfileDirsOptions opts);

View File

@@ -11,8 +11,7 @@ struct S3BinaryCacheStoreConfig : HttpBinaryCacheStoreConfig
{
using HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig;
S3BinaryCacheStoreConfig(
nix::Settings & settings, std::string_view uriScheme, std::string_view bucketName, const Params & params);
S3BinaryCacheStoreConfig(std::string_view uriScheme, std::string_view bucketName, const Params & params);
Setting<std::string> profile{
this,

View File

@@ -15,8 +15,7 @@ struct SSHStoreConfig : std::enable_shared_from_this<SSHStoreConfig>,
using CommonSSHStoreConfig::CommonSSHStoreConfig;
using RemoteStoreConfig::RemoteStoreConfig;
SSHStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params);
SSHStoreConfig(std::string_view scheme, std::string_view authority, const Params & params);
Setting<Strings> remoteProgram{
this, {"nix-daemon"}, "remote-program", "Path to the `nix-daemon` executable on the remote machine."};
@@ -40,8 +39,8 @@ struct SSHStoreConfig : std::enable_shared_from_this<SSHStoreConfig>,
struct MountedSSHStoreConfig : virtual SSHStoreConfig, virtual LocalFSStoreConfig
{
MountedSSHStoreConfig(nix::Settings & settings, StringMap params);
MountedSSHStoreConfig(nix::Settings & settings, std::string_view scheme, std::string_view host, StringMap params);
MountedSSHStoreConfig(StringMap params);
MountedSSHStoreConfig(std::string_view scheme, std::string_view host, StringMap params);
static const std::string name()
{

View File

@@ -44,8 +44,6 @@ struct NarInfoDiskCache;
struct NarInfoDiskCacheSettings;
class Store;
class Settings;
typedef std::map<std::string, StorePath> OutputPathMap;
enum CheckSigsFlag : bool { NoCheckSigs = false, CheckSigs = true };
@@ -82,32 +80,19 @@ struct StoreConfigBase : Config
{
using Config::Config;
/**
* Global settings, which are we trying to triage and give better
* homes. Putting a reference to them on stores will help a bit with
* this, though we may find that some of the things here should
* *not* be per-store.
*/
nix::Settings & settings;
StoreConfigBase(nix::Settings & settings)
: settings{settings}
{
}
private:
/**
* Compute the default Nix store directory from environment variables
* (`NIX_STORE_DIR`, `NIX_STORE`) or the compile-time default.
*/
static Path getDefaultNixStoreDir(const nix::Settings & settings);
static Path getDefaultNixStoreDir();
public:
const PathSetting storeDir_{
this,
getDefaultNixStoreDir(settings),
getDefaultNixStoreDir(),
"store",
R"(
Logical location of the Nix store, usually
@@ -152,13 +137,13 @@ struct StoreConfig : public StoreConfigBase, public StoreDirConfig
{
using Params = StoreReference::Params;
StoreConfig(nix::Settings & settings, const Params & params);
StoreConfig(const Params & params);
StoreConfig() = delete;
virtual ~StoreConfig() {}
static StringSet getDefaultSystemFeatures(const nix::Settings & settings);
static StringSet getDefaultSystemFeatures();
/**
* Documentation for this type of store.
@@ -223,7 +208,7 @@ struct StoreConfig : public StoreConfigBase, public StoreDirConfig
Setting<StringSet> systemFeatures{
this,
getDefaultSystemFeatures(settings),
getDefaultSystemFeatures(),
"system-features",
R"(
Optional [system features](@docroot@/command-ref/conf-file.md#conf-system-features) available on the system this store uses to build derivations.

View File

@@ -11,7 +11,6 @@
namespace nix {
class Settings;
struct SourcePath;
MakeError(BadStorePath, Error);
@@ -90,7 +89,6 @@ struct StoreDirConfig
* path for the given file system object.
*/
std::pair<StorePath, Hash> computeStorePath(
const Settings & settings,
std::string_view name,
const SourcePath & path,
ContentAddressMethod method = ContentAddressMethod::Raw::NixArchive,

View File

@@ -15,37 +15,32 @@
namespace nix {
class Settings;
/**
* @return The store config denoted by `storeURI` (slight misnomer...).
*/
ref<StoreConfig> resolveStoreConfig(Settings & settings, StoreReference && storeURI);
ref<StoreConfig> resolveStoreConfig(StoreReference && storeURI);
/**
* @return a Store object to access the Nix store denoted by
* 'uri' (slight misnomer...).
* uri (slight misnomer...).
*/
ref<Store> openStore(Settings & settings, StoreReference && storeURI);
ref<Store> openStore(StoreReference && storeURI);
/**
* Opens the store at `uri`, where `uri` is in the format expected by
* `StoreReference::parse`
*/
ref<Store> openStore(
Settings & settings,
const std::string & uri,
const StoreReference::Params & extraParams = StoreReference::Params());
ref<Store> openStore(const std::string & uri, const StoreReference::Params & extraParams = StoreReference::Params());
/**
* Short-hand which opens the default store, according to global settings
*/
ref<Store> openStore(Settings & settings);
ref<Store> openStore();
/**
* @return the default substituter stores, defined by the
* 'substituters' option and various legacy options.
* substituters option and various legacy options.
*/
std::list<ref<Store>> getDefaultSubstituters(Settings & settings);
std::list<ref<Store>> getDefaultSubstituters();
} // namespace nix

View File

@@ -14,7 +14,6 @@
*/
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
namespace nix {
@@ -41,10 +40,7 @@ struct StoreFactory
* whatever comes after `<scheme>://` and before `?<query-params>`.
*/
std::function<ref<StoreConfig>(
Settings & settings,
std::string_view scheme,
std::string_view authorityPath,
const Store::Config::Params & params)>
std::string_view scheme, std::string_view authorityPath, const Store::Config::Params & params)>
parseConfig;
/**
@@ -68,13 +64,10 @@ struct Implementations
.doc = TConfig::doc(),
.uriSchemes = TConfig::uriSchemes(),
.experimentalFeature = TConfig::experimentalFeature(),
.parseConfig = ([](Settings & settings, auto scheme, auto uri, auto & params) -> ref<StoreConfig> {
return make_ref<TConfig>(settings, scheme, uri, params);
}),
.getConfig = ([]() -> ref<StoreConfig> {
Settings settings;
return make_ref<TConfig>(settings, Store::Config::Params{});
.parseConfig = ([](auto scheme, auto uri, auto & params) -> ref<StoreConfig> {
return make_ref<TConfig>(scheme, uri, params);
}),
.getConfig = ([]() -> ref<StoreConfig> { return make_ref<TConfig>(Store::Config::Params{}); }),
};
auto [it, didInsert] = registered().insert({TConfig::name(), std::move(factory)});
if (!didInsert) {

View File

@@ -19,10 +19,9 @@ struct UDSRemoteStoreConfig : std::enable_shared_from_this<UDSRemoteStoreConfig>
/**
* @param authority is the socket path.
*/
UDSRemoteStoreConfig(
nix::Settings & settings, std::string_view scheme, std::string_view authority, const Params & params);
UDSRemoteStoreConfig(std::string_view scheme, std::string_view authority, const Params & params);
UDSRemoteStoreConfig(nix::Settings & settings, const Params & params);
UDSRemoteStoreConfig(const Params & params);
static const std::string name()
{

Some files were not shown because too many files have changed in this diff Show More