Compare commits

...
Author SHA1 Message Date
John Ericson 97d29276b9 WIP 2026-01-28 18:06:36 -05:00
John Ericson 2545b6716a EvalState::coerceToSingleDerivedPath also alow taking "packages"
This relaxes `builtins.outputOf` and also allows it to be used for
`builtins.derivationOf`
2026-01-28 17:47:13 -05:00
John Ericson 711e6b3476 Merge pull request #15111 from obsidiansystems/fstat-windows
Refactor fstat into a wrapper in libutil
2026-01-28 18:45:15 +00:00
Artemis Tosini 0fc20e3e20 Refactor fstat into a wrapper in libutil
We use a different fstat on posix and windows systems,
and not all fstat users were using the correct one.
Factor out fstat to make the change easier.

See also a13de50df3 for other stat
functions
2026-01-28 12:53:54 -05:00
John Ericson d5e4b0b4b8 Merge pull request #15104 from obsidiansystems/refactor-buildlocally
libstore: decide how to build in one spot
2026-01-28 00:19:22 +00:00
Amaan Qureshi 00d0e6dff3 libstore: decide how to build in one spot
This cleans up the logic for checking if the worker's store is a valid
local store when we're not hooking it. If we have a local store, we then
pass that as an argument to `DerivationBuildingGoal::buildLocally`,
rather than checking inside the function itself.
2026-01-27 18:35:05 -05:00
John Ericson f326f02813 Merge pull request #15099 from obsidiansystems/split-command-specific-settings
libstore: move command-specific settings to their own files
2026-01-27 19:01:32 +00:00
John Ericson 9e9b6d44f8 Merge pull request #15094 from amaanq/git-signing-isolation
tests/functional: isolate git tests from host signing config
2026-01-27 18:11:25 +00:00
Amaan Qureshi 11f6f07598 libstore: move command-specific settings to their own files
The two settings `envKeepDerivations` and `upgradeNixStorePathUrl` were
only used in one command each, so it makes more sense to move them to
their own files. This commit moves them both into a small self-contained
settings struct and registers them with the global config.
2026-01-27 12:36:31 -05:00
Amaan Qureshi ac9682c52f tests/functional: isolate git tests from host signing config
Currently, tests fail when the host system has `commit.gpgsign` or
`tag.gpgsign` enabled at the system level (in my case, a custom path
located at `/etc/git/config`), since the signing key is unavailable in
the test sandbox.

The tests set `HOME=$TEST_HOME` to isolate themselves, which bypasses
the user-level git config (`~/.gitconfig`). However, if a user sets the
system-level config via `GIT_CONFIG_GLOBAL` or `GIT_CONFIG_SYSTEM`, it
still applies, causing commits to fail when signing is enabled there.

In this PR, I explicitly set `GIT_CONFIG_GLOBAL` and `GIT_CONFIG_SYSTEM`
to `/dev/null` so that the user's system config is never read from or
written to. I've also replaced `git config --global protocol.file.allow
always` with `GIT_CONFIG_*` environment variables to avoid writing to
`/dev/null`.
2026-01-27 12:32:41 -05:00
John Ericson c7f1036bcb Merge pull request #15098 from amaanq/fix-nix-shell-test
tests/functional: fix nix-shell fixed-output derivation test
2026-01-27 17:26:11 +00:00
Amaan Qureshi d1348a2477 tests/functional: fix nix-shell fixed-output derivation test
The test was checking for `$stdenv` but the `fixed` derivation doesn't
actually have stdenv, it just has `FOO`. I've updated it to check the
value of `FOO` instead.
2026-01-27 10:35:06 -05:00
John Ericson e5536c8935 Merge pull request #15091 from obsidiansystems/split-diff-hook-settings
globals: refactor `diffHook` settings
2026-01-27 00:24:17 +00:00
John Ericson 929022c8f8 Merge pull request #15092 from NixOS/improve-error-messages
libexpr/parser: Use readable tokens in error messages instead of inte…
2026-01-27 00:01:21 +00:00
John Ericson bad1a005ed Merge pull request #15079 from NixOS/auto-cleanup-cleanups
Clean up `AutoRemoveJail`, `AutoDelete`, and `AutoUnmount`
2026-01-26 23:46:59 +00:00
Amaan Qureshi 692102f0dc globals: refactor diffHook settings
The settings related to diff hook (`run-diff-hook` and `diff-hook`) are
a little redundant and don't need to be leaked in derivation-builder
when computing the diff hook path to execute.

Instead of directly using both `runDiffHook` and `diffHook` settings in
derivation-builder, we can just encapsulate the logic to determine
whether or not we have a diff hook executable to run in a helper
function. We also mark `handleDiffHook` as static.
2026-01-26 18:37:13 -05:00
Sergei Zimmerman 68cf0a7f8a libexpr/parser: Use readable tokens in error messages instead of internal token names
Very low-hanging fruit for improving parser error messages.
2026-01-27 02:11:40 +03:00
John EricsonandSergei Zimmerman 5dfd81cbc0 Clean up AutoRemoveJail, AutoDelete, and AutoUnmount
- Extract destructor logic into named methods (`deletePath()`,
  `unmount()`, and `remove()`) that can be called explicitly. These ones
  will throw exceptions normally, unlike the destructor which must quash
  them to avoid double-exceptions.

- Use `std::filesystem::path` in `AutoUnmount` (changed from `Path`)

- Remove `del` field from `AutoRemoveJail`, using `INVALID_JAIL`
  sentinel value instead.

- Add move assignment operators implemented via friend `swap` functions
  for all three RAII classes.

- Remove old `reset(...)` methods that took parameters. These were a bit
  misleading --- do they cancel or immediately destroy? --- and doing it
  explicitly with cancel and then assignment is not hard.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2026-01-26 17:26:59 -05:00
John Ericson 395eef30f1 Merge pull request #14688 from NixOS/nixpkgs-25.11
flake: Bump nixpkgs to 25.11
2026-01-26 22:21:25 +00:00
John Ericson c7098ec8da Merge pull request #14329 from Mic92/nix-store-print-env
nix-store --print-env: fix shell quoting on _args output
2026-01-26 20:55:40 +00:00
John Ericson e8d1cb0668 Merge pull request #15086 from JustAGuyTryingHisBest/darwin-pathfmt
libutil: unix filesystem add more PathFmt
2026-01-26 20:49:51 +00:00
John Ericson a7c043b95d Merge pull request #14749 from obsidiansystems/build-status-serializer
Create proper serializer for BuildResult status
2026-01-26 20:37:57 +00:00
Jörg Thalheim ffe97db4f9 nix-store --print-env: fix shell quoting on _args output
The previous implementation double-quoted the _args variable by escaping
each argument individually and then wrapping them all in single quotes,
producing output like: _args=''-e' 'arg1' 'arg2''

This fix concatenates all arguments into a single string first, then
escapes that string once, producing correct output like:
_args='-e arg1 arg2'

This prevents potential command injection issues when the output is
sourced in shell scripts.

Fixes #14327
2026-01-26 15:08:08 -05:00
John Ericson 252e4ee5ca Merge pull request #15077 from NixOS/stat-wrapper
Cleanup stat usage
2026-01-26 19:55:00 +00:00
John Ericson 623360d07f Merge pull request #15083 from lisanna-dettwyler/fix-build-hook-setting
Fix build-hook setting being clobbered
2026-01-26 19:50:38 +00:00
Some Guy 7f95112fac libutil: unix filesystem add more PathFmt 2026-01-26 11:39:34 -08:00
John Ericson d3116dc764 Merge pull request #15043 from obsidiansystems/settings-split-0
Split out `AutoAllocateUidSettings`
2026-01-26 19:25:51 +00:00
John Ericson b190548c83 Merge pull request #15040 from NixOS/factor-out-nar-cache-0
Factor out `NarCache` from `RemoteFSAccessor`
2026-01-26 19:19:39 +00:00
John Ericson a13de50df3 Cleanup stat usage
Use wrappers to make error handling easier.

On Windows we are using proper 64-bit time and size info.

We still have the problem of no `lstat` on Windows, but this will be
dealt with in future PRs.
2026-01-26 14:00:19 -05:00
John EricsonandAmaan Qureshi ab56ac49e3 libstore: split out AutoAllocateUidSettings
Follows the same pattern as `GCSettings`: extract UID allocation
settings
into a dedicated struct that Settings inherits privately from.

The current settings infrastructure prevents correct data modeling that
would allow `autoAllocateUids` to be a
`std::optional<AutoAllocateUidSettings>`.
To compensate, the getter `getAutoAllocateUidSettings()` returns a
pointer -
nullptr when disabled, providing the optional-like semantics we want.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-01-26 13:43:41 -05:00
Lisanna Dettwyler 0e3a620374 Fix build-hook setting being clobbered
settings.buildHook.setDefault was running after nix.conf was parsed,
causing whatever value settings.buildHook had to be clobbered.
Re-arrange the logic so that the default is set before nix.conf is
parsed, so that custom build hooks can be used by specifying them in
nix.conf.

Signed-off-by: Lisanna Dettwyler <lisanna.dettwyler@gmail.com>
2026-01-26 12:13:20 -05:00
Sergei Zimmerman 00f67ee5d5 tests/functional: Require newer daemon version for empty error message bugfix 2026-01-25 23:56:44 +03:00
Sergei Zimmerman d69001600b tests/nixos/functional/unpriviledged-daemon: Use nixStoreMountOpts instead of readOnlyNixStore
This option is not available in 25.11:

> Please use the `boot.nixStoreMountOpts' option to define mount options for the Nix store, including 'ro'
2026-01-25 22:24:34 +03:00
Taeer Bar-Yam c1ab73f921 tests: Update version requirements on tests 2026-01-25 22:19:29 +03:00
Taeer Bar-Yam 3cb27988fb update error message of new daemon 2026-01-25 22:19:22 +03:00
Sergei Zimmerman f43566f4d7 packaging/components: Drop hardeningDisable
This is no longer necessary and produces an eval warning:

> evaluation warning: The 'pie' hardening flag has been removed in favor of enabling PIE by default in compilers and should no longer be used.

This was first introduced in 2200f315da, but
is no longer necessary since the switch to meson.
2026-01-25 22:12:32 +03:00
Taeer Bar-Yam fb6274b312 fix nix-serve with hacky workaround 2026-01-25 22:09:13 +03:00
Taeer Bar-Yam e72a8bebb8 update .gitignore
new version of meson creates some state file
2026-01-25 22:08:40 +03:00
Taeer Bar-Yam dad793fcfd fix perl dependencies error 2026-01-25 22:08:33 +03:00
Taeer Bar-Yam 7985873f73 inputDerivation is fixed upstream
fixed in nixpkgs PR #469652
2026-01-25 22:08:21 +03:00
Taeer Bar-Yam db576d599c fix infinite recursion 2026-01-25 22:07:18 +03:00
Taeer Bar-Yam d5544919e4 tests: minio: mc config host add -> mc alias set
`mc config host add` has been removed
SEE: https://github.com/minio/mc/issues/5206
2026-01-25 22:07:11 +03:00
Taeer Bar-Yam 8928cb4fb8 separateDebugInfo implies __structuredAttrs 2026-01-25 22:07:00 +03:00
Sergei Zimmerman 0dd38bc8b6 packaging/dependencies: Override fixes
- nghttp3 is not supported on mingw
- onetbb doesn't build on mingw
- lowdown override is no longer needed, same for toml11
2026-01-25 22:05:53 +03:00
Sergei Zimmerman d45004f5ec treewide: Apply formatter diffs
Also disable some churny formatters on some specific files.
2026-01-25 22:03:16 +03:00
Sergei Zimmerman 50050b5ef1 flake: Bump nixpkgs to 25.11 2026-01-25 21:59:36 +03:00
Sergei Zimmerman d0c194efc1 maintainers/flake-module: Pin clang-format to 21
We don't want too much unnecessary formatting churn.
2026-01-25 21:57:18 +03:00
Sergei Zimmerman ed9d8af93d Merge pull request #15059 from lovesegfault/fix-aws-logs
feat(libstore/aws-creds): route AWS CRT logs through Nix logger
2026-01-25 18:06:03 +00:00
Sergei Zimmerman e3b788b4ca tests/nixos/s3-binary-cache-store: Drop superfluous prints
As requested in review.
2026-01-25 19:40:30 +03:00
Bernardo Meurer Costa 3b8b764e29 feat(libstore/aws-creds): route AWS CRT logs through Nix logger
Previously AWS CRT logs went directly to stderr via ApiHandle::InitializeLogging,
causing log spam that didn't respect Nix's verbosity settings.

This implements a custom aws_logger using the aws-c-common C API that:
- Routes all AWS logs through nix::logger
- Maps AWS log levels conservatively (ERROR/WARN -> lvlInfo) since the SDK
  treats expected conditions like missing IMDS as errors
- Prefixes messages with (aws) for clarity
- Respects Nix's verbosity flags (-v, -vv, etc.)
2026-01-25 19:40:29 +03:00
Jörg Thalheim 2eb19a6353 Merge pull request #13030 from vlaci/mtls-auth
libstore/filetransfer: add support for MTLS authentication
2026-01-25 13:58:12 +00:00
John Ericson e8e3c30dfc Merge pull request #15076 from NixOS/prepare-for-25.11
Prepare for nixpkgs 25.11, enable S3 support in static builds
2026-01-24 23:11:18 +00:00
John Ericson a3f2d2b3e9 Merge pull request #15075 from NixOS/chmod-wrapper
Share the exception-using `chmod` wrapper with more code
2026-01-24 22:51:42 +00:00
Sergei Zimmerman 64458acde2 packaging: Fix static builds with S3 support, enable by default
aws-crt-cpp doesn't provide pkg-config files and has a bunch of transitive
deps, so switch to cmake for resolving the dependency.
2026-01-25 01:26:23 +03:00
John Ericson 6e2e53a8d2 Share the exception-using chmod wrapper with more code
It is not just useful to `DerivationBuilder`.
2026-01-24 17:03:48 -05:00
Sergei Zimmerman dcaaf2c65f dev-shell: Use stdenv.hostPlatform instead of hostPlatform
This is now a warning in 25.11:

> evaluation warning: 'hostPlatform' has been renamed to/replaced by 'stdenv.hostPlatform'
2026-01-25 00:50:56 +03:00
Sergei Zimmerman c4c0aee4f1 tests/nixos: Drop otherNixes.nix_2_3, replace with 2_18
Since [1] there's no way to run 2.3 anymore and overrides wouldn't be very
helpful. Let's instead use 2.18, which is the baseline for nixpkgs.
2026-01-25 00:50:55 +03:00
Sergei Zimmerman 0f22d60c7e tests/nixos: Specify -f argument to mount
Otherwise we barf on btrfs:

machine # [   17.027621] EXT4-fs error (device vdb): ext4_lookup:1819: inode #2476: comm nix: iget: checksum invalid
machine # error: getting status of '/mnt/nix/store/j8645yndikbrvn292zgvyv64xrrmwdcb-bash-5.3p3': Bad message
machine # checking '/nix/store/m3954qff15v7z1l6lpyqf8v2h47c7hv2-mailcap-2.1.54'...
machine # checking '/nix/store/xh1ff9c9c0yv1wxrwa5gnfp092yagh7v-tzdata-2025b'...
machine # [   17.172793] EXT4-fs error (device vdb): ext4_lookup:1819: inode #1777: comm nix: iget: checksum invalid
machine # error: getting status of '/mnt/nix/store/xh1ff9c9c0yv1wxrwa5gnfp092yagh7v-tzdata-2025b/share/zoneinfo/leap-seconds.list': Bad message
2026-01-25 00:50:54 +03:00
John Ericson 943c18f9fe Merge pull request #15072 from NixOS/fix-interrupted-linux-derivation-builder
Fix destruction of DerivationBuilder implementations
2026-01-24 21:16:34 +00:00
Sergei Zimmerman b752c5cb64 Fix destruction of DerivationBuilder implementations
This unsures that we call the correct virtual functions when destroying a particular
DerivationBuilder.

Usually the order of destructors is in the reverse order of inheritance:

ChrootLinuxDerivationBuilder -> ChrootDerivationBuilder -> DerivationBuilderImpl

autoDelChroot was being destroyed before the DerivationBuilderImpl::killChild was
run and it would fail to clean up the chroot directory, since there were still processes
writing to it. Note that ChrootLinuxDerivationBuilder::killSandbox was never run in
the interrupted case at all, since virtual functions in destructors do not call derived class
methods.

I could reproduce the issue with the following derivation:

let
  pkgs = import <nixpkgs> { };
in
pkgs.runCommand "chroot-cleanup-race" { } ''
  mkdir -p $out

  for i in $(seq 1 200); do
    (
      mkfifo $out/fifo$i
      cat $out/fifo$i > /dev/null &

      while true; do
        : > $out/file$i
      done
    ) &
  done

  sleep 0.05
  echo done > $out/main
''

While interrupting it manually when it would hang.

Wrapping the unique pointer in a custom deleter function we can run all
of the necessary clean up code consistently and calling the right virtual
functions. Ideally we'd have a lint that bans the usage of virtual functions
in destructors completely.
2026-01-24 23:31:11 +03:00
John Ericson b7d07e42dc Merge pull request #15071 from roberth/fix-concurrent-failure-bug
tests: fix sandbox-paths in cancelled-builds test
2026-01-24 19:48:25 +00:00
Sergei Zimmerman 0f17a1f655 libutil-tests: Add unit tests for https binary cache stores with mTLS
This addresses the concerns with network isolation that have been raised
previously [1] by only running the tests by default in a network namespace.
This way all networks tests are independent of each other and do not bind
to ports in the host namespace.

This is much neater than doing these sorts of tests in functional suite.

[1]: https://github.com/NixOS/nix/pull/14266#issuecomment-3411261285
2026-01-24 21:59:59 +03:00
Damien Diederen 36b0bebe25 http-binary-cache-store: Add 'tls-certificate' and 'tls-private-key' settings
Those are set via the store's URI, e.g.:

    https://substituter.invalid?tls-certificate=/path/to/cert.pem&tls-private-key=/path/to/key.pem
2026-01-24 21:59:58 +03:00
Robert Hensing 7b4444f174 tests: fix sandbox-paths in cancelled-builds test
Don't add the whole store to sandbox-paths unconditionally. Exposing
the entire store defeats the purpose of sandboxing, and when the test
store is the same as the system store (NixOS VM), it causes an obscure
"Permission denied" error.

Only add sandbox-paths when not on NixOS, indicating a separate test
store that needs access to system store build tools.
2026-01-24 19:55:50 +01:00
John Ericson bfdd124837 Create proper serializer for BuildResult status
The casts were not safe with respect to unknown values, but these are.
2026-01-23 16:27:26 -05:00
John Ericson aa17b75601 Merge pull request #15054 from obsidiansystems/unprivileged-test
Add new VM test with unprivileged daemon user
2026-01-23 19:49:46 +00:00
Jörg Thalheim 18176d2678 ignore-gc-delete-failure: add release note 2026-01-23 14:08:05 -05:00
John Ericson 1a17c9d02b Merge pull request #15051 from amaanq/split-build-log-settings
libstore: split out `LogFileSettings`
2026-01-23 18:58:51 +00:00
Amaan Qureshi 98178e24d0 libstore: split out LogFileSettings 2026-01-23 13:17:15 -05:00
John Ericson fe8f574471 Clean up NarAccessor construction
We had a minor combinatorial explosion of ways to do things. We can get
rid of those by just having the caller call `parseNarListing` intead.
2026-01-21 19:25:52 -05:00
Eelco DolstraandJohn Ericson 0b2dffefea Factor out NarCache from RemoteFSAccessor
Use

```
git show --color-moved --patience --color-moved-ws=ignore-all-space
```

to review and see that this is mostly code motion.

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
2026-01-21 19:15:51 -05:00
163 changed files with 2171 additions and 873 deletions

No files matched your search

+2
View File
@@ -1,5 +1,7 @@
# Default meson build dir
/build
# Meson creates this file too
src/.wraplock
# /tests/functional/
/tests/functional/common/subst-vars.sh
@@ -0,0 +1,10 @@
---
synopsis: "New setting `ignore-gc-delete-failure` for local stores"
prs: [15054]
---
A new local store setting [`ignore-gc-delete-failure`](@docroot@/store/types/local-store.md#store-local-store-ignore-gc-delete-failure) has been added.
When enabled, garbage collection will log warnings instead of failing when it cannot delete store paths.
This is useful when running Nix as an unprivileged user that may not have write access to all paths in the store.
This setting is experimental and requires the [`local-overlay-store`](@docroot@/development/experimental-features.md#xp-feature-local-overlay-store) experimental feature.
+15
View File
@@ -0,0 +1,15 @@
---
synopsis: Support HTTPS binary caches using mTLS (client certificate) authentication
issues: [13002]
prs: [13030]
---
Added support for `tls-certificate` and `tls-private-key` options in substituter URLs.
Example:
```
https://substituter.invalid?tls-certificate=/path/to/cert.pem&tls-private-key=/path/to/key.pem
```
When these options are configured, Nix will use this certificate/private key pair to authenticate to the server.
Generated
+5 -5
View File
@@ -63,15 +63,15 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1763948260,
"narHash": "sha256-zZk7fn2ARAqmLwaYTpxBJmj81KIdz11NiWt7ydHHD/M=",
"rev": "1c8ba8d3f7634acac4a2094eef7c32ad9106532c",
"lastModified": 1769089682,
"narHash": "sha256-Xu+7iYcAuOvsI2wdkUcIEmkqEJbvvE6n7qR9QNjJyP4=",
"rev": "078d69f03934859a181e81ba987c2bb033eebfc5",
"type": "tarball",
"url": "https://releases.nixos.org/nixos/25.05/nixos-25.05.813095.1c8ba8d3f763/nixexprs.tar.xz"
"url": "https://releases.nixos.org/nixos/25.11/nixos-25.11.4506.078d69f03934/nixexprs.tar.xz"
},
"original": {
"type": "tarball",
"url": "https://channels.nixos.org/nixos-25.05/nixexprs.tar.xz"
"url": "https://channels.nixos.org/nixos-25.11/nixexprs.tar.xz"
}
},
"nixpkgs-23-11": {
+1 -1
View File
@@ -1,7 +1,7 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "https://channels.nixos.org/nixos-25.05/nixexprs.tar.xz";
inputs.nixpkgs.url = "https://channels.nixos.org/nixos-25.11/nixexprs.tar.xz";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
+9 -2
View File
@@ -88,16 +88,23 @@
''^tests/functional/lang/eval-fail-path-slash\.nix$''
''^tests/functional/lang/eval-fail-toJSON-non-utf-8\.nix$''
''^tests/functional/lang/eval-fail-set\.nix$''
# Language tests, don't churn the formatting of strings
''^tests/functional/lang/eval-fail-fromTOML-overflow\.nix$''
''^tests/functional/lang/eval-fail-fromTOML-underflow\.nix$''
''^tests/functional/lang/eval-fail-bad-string-interpolation-3\.nix$''
''^tests/functional/lang/eval-fail-bad-string-interpolation-4\.nix$''
''^tests/functional/lang/eval-okay-regex-match2\.nix$''
];
};
clang-format = {
enable = true;
# https://github.com/cachix/git-hooks.nix/pull/532
package = pkgs.llvmPackages_latest.clang-tools;
package = pkgs.llvmPackages_21.clang-tools;
excludes = [
# We don't want to format test data
# ''tests/(?!nixos/).*\.nix''
''^src/[^/]*-tests/data/.*$''
"^src/[^/]*-tests/data/.*$"
# Don't format vendored code
''^doc/manual/redirects\.js$''
@@ -1,7 +1,7 @@
# Clang gets grumpy about missing libasan symbols if -shared-libasan is not
# passed when building shared libs, at least on Linux
if cxx.get_id() == 'clang' and ('address' in get_option('b_sanitize') or 'undefined' in get_option(
'b_sanitize',
'b_sanitize',
))
add_project_link_arguments('-shared-libasan', language : 'cpp')
endif
+4 -2
View File
@@ -155,12 +155,14 @@ let
];
};
mesonBuildLayer = finalAttrs: prevAttrs: {
mesonBuildLayer = finalAttrs: prevAttrs: rec {
nativeBuildInputs = prevAttrs.nativeBuildInputs or [ ] ++ [
pkg-config
];
separateDebugInfo = !stdenv.hostPlatform.isStatic;
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
# needed by separateDebugInfo
# SEE: https://github.com/NixOS/nixpkgs/pull/394674/commits/a4d355342976e9e9823fb94f133bc43ebec9da5b
__structuredAttrs = separateDebugInfo;
};
mesonLibraryLayer = finalAttrs: prevAttrs: {
+6 -26
View File
@@ -30,33 +30,13 @@ scope: {
NIX_CFLAGS_COMPILE = "-DINITIAL_MARK_STACK_SIZE=1048576";
});
lowdown = pkgs.lowdown.overrideAttrs (prevAttrs: rec {
version = "2.0.2";
src = pkgs.fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${version}.tar.gz";
hash = "sha512-cfzhuF4EnGmLJf5EGSIbWqJItY3npbRSALm+GarZ7SMU7Hr1xw0gtBFMpOdi5PBar4TgtvbnG4oRPh+COINGlA==";
};
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ pkgs.buildPackages.bmake ];
postInstall =
lib.replaceStrings [ "lowdown.so.1" "lowdown.1.dylib" ] [ "lowdown.so.2" "lowdown.2.dylib" ]
(prevAttrs.postInstall or "");
patches = [ ];
});
curl = pkgs.curl.override {
http3Support = !pkgs.stdenv.hostPlatform.isWindows;
};
# TODO: Remove this when https://github.com/NixOS/nixpkgs/pull/442682 is included in a stable release
toml11 =
if lib.versionAtLeast pkgs.toml11.version "4.4.0" then
pkgs.toml11
else
pkgs.toml11.overrideAttrs rec {
version = "4.4.0";
src = pkgs.fetchFromGitHub {
owner = "ToruNiina";
repo = "toml11";
tag = "v${version}";
hash = "sha256-sgWKYxNT22nw376ttGsTdg0AMzOwp8QH3E8mx0BZJTQ=";
};
};
libblake3 = pkgs.libblake3.override {
useTBB = !(stdenv.hostPlatform.isWindows || stdenv.hostPlatform.isStatic);
};
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost =
+3 -3
View File
@@ -131,7 +131,7 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
ignoreCrossFile = flags: builtins.filter (flag: !(lib.strings.hasInfix "cross-file" flag)) flags;
availableComponents = lib.filterAttrs (
k: v: lib.meta.availableOn pkgs.hostPlatform v
k: v: lib.meta.availableOn pkgs.stdenv.hostPlatform v
) allComponents;
activeComponents = buildInputsClosureCond isInternal (
@@ -323,7 +323,7 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
pkgs.buildPackages.shellcheck
pkgs.buildPackages.include-what-you-use
]
++ lib.optional pkgs.hostPlatform.isUnix pkgs.buildPackages.gdb
++ lib.optional stdenv.hostPlatform.isUnix pkgs.buildPackages.gdb
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
)
@@ -341,7 +341,7 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
buildInputs =
# TODO change Nixpkgs to mark gbenchmark as building on Windows
lib.optional pkgs.hostPlatform.isUnix pkgs.gbenchmark
lib.optional stdenv.hostPlatform.isUnix pkgs.gbenchmark
++ dedupByString (v: "${v}") (
lib.filter (x: !isInternal x) (lib.lists.concatMap (c: c.buildInputs) activeComponents)
)
+1
View File
@@ -38,6 +38,7 @@ escape_systemd_env() {
create_systemd_proxy_env() {
vars="http_proxy https_proxy ftp_proxy all_proxy no_proxy HTTP_PROXY HTTPS_PROXY FTP_PROXY ALL_PROXY NO_PROXY"
for v in $vars; do
# shellcheck disable=SC2268
if [ "x${!v:-}" != "x" ]; then
echo "Environment=${v}=$(escape_systemd_env "${!v}")"
fi
+2 -2
View File
@@ -43,7 +43,7 @@ RC_GTEST_FIXTURE_PROP(
auto * v = state.allocValue();
state.mkOutputString(*v, b, std::nullopt, mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
auto d = state.coerceToSingleDerivedPath(noPos, *v, "", mockXpSettings, true);
RC_ASSERT(SingleDerivedPath{b} == d);
}
@@ -57,7 +57,7 @@ RC_GTEST_FIXTURE_PROP(
auto * v = state.allocValue();
state.mkOutputString(*v, b, outPath, mockXpSettings);
auto [d, _] = state.coerceToSingleDerivedPathUnchecked(noPos, *v, "", mockXpSettings);
auto d = state.coerceToSingleDerivedPath(noPos, *v, "", mockXpSettings, true);
RC_ASSERT(SingleDerivedPath{b} == d);
}
+139 -55
View File
@@ -2406,7 +2406,8 @@ BackedStringView EvalState::coerceToString(
std::string_view errorCtx,
bool coerceMore,
bool copyToStore,
bool canonicalizePath)
bool canonicalizePath,
bool strict)
{
auto _level = addCallDepth(pos);
@@ -2433,14 +2434,28 @@ BackedStringView EvalState::coerceToString(
auto maybeString = tryAttrsToString(pos, v, context, coerceMore, copyToStore);
if (maybeString)
return std::move(*maybeString);
auto i = v.attrs()->get(s.outPath);
if (!i) {
auto outPathAttr = v.attrs()->get(s.outPath);
if (!outPathAttr) {
error<TypeError>(
"cannot coerce %1% to a string: %2%", showType(v), ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
return coerceToString(pos, *i->value, context, errorCtx, coerceMore, copyToStore, canonicalizePath);
try {
auto derivedPath = coerceToSingleDerivedPath(pos, v, errorCtx);
auto result = mkSingleDerivedPathStringRaw(derivedPath);
context.insert(
std::visit([](auto && o) -> NixStringContextElem { return std::move(o); }, std::move(derivedPath)));
return result;
} catch (Error & e) {
if (strict)
throw;
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
return coerceToString(
pos, *outPathAttr->value, context, errorCtx, coerceMore, copyToStore, canonicalizePath, strict);
}
}
if (v.type() == nExternal) {
@@ -2478,7 +2493,8 @@ BackedStringView EvalState::coerceToString(
"while evaluating one element of the list",
coerceMore,
copyToStore,
canonicalizePath);
canonicalizePath,
strict);
} catch (Error & e) {
e.addTrace(positions[pos], errorCtx);
throw;
@@ -2565,63 +2581,131 @@ EvalState::coerceToStorePath(const PosIdx pos, Value & v, NixStringContext & con
error<EvalError>("path '%1%' is not in the Nix store", path).withTrace(pos, errorCtx).debugThrow();
}
std::pair<SingleDerivedPath, std::string_view> EvalState::coerceToSingleDerivedPathUnchecked(
const PosIdx pos, Value & v, std::string_view errorCtx, const ExperimentalFeatureSettings & xpSettings)
SingleDerivedPath EvalState::coerceToSingleDerivedPath(
const PosIdx pos,
Value & v,
std::string_view errorCtx,
const ExperimentalFeatureSettings & xpSettings,
bool skipStringValidation)
{
NixStringContext context;
auto s = forceString(v, context, pos, errorCtx, xpSettings);
auto csize = context.size();
if (csize != 1)
error<EvalError>("string '%s' has %d entries in its context. It should only have exactly one entry", s, csize)
.withTrace(pos, errorCtx)
.debugThrow();
auto derivedPath = std::visit(
overloaded{
[&](NixStringContextElem::Opaque && o) -> SingleDerivedPath { return std::move(o); },
[&](NixStringContextElem::DrvDeep &&) -> SingleDerivedPath {
error<EvalError>(
"string '%s' has a context which refers to a complete source and binary closure. This is not supported at this time",
s)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](NixStringContextElem::Built && b) -> SingleDerivedPath { return std::move(b); },
},
((NixStringContextElem &&) *context.begin()).raw);
return {
std::move(derivedPath),
std::move(s),
};
}
forceValue(v, pos);
SingleDerivedPath EvalState::coerceToSingleDerivedPath(const PosIdx pos, Value & v, std::string_view errorCtx)
{
auto [derivedPath, s_] = coerceToSingleDerivedPathUnchecked(pos, v, errorCtx);
auto s = s_;
auto sExpected = mkSingleDerivedPathStringRaw(derivedPath);
if (s != sExpected) {
/* `std::visit` is used here just to provide a more precise
error message. */
std::visit(
switch (v.type()) {
case nAttrs: {
if (!isDerivation(v)) {
error<TypeError>(
"cannot coerce a set that is not a package to a single derived path: %1%",
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
auto outPathAttr = v.attrs()->get(s.outPath);
if (!outPathAttr) {
error<TypeError>(
"cannot coerce %1% to a single derived path: no 'outPath' attribute: %2%",
showType(v),
ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
auto outPathDerivedPath =
coerceToSingleDerivedPath(pos, *outPathAttr->value, errorCtx, xpSettings, skipStringValidation);
auto drvPathAttr = v.attrs()->get(s.drvPath);
if (drvPathAttr) {
auto drvPathDerivedPath =
coerceToSingleDerivedPath(pos, *drvPathAttr->value, errorCtx, xpSettings, skipStringValidation);
std::visit(
overloaded{
[&](const SingleDerivedPath::Built & built) {
if (*built.drvPath != drvPathDerivedPath) {
error<EvalError>(
"relative outPath '%s' does not refer to drvPath '%s'",
outPathDerivedPath.to_string(*store),
drvPathDerivedPath.to_string(*store))
.withTrace(pos, errorCtx)
.debugThrow();
}
},
[&](const SingleDerivedPath::Opaque &) {
error<EvalError>(
"relative outPath '%s' does not refer to drvPath '%s'",
outPathDerivedPath.to_string(*store),
drvPathDerivedPath.to_string(*store))
.withTrace(pos, errorCtx)
.debugThrow();
},
},
outPathDerivedPath.raw());
}
return outPathDerivedPath;
}
case nString: {
auto s = v.string_view();
NixStringContext context;
copyContext(v, context, xpSettings);
auto csize = context.size();
if (csize != 1)
error<EvalError>(
"string '%s' has %d entries in its context. It should only have exactly one entry", s, csize)
.withTrace(pos, errorCtx)
.debugThrow();
auto derivedPath = std::visit(
overloaded{
[&](const SingleDerivedPath::Opaque & o) {
error<EvalError>("path string '%s' has context with the different path '%s'", s, sExpected)
[&](NixStringContextElem::Opaque && o) -> SingleDerivedPath { return std::move(o); },
[&](NixStringContextElem::DrvDeep &&) -> SingleDerivedPath {
error<EvalError>(
"string '%s' has a context which refers to a complete source and binary closure. This is not "
"supported at this time",
s)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](const SingleDerivedPath::Built & b) {
error<EvalError>(
"string '%s' has context with the output '%s' from derivation '%s', but the string is not the right placeholder for this derivation output. It should be '%s'",
s,
b.output,
b.drvPath->to_string(*store),
sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
}},
derivedPath.raw());
[&](NixStringContextElem::Built && b) -> SingleDerivedPath { return std::move(b); },
},
((NixStringContextElem &&) *context.begin()).raw);
if (!skipStringValidation) {
auto sExpected = mkSingleDerivedPathStringRaw(derivedPath);
if (s != sExpected) {
/* `std::visit` is used here just to provide a more precise
error message. */
std::visit(
overloaded{
[&](const SingleDerivedPath::Opaque & o) {
error<EvalError>("path string '%s' has context with the different path '%s'", s, sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
},
[&](const SingleDerivedPath::Built & b) {
error<EvalError>(
"string '%s' has context with the output '%s' from derivation '%s', but the string is "
"not the right placeholder for this derivation output. It should be '%s'",
s,
b.output,
b.drvPath->to_string(*store),
sExpected)
.withTrace(pos, errorCtx)
.debugThrow();
}},
derivedPath.raw());
}
}
return derivedPath;
}
default:
error<TypeError>(
"cannot coerce %1% to a single derived path: %2%", showType(v), ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
.debugThrow();
}
return derivedPath;
}
// NOTE: This implementation must match eqValues!
+97 -23
View File
@@ -1,5 +1,6 @@
#include "nix/expr/get-drvs.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/store/derived-path.hh"
#include "nix/store/derivations.hh"
#include "nix/store/store-api.hh"
#include "nix/store/path-with-outputs.hh"
@@ -21,13 +22,13 @@ PackageInfo::PackageInfo(EvalState & state, ref<Store> store, const std::string
, attrs(nullptr)
, attrPath("")
{
auto [drvPath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
auto [drvStorePath, selectedOutputs] = parsePathWithOutputs(*store, drvPathWithOutputs);
this->drvPath = drvPath;
this->drvPath = {{Path{SingleDerivedPath::Opaque{drvStorePath}, drvStorePath}}};
auto drv = store->derivationFromPath(drvPath);
auto drv = store->derivationFromPath(drvStorePath);
name = drvPath.name();
name = drvStorePath.name();
if (selectedOutputs.size() > 1)
throw Error("building more than one derivation output is not supported, in '%s'", drvPathWithOutputs);
@@ -36,10 +37,18 @@ PackageInfo::PackageInfo(EvalState & state, ref<Store> store, const std::string
auto i = drv.outputs.find(outputName);
if (i == drv.outputs.end())
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvPath), outputName);
auto & [outputName, output] = *i;
throw Error("derivation '%s' does not have output '%s'", store->printStorePath(drvStorePath), outputName);
auto & [outputName_, output] = *i;
outPath = {output.path(*store, drv.name, outputName)};
auto outStorePath = output.path(*store, drv.name, outputName_);
if (outStorePath) {
outPath = Path{SingleDerivedPath::Opaque{*outStorePath}, *outStorePath};
} else {
// CA derivation with unknown output path
outPath = Path{
SingleDerivedPath::Built{makeConstantStorePathRef(drvStorePath), outputName_},
std::nullopt};
}
}
std::string PackageInfo::queryName() const
@@ -64,26 +73,60 @@ std::string PackageInfo::querySystem() const
return system;
}
std::optional<StorePath> PackageInfo::queryDrvPath() const
std::optional<PackageInfo::Path> PackageInfo::queryDrvPathFlexible() const
{
if (!drvPath && attrs) {
if (auto i = attrs->get(state->s.drvPath)) {
NixStringContext context;
auto found = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the 'drvPath' attribute of a derivation");
auto i = attrs->get(state->s.drvPath);
if (i) {
Value v;
v.mkAttrs(const_cast<Bindings *>(attrs));
std::optional<SingleDerivedPath> outPathDerivedPath;
try {
found.requireDerivation();
// Validate derivation structure
outPathDerivedPath = state->coerceToSingleDerivedPath(
noPos, v, "while evaluating the derivation");
} catch (Error & e) {
e.addTrace(state->positions[i->pos], "while evaluating the 'drvPath' attribute of a derivation");
throw;
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
}
if (outPathDerivedPath) {
// Validation passed. Now get drvPath.
auto drvPathDerivedPath = state->coerceToSingleDerivedPath(
i->pos, *i->value, "while evaluating the 'drvPath' attribute of a derivation");
if (auto * opaque = std::get_if<SingleDerivedPath::Opaque>(&drvPathDerivedPath.raw())) {
opaque->path.requireDerivation();
drvPath = {{Path{std::move(drvPathDerivedPath), opaque->path}}};
} else {
// Dynamic derivation - drvPath is itself a derivation output
drvPath = {{Path{std::move(drvPathDerivedPath), std::nullopt}}};
}
} else {
// Fall back to old behavior
NixStringContext context;
auto found = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the 'drvPath' attribute of a derivation");
try {
found.requireDerivation();
} catch (Error & e) {
e.addTrace(state->positions[i->pos], "while evaluating the 'drvPath' attribute of a derivation");
throw;
}
drvPath = {{Path{SingleDerivedPath::Opaque{found}, std::move(found)}}};
}
drvPath = {std::move(found)};
} else
drvPath = {std::nullopt};
}
return drvPath.value_or(std::nullopt);
}
std::optional<StorePath> PackageInfo::queryDrvPath() const
{
if (auto path = queryDrvPathFlexible())
return path->storePath;
return std::nullopt;
}
StorePath PackageInfo::requireDrvPath() const
{
if (auto drvPath = queryDrvPath())
@@ -91,20 +134,51 @@ StorePath PackageInfo::requireDrvPath() const
throw Error("derivation does not contain a 'drvPath' attribute");
}
StorePath PackageInfo::queryOutPath() const
PackageInfo::Path PackageInfo::queryOutPathFlexible() const
{
if (!outPath && attrs) {
auto i = attrs->get(state->s.outPath);
NixStringContext context;
if (i)
outPath = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation");
Value v;
v.mkAttrs(const_cast<Bindings *>(attrs));
std::optional<SingleDerivedPath> derivedPath;
try {
derivedPath = state->coerceToSingleDerivedPath(
noPos, v, "while evaluating the derivation");
} catch (Error & e) {
auto info = e.info();
info.msg = HintFmt("in a future version of Nix this will be an error: %s", Uncolored(info.msg.str()));
logWarning(info);
}
if (derivedPath) {
if (auto * opaque = std::get_if<SingleDerivedPath::Opaque>(&derivedPath->raw())) {
outPath = Path{std::move(*derivedPath), opaque->path};
} else {
// Built path - no concrete output path available (placeholder)
outPath = Path{std::move(*derivedPath), std::nullopt};
}
} else {
// Fall back to old behavior
auto i = attrs->get(state->s.outPath);
NixStringContext context;
if (i) {
auto path = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation");
outPath = Path{SingleDerivedPath::Opaque{path}, std::move(path)};
}
}
}
if (!outPath)
throw UnimplementedError("CA derivations are not yet supported");
throw Error("derivation does not have an 'outPath' attribute");
return *outPath;
}
StorePath PackageInfo::queryOutPath() const
{
auto path = queryOutPathFlexible();
if (!path.storePath)
throw UnimplementedError("CA derivations are not yet supported");
return *path.storePath;
}
PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsToInstall)
{
if (outputs.empty()) {
+28 -21
View File
@@ -715,6 +715,9 @@ public:
* string. If `coerceMore` is set, also converts nulls, integers,
* booleans and lists to a string. If `copyToStore` is set,
* referenced paths are copied to the Nix store as a side effect.
* If `strict` is set, ensures that when coercing an attrset with
* both `outPath` and `drvPath`, the string context for `outPath`
* agrees with the `drvPath` attribute.
*/
BackedStringView coerceToString(
const PosIdx pos,
@@ -723,7 +726,8 @@ public:
std::string_view errorCtx,
bool coerceMore = false,
bool copyToStore = true,
bool canonicalizePath = true);
bool canonicalizePath = true,
bool strict = false);
StorePath copyPathToStore(NixStringContext & context, const SourcePath & path);
@@ -742,30 +746,33 @@ public:
StorePath coerceToStorePath(const PosIdx pos, Value & v, NixStringContext & context, std::string_view errorCtx);
/**
* Part of `coerceToSingleDerivedPath()` without any store IO which is exposed for unit testing only.
* Coerce to `SingleDerivedPath`.
*
* For strings: must be either a literal store path or a
* placeholder (see `DownstreamPlaceholder`). The string context
* must be exactly one element, which is either a
* `NixStringContextElem::Opaque` or `NixStringContextElem::Built`.
* (`NixStringContextElem::DrvDeep` is not permitted). The string
* is parsed based on the context --- the context is the source of
* truth, and ultimately tells us what we want, and then we ensure
* the string corresponds to it.
*
* For attrsets: must have an `outPath` attribute which is
* recursively coerced. If a `drvPath` attribute is present, it is
* also coerced and validated to agree with the `outPath` context.
*
* @param xpSettings Stop-gap to avoid globals during unit tests.
*
* @param skipStringValidation Skip validating that the string
* matches the expected placeholder for the context. This is
* exposed for unit testing only to avoid store IO.
*/
std::pair<SingleDerivedPath, std::string_view> coerceToSingleDerivedPathUnchecked(
SingleDerivedPath coerceToSingleDerivedPath(
const PosIdx pos,
Value & v,
std::string_view errorCtx,
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings);
/**
* Coerce to `SingleDerivedPath`.
*
* Must be a string which is either a literal store path or a
* "placeholder (see `DownstreamPlaceholder`).
*
* Even more importantly, the string context must be exactly one
* element, which is either a `NixStringContextElem::Opaque` or
* `NixStringContextElem::Built`. (`NixStringContextEleme::DrvDeep`
* is not permitted).
*
* The string is parsed based on the context --- the context is the
* source of truth, and ultimately tells us what we want, and then
* we ensure the string corresponds to it.
*/
SingleDerivedPath coerceToSingleDerivedPath(const PosIdx pos, Value & v, std::string_view errorCtx);
const ExperimentalFeatureSettings & xpSettings = experimentalFeatureSettings,
bool skipStringValidation = false);
#if NIX_USE_BOEHMGC
/** A GC root for the baseEnv reference. */
+20 -4
View File
@@ -2,6 +2,7 @@
///@file
#include "nix/expr/eval.hh"
#include "nix/store/derived-path.hh"
#include "nix/store/path.hh"
#include <string>
@@ -20,10 +21,23 @@ public:
private:
EvalState * state;
struct Path {
/**
* Underlying semantic path, valid in CA and IA cases.
*/
SingleDerivedPath derivedPath;
/**
* If we don't need a placeholder, we'll have a real store path, which we can remember here.
* For CA derivations using placeholders, this will be std::nullopt.
*/
std::optional<StorePath> storePath;
};
mutable std::string name;
mutable std::string system;
mutable std::optional<std::optional<StorePath>> drvPath;
mutable std::optional<StorePath> outPath;
mutable std::optional<std::optional<Path>> drvPath;
mutable std::optional<Path> outPath;
mutable std::string outputName;
Outputs outputs;
@@ -52,8 +66,10 @@ public:
std::string queryName() const;
std::string querySystem() const;
std::optional<StorePath> queryDrvPath() const;
std::optional<Path> queryDrvPathFlexible() const;
StorePath requireDrvPath() const;
StorePath queryOutPath() const;
Path queryOutPathFlexible() const;
std::string queryOutputName() const;
/**
* Return the unordered map of output names to (optional) output paths.
@@ -81,12 +97,12 @@ public:
void setDrvPath(StorePath path)
{
drvPath = {{std::move(path)}};
drvPath = {{Path{SingleDerivedPath::Opaque{path}, std::move(path)}}};
}
void setOutPath(StorePath path)
{
outPath = {{std::move(path)}};
outPath = Path{SingleDerivedPath::Opaque{path}, std::move(path)};
}
void setFailed()
+2 -2
View File
@@ -533,8 +533,8 @@ inline constexpr bool useBitPackedValueStorage = (ptrSize == 8) && (__STDCPP_DEF
* Packs discriminator bits into the pointer alignment niches.
*/
template<std::size_t ptrSize>
class alignas(16) ValueStorage<ptrSize, std::enable_if_t<detail::useBitPackedValueStorage<ptrSize>>>
: public detail::ValueBase
class alignas(16)
ValueStorage<ptrSize, std::enable_if_t<detail::useBitPackedValueStorage<ptrSize>>> : public detail::ValueBase
{
/* Needs a dependent type name in order for member functions (and
* potentially ill-formed bit casts) to be SFINAE'd out.
+36 -13
View File
@@ -3,7 +3,7 @@
%define api.namespace { ::nix::parser }
%define api.parser.class { BisonParser }
%locations
%define parse.error verbose
%define parse.error detailed
%defines
/* %no-lines */
%parse-param { void * scanner }
@@ -140,18 +140,41 @@ static Expr * makeCall(Exprs & exprs, PosIdx pos, Expr * fn, Expr * arg) {
%type <Expr *> path_start
%type <ToBeStringyExpr> string_parts string_attr
%type <StringToken> attr
%token <StringToken> ID
%token <StringToken> STR IND_STR
%token <NixInt> INT_LIT
%token <NixFloat> FLOAT_LIT
%token <StringToken> PATH HPATH SPATH PATH_END
%token <StringToken> URI
%token IF THEN ELSE ASSERT WITH LET IN_KW REC INHERIT EQ NEQ AND OR IMPL OR_KW
%token PIPE_FROM PIPE_INTO /* <| and |> */
%token DOLLAR_CURLY /* == ${ */
%token IND_STRING_OPEN IND_STRING_CLOSE
%token ELLIPSIS
%token <StringToken> ID "identifier"
%token <StringToken> STR "string"
%token <StringToken> IND_STR "indented string"
%token <NixInt> INT_LIT "integer"
%token <NixFloat> FLOAT_LIT "floating-point literal"
%token <StringToken> PATH "path"
%token <StringToken> HPATH "'~/…' path"
%token <StringToken> SPATH "'<…>' path"
%token <StringToken> PATH_END "end of path"
%token <StringToken> URI "URI"
%token IF "'if'"
%token THEN "'then'"
%token ELSE "'else'"
%token ASSERT "'assert'"
%token WITH "'with'"
%token LET "'let'"
%token IN_KW "'in'"
%token REC "'rec'"
%token INHERIT "'inherit'"
%token EQ "'=='"
%token NEQ "'!='"
%token LEQ "'<='"
%token GEQ "'>='"
%token UPDATE "'//'"
%token CONCAT "'++'"
%token AND "'&&'"
%token OR "'||'"
%token IMPL "'->'"
%token OR_KW "'or'"
%token PIPE_FROM "'<|'"
%token PIPE_INTO "'|>'"
%token DOLLAR_CURLY "'${'"
%token IND_STRING_OPEN "start of an indented string"
%token IND_STRING_CLOSE "end of an indented string"
%token ELLIPSIS "'...'"
%right IMPL
%left OR
+7 -7
View File
@@ -37,7 +37,7 @@ namespace {
// old version of git, which will ignore unrecognized `-c` options.
const std::string gitInitialBranch = "__nix_dummy_branch";
bool isCacheFileWithinTtl(time_t now, const struct stat & st)
bool isCacheFileWithinTtl(time_t now, const PosixStat & st)
{
return st.st_mtime + static_cast<time_t>(settings.tarballTtl) > now;
}
@@ -113,11 +113,11 @@ std::optional<std::string> readHeadCached(const std::string & actualUrl, bool sh
std::filesystem::path headRefFile = cacheDir / "HEAD";
time_t now = time(0);
struct stat st;
auto st = maybeStat(headRefFile);
std::optional<std::string> cachedRef;
if (stat(headRefFile.string().c_str(), &st) == 0) {
if (st) {
cachedRef = readHead(cacheDir);
if (cachedRef != std::nullopt && *cachedRef != gitInitialBranch && isCacheFileWithinTtl(now, st)) {
if (cachedRef != std::nullopt && *cachedRef != gitInitialBranch && isCacheFileWithinTtl(now, *st)) {
debug("using cached HEAD ref '%s' for repo '%s'", *cachedRef, actualUrl);
return cachedRef;
}
@@ -819,10 +819,10 @@ struct GitInputScheme : InputScheme
if (getAllRefsAttr(input)) {
doFetch = true;
} else {
/* If the local ref is older than ‘tarball-ttl’ seconds, do a
/* If the local ref is older than 'tarball-ttl' seconds, do a
git fetch to update the local ref to the remote ref. */
struct stat st;
doFetch = stat(localRefFile.string().c_str(), &st) != 0 || !isCacheFileWithinTtl(now, st);
auto st = maybeStat(localRefFile);
doFetch = !st || !isCacheFileWithinTtl(now, *st);
}
}
+2 -1
View File
@@ -177,7 +177,8 @@ static DownloadTarballResult downloadTarball_(
the entire file to disk so libarchive can access it
in random-access mode. */
auto [fdTemp, path] = createTempFile("nix-zipfile");
cleanupTemp.reset(path);
cleanupTemp.cancel();
cleanupTemp = {path};
debug("downloading '%s' into '%s'...", url, path);
{
FdSink sink(fdTemp.get());
+125
View File
@@ -0,0 +1,125 @@
#include "nix/store/tests/https-store.hh"
#include <thread>
namespace nix::testing {
void TestHttpBinaryCacheStore::init()
{
BinaryCacheStore::init();
}
ref<TestHttpBinaryCacheStore> TestHttpBinaryCacheStoreConfig::openTestStore() const
{
auto store = make_ref<TestHttpBinaryCacheStore>(
ref{// FIXME we shouldn't actually need a mutable config
std::const_pointer_cast<HttpBinaryCacheStore::Config>(shared_from_this())});
store->init();
return store;
}
void HttpsBinaryCacheStoreTest::openssl(Strings args)
{
runProgram("openssl", /*lookupPath=*/true, args);
}
void HttpsBinaryCacheStoreTest::SetUp()
{
LibStoreNetworkTest::SetUp();
#ifdef _WIN32
GTEST_SKIP() << "HTTPS store tests are not supported on Windows";
#endif
tmpDir = createTempDir();
cacheDir = tmpDir / "cache";
delTmpDir = std::make_unique<AutoDelete>(tmpDir);
localCacheStore =
make_ref<LocalBinaryCacheStoreConfig>("file", cacheDir.string(), LocalBinaryCacheStoreConfig::Params{})
->openStore();
caCert = tmpDir / "ca.crt";
caKey = tmpDir / "ca.key";
serverCert = tmpDir / "server.crt";
serverKey = tmpDir / "server.key";
clientCert = tmpDir / "client.crt";
clientKey = tmpDir / "client.key";
// clang-format off
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", caKey.string()});
openssl({"req", "-new", "-x509", "-days", "1", "-key", caKey.string(), "-out", caCert.string(), "-subj", "/CN=TestCA"});
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", serverKey.string()});
openssl({"req", "-new", "-key", serverKey.string(), "-out", (tmpDir / "server.csr").string(), "-subj", "/CN=localhost"});
openssl({"x509", "-req", "-in", (tmpDir / "server.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", serverCert.string(), "-days", "1"});
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", clientKey.string()});
openssl({"req", "-new", "-key", clientKey.string(), "-out", (tmpDir / "client.csr").string(), "-subj", "/CN=TestClient"});
openssl({"x509", "-req", "-in", (tmpDir / "client.csr").string(), "-CA", caCert.string(), "-CAkey", caKey.string(), "-CAcreateserial", "-out", clientCert.string(), "-days", "1"});
// clang-format on
#ifndef _WIN32 /* FIXME: Can't yet start processes on windows */
auto args = serverArgs();
serverPid = startProcess(
[&] {
if (chdir(cacheDir.c_str()) == -1)
_exit(1);
std::vector<char *> argv;
argv.push_back(const_cast<char *>("openssl"));
for (auto & a : args)
argv.push_back(const_cast<char *>(a.c_str()));
argv.push_back(nullptr);
execvp("openssl", argv.data());
_exit(1);
},
{.dieWithParent = true});
#endif
/* As an optimization, sleep for a bit to allow the server to come up to avoid retrying when connecting.
This won't make the tests fail, but does make them run faster. We don't need to overcomplicate by waiting
for the port explicitly - this is enough. */
std::this_thread::sleep_for(std::chrono::milliseconds(50));
/* FIXME: Don't use global settings. Tests are not run concurrently, so this is fine for now. */
oldCaCert = settings.caFile;
settings.caFile = caCert.string();
}
void HttpsBinaryCacheStoreTest::TearDown()
{
settings.caFile = oldCaCert;
serverPid.kill();
delTmpDir.reset();
}
std::vector<std::string> HttpsBinaryCacheStoreTest::serverArgs()
{
return {
"s_server",
"-accept",
std::to_string(port),
"-cert",
serverCert.string(),
"-key",
serverKey.string(),
"-WWW", /* Serve from current directory. */
"-quiet",
};
}
std::vector<std::string> HttpsBinaryCacheStoreMtlsTest::serverArgs()
{
auto args = HttpsBinaryCacheStoreTest::serverArgs();
/* With the -Verify option the client must supply a certificate or an error occurs, which is not the
case with -verify. */
args.insert(args.end(), {"-CAfile", caCert.string(), "-Verify", "1", "-verify_return_error"});
return args;
}
ref<TestHttpBinaryCacheStoreConfig> HttpsBinaryCacheStoreTest::makeConfig(BinaryCacheStoreConfig::Params params)
{
auto res = make_ref<TestHttpBinaryCacheStoreConfig>("https", fmt("localhost:%d", port), std::move(params));
res->pathInfoCacheSize = 0; /* We don't want any caching in tests. */
return res;
}
} // namespace nix::testing
@@ -0,0 +1,93 @@
#pragma once
///@file
#include <gtest/gtest.h>
#include <gmock/gmock.h>
#include "nix/store/tests/libstore-network.hh"
#include "nix/store/http-binary-cache-store.hh"
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
#include "nix/store/local-binary-cache-store.hh"
#include "nix/util/file-system.hh"
#include "nix/util/processes.hh"
namespace nix::testing {
class TestHttpBinaryCacheStoreConfig;
/**
* Test shim for testing. We don't want to use the on-disk narinfo cache in unit
* tests.
*/
class TestHttpBinaryCacheStore : public HttpBinaryCacheStore
{
public:
TestHttpBinaryCacheStore(const TestHttpBinaryCacheStore &) = delete;
TestHttpBinaryCacheStore(TestHttpBinaryCacheStore &&) = delete;
TestHttpBinaryCacheStore & operator=(const TestHttpBinaryCacheStore &) = delete;
TestHttpBinaryCacheStore & operator=(TestHttpBinaryCacheStore &&) = delete;
TestHttpBinaryCacheStore(ref<HttpBinaryCacheStoreConfig> config)
: Store{*config}
, BinaryCacheStore{*config}
, HttpBinaryCacheStore(config)
{
diskCache = nullptr; /* Disable caching, we'll be creating a new binary cache for each test. */
}
void init() override;
};
class TestHttpBinaryCacheStoreConfig : public HttpBinaryCacheStoreConfig
{
public:
TestHttpBinaryCacheStoreConfig(
std::string_view scheme, std::string_view cacheUri, const Store::Config::Params & params)
: StoreConfig(params)
, HttpBinaryCacheStoreConfig(scheme, cacheUri, params)
{
}
ref<TestHttpBinaryCacheStore> openTestStore() const;
ref<Store> openStore() const override
{
return openTestStore();
}
};
class HttpsBinaryCacheStoreTest : public virtual LibStoreNetworkTest
{
std::unique_ptr<AutoDelete> delTmpDir;
public:
static void SetUpTestSuite()
{
initLibStore(/*loadConfig=*/false);
}
protected:
std::filesystem::path tmpDir, cacheDir;
std::filesystem::path caCert, caKey, serverCert, serverKey;
std::filesystem::path clientCert, clientKey;
std::string oldCaCert;
Pid serverPid;
uint16_t port = 8443;
std::shared_ptr<Store> localCacheStore;
static void openssl(Strings args);
void SetUp() override;
void TearDown() override;
virtual std::vector<std::string> serverArgs();
ref<TestHttpBinaryCacheStoreConfig> makeConfig(BinaryCacheStoreConfig::Params params);
};
class HttpsBinaryCacheStoreMtlsTest : public HttpsBinaryCacheStoreTest
{
protected:
std::vector<std::string> serverArgs() override;
};
} // namespace nix::testing
@@ -0,0 +1,39 @@
#pragma once
/// @file
#include <gtest/gtest.h>
namespace nix::testing {
/**
* Whether to run network tests. This is global so that the test harness can
* enable this by default if we can run tests in isolation.
*/
extern bool networkTestsAvailable;
/**
* Set up network tests and, if on linux, create a new network namespace for
* tests with a loopback interface. This is to avoid binding to ports in the
* host's namespace.
*/
void setupNetworkTests();
class LibStoreNetworkTest : public virtual ::testing::Test
{
protected:
void SetUp() override
{
if (networkTestsAvailable)
return;
static bool warned = false;
if (!warned) {
warned = true;
GTEST_SKIP()
<< "Network tests not enabled by default without user namespaces, use NIX_TEST_FORCE_NETWORK_TESTS=1 to override";
} else {
GTEST_SKIP();
}
}
};
} // namespace nix::testing
@@ -4,6 +4,8 @@ include_dirs = [ include_directories('../../..') ]
headers = files(
'derived-path.hh',
'https-store.hh',
'libstore-network.hh',
'libstore.hh',
'nix_api_store.hh',
'outputs-spec.hh',
@@ -0,0 +1,60 @@
#include "nix/store/tests/libstore-network.hh"
#include "nix/util/error.hh"
#include "nix/util/environment-variables.hh"
#ifdef __linux__
# include "nix/util/file-system.hh"
# include "nix/util/linux-namespaces.hh"
# include <sched.h>
# include <sys/ioctl.h>
# include <net/if.h>
# include <netinet/in.h>
#endif
namespace nix::testing {
bool networkTestsAvailable = false;
#ifdef __linux__
static void enterNetworkNamespace()
{
auto uid = ::getuid();
auto gid = ::getgid();
if (::unshare(CLONE_NEWUSER | CLONE_NEWNET) == -1)
throw SysError("setting up a private network namespace for tests");
std::filesystem::path procSelf = "/proc/self";
writeFile(procSelf / "setgroups", "deny");
writeFile(procSelf / "uid_map", fmt("%d %d 1", uid, uid));
writeFile(procSelf / "gid_map", fmt("%d %d 1", gid, gid));
AutoCloseFD fd(::socket(PF_INET, SOCK_DGRAM, IPPROTO_IP));
if (!fd)
throw SysError("cannot open IP socket for loopback interface");
struct ::ifreq ifr = {};
strcpy(ifr.ifr_name, "lo");
ifr.ifr_flags = IFF_UP | IFF_LOOPBACK | IFF_RUNNING;
if (::ioctl(fd.get(), SIOCSIFFLAGS, &ifr) == -1)
throw SysError("cannot set loopback interface flags");
}
#endif
void setupNetworkTests()
try {
networkTestsAvailable = getEnvOs(OS_STR("NIX_TEST_FORCE_NETWORK_TESTS")).has_value();
#ifdef __linux__
if (!networkTestsAvailable && userNamespacesSupported()) {
enterNetworkNamespace();
networkTestsAvailable = true;
}
#endif
} catch (SystemError & e) {
/* Ignore any set up errors. */
}
} // namespace nix::testing
+5
View File
@@ -28,10 +28,15 @@ subdir('nix-meson-build-support/subprojects')
rapidcheck = dependency('rapidcheck')
deps_public += rapidcheck
gtest = dependency('gtest')
deps_public += gtest
subdir('nix-meson-build-support/common')
sources = files(
'derived-path.cc',
'https-store.cc',
'libstore-network.cc',
'outputs-spec.cc',
'path.cc',
'test-main.cc',
+2
View File
@@ -7,6 +7,7 @@
nix-store-c,
rapidcheck,
gtest,
# Configuration Options
@@ -39,6 +40,7 @@ mkMesonLibrary (finalAttrs: {
nix-store
nix-store-c
rapidcheck
gtest
];
mesonFlags = [
@@ -1,6 +1,9 @@
#include <gtest/gtest.h>
#include <regex>
#include "nix/store/http-binary-cache-store.hh"
#include "nix/store/tests/https-store.hh"
#include "nix/util/fs-sink.hh"
namespace nix {
@@ -34,4 +37,88 @@ TEST(HttpBinaryCacheStore, constructConfigWithParamsAndUrlWithParams)
EXPECT_EQ(config.getReference().params, params);
}
using testing::HttpsBinaryCacheStoreMtlsTest;
using testing::HttpsBinaryCacheStoreTest;
using namespace std::string_view_literals;
using namespace std::string_literals;
TEST_F(HttpsBinaryCacheStoreTest, queryPathInfo)
{
auto config = makeConfig({});
auto store = config->openStore();
StringSource dump{"test"sv};
auto path = localCacheStore->addToStoreFromDump(dump, "test-name", FileSerialisationMethod::Flat);
EXPECT_NO_THROW(store->queryPathInfo(path));
}
auto withNoRetries()
{
auto oldTries = fileTransferSettings.tries.get();
Finally restoreTries = [=]() { fileTransferSettings.tries = oldTries; };
fileTransferSettings.tries = 1; /* FIXME: Don't use global settings. */
return restoreTries;
}
TEST_F(HttpsBinaryCacheStoreMtlsTest, queryPathInfo)
{
auto config = makeConfig({
{"tls-certificate"s, clientCert.string()},
{"tls-private-key"s, clientKey.string()},
});
auto store = config->openStore();
StringSource dump{"test"sv};
auto path = localCacheStore->addToStoreFromDump(dump, "test-name", FileSerialisationMethod::Flat);
EXPECT_NO_THROW(store->queryPathInfo(path));
}
TEST_F(HttpsBinaryCacheStoreMtlsTest, rejectsWithoutClientCert)
{
auto restoreTries = withNoRetries();
auto config = makeConfig({});
EXPECT_THROW(config->openStore(), Error);
}
TEST_F(HttpsBinaryCacheStoreMtlsTest, rejectsWrongClientCert)
{
auto wrongKey = tmpDir / "wrong.key";
auto wrongCert = tmpDir / "wrong.crt";
// clang-format off
openssl({"ecparam", "-genkey", "-name", "prime256v1", "-out", wrongKey.string()});
openssl({"req", "-new", "-x509", "-days", "1", "-key", wrongKey.string(), "-out", wrongCert.string(), "-subj", "/CN=WrongClient"});
// clang-format on
auto config = makeConfig({
{"tls-certificate"s, wrongCert.string()},
{"tls-private-key"s, wrongKey.string()},
});
auto restoreTries = withNoRetries();
EXPECT_THROW(config->openStore(), Error);
}
TEST_F(HttpsBinaryCacheStoreMtlsTest, doesNotSendCertOnRedirectToDifferentAuthority)
{
StringSource dump{"test"sv};
auto path = localCacheStore->addToStoreFromDump(dump, "test-name", FileSerialisationMethod::Flat);
for (auto & entry : DirectoryIterator{cacheDir})
if (entry.path().extension() == ".narinfo") {
auto content = readFile(entry.path());
content = std::regex_replace(content, std::regex("URL: nar/"), fmt("URL: https://127.0.0.1:%d/nar/", port));
writeFile(entry.path(), content);
}
auto config = makeConfig({
{"tls-certificate"s, clientCert.string()},
{"tls-private-key"s, clientKey.string()},
});
auto store = config->openStore();
auto restoreTries = withNoRetries();
auto info = store->queryPathInfo(path);
NullSink null;
EXPECT_THROW(store->narFromPath(path, null), Error);
}
} // namespace nix
+2
View File
@@ -1,6 +1,7 @@
#include <gtest/gtest.h>
#include "nix/store/tests/test-main.hh"
#include "nix/store/tests/libstore-network.hh"
using namespace nix;
@@ -10,6 +11,7 @@ int main(int argc, char ** argv)
if (res)
return res;
nix::testing::setupNetworkTests();
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
+5 -1
View File
@@ -9,6 +9,7 @@
nix-store-c,
nix-store-test-support,
sqlite,
openssl,
rapidcheck,
gtest,
@@ -75,7 +76,10 @@ mkMesonExecutable (finalAttrs: {
runCommand "${finalAttrs.pname}-run"
{
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
buildInputs = [ writableTmpDirAsHomeHook ];
nativeBuildInputs = [
writableTmpDirAsHomeHook
openssl
];
}
(
''
+103 -12
View File
@@ -13,6 +13,11 @@
// C library headers for SSO provider support
# include <aws/auth/credentials.h>
// C library headers for custom logging
# include <aws/common/logging.h>
# include <cstdarg>
# include <boost/unordered/concurrent_flat_map.hpp>
# include <chrono>
@@ -30,6 +35,101 @@ AwsAuthError::AwsAuthError(int errorCode)
namespace {
/**
* Map AWS log level to Nix verbosity.
* AWS levels: AWS_LL_NONE=0, AWS_LL_FATAL=1, AWS_LL_ERROR=2, AWS_LL_WARN=3,
* AWS_LL_INFO=4, AWS_LL_DEBUG=5, AWS_LL_TRACE=6
*
* We map very conservatively because the AWS SDK is extremely noisy. What AWS
* considers "info" includes low-level details like "Initializing epoll" and
* "Starting event-loop thread". What it considers "errors" includes expected
* conditions like missing ~/.aws/config or IMDS being unavailable on non-EC2.
*
* To avoid spamming users, we only show FATAL at default verbosity. Everything
* else requires -vvvvv (lvlDebug) or higher to see.
*/
static Verbosity awsLogLevelToVerbosity(enum aws_log_level level)
{
switch (level) {
case AWS_LL_FATAL:
return lvlError;
case AWS_LL_ERROR:
case AWS_LL_WARN:
case AWS_LL_INFO:
return lvlDebug;
case AWS_LL_DEBUG:
case AWS_LL_TRACE:
return lvlVomit;
// AWS_LL_NONE and AWS_LL_COUNT are enum sentinels, not real log levels
case AWS_LL_NONE:
case AWS_LL_COUNT:
return lvlDebug;
}
unreachable();
}
/**
* Custom AWS logger that routes logs through Nix's logging infrastructure.
*
* The AWS CRT C++ wrapper (ApiHandle::InitializeLogging) only supports FILE*
* or filename-based logging. The underlying C library supports custom loggers
* via aws_logger struct with a vtable containing callback functions.
*/
static int nixAwsLoggerLog(
struct aws_logger * logger, enum aws_log_level logLevel, aws_log_subject_t subject, const char * format, ...)
{
Verbosity nixLevel = awsLogLevelToVerbosity(logLevel);
if (nixLevel > verbosity)
return AWS_OP_SUCCESS; /* Bail out early to avoid formatting the message unnecessarily. */
va_list args;
va_start(args, format);
std::array<char, 4096> buffer{};
auto res = vsnprintf(buffer.data(), buffer.size(), format, args);
va_end(args);
if (res < 0) /* Skip garbage debug messages in case the SDK is busted. */
return AWS_OP_SUCCESS;
const char * subjectName = aws_log_subject_name(subject);
printMsgUsing(nix::logger, nixLevel, "(aws:%s) %s", subjectName ? subjectName : "unknown", chomp(buffer.data()));
return AWS_OP_SUCCESS;
}
/**
* Get current log level for a subject - determines which messages will be logged.
* Must be consistent with awsLogLevelToVerbosity mapping.
*/
static aws_log_level nixAwsLoggerGetLevel(struct aws_logger * logger, aws_log_subject_t subject)
{
// Map Nix verbosity back to AWS log level (inverse of awsLogLevelToVerbosity)
if (verbosity >= lvlVomit)
return AWS_LL_TRACE;
if (verbosity >= lvlDebug)
return AWS_LL_INFO; // error/warn/info are all mapped to lvlDebug
return AWS_LL_FATAL;
}
static void initialiseAwsLogger()
{
static std::once_flag initialised; /* aws_logger_set must only be called once */
std::call_once(initialised, []() {
static aws_logger_vtable nixAwsLoggerVtable = {
.log = nixAwsLoggerLog,
.get_log_level = nixAwsLoggerGetLevel,
.clean_up = [](struct aws_logger *) {}, // No resources to clean up
.set_log_level = nullptr,
};
static aws_logger nixAwsLogger = {
.vtable = &nixAwsLoggerVtable,
.allocator = nullptr,
.p_impl = nullptr,
};
aws_logger_set(&nixAwsLogger);
});
}
/**
* Helper function to wrap a C credentials provider in the C++ interface.
* This replicates the static s_CreateWrappedProvider from aws-crt-cpp.
@@ -119,18 +219,9 @@ class AwsCredentialProviderImpl : public AwsCredentialProvider
public:
AwsCredentialProviderImpl()
{
// Map Nix's verbosity to AWS CRT log level
Aws::Crt::LogLevel logLevel;
if (verbosity >= lvlVomit) {
logLevel = Aws::Crt::LogLevel::Trace;
} else if (verbosity >= lvlDebug) {
logLevel = Aws::Crt::LogLevel::Debug;
} else if (verbosity >= lvlChatty) {
logLevel = Aws::Crt::LogLevel::Info;
} else {
logLevel = Aws::Crt::LogLevel::Warn;
}
apiHandle.InitializeLogging(logLevel, stderr);
// Install custom logger that routes AWS CRT logs through Nix's logging infrastructure.
// This ensures AWS logs respect Nix's verbosity settings and are formatted consistently.
initialiseAwsLogger();
// Create a shared TLS context for SSO (required for HTTPS connections)
auto allocator = Aws::Crt::ApiAllocator();
+1 -1
View File
@@ -156,7 +156,7 @@ ref<const ValidPathInfo> BinaryCacheStore::addToStoreCommon(
config.compression, teeSinkCompressed, config.parallelCompression, config.compressionLevel);
TeeSink teeSinkUncompressed{*compressionSink, narHashSink};
TeeSource teeSource{narSource, teeSinkUncompressed};
narAccessor = makeNarAccessor(teeSource);
narAccessor = makeNarAccessor(parseNarListing(teeSource));
compressionSink->finish();
fileSink.flush();
}
+24 -25
View File
@@ -175,7 +175,7 @@ struct LogFile
AutoCloseFD fd;
std::shared_ptr<BufferedSink> fileSink, sink;
LogFile(Store & store, const StorePath & drvPath);
LogFile(Store & store, const StorePath & drvPath, const LogFileSettings & logSettings);
~LogFile();
};
@@ -396,13 +396,22 @@ Goal::Co DerivationBuildingGoal::tryToBuild(StorePathSet inputPaths)
if (useHook) {
co_return buildWithHook(
std::move(inputPaths), std::move(initialOutputs), std::move(drvOptions), std::move(outputLocks));
} else {
} else if (auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store)) {
co_return buildLocally(
*localStoreP,
std::move(inputPaths),
std::move(initialOutputs),
std::move(drvOptions),
std::move(outputLocks),
externalBuilder);
} else {
throw Error(
R"(
Unable to build with a primary store that isn't a local store;
either pass a different '--store' or enable remote builds.
For more information check 'man nix.conf' and search for '/machines'.
)");
}
}
@@ -451,7 +460,7 @@ Goal::Co DerivationBuildingGoal::buildWithHook(
hook->toHook.writeSide.close();
/* Create the log file and pipe. */
std::unique_ptr<LogFile> logFile = std::make_unique<LogFile>(worker.store, drvPath);
std::unique_ptr<LogFile> logFile = std::make_unique<LogFile>(worker.store, drvPath, settings.getLogFileSettings());
std::set<MuxablePipePollState::CommChannel> fds;
fds.insert(hook->fromHook.readSide.get());
@@ -611,6 +620,7 @@ Goal::Co DerivationBuildingGoal::buildWithHook(
}
Goal::Co DerivationBuildingGoal::buildLocally(
LocalStore & localStore,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
@@ -619,23 +629,15 @@ Goal::Co DerivationBuildingGoal::buildLocally(
{
co_await yield();
if (!dynamic_cast<LocalStore *>(&worker.store)) {
throw Error(
R"(
Unable to build with a primary store that isn't a local store;
either pass a different '--store' or enable remote builds.
For more information check 'man nix.conf' and search for '/machines'.
)");
}
#ifdef _WIN32 // TODO enable `DerivationBuilder` on Windows
throw UnimplementedError("building derivations is not yet implemented on Windows");
#else
std::unique_ptr<BuildLog> buildLog;
std::unique_ptr<LogFile> logFile;
auto openLogFile = [&]() { logFile = std::make_unique<LogFile>(worker.store, drvPath); };
auto openLogFile = [&]() {
logFile = std::make_unique<LogFile>(worker.store, drvPath, settings.getLogFileSettings());
};
auto closeLogFile = [&]() { logFile.reset(); };
@@ -654,7 +656,7 @@ Goal::Co DerivationBuildingGoal::buildLocally(
};
std::unique_ptr<Activity> actLock;
std::unique_ptr<DerivationBuilder> builder;
DerivationBuilderUnique builder;
Descriptor builderOut;
// Will continue here while waiting for a build user below
@@ -706,9 +708,6 @@ Goal::Co DerivationBuildingGoal::buildLocally(
}
};
auto * localStoreP = dynamic_cast<LocalStore *>(&worker.store);
assert(localStoreP);
decltype(DerivationBuilderParams::defaultPathsInChroot) defaultPathsInChroot = settings.sandboxPaths.get();
DesugaredEnv desugaredEnv;
@@ -754,12 +753,12 @@ Goal::Co DerivationBuildingGoal::buildLocally(
already be created, so we don't need to create it again. */
builder = externalBuilder
? makeExternalDerivationBuilder(
*localStoreP,
localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params),
*externalBuilder)
: makeDerivationBuilder(
*localStoreP,
localStore,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, openLogFile, closeLogFile),
std::move(params));
}
@@ -1041,9 +1040,9 @@ HookReply DerivationBuildingGoal::tryBuildHook(const DerivationOptions<StorePath
#endif
}
LogFile::LogFile(Store & store, const StorePath & drvPath)
LogFile::LogFile(Store & store, const StorePath & drvPath, const LogFileSettings & logSettings)
{
if (!settings.keepLog)
if (!logSettings.keepLog)
return;
auto baseName = std::string(baseNameOf(store.printStorePath(drvPath)));
@@ -1052,11 +1051,11 @@ LogFile::LogFile(Store & store, const StorePath & drvPath)
if (auto localStore = dynamic_cast<LocalStore *>(&store))
logDir = localStore->config->logDir;
else
logDir = settings.nixLogDir;
logDir = logSettings.nixLogDir;
Path dir = fmt("%s/%s/%s/", logDir, LocalFSStore::drvsLogDir, baseName.substr(0, 2));
createDirs(dir);
Path logFileName = fmt("%s/%s%s", dir, baseName.substr(2), settings.compressLog ? ".bz2" : "");
Path logFileName = fmt("%s/%s%s", dir, baseName.substr(2), logSettings.compressLog ? ".bz2" : "");
fd = toDescriptor(open(
logFileName.c_str(),
@@ -1071,7 +1070,7 @@ LogFile::LogFile(Store & store, const StorePath & drvPath)
fileSink = std::make_shared<FdSink>(fd.get());
if (settings.compressLog)
if (logSettings.compressLog)
sink = std::shared_ptr<CompressionSink>(makeCompressionSink(CompressionAlgo::bzip2, *fileSink));
else
sink = fileSink;
+5 -10
View File
@@ -50,17 +50,12 @@ static void createLinks(State & state, const Path & srcDir, const Path & dstDir,
auto srcFile = (std::filesystem::path{srcDir} / name).string();
auto dstFile = (std::filesystem::path{dstDir} / name).string();
struct stat srcSt;
try {
if (stat(srcFile.c_str(), &srcSt) == -1)
throw SysError("getting status of '%1%'", srcFile);
} catch (SystemError & e) {
if (e.is(std::errc::no_such_file_or_directory) || e.is(std::errc::not_a_directory)) {
warn("skipping dangling symlink '%s'", dstFile);
continue;
}
throw;
auto srcStOpt = maybeStat(srcFile.c_str());
if (!srcStOpt) {
warn("skipping dangling symlink '%s'", dstFile);
continue;
}
auto & srcSt = *srcStOpt;
/* The files below are special-cased to that they don't show
* up in user profiles, either because they are useless, or
+2 -2
View File
@@ -4,6 +4,7 @@
#include "nix/store/globals.hh"
#include "nix/util/archive.hh"
#include "nix/util/compression.hh"
#include "nix/util/file-system.hh"
namespace nix {
@@ -65,8 +66,7 @@ static void builtinFetchurl(const BuiltinBuilderContext & ctx)
auto executable = ctx.drv.env.find("executable");
if (executable != ctx.drv.env.end() && executable->second == "1") {
if (chmod(storePath.c_str(), 0755) == -1)
throw SysError("making '%1%' executable", storePath);
chmod(storePath, 0755);
}
};
+47
View File
@@ -111,4 +111,51 @@ void CommonProto::Serialise<Signature>::write(
conn.to << sig.to_string();
}
/**
* Mapping from protocol wire values to BuildResultStatus.
*
* The array index is the wire value.
* Note: HashMismatch is not in the protocol; it gets converted
* to OutputRejected before serialization.
*/
constexpr static BuildResultStatus buildResultStatusTable[] = {
BuildResultSuccessStatus::Built, // 0
BuildResultSuccessStatus::Substituted, // 1
BuildResultSuccessStatus::AlreadyValid, // 2
BuildResultFailureStatus::PermanentFailure, // 3
BuildResultFailureStatus::InputRejected, // 4
BuildResultFailureStatus::OutputRejected, // 5
BuildResultFailureStatus::TransientFailure, // 6
BuildResultFailureStatus::CachedFailure, // 7
BuildResultFailureStatus::TimedOut, // 8
BuildResultFailureStatus::MiscFailure, // 9
BuildResultFailureStatus::DependencyFailed, // 10
BuildResultFailureStatus::LogLimitExceeded, // 11
BuildResultFailureStatus::NotDeterministic, // 12
BuildResultSuccessStatus::ResolvesToAlreadyValid, // 13
BuildResultFailureStatus::NoSubstituters, // 14
};
BuildResultStatus
CommonProto::Serialise<BuildResultStatus>::read(const StoreDirConfig & store, CommonProto::ReadConn conn)
{
auto rawStatus = readNum<uint8_t>(conn.from);
if (rawStatus >= std::size(buildResultStatusTable))
throw Error("Invalid BuildResult status %d from remote", rawStatus);
return buildResultStatusTable[rawStatus];
}
void CommonProto::Serialise<BuildResultStatus>::write(
const StoreDirConfig & store, CommonProto::WriteConn conn, const BuildResultStatus & status)
{
for (auto && [wire, val] : enumerate(buildResultStatusTable))
if (val == status) {
conn.to << uint8_t(wire);
return;
}
unreachable();
}
} // namespace nix
+15
View File
@@ -531,6 +531,21 @@ struct curlFileTransfer : public FileTransfer
if (writtenToSink)
curl_easy_setopt(req, CURLOPT_RESUME_FROM_LARGE, writtenToSink);
/* Note that the underlying strings get copied by libcurl, so the path -> string conversion is ok:
> The application does not have to keep the string around after setting this option.
https://curl.se/libcurl/c/CURLOPT_SSLKEY.html
https://curl.se/libcurl/c/CURLOPT_SSLCERT.html */
if (request.tlsCert) {
curl_easy_setopt(req, CURLOPT_SSLCERTTYPE, "PEM");
curl_easy_setopt(req, CURLOPT_SSLCERT, request.tlsCert->string().c_str());
}
if (request.tlsKey) {
curl_easy_setopt(req, CURLOPT_SSLKEYTYPE, "PEM");
curl_easy_setopt(req, CURLOPT_SSLKEY, request.tlsKey->string().c_str());
}
curl_easy_setopt(req, CURLOPT_ERRORBUFFER, errbuf);
errbuf[0] = 0;
+3 -6
View File
@@ -7,6 +7,7 @@
#include "nix/util/signals.hh"
#include "nix/util/serialise.hh"
#include "nix/util/util.hh"
#include "nix/util/file-system.hh"
#include "nix/store/posix-fs-canonicalise.hh"
#include "store-config-private.hh"
@@ -68,9 +69,7 @@ void LocalStore::createTempRootsFile()
/* Check whether the garbage collector didn't get in our
way. */
struct stat st;
if (fstat(fromDescriptorReadOnly(fdTempRoots->get()), &st) == -1)
throw SysError("statting '%1%'", fnTempRoots);
auto st = nix::fstat(fromDescriptorReadOnly(fdTempRoots->get()));
if (st.st_size == 0)
break;
@@ -901,9 +900,7 @@ void LocalStore::collectGarbage(const GCOptions & options, GCResults & results)
accounting. */
}
struct stat st;
if (stat(linksDir.c_str(), &st) == -1)
throw SysError("statting '%1%'", linksDir);
auto st = stat(linksDir);
int64_t overhead =
#ifdef _WIN32
0
+5 -1
View File
@@ -69,6 +69,11 @@ static std::filesystem::path resolveNixConfDir()
#endif
}
LogFileSettings::LogFileSettings()
: nixLogDir(canonPath(getEnvNonEmpty("NIX_LOG_DIR").value_or(NIX_LOG_DIR)))
{
}
Settings settings;
static GlobalConfig::Register rSettings(&settings);
@@ -81,7 +86,6 @@ Settings::Settings()
canonPath
#endif
(getEnvNonEmpty("NIX_STORE_DIR").value_or(getEnvNonEmpty("NIX_STORE").value_or(NIX_STORE_DIR))))
, nixLogDir(canonPath(getEnvNonEmpty("NIX_LOG_DIR").value_or(NIX_LOG_DIR)))
, nixStateDir(canonPath(getEnvNonEmpty("NIX_STATE_DIR").value_or(NIX_STATE_DIR)))
, nixConfDir(canonPath(getEnvOsNonEmpty(OS_STR("NIX_CONF_DIR"))
.transform([](auto && s) { return std::filesystem::path(s); })
+17 -1
View File
@@ -195,7 +195,23 @@ FileTransferRequest HttpBinaryCacheStore::makeRequest(std::string_view path)
result.query = config->cacheUri.query;
}
return FileTransferRequest(result);
FileTransferRequest request(result);
/* Only use the specified SSL certificate and private key if the resolved URL names the same
authority and uses the same protocol. */
if (result.scheme == config->cacheUri.scheme && result.authority == config->cacheUri.authority) {
if (const auto & cert = config->tlsCert.get()) {
debug("using TLS client certificate %s for '%s'", PathFmt(*cert), request.uri);
request.tlsCert = *cert;
}
if (const auto & key = config->tlsKey.get()) {
debug("using TLS client key '%s' for '%s'", PathFmt(*key), request.uri);
request.tlsKey = *key;
}
}
return request;
}
void HttpBinaryCacheStore::getFile(const std::string & path, Sink & sink)
+45 -47
View File
@@ -11,24 +11,52 @@
namespace nix {
/**
* Names must be disjoint with `BuildResultFailureStatus`.
*
* @note Prefer using `BuildResult::Success::Status`, this name is just
* for sake of forward declarations.
*/
enum struct BuildResultSuccessStatus : uint8_t {
Built,
Substituted,
AlreadyValid,
ResolvesToAlreadyValid,
};
/**
* Names must be disjoint with `BuildResultSuccessStatus`.
*
* @note Prefer using `BuildResult::Failure::Status`, this name is just
* for sake of forward declarations.
*/
enum struct BuildResultFailureStatus : uint8_t {
PermanentFailure,
InputRejected,
OutputRejected,
/// possibly transient
TransientFailure,
/// no longer used
CachedFailure,
TimedOut,
MiscFailure,
DependencyFailed,
LogLimitExceeded,
NotDeterministic,
NoSubstituters,
/// A certain type of `OutputRejected`. The protocols do not yet
/// know about this one, so change it back to `OutputRejected`
/// before serialization.
HashMismatch,
};
struct BuildResult
{
struct Success
{
/**
* @note This is directly used in the nix-store --serve protocol.
* That means we need to worry about compatibility across versions.
* Therefore, don't remove status codes, and only add new status
* codes at the end of the list.
*
* Must be disjoint with `Failure::Status`.
*/
enum Status : uint8_t {
Built = 0,
Substituted = 1,
AlreadyValid = 2,
ResolvesToAlreadyValid = 13,
} status;
using Status = enum BuildResultSuccessStatus;
using enum Status;
Status status;
/**
* For derivations, a mapping from the names of the wanted outputs
@@ -38,43 +66,13 @@ struct BuildResult
bool operator==(const BuildResult::Success &) const noexcept;
std::strong_ordering operator<=>(const BuildResult::Success &) const noexcept;
static bool statusIs(uint8_t status)
{
return status == Built || status == Substituted || status == AlreadyValid
|| status == ResolvesToAlreadyValid;
}
};
struct Failure
{
/**
* @note This is directly used in the nix-store --serve protocol.
* That means we need to worry about compatibility across versions.
* Therefore, don't remove status codes, and only add new status
* codes at the end of the list.
*
* Must be disjoint with `Success::Status`.
*/
enum Status : uint8_t {
PermanentFailure = 3,
InputRejected = 4,
OutputRejected = 5,
/// possibly transient
TransientFailure = 6,
/// no longer used
CachedFailure = 7,
TimedOut = 8,
MiscFailure = 9,
DependencyFailed = 10,
LogLimitExceeded = 11,
NotDeterministic = 12,
NoSubstituters = 14,
/// A certain type of `OutputRejected`. The protocols do not yet
/// know about this one, so change it back to `OutputRejected`
/// before serialization.
HashMismatch = 15,
} status = MiscFailure;
using Status = enum BuildResultFailureStatus;
using enum Status;
Status status = MiscFailure;
/**
* Information about the error if the build failed.
@@ -190,15 +190,22 @@ struct ExternalBuilder
std::vector<std::string> args;
};
struct DerivationBuilderDeleter
{
void operator()(DerivationBuilder * builder) noexcept;
};
using DerivationBuilderUnique = std::unique_ptr<DerivationBuilder, DerivationBuilderDeleter>;
#ifndef _WIN32 // TODO enable `DerivationBuilder` on Windows
std::unique_ptr<DerivationBuilder> makeDerivationBuilder(
DerivationBuilderUnique makeDerivationBuilder(
LocalStore & store, std::unique_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params);
/**
* @param handler Must be chosen such that it supports the given
* derivation.
*/
std::unique_ptr<DerivationBuilder> makeExternalDerivationBuilder(
DerivationBuilderUnique makeExternalDerivationBuilder(
LocalStore & store,
std::unique_ptr<DerivationBuilderCallbacks> miscMethods,
DerivationBuilderParams params,
@@ -2,10 +2,10 @@
///@file
#include "nix/store/derivations.hh"
#include "nix/store/local-store.hh"
#include "nix/store/parsed-derivations.hh"
#include "nix/store/derivation-options.hh"
#include "nix/store/build/derivation-building-misc.hh"
#include "nix/store/outputs-spec.hh"
#include "nix/store/store-api.hh"
#include "nix/store/pathlocks.hh"
#include "nix/store/build/goal.hh"
@@ -75,6 +75,7 @@ private:
DerivationOptions<StorePath> drvOptions,
PathLocks outputLocks);
Co buildLocally(
LocalStore & localStore,
StorePathSet inputPaths,
std::map<std::string, InitialOutput> initialOutputs,
DerivationOptions<StorePath> drvOptions,
@@ -3,6 +3,8 @@
#include "nix/util/serialise.hh"
#include <variant>
namespace nix {
struct StoreDirConfig;
@@ -14,6 +16,8 @@ struct ContentAddress;
struct DrvOutput;
struct Realisation;
struct Signature;
enum struct BuildResultSuccessStatus : uint8_t;
enum struct BuildResultFailureStatus : uint8_t;
/**
* Shared serializers between the worker protocol, serve protocol, and a
@@ -86,7 +90,6 @@ DECLARE_COMMON_SERIALISER(std::tuple<Ts...>);
template<typename K, typename V>
DECLARE_COMMON_SERIALISER(std::map<K COMMA_ V>);
#undef COMMA_
/**
* These use the empty string for the null case, relying on the fact
@@ -107,4 +110,14 @@ DECLARE_COMMON_SERIALISER(std::optional<StorePath>);
template<>
DECLARE_COMMON_SERIALISER(std::optional<ContentAddress>);
/**
* The success and failure codes never overlay in enum tag values in the wire formats
*/
using BuildResultStatus = std::variant<BuildResultSuccessStatus, BuildResultFailureStatus>;
template<>
DECLARE_COMMON_SERIALISER(BuildResultStatus);
#undef COMMA_
} // namespace nix
@@ -121,6 +121,16 @@ struct FileTransferRequest
ActivityId parentAct;
bool decompress = true;
/**
* Optional path to the client certificate in "PEM" format. Only used for TLS-based protocols.
*/
std::optional<std::filesystem::path> tlsCert;
/**
* Optional path to the client private key in "PEM" format. Only used for TLS-based protocols.
*/
std::optional<std::filesystem::path> tlsKey;
struct UploadData
{
UploadData(StringSource & s)
+92 -81
View File
@@ -143,9 +143,66 @@ struct GCSettings : public virtual Config
};
};
class Settings : public virtual Config, private GCSettings
struct LogFileSettings : public virtual Config
{
/**
* The directory where we log various operations.
*/
const Path nixLogDir;
protected:
LogFileSettings();
public:
Setting<bool> keepLog{
this,
true,
"keep-build-log",
R"(
If set to `true` (the default), Nix writes the build log of a
derivation (i.e. the standard output and error of its builder) to
the directory `/nix/var/log/nix/drvs`. The build log can be
retrieved using the command `nix-store -l path`.
)",
{"build-keep-log"}};
Setting<bool> compressLog{
this,
true,
"compress-build-log",
R"(
If set to `true` (the default), build logs written to
`/nix/var/log/nix/drvs` are compressed on the fly using bzip2.
Otherwise, they are not compressed.
)",
{"build-compress-log"}};
};
struct AutoAllocateUidSettings : public virtual Config
{
Setting<uint32_t> startId{
this,
#ifdef __linux__
0x34000000,
#else
56930,
#endif
"start-id",
"The first UID and GID to use for dynamic ID allocation."};
Setting<uint32_t> uidCount{
this,
#ifdef __linux__
maxIdsPerBuild * 128,
#else
128,
#endif
"id-count",
"The number of UIDs/GIDs to use for dynamic ID allocation."};
};
class Settings : public virtual Config, private AutoAllocateUidSettings, private GCSettings, private LogFileSettings
{
StringSet getDefaultSystemFeatures();
StringSet getDefaultExtraPlatforms();
@@ -171,6 +228,28 @@ public:
return *this;
}
/**
* Get the log file settings.
*/
LogFileSettings & getLogFileSettings()
{
return *this;
}
const LogFileSettings & getLogFileSettings() const
{
return *this;
}
/**
* Get AutoAllocateUidSettings if auto-allocate-uids is enabled.
* @return Pointer to settings if enabled, nullptr otherwise.
*/
const AutoAllocateUidSettings * getAutoAllocateUidSettings() const
{
return autoAllocateUids ? this : nullptr;
}
static unsigned int getDefaultCores();
/**
@@ -178,11 +257,6 @@ public:
*/
Path nixStore;
/**
* The directory where we log various operations.
*/
Path nixLogDir;
/**
* The directory where state is stored.
*/
@@ -615,26 +689,6 @@ public:
true,
Xp::AutoAllocateUids};
Setting<uint32_t> startId{
this,
#ifdef __linux__
0x34000000,
#else
56930,
#endif
"start-id",
"The first UID and GID to use for dynamic ID allocation."};
Setting<uint32_t> uidCount{
this,
#ifdef __linux__
maxIdsPerBuild * 128,
#else
128,
#endif
"id-count",
"The number of UIDs/GIDs to use for dynamic ID allocation."};
#ifdef __linux__
Setting<bool> useCgroups{
this,
@@ -656,29 +710,6 @@ public:
"Whether to impersonate a Linux 2.6 machine on newer kernels.",
{"build-impersonate-linux-26"}};
Setting<bool> keepLog{
this,
true,
"keep-build-log",
R"(
If set to `true` (the default), Nix writes the build log of a
derivation (i.e. the standard output and error of its builder) to
the directory `/nix/var/log/nix/drvs`. The build log can be
retrieved using the command `nix-store -l path`.
)",
{"build-keep-log"}};
Setting<bool> compressLog{
this,
true,
"compress-build-log",
R"(
If set to `true` (the default), build logs written to
`/nix/var/log/nix/drvs` are compressed on the fly using bzip2.
Otherwise, they are not compressed.
)",
{"build-compress-log"}};
Setting<unsigned long> maxLogSize{
this,
0,
@@ -704,29 +735,6 @@ public:
duplicate files.
)"};
Setting<bool> envKeepDerivations{
this,
false,
"keep-env-derivations",
R"(
If `false` (default), derivations are not stored in Nix user
environments. That is, the derivations of any build-time-only
dependencies may be garbage-collected.
If `true`, when you add a Nix derivation to a user environment, the
path of the derivation is stored in the user environment. Thus, the
derivation isn't garbage-collected until the user environment
generation is deleted (`nix-env --delete-generations`). To prevent
build-time-only dependencies from being collected, you should also
turn on `keep-outputs`.
The difference between this option and `keep-derivations` is that
this one is “sticky”: it applies to any user environment created
while this option was enabled, while `keep-derivations` only applies
at the moment the garbage collector is run.
)",
{"env-keep-derivations"}};
Setting<SandboxMode> sandboxMode{
this,
#ifdef __linux__
@@ -875,6 +883,8 @@ public:
line.
)"};
private:
OptionalPathSetting diffHook{
this,
std::nullopt,
@@ -907,6 +917,16 @@ public:
configuration file, and cannot be passed at the command line.
)"};
public:
const Path * getDiffHook() const
{
if (!runDiffHook.get()) {
return nullptr;
}
return get(diffHook.get());
}
Setting<Strings> trustedPublicKeys{
this,
{"cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="},
@@ -1380,15 +1400,6 @@ public:
true, // document default
Xp::ConfigurableImpureEnv};
Setting<std::string> upgradeNixStorePathUrl{
this,
"https://github.com/NixOS/nixpkgs/raw/master/nixos/modules/installer/tools/nix-fallback-paths.nix",
"upgrade-nix-store-path-url",
R"(
Used by `nix upgrade-nix`, the URL of the file that contains the
store paths of the latest Nix release.
)"};
Setting<uint64_t> warnLargePathThreshold{
this,
0,
@@ -37,6 +37,12 @@ struct HttpBinaryCacheStoreConfig : std::enable_shared_from_this<HttpBinaryCache
(e.g. `brotli`).
)"};
const Setting<std::optional<std::filesystem::path>> tlsCert{
this, std::nullopt, "tls-certificate", "Path to an optional TLS client certificate in PEM format."};
const Setting<std::optional<std::filesystem::path>> tlsKey{
this, std::nullopt, "tls-private-key", "Path to an optional TLS client certificate private key in PEM format."};
static const std::string name()
{
return "HTTP Binary Cache Store";
@@ -3,6 +3,7 @@
#include "nix/util/source-accessor.hh"
#include "nix/util/ref.hh"
#include "nix/util/nar-cache.hh"
#include "nix/store/store-api.hh"
namespace nix {
@@ -13,20 +14,15 @@ class RemoteFSAccessor : public SourceAccessor
/**
* Map from store path hash part to NAR hash. Used to then look up
* in `nars`. The indirection allows avoiding opening multiple
* in the NAR cache. The indirection allows avoiding opening multiple
* redundant NAR accessors for the same NAR.
*/
std::map<std::string, Hash, std::less<>> narHashes;
/**
* Map from NAR hash to NAR accessor.
*/
std::map<Hash, ref<SourceAccessor>> nars;
NarCache narCache;
bool requireValidPath;
std::optional<std::filesystem::path> cacheDir;
std::pair<ref<SourceAccessor>, CanonPath> fetch(const CanonPath & path);
friend struct BinaryCacheStore;
+6 -7
View File
@@ -3,6 +3,7 @@
#include "nix/util/archive.hh"
#include "nix/util/pool.hh"
#include "nix/store/remote-store.hh"
#include "nix/store/common-protocol.hh"
#include "nix/store/serve-protocol.hh"
#include "nix/store/serve-protocol-connection.hh"
#include "nix/store/serve-protocol-impl.hh"
@@ -243,13 +244,11 @@ void LegacySSHStore::buildPaths(
conn->to.flush();
auto status = readInt(conn->from);
if (!BuildResult::Success::statusIs(status)) {
BuildResult::Failure failure{
.status = (BuildResult::Failure::Status) status,
};
conn->from >> failure.errorMsg;
throw Error(failure.status, std::move(failure.errorMsg));
auto status = CommonProto::Serialise<BuildResultStatus>::read(*this, {conn->from});
if (auto * failure = std::get_if<BuildResultFailureStatus>(&status)) {
std::string errorMsg;
conn->from >> errorMsg;
throw BuildError(*failure, std::move(errorMsg));
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ Path LocalFSStoreConfig::getDefaultStateDir()
Path LocalFSStoreConfig::getDefaultLogDir()
{
return settings.nixLogDir;
return settings.getLogFileSettings().nixLogDir;
}
LocalFSStoreConfig::LocalFSStoreConfig(PathView rootDir, const Params & params)
+4 -7
View File
@@ -167,15 +167,12 @@ LocalStore::LocalStore(ref<const Config> config)
printError(
"warning: the group '%1%' specified in 'build-users-group' does not exist", settings.buildUsersGroup);
else if (!config->readOnly) {
struct stat st;
if (stat(config->realStoreDir.get().c_str(), &st))
throw SysError("getting attributes of path '%1%'", config->realStoreDir);
auto st = stat(config->realStoreDir.get());
if (st.st_uid != 0 || st.st_gid != gr->gr_gid || (st.st_mode & ~S_IFMT) != perm) {
if (chown(config->realStoreDir.get().c_str(), 0, gr->gr_gid) == -1)
throw SysError("changing ownership of path '%1%'", config->realStoreDir);
if (chmod(config->realStoreDir.get().c_str(), perm) == -1)
throw SysError("changing permissions on path '%1%'", config->realStoreDir);
chmod(config->realStoreDir.get(), perm);
}
}
}
@@ -200,8 +197,8 @@ LocalStore::LocalStore(ref<const Config> config)
needed, we reserve some dummy space that we can free just
before doing a garbage collection. */
try {
struct stat st;
if (stat(reservedPath.c_str(), &st) == -1 || st.st_size != gcSettings.reservedSize) {
auto st = maybeStat(reservedPath);
if (!st || st->st_size != gcSettings.reservedSize) {
AutoCloseFD fd = toDescriptor(open(
reservedPath.c_str(),
O_WRONLY | O_CREAT
+6 -5
View File
@@ -158,14 +158,15 @@ sqlite = dependency('sqlite3', 'sqlite', version : '>=3.6.19')
deps_private += sqlite
s3_aws_auth = get_option('s3-aws-auth')
aws_crt_cpp = cxx.find_library('aws-crt-cpp', required : s3_aws_auth)
aws_crt_cpp = dependency(
'aws-crt-cpp',
required : s3_aws_auth,
method : 'cmake',
modules : [ 'AWS::aws-crt-cpp' ],
)
if s3_aws_auth.enabled()
deps_other += aws_crt_cpp
aws_c_common = cxx.find_library('aws-c-common', required : true)
deps_other += aws_c_common
aws_c_auth = cxx.find_library('aws-c-auth', required : true)
deps_other += aws_c_auth
endif
configdata_pub.set('NIX_WITH_AWS_AUTH', s3_aws_auth.enabled().to_int())
+4 -4
View File
@@ -3,6 +3,7 @@
#include "nix/util/signals.hh"
#include "nix/store/posix-fs-canonicalise.hh"
#include "nix/util/posix-source-accessor.hh"
#include "nix/util/file-system.hh"
#include <cstdlib>
#include <cstring>
@@ -20,8 +21,7 @@ namespace nix {
static void makeWritable(const Path & path)
{
auto st = lstat(path);
if (chmod(path.c_str(), st.st_mode | S_IWUSR) == -1)
throw SysError("changing writability of '%1%'", path);
chmod(path, st.st_mode | S_IWUSR);
}
struct MakeReadOnly
@@ -169,7 +169,7 @@ void LocalStore::optimisePath_(
/* Maybe delete the link, if it has been corrupted. */
if (std::filesystem::exists(std::filesystem::symlink_status(linkPath))) {
auto stLink = lstat(linkPath.string());
auto stLink = lstat(linkPath);
if (st.st_size != stLink.st_size || (repair && hash != ({
hashPath(
makeFSSourceAccessor(linkPath),
@@ -215,7 +215,7 @@ void LocalStore::optimisePath_(
/* Yes! We've seen a file with the same contents. Replace the
current file with a hard link to that file. */
auto stLink = lstat(linkPath.string());
auto stLink = lstat(linkPath);
if (st.st_ino == stLink.st_ino) {
debug("%1% is already linked to %2%", PathFmt(path), PathFmt(linkPath));
+4 -2
View File
@@ -14,6 +14,7 @@
libseccomp,
nlohmann_json,
sqlite,
cmake, # for resolving aws-crt-cpp dep
busybox-sandbox-shell ? null,
@@ -25,7 +26,7 @@
withAWS ?
# Default is this way because there have been issues building this dependency
lib.meta.availableOn stdenv.hostPlatform aws-c-common,
(lib.meta.availableOn stdenv.hostPlatform aws-c-common),
}:
let
@@ -57,7 +58,8 @@ mkMesonLibrary (finalAttrs: {
(fileset.fileFilter (file: file.hasExt "sql") ./.)
];
nativeBuildInputs = lib.optional embeddedSandboxShell unixtools.hexdump;
nativeBuildInputs =
lib.optional withAWS cmake ++ lib.optional embeddedSandboxShell unixtools.hexdump;
buildInputs = [
boost
+3 -4
View File
@@ -15,7 +15,7 @@ namespace nix {
const time_t mtimeStore = 1; /* 1 second into the epoch */
static void canonicaliseTimestampAndPermissions(const Path & path, const struct stat & st)
static void canonicaliseTimestampAndPermissions(const Path & path, const PosixStat & st)
{
if (!S_ISLNK(st.st_mode)) {
@@ -24,14 +24,13 @@ static void canonicaliseTimestampAndPermissions(const Path & path, const struct
bool isDir = S_ISDIR(st.st_mode);
if ((mode != 0444 || isDir) && mode != 0555) {
mode = (st.st_mode & S_IFMT) | 0444 | (st.st_mode & S_IXUSR || isDir ? 0111 : 0);
if (chmod(path.c_str(), mode) == -1)
throw SysError("changing mode of '%1%' to %2$o", path, mode);
chmod(path, mode);
}
}
#ifndef _WIN32 // TODO implement
if (st.st_mtime != mtimeStore) {
struct stat st2 = st;
PosixStat st2 = st;
st2.st_mtime = mtimeStore, setWriteTime(path, st2);
}
#endif
+10 -72
View File
@@ -1,21 +1,13 @@
#include <nlohmann/json.hpp>
#include "nix/store/remote-fs-accessor.hh"
#include "nix/util/nar-accessor.hh"
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
namespace nix {
RemoteFSAccessor::RemoteFSAccessor(
ref<Store> store, bool requireValidPath, std::optional<std::filesystem::path> cacheDir_)
ref<Store> store, bool requireValidPath, std::optional<std::filesystem::path> cacheDir)
: store(store)
, narCache(cacheDir)
, requireValidPath(requireValidPath)
, cacheDir(std::move(cacheDir_))
{
if (cacheDir)
createDirs(*cacheDir);
}
std::pair<ref<SourceAccessor>, CanonPath> RemoteFSAccessor::fetch(const CanonPath & path)
@@ -28,72 +20,18 @@ std::pair<ref<SourceAccessor>, CanonPath> RemoteFSAccessor::fetch(const CanonPat
std::shared_ptr<SourceAccessor> RemoteFSAccessor::accessObject(const StorePath & storePath)
{
if (auto * narHash = get(narHashes, storePath.hashPart())) {
if (auto * accessor = get(nars, *narHash))
return *accessor;
}
// Check if we already have the NAR hash for this store path
if (auto * narHash = get(narHashes, storePath.hashPart()))
return narCache.getOrInsert(*narHash, [&](Sink & sink) { store->narFromPath(storePath, sink); });
// Query the path info to get the NAR hash
auto info = store->queryPathInfo(storePath);
auto cacheAccessor = [&](ref<SourceAccessor> accessor) {
narHashes.emplace(storePath.hashPart(), info->narHash);
nars.emplace(info->narHash, accessor);
return accessor;
};
// Cache the mapping from store path to NAR hash
narHashes.emplace(storePath.hashPart(), info->narHash);
auto getNar = [&]() {
StringSink sink;
store->narFromPath(storePath, sink);
return std::move(sink.s);
};
if (cacheDir) {
auto makeCacheFile = [&](const std::string & ext) {
auto res = *cacheDir / info->narHash.to_string(HashFormat::Nix32, false);
res += ".";
res += ext;
return res;
};
auto cacheFile = makeCacheFile("nar");
auto listingFile = makeCacheFile("ls");
if (nix::pathExists(cacheFile)) {
try {
return cacheAccessor(makeLazyNarAccessor(
nlohmann::json::parse(nix::readFile(listingFile)).template get<NarListing>(),
seekableGetNarBytes(cacheFile)));
} catch (SystemError &) {
}
try {
return cacheAccessor(makeNarAccessor(nix::readFile(cacheFile)));
} catch (SystemError &) {
}
}
auto nar = getNar();
try {
/* FIXME: do this asynchronously. */
writeFile(cacheFile, nar);
} catch (...) {
ignoreExceptionExceptInterrupt();
}
auto narAccessor = makeNarAccessor(std::move(nar));
try {
nlohmann::json j = narAccessor->getListing();
writeFile(listingFile, j.dump());
} catch (...) {
ignoreExceptionExceptInterrupt();
}
return cacheAccessor(narAccessor);
}
return cacheAccessor(makeNarAccessor(getNar()));
// Get or create the NAR accessor
return narCache.getOrInsert(info->narHash, [&](Sink & sink) { store->narFromPath(storePath, sink); });
}
std::optional<SourceAccessor::Stat> RemoteFSAccessor::maybeLstat(const CanonPath & path)
+19 -13
View File
@@ -2,6 +2,7 @@
#include "nix/store/path-with-outputs.hh"
#include "nix/store/store-api.hh"
#include "nix/store/build-result.hh"
#include "nix/store/common-protocol.hh"
#include "nix/store/serve-protocol.hh"
#include "nix/store/serve-protocol-impl.hh"
#include "nix/util/archive.hh"
@@ -15,30 +16,35 @@ namespace nix {
BuildResult ServeProto::Serialise<BuildResult>::read(const StoreDirConfig & store, ServeProto::ReadConn conn)
{
BuildResult status;
BuildResult res;
BuildResult::Success success;
BuildResult::Failure failure;
auto rawStatus = readInt(conn.from);
auto status = ServeProto::Serialise<BuildResultStatus>::read(store, {conn.from});
conn.from >> failure.errorMsg;
if (GET_PROTOCOL_MINOR(conn.version) >= 3)
conn.from >> status.timesBuilt >> failure.isNonDeterministic >> status.startTime >> status.stopTime;
conn.from >> res.timesBuilt >> failure.isNonDeterministic >> res.startTime >> res.stopTime;
if (GET_PROTOCOL_MINOR(conn.version) >= 6) {
auto builtOutputs = ServeProto::Serialise<DrvOutputs>::read(store, conn);
for (auto && [output, realisation] : builtOutputs)
success.builtOutputs.insert_or_assign(std::move(output.outputName), std::move(realisation));
}
if (BuildResult::Success::statusIs(rawStatus)) {
success.status = static_cast<BuildResult::Success::Status>(rawStatus);
status.inner = std::move(success);
} else {
failure.status = static_cast<BuildResult::Failure::Status>(rawStatus);
status.inner = std::move(failure);
}
res.inner = std::visit(
overloaded{
[&](BuildResult::Success::Status s) -> decltype(res.inner) {
success.status = s;
return std::move(success);
},
[&](BuildResult::Failure::Status s) -> decltype(res.inner) {
failure.status = s;
return std::move(failure);
},
},
status);
return status;
return res;
}
void ServeProto::Serialise<BuildResult>::write(
@@ -63,11 +69,11 @@ void ServeProto::Serialise<BuildResult>::write(
std::visit(
overloaded{
[&](const BuildResult::Failure & failure) {
conn.to << failure.status;
ServeProto::write(store, {conn.to}, BuildResultStatus{failure.status});
common(failure.errorMsg, failure.isNonDeterministic, decltype(BuildResult::Success::builtOutputs){});
},
[&](const BuildResult::Success & success) {
conn.to << success.status;
ServeProto::write(store, {conn.to}, BuildResultStatus{success.status});
common(/*errorMsg=*/"", /*isNonDeterministic=*/false, success.builtOutputs);
},
},
@@ -85,7 +85,7 @@ struct ChrootDerivationBuilder : virtual DerivationBuilderImpl
instead.) */
std::filesystem::path chrootTmpDir = chrootRootDir / "tmp";
createDirs(chrootTmpDir);
chmod_(chrootTmpDir, 01777);
chmod(chrootTmpDir, 01777);
/* Create a /etc/passwd with entries for the build user and the
nobody account. The latter is kind of a hack to support
@@ -119,7 +119,7 @@ struct ChrootDerivationBuilder : virtual DerivationBuilderImpl
build user. */
std::filesystem::path chrootStoreDir = chrootRootDir / std::filesystem::path(store.storeDir).relative_path();
createDirs(chrootStoreDir);
chmod_(chrootStoreDir, 01775);
chmod(chrootStoreDir, 01775);
if (buildUser && chown(chrootStoreDir.c_str(), 0, buildUser->getGID()) == -1)
throw SysError("cannot change ownership of %1%", PathFmt(chrootStoreDir));
+66 -54
View File
@@ -98,10 +98,13 @@ public:
{
}
~DerivationBuilderImpl()
/**
* Cleanup code to run when destroying any DerivationBuilderImpl implementation.
*/
void cleanupOnDestruction() noexcept
{
/* Careful: we should never ever throw an exception from a
destructor. */
noexcept function. */
try {
killChild();
} catch (...) {
@@ -232,7 +235,7 @@ protected:
*/
virtual std::unique_ptr<UserLock> getBuildUser()
{
return acquireUserLock(1, false);
return acquireUserLock(settings.buildUsersGroup, 1, false);
}
/**
@@ -436,7 +439,8 @@ private:
StorePath makeFallbackPath(OutputNameView outputName);
};
void handleDiffHook(
static void handleDiffHook(
const Path & diffHook,
uid_t uid,
uid_t gid,
const std::filesystem::path & tryA,
@@ -444,29 +448,25 @@ void handleDiffHook(
const std::filesystem::path & drvPath,
const std::filesystem::path & tmpDir)
{
auto & diffHookOpt = settings.diffHook.get();
if (diffHookOpt && settings.runDiffHook) {
auto & diffHook = *diffHookOpt;
try {
auto diffRes = runProgram(
RunOptions{
.program = diffHook,
.lookupPath = true,
.args = {tryA, tryB, drvPath, tmpDir},
.uid = uid,
.gid = gid,
.chdir = "/"});
if (!statusOk(diffRes.first))
throw ExecError(diffRes.first, "diff-hook program '%1%' %2%", diffHook, statusToString(diffRes.first));
try {
auto diffRes = runProgram(
RunOptions{
.program = diffHook,
.lookupPath = true,
.args = {tryA, tryB, drvPath, tmpDir},
.uid = uid,
.gid = gid,
.chdir = "/"});
if (!statusOk(diffRes.first))
throw ExecError(diffRes.first, "diff-hook program '%1%' %2%", diffHook, statusToString(diffRes.first));
if (diffRes.second != "")
printError(chomp(diffRes.second));
} catch (Error & error) {
ErrorInfo ei = error.info();
// FIXME: wrap errors.
ei.msg = HintFmt("diff hook execution failed: %s", ei.msg.str());
logError(ei);
}
if (diffRes.second != "")
printError(chomp(diffRes.second));
} catch (Error & error) {
ErrorInfo ei = error.info();
// FIXME: wrap errors.
ei.msg = HintFmt("diff hook execution failed: %s", ei.msg.str());
logError(ei);
}
}
@@ -567,12 +567,6 @@ SingleDrvOutputs DerivationBuilderImpl::unprepareBuild()
return builtOutputs;
}
static void chmod_(const std::filesystem::path & path, mode_t mode)
{
if (chmod(path.c_str(), mode) == -1)
throw SysError("setting permissions on %s", PathFmt(path));
}
/* Move/rename path 'src' to 'dst'. Temporarily make 'src' writable if
it's a directory and we're not root (to be able to update the
directory's parent link ".."). */
@@ -583,12 +577,12 @@ static void movePath(const std::filesystem::path & src, const std::filesystem::p
bool changePerm = (geteuid() && S_ISDIR(st.st_mode) && !(st.st_mode & S_IWUSR));
if (changePerm)
chmod_(src, st.st_mode | S_IWUSR);
chmod(src, st.st_mode | S_IWUSR);
std::filesystem::rename(src, dst);
if (changePerm)
chmod_(dst, st.st_mode);
chmod(dst, st.st_mode);
}
static void replaceValidPath(const std::filesystem::path & storePath, const std::filesystem::path & tmpPath)
@@ -821,8 +815,7 @@ std::optional<Descriptor> DerivationBuilderImpl::startBuild()
std::string slaveName = getPtsName(builderOut.get());
if (buildUser) {
if (chmod(slaveName.c_str(), 0600))
throw SysError("changing mode of pseudoterminal slave");
chmod(slaveName, 0600);
if (chown(slaveName.c_str(), buildUser->getUID(), 0))
throw SysError("changing owner of pseudoterminal slave");
@@ -1416,7 +1409,7 @@ SingleDrvOutputs DerivationBuilderImpl::registerOutputs()
scratchOutputsInverse.insert_or_assign(path, outputName);
std::map<std::string, std::variant<AlreadyRegistered, PerhapsNeedToRegister>> outputReferencesIfUnregistered;
std::map<std::string, struct stat> outputStats;
std::map<std::string, PosixStat> outputStats;
for (auto & [outputName, _] : drv.outputs) {
auto scratchOutput = get(scratchOutputs, outputName);
assert(scratchOutput);
@@ -1445,7 +1438,7 @@ SingleDrvOutputs DerivationBuilderImpl::registerOutputs()
store.printStorePath(drvPath),
outputName,
PathFmt(actualPath));
struct stat & st = *optSt;
PosixStat & st = *optSt;
#ifndef __CYGWIN__
/* Check that the output is not group or world writable, as
@@ -1789,18 +1782,22 @@ SingleDrvOutputs DerivationBuilderImpl::registerOutputs()
if (store.isValidPath(newInfo.path)) {
ValidPathInfo oldInfo(*store.queryPathInfo(newInfo.path));
if (newInfo.narHash != oldInfo.narHash) {
if (settings.runDiffHook || settings.keepFailed) {
auto * diffHook = settings.getDiffHook();
if (diffHook || settings.keepFailed) {
auto dst = store.toRealPath(finalDestPath + ".check");
deletePath(dst);
movePath(actualPath, dst);
handleDiffHook(
buildUser ? buildUser->getUID() : getuid(),
buildUser ? buildUser->getGID() : getgid(),
finalDestPath,
dst,
store.printStorePath(drvPath),
tmpDir);
if (diffHook) {
handleDiffHook(
*diffHook,
buildUser ? buildUser->getUID() : getuid(),
buildUser ? buildUser->getGID() : getgid(),
finalDestPath,
dst,
store.printStorePath(drvPath),
tmpDir);
}
throw NotDeterministic(
"derivation '%s' may not be deterministic: output '%s' differs from '%s'",
@@ -1923,14 +1920,14 @@ void DerivationBuilderImpl::cleanupBuild(bool force)
* This hardens against an attack which smuggles a file descriptor
* to make use of the temporary directory.
*/
chmod(topTmpDir.c_str(), 0000);
chmod(topTmpDir, 0000);
/* Don't keep temporary directories for builtins because they
might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv.isBuiltin()) {
printError("note: keeping build directory %s", PathFmt(tmpDir));
chmod(topTmpDir.c_str(), 0755);
chmod(tmpDir.c_str(), 0755);
chmod(topTmpDir, 0755);
chmod(tmpDir, 0755);
} else
deletePath(topTmpDir);
topTmpDir = "";
@@ -1974,7 +1971,20 @@ StorePath DerivationBuilderImpl::makeFallbackPath(const StorePath & path)
namespace nix {
std::unique_ptr<DerivationBuilder> makeDerivationBuilder(
void DerivationBuilderDeleter::operator()(DerivationBuilder * builder) noexcept
{
if (!builder) /* Idempotent and handles nullptr as any deleter must. */
return;
if (auto builderImpl = dynamic_cast<DerivationBuilderImpl *>(builder))
/* Note that this might call into virtual functions, which we can't do in a destructor of
the DerivationBuilderImpl itself. */
builderImpl->cleanupOnDestruction();
delete builder;
}
std::unique_ptr<DerivationBuilder, DerivationBuilderDeleter> makeDerivationBuilder(
LocalStore & store, std::unique_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params)
{
bool useSandbox = false;
@@ -2025,17 +2035,19 @@ std::unique_ptr<DerivationBuilder> makeDerivationBuilder(
throw Error("feature 'uid-range' is only supported in sandboxed builds");
#ifdef __APPLE__
return std::make_unique<DarwinDerivationBuilder>(store, std::move(miscMethods), std::move(params), useSandbox);
return DerivationBuilderUnique(
new DarwinDerivationBuilder(store, std::move(miscMethods), std::move(params), useSandbox));
#elif defined(__linux__)
if (useSandbox)
return std::make_unique<ChrootLinuxDerivationBuilder>(store, std::move(miscMethods), std::move(params));
return DerivationBuilderUnique(
new ChrootLinuxDerivationBuilder(store, std::move(miscMethods), std::move(params)));
return std::make_unique<LinuxDerivationBuilder>(store, std::move(miscMethods), std::move(params));
return DerivationBuilderUnique(new LinuxDerivationBuilder(store, std::move(miscMethods), std::move(params)));
#else
if (useSandbox)
throw Error("sandboxing builds is not supported on this platform");
return std::make_unique<DerivationBuilderImpl>(store, std::move(miscMethods), std::move(params));
return DerivationBuilderUnique(new DerivationBuilderImpl(store, std::move(miscMethods), std::move(params)));
#endif
}
@@ -106,13 +106,14 @@ struct ExternalDerivationBuilder : DerivationBuilderImpl
}
};
std::unique_ptr<DerivationBuilder> makeExternalDerivationBuilder(
DerivationBuilderUnique makeExternalDerivationBuilder(
LocalStore & store,
std::unique_ptr<DerivationBuilderCallbacks> miscMethods,
DerivationBuilderParams params,
const ExternalBuilder & handler)
{
return std::make_unique<ExternalDerivationBuilder>(store, std::move(miscMethods), std::move(params), handler);
return DerivationBuilderUnique(
new ExternalDerivationBuilder(store, std::move(miscMethods), std::move(params), handler));
}
} // namespace nix
@@ -215,7 +215,7 @@ struct ChrootLinuxDerivationBuilder : ChrootDerivationBuilder, LinuxDerivationBu
std::unique_ptr<UserLock> getBuildUser() override
{
return acquireUserLock(drvOptions.useUidRange(drv) ? 65536 : 1, true);
return acquireUserLock(settings.buildUsersGroup, drvOptions.useUidRange(drv) ? 65536 : 1, true);
}
void prepareUser() override
@@ -586,7 +586,7 @@ struct ChrootLinuxDerivationBuilder : ChrootDerivationBuilder, LinuxDerivationBu
auto dst = chrootRootDir / i.first.relative_path();
createDirs(dst.parent_path());
writeFile(dst, std::string_view((const char *) sh, sizeof(sh)));
chmod_(dst, 0555);
chmod(dst, 0555);
} else
# endif
{
@@ -629,7 +629,7 @@ struct ChrootLinuxDerivationBuilder : ChrootDerivationBuilder, LinuxDerivationBu
/* Make sure /dev/pts/ptmx is world-writable. With some
Linux versions, it is created with permissions 0. */
chmod_(chrootRootDir / "dev" / "pts" / "ptmx", 0666);
chmod(chrootRootDir / "dev" / "pts" / "ptmx", 0666);
} else {
if (errno != EINVAL)
throw SysError("mounting /dev/pts");
@@ -640,7 +640,7 @@ struct ChrootLinuxDerivationBuilder : ChrootDerivationBuilder, LinuxDerivationBu
/* Make /etc unwritable */
if (!drvOptions.useUidRange(drv))
chmod_(chrootRootDir / "etc", 0555);
chmod(chrootRootDir / "etc", 0555);
/* Unshare this mount namespace. This is necessary because
pivot_root() below changes the root of the mount
@@ -36,7 +36,7 @@ struct UserLock
* Acquire a user lock for a UID range of size `nrIds`. Note that this
* may return nullptr if no user is available.
*/
std::unique_ptr<UserLock> acquireUserLock(uid_t nrIds, bool useUserNamespace);
std::unique_ptr<UserLock> acquireUserLock(const std::string & userGroup, uid_t nrIds, bool useUserNamespace);
bool useBuildUsers();
+2 -3
View File
@@ -1,4 +1,5 @@
#include "nix/store/pathlocks.hh"
#include "nix/util/file-system.hh"
#include "nix/util/util.hh"
#include "nix/util/sync.hh"
#include "nix/util/signals.hh"
@@ -110,9 +111,7 @@ bool PathLocks::lockPaths(const std::set<std::filesystem::path> & paths, const s
/* Check that the lock file hasn't become stale (i.e.,
hasn't been unlinked). */
struct stat st;
if (fstat(fd.get(), &st) == -1)
throw SysError("statting lock file %1%", PathFmt(lockPath));
auto st = nix::fstat(fd.get());
if (st.st_size != 0)
/* This lock file has been unlinked, so we're holding
a lock on a deleted file. This means that other
+35 -30
View File
@@ -64,15 +64,15 @@ struct SimpleUserLock : UserLock
return supplementaryGIDs;
}
static std::unique_ptr<UserLock> acquire()
static std::unique_ptr<UserLock>
acquire(const std::filesystem::path & userPoolDir, const std::string & buildUsersGroup)
{
assert(settings.buildUsersGroup != "");
createDirs(settings.nixStateDir + "/userpool");
assert(buildUsersGroup != "");
/* Get the members of the build-users-group. */
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
struct group * gr = getgrnam(buildUsersGroup.c_str());
if (!gr)
throw Error("the group '%s' specified in 'build-users-group' does not exist", settings.buildUsersGroup);
throw Error("the group '%s' specified in 'build-users-group' does not exist", buildUsersGroup);
/* Copy the result of getgrnam. */
Strings users;
@@ -82,7 +82,7 @@ struct SimpleUserLock : UserLock
}
if (users.empty())
throw Error("the build users group '%s' has no members", settings.buildUsersGroup);
throw Error("the build users group '%s' has no members", buildUsersGroup);
/* Find a user account that isn't currently in use for another
build. */
@@ -91,13 +91,13 @@ struct SimpleUserLock : UserLock
struct passwd * pw = getpwnam(i.c_str());
if (!pw)
throw Error("the user '%s' in the group '%s' does not exist", i, settings.buildUsersGroup);
throw Error("the user '%s' in the group '%s' does not exist", i, buildUsersGroup);
auto fnUserLock = fmt("%s/userpool/%s", settings.nixStateDir, pw->pw_uid);
auto fnUserLock = userPoolDir / std::to_string(pw->pw_uid);
AutoCloseFD fd = open(fnUserLock.c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (!fd)
throw SysError("opening user lock '%s'", fnUserLock);
throw SysError("opening user lock %s", PathFmt(fnUserLock));
if (lockFile(fd.get(), ltWrite, false)) {
auto lock = std::make_unique<SimpleUserLock>();
@@ -108,7 +108,7 @@ struct SimpleUserLock : UserLock
/* Sanity check... */
if (lock->uid == getuid() || lock->uid == geteuid())
throw Error("the Nix user should not be a member of '%s'", settings.buildUsersGroup);
throw Error("the Nix user should not be a member of '%s'", buildUsersGroup);
#ifdef __linux__
/* Get the list of supplementary groups of this user. This is
@@ -158,36 +158,37 @@ struct AutoUserLock : UserLock
return {};
}
static std::unique_ptr<UserLock> acquire(uid_t nrIds, bool useUserNamespace)
static std::unique_ptr<UserLock> acquire(
const std::filesystem::path & userPoolDir,
const std::string & buildUsersGroup,
uid_t nrIds,
bool useUserNamespace,
const AutoAllocateUidSettings & uidSettings)
{
#if !defined(__linux__)
useUserNamespace = false;
#endif
experimentalFeatureSettings.require(Xp::AutoAllocateUids);
assert(settings.startId > 0);
assert(settings.uidCount % maxIdsPerBuild == 0);
assert((uint64_t) settings.startId + (uint64_t) settings.uidCount <= std::numeric_limits<uid_t>::max());
assert(uidSettings.startId > 0);
assert(uidSettings.uidCount % maxIdsPerBuild == 0);
assert((uint64_t) uidSettings.startId + (uint64_t) uidSettings.uidCount <= std::numeric_limits<uid_t>::max());
assert(nrIds <= maxIdsPerBuild);
createDirs(settings.nixStateDir + "/userpool2");
size_t nrSlots = settings.uidCount / maxIdsPerBuild;
size_t nrSlots = uidSettings.uidCount / maxIdsPerBuild;
for (size_t i = 0; i < nrSlots; i++) {
debug("trying user slot '%d'", i);
createDirs(settings.nixStateDir + "/userpool2");
auto fnUserLock = fmt("%s/userpool2/slot-%d", settings.nixStateDir, i);
auto fnUserLock = userPoolDir / fmt("slot-%d", i);
AutoCloseFD fd = open(fnUserLock.c_str(), O_RDWR | O_CREAT | O_CLOEXEC, 0600);
if (!fd)
throw SysError("opening user lock '%s'", fnUserLock);
throw SysError("opening user lock %s", PathFmt(fnUserLock));
if (lockFile(fd.get(), ltWrite, false)) {
auto firstUid = settings.startId + i * maxIdsPerBuild;
auto firstUid = uidSettings.startId + i * maxIdsPerBuild;
auto pw = getpwuid(firstUid);
if (pw)
@@ -199,10 +200,9 @@ struct AutoUserLock : UserLock
if (useUserNamespace)
lock->firstGid = firstUid;
else {
struct group * gr = getgrnam(settings.buildUsersGroup.get().c_str());
struct group * gr = getgrnam(buildUsersGroup.c_str());
if (!gr)
throw Error(
"the group '%s' specified in 'build-users-group' does not exist", settings.buildUsersGroup);
throw Error("the group '%s' specified in 'build-users-group' does not exist", buildUsersGroup);
lock->firstGid = gr->gr_gid;
}
lock->nrIds = nrIds;
@@ -214,12 +214,17 @@ struct AutoUserLock : UserLock
}
};
std::unique_ptr<UserLock> acquireUserLock(uid_t nrIds, bool useUserNamespace)
std::unique_ptr<UserLock> acquireUserLock(const std::string & userGroup, uid_t nrIds, bool useUserNamespace)
{
if (settings.autoAllocateUids)
return AutoUserLock::acquire(nrIds, useUserNamespace);
else
return SimpleUserLock::acquire();
if (auto * uidSettings = settings.getAutoAllocateUidSettings()) {
auto userPoolDir = std::filesystem::path{settings.nixStateDir} / "userpool2";
createDirs(userPoolDir);
return AutoUserLock::acquire(userPoolDir, userGroup, nrIds, useUserNamespace, *uidSettings);
} else {
auto userPoolDir = std::filesystem::path{settings.nixStateDir} / "userpool";
createDirs(userPoolDir);
return SimpleUserLock::acquire(userPoolDir, userGroup);
}
}
bool useBuildUsers()
+16 -10
View File
@@ -3,6 +3,7 @@
#include "nix/store/store-api.hh"
#include "nix/store/gc-store.hh"
#include "nix/store/build-result.hh"
#include "nix/store/common-protocol.hh"
#include "nix/store/worker-protocol.hh"
#include "nix/store/worker-protocol-impl.hh"
#include "nix/util/archive.hh"
@@ -209,7 +210,7 @@ BuildResult WorkerProto::Serialise<BuildResult>::read(const StoreDirConfig & sto
BuildResult::Success success;
BuildResult::Failure failure;
auto rawStatus = readInt(conn.from);
auto status = WorkerProto::Serialise<BuildResultStatus>::read(store, {conn.from});
conn.from >> failure.errorMsg;
if (GET_PROTOCOL_MINOR(conn.version) >= 29) {
@@ -225,13 +226,18 @@ BuildResult WorkerProto::Serialise<BuildResult>::read(const StoreDirConfig & sto
success.builtOutputs.insert_or_assign(std::move(output.outputName), std::move(realisation));
}
if (BuildResult::Success::statusIs(rawStatus)) {
success.status = static_cast<BuildResult::Success::Status>(rawStatus);
res.inner = std::move(success);
} else {
failure.status = static_cast<BuildResult::Failure::Status>(rawStatus);
res.inner = std::move(failure);
}
res.inner = std::visit(
overloaded{
[&](BuildResult::Success::Status s) -> decltype(res.inner) {
success.status = s;
return std::move(success);
},
[&](BuildResult::Failure::Status s) -> decltype(res.inner) {
failure.status = s;
return std::move(failure);
},
},
status);
return res;
}
@@ -263,11 +269,11 @@ void WorkerProto::Serialise<BuildResult>::write(
std::visit(
overloaded{
[&](const BuildResult::Failure & failure) {
conn.to << failure.status;
WorkerProto::write(store, {conn.to}, BuildResultStatus{failure.status});
common(failure.errorMsg, failure.isNonDeterministic, decltype(BuildResult::Success::builtOutputs){});
},
[&](const BuildResult::Success & success) {
conn.to << success.status;
WorkerProto::write(store, {conn.to}, BuildResultStatus{success.status});
common(/*errorMsg=*/"", /*isNonDeterministic=*/false, success.builtOutputs);
},
},
+3 -3
View File
@@ -97,8 +97,8 @@ TEST(fchmodatTryNoFollow, works)
sink.createSymlink(CanonPath("dirlink"), "dir");
}
ASSERT_EQ(chmod((tmpDir / "file").c_str(), 0644), 0);
ASSERT_EQ(chmod((tmpDir / "dir").c_str(), 0755), 0);
ASSERT_NO_THROW(chmod(tmpDir / "file", 0644));
ASSERT_NO_THROW(chmod(tmpDir / "dir", 0755));
AutoCloseFD dirFd = openDirectory(tmpDir);
ASSERT_TRUE(dirFd);
@@ -152,7 +152,7 @@ TEST(fchmodatTryNoFollow, fallbackWithoutProc)
sink.createSymlink(CanonPath("link"), "file");
}
ASSERT_EQ(chmod((tmpDir / "file").c_str(), 0644), 0);
ASSERT_NO_THROW(chmod(tmpDir / "file", 0644));
Pid pid = startProcess(
[&] {
+88 -42
View File
@@ -193,40 +193,71 @@ bool isDirOrInDir(const std::filesystem::path & path, const std::filesystem::pat
return path == dir || isInDir(path, dir);
}
struct stat stat(const Path & path)
{
struct stat st;
if (stat(path.c_str(), &st))
throw SysError("getting status of '%1%'", path);
return st;
}
#ifdef _WIN32
# define STAT stat
# define STAT _wstat64
# define LSTAT _wstat64
#else
# define STAT lstat
# define STAT stat
# define LSTAT lstat
#endif
struct stat lstat(const Path & path)
PosixStat stat(const std::filesystem::path & path)
{
struct stat st;
PosixStat st;
if (STAT(path.c_str(), &st))
throw SysError("getting status of '%1%'", path);
throw SysError("getting status of %s", PathFmt(path));
return st;
}
std::optional<struct stat> maybeLstat(const Path & path)
PosixStat lstat(const std::filesystem::path & path)
{
std::optional<struct stat> st{std::in_place};
PosixStat st;
if (LSTAT(path.c_str(), &st))
throw SysError("getting status of %s", PathFmt(path));
return st;
}
PosixStat fstat(int fd)
{
PosixStat st;
if (
#ifdef _WIN32
_fstat64
#else
::fstat
#endif
(fd, &st))
throw SysError("getting status of fd %d", fd);
return st;
}
std::optional<PosixStat> maybeStat(const std::filesystem::path & path)
{
std::optional<PosixStat> st{std::in_place};
if (STAT(path.c_str(), &*st)) {
if (errno == ENOENT || errno == ENOTDIR)
st.reset();
else
throw SysError("getting status of '%s'", path);
throw SysError("getting status of %s", PathFmt(path));
}
return st;
}
std::optional<PosixStat> maybeLstat(const std::filesystem::path & path)
{
std::optional<PosixStat> st{std::in_place};
if (LSTAT(path.c_str(), &*st)) {
if (errno == ENOENT || errno == ENOTDIR)
st.reset();
else
throw SysError("getting status of %s", PathFmt(path));
}
return st;
}
#undef STAT
#undef LSTAT
bool pathExists(const std::filesystem::path & path)
{
return maybeLstat(path.string()).has_value();
@@ -448,33 +479,31 @@ AutoDelete::AutoDelete(const std::filesystem::path & p, bool recursive)
this->recursive = recursive;
}
void AutoDelete::deletePath()
{
if (del) {
if (recursive)
nix::deletePath(_path);
else
std::filesystem::remove(_path);
cancel();
}
}
AutoDelete::~AutoDelete()
{
try {
if (del) {
if (recursive)
deletePath(_path);
else {
std::filesystem::remove(_path);
}
}
deletePath();
} catch (...) {
ignoreExceptionInDestructor();
}
}
void AutoDelete::cancel()
void AutoDelete::cancel() noexcept
{
del = false;
}
void AutoDelete::reset(const std::filesystem::path & p, bool recursive)
{
_path = p;
this->recursive = recursive;
del = true;
}
//////////////////////////////////////////////////////////////////////
#ifdef __FreeBSD__
@@ -483,7 +512,7 @@ AutoUnmount::AutoUnmount()
{
}
AutoUnmount::AutoUnmount(Path & p)
AutoUnmount::AutoUnmount(const std::filesystem::path & p)
: path(p)
, del(true)
{
@@ -492,20 +521,26 @@ AutoUnmount::AutoUnmount(Path & p)
AutoUnmount::~AutoUnmount()
{
try {
if (del) {
if (unmount(path.c_str(), 0) < 0) {
throw SysError("Failed to unmount path %1%", path);
}
}
unmount();
} catch (...) {
ignoreExceptionInDestructor();
}
}
void AutoUnmount::cancel()
void AutoUnmount::cancel() noexcept
{
del = false;
}
void AutoUnmount::unmount()
{
if (del) {
if (::unmount(path.c_str(), 0) < 0) {
throw SysError("Failed to unmount path %1%", PathFmt(path));
}
}
cancel();
}
#endif
//////////////////////////////////////////////////////////////////////
@@ -641,7 +676,7 @@ void replaceSymlink(const std::filesystem::path & target, const std::filesystem:
}
}
void setWriteTime(const std::filesystem::path & path, const struct stat & st)
void setWriteTime(const std::filesystem::path & path, const PosixStat & st)
{
setWriteTime(path, st.st_atime, st.st_mtime, S_ISLNK(st.st_mode));
}
@@ -739,6 +774,19 @@ std::filesystem::path makeParentCanonical(const std::filesystem::path & rawPath)
}
}
void chmod(const std::filesystem::path & path, mode_t mode)
{
if (
#ifdef _WIN32
::_wchmod
#else
::chmod
#endif
(path.c_str(), mode)
== -1)
throw SysError("setting permissions on %s", PathFmt(path));
}
bool chmodIfNeeded(const std::filesystem::path & path, mode_t mode, mode_t mask)
{
auto pathString = path.string();
@@ -747,9 +795,7 @@ bool chmodIfNeeded(const std::filesystem::path & path, mode_t mode, mode_t mask)
if (((prevMode ^ mode) & mask) == 0)
return false;
if (chmod(pathString.c_str(), mode) != 0)
throw SysError("could not set permissions on '%s' to %o", pathString, mode);
chmod(path, mode);
return true;
}
+14 -18
View File
@@ -11,39 +11,35 @@
namespace nix {
AutoRemoveJail::AutoRemoveJail()
: del{false}
{
}
AutoRemoveJail::AutoRemoveJail() = default;
AutoRemoveJail::AutoRemoveJail(int jid)
: jid(jid)
, del(true)
{
}
void AutoRemoveJail::remove()
{
if (jid != INVALID_JAIL) {
if (jail_remove(jid) < 0) {
throw SysError("Failed to remove jail %1%", jid);
}
}
cancel();
}
AutoRemoveJail::~AutoRemoveJail()
{
try {
if (del) {
if (jail_remove(jid) < 0) {
throw SysError("Failed to remove jail %1%", jid);
}
}
remove();
} catch (...) {
ignoreExceptionInDestructor();
}
}
void AutoRemoveJail::cancel()
void AutoRemoveJail::cancel() noexcept
{
del = false;
}
void AutoRemoveJail::reset(int j)
{
del = true;
jid = j;
jid = INVALID_JAIL;
}
//////////////////////////////////////////////////////////////////////
@@ -7,32 +7,51 @@ namespace nix {
class AutoRemoveJail
{
int jid;
bool del;
static constexpr int INVALID_JAIL = -1;
int jid = INVALID_JAIL;
public:
AutoRemoveJail() = default;
AutoRemoveJail(int jid);
AutoRemoveJail(const AutoRemoveJail &) = delete;
AutoRemoveJail & operator=(const AutoRemoveJail &) = delete;
AutoRemoveJail(AutoRemoveJail && other) noexcept
: jid(other.jid)
, del(other.del)
{
other.cancel();
}
AutoRemoveJail & operator=(AutoRemoveJail && other) noexcept
{
jid = other.jid;
del = other.del;
other.cancel();
swap(*this, other);
return *this;
}
AutoRemoveJail();
friend void swap(AutoRemoveJail & lhs, AutoRemoveJail & rhs) noexcept
{
using std::swap;
swap(lhs.jid, rhs.jid);
}
operator int() const
{
return jid;
}
~AutoRemoveJail();
void cancel();
void reset(int j);
/**
* Remove the jail and cancel this `AutoRemoveJail`, so jail removal is not
* attempted a second time by the destructor.
*
* The destructor calls this ignoring any exception.
*/
void remove();
/**
* Cancel the jail removal.
*/
void cancel() noexcept;
};
} // namespace nix
+1 -3
View File
@@ -189,9 +189,7 @@ void RestoreRegularFile::isExecutable()
// Windows doesn't have a notion of executable file permissions we
// care about here, right?
#ifndef _WIN32
struct stat st;
if (fstat(fd.get(), &st) == -1)
throw SysError("fstat");
auto st = nix::fstat(fd.get());
if (fchmod(fd.get(), st.st_mode | (S_IXUSR | S_IXGRP | S_IXOTH)) == -1)
throw SysError("fchmod");
#endif
+84 -13
View File
@@ -15,6 +15,7 @@
#include <unistd.h>
#ifdef _WIN32
# include <windef.h>
# include <wchar.h>
#endif
#include <functional>
@@ -110,16 +111,35 @@ bool isInDir(const std::filesystem::path & path, const std::filesystem::path & d
*/
bool isDirOrInDir(const std::filesystem::path & path, const std::filesystem::path & dir);
/**
* `struct stat` is not 64-bit everywhere on Windows.
*/
using PosixStat =
#ifdef _WIN32
struct ::__stat64
#else
struct ::stat
#endif
;
/**
* Get status of `path`.
*/
struct stat stat(const Path & path);
struct stat lstat(const Path & path);
PosixStat lstat(const std::filesystem::path & path);
/**
* Get status of `path` following symlinks.
*/
PosixStat stat(const std::filesystem::path & path);
/**
* Get status of an open file descriptor.
*/
PosixStat fstat(int fd);
/**
* `lstat` the given path if it exists.
* @return std::nullopt if the path doesn't exist, or an optional containing the result of `lstat` otherwise
*/
std::optional<struct stat> maybeLstat(const Path & path);
std::optional<PosixStat> maybeLstat(const std::filesystem::path & path);
std::optional<PosixStat> maybeStat(const std::filesystem::path & path);
/**
* @return true iff the given path exists.
@@ -260,9 +280,9 @@ void setWriteTime(
std::optional<bool> isSymlink = std::nullopt);
/**
* Convenience wrapper that takes all arguments from the `struct stat`.
* Convenience wrapper that takes all arguments from the `PosixStat`.
*/
void setWriteTime(const std::filesystem::path & path, const struct stat & st);
void setWriteTime(const std::filesystem::path & path, const PosixStat & st);
/**
* Create a symlink.
@@ -316,15 +336,37 @@ public:
x.del = false;
}
AutoDelete & operator=(AutoDelete && x) noexcept
{
swap(*this, x);
return *this;
}
friend void swap(AutoDelete & lhs, AutoDelete & rhs) noexcept
{
using std::swap;
swap(lhs._path, rhs._path);
swap(lhs.del, rhs.del);
swap(lhs.recursive, rhs.recursive);
}
AutoDelete(const std::filesystem::path & p, bool recursive = true);
AutoDelete(const AutoDelete &) = delete;
AutoDelete & operator=(AutoDelete &&) = delete;
AutoDelete & operator=(const AutoDelete &) = delete;
~AutoDelete();
void cancel();
/**
* Delete the file the path points to, and cancel this `AutoDelete`,
* so deletion is not attempted a second time by the destructor.
*
* The destructor calls this, but ignoring any exception.
*/
void deletePath();
void reset(const std::filesystem::path & p, bool recursive = true);
/**
* Cancel the pending deletion
*/
void cancel() noexcept;
const std::filesystem::path & path() const
{
@@ -418,6 +460,17 @@ extern PathFilter defaultPathFilter;
*/
bool chmodIfNeeded(const std::filesystem::path & path, mode_t mode, mode_t mask = S_IRWXU | S_IRWXG | S_IRWXO);
/**
* Set permissions on a path, throwing an exception on error.
*
* @param path Path to the file to change the permissions for.
* @param mode New file mode.
*
* @todo stop using this and start using `fchmodatTryNoFollow` (or a different
* wrapper) to avoid TOCTOU issues.
*/
void chmod(const std::filesystem::path & path, mode_t mode);
/**
* @brief A directory iterator that can be used to iterate over the
* contents of a directory. It is similar to std::filesystem::directory_iterator
@@ -493,11 +546,11 @@ private:
#ifdef __FreeBSD__
class AutoUnmount
{
Path path;
std::filesystem::path path;
bool del;
public:
AutoUnmount();
AutoUnmount(Path &);
AutoUnmount(const std::filesystem::path &);
AutoUnmount(const AutoUnmount &) = delete;
AutoUnmount(AutoUnmount && other) noexcept
@@ -508,13 +561,31 @@ public:
AutoUnmount & operator=(AutoUnmount && other) noexcept
{
path = std::move(other.path);
del = std::exchange(other.del, false);
swap(*this, other);
return *this;
}
friend void swap(AutoUnmount & lhs, AutoUnmount & rhs) noexcept
{
using std::swap;
swap(lhs.path, rhs.path);
swap(lhs.del, rhs.del);
}
~AutoUnmount();
void cancel();
/**
* Cancel the unmounting
*/
void cancel() noexcept;
/**
* Unmount the mountpoint right away (if it exists), resetting the
* `AutoUnmount`
*
* The destructor calls this, but ignoring any exception.
*/
void unmount();
};
#endif
+1
View File
@@ -53,6 +53,7 @@ headers = files(
'mounted-source-accessor.hh',
'muxable-pipe.hh',
'nar-accessor.hh',
'nar-cache.hh',
'nar-listing.hh',
'os-string.hh',
'pool.hh',
+9 -7
View File
@@ -27,7 +27,12 @@ struct NarAccessor : SourceAccessor
*/
ref<NarAccessor> makeNarAccessor(std::string && nar);
ref<NarAccessor> makeNarAccessor(Source & source);
/**
* This NAR accessor doesn't actually access a NAR, and thus cannot read
* the contents of files. It just conveys the information which is
* gotten from `listing`.
*/
ref<NarAccessor> makeNarAccessor(NarListing listing);
/**
* Create a NAR accessor from a NAR listing (in the format produced by
@@ -44,12 +49,9 @@ GetNarBytes seekableGetNarBytes(const std::filesystem::path & path);
GetNarBytes seekableGetNarBytes(Descriptor fd);
/**
* Creates a NAR accessor from a given listing and a `GetNarBytes` getter.
*/
ref<NarAccessor> makeLazyNarAccessor(NarListing listing, GetNarBytes getNarBytes);
/**
* Creates a NAR accessor from a given stream and a GetNarBytes getter.
* @param source Consumed eagerly. References to it are not persisted in the resulting SourceAccessor.
*/
ref<NarAccessor> makeLazyNarAccessor(Source & source, GetNarBytes getNarBytes);
} // namespace nix
+47
View File
@@ -0,0 +1,47 @@
#pragma once
#include "nix/util/hash.hh"
#include "nix/util/nar-accessor.hh"
#include "nix/util/ref.hh"
#include "nix/util/source-accessor.hh"
#include <filesystem>
#include <functional>
#include <map>
#include <optional>
namespace nix {
/**
* A cache for NAR accessors with optional disk caching.
*/
class NarCache
{
/**
* Optional directory for caching NARs and listings on disk.
*/
std::optional<std::filesystem::path> cacheDir;
/**
* Map from NAR hash to NAR accessor.
*/
std::map<Hash, ref<SourceAccessor>> nars;
public:
/**
* Create a NAR cache with an optional cache directory for disk storage.
*/
NarCache(std::optional<std::filesystem::path> cacheDir = {});
/**
* Lookup or create a NAR accessor, optionally using disk cache.
*
* @param narHash The NAR hash to use as cache key
* @param populate Function called with a Sink to populate the NAR if not cached
* @return The cached or newly created accessor
*/
ref<SourceAccessor> getOrInsert(const Hash & narHash, std::function<void(Sink &)> populate);
};
} // namespace nix
@@ -87,7 +87,7 @@ private:
*/
void assertNoSymlinks(CanonPath path);
std::optional<struct stat> cachedLstat(const CanonPath & path);
std::optional<PosixStat> cachedLstat(const CanonPath & path);
std::filesystem::path makeAbsPath(const CanonPath & path);
};
+1
View File
@@ -151,6 +151,7 @@ sources = [ config_priv_h ] + files(
'memory-source-accessor/json.cc',
'mounted-source-accessor.cc',
'nar-accessor.cc',
'nar-cache.cc',
'nar-listing.cc',
'pos-table.cc',
'position.cc',
+4 -15
View File
@@ -34,14 +34,8 @@ struct NarAccessorImpl : NarAccessor
{
}
NarAccessorImpl(Source & source)
: root{parseNarListing(source)}
{
}
NarAccessorImpl(Source & source, GetNarBytes getNarBytes)
: root{parseNarListing(source)}
, getNarBytes{std::move(getNarBytes)}
NarAccessorImpl(NarListing && listing)
: root{std::move(listing)}
{
}
@@ -147,9 +141,9 @@ ref<NarAccessor> makeNarAccessor(std::string && nar)
return make_ref<NarAccessorImpl>(std::move(nar));
}
ref<NarAccessor> makeNarAccessor(Source & source)
ref<NarAccessor> makeNarAccessor(NarListing listing)
{
return make_ref<NarAccessorImpl>(source);
return make_ref<NarAccessorImpl>(std::move(listing));
}
ref<NarAccessor> makeLazyNarAccessor(NarListing listing, GetNarBytes getNarBytes)
@@ -157,11 +151,6 @@ ref<NarAccessor> makeLazyNarAccessor(NarListing listing, GetNarBytes getNarBytes
return make_ref<NarAccessorImpl>(std::move(listing), getNarBytes);
}
ref<NarAccessor> makeLazyNarAccessor(Source & source, GetNarBytes getNarBytes)
{
return make_ref<NarAccessorImpl>(source, getNarBytes);
}
GetNarBytes seekableGetNarBytes(const std::filesystem::path & path)
{
AutoCloseFD fd = openFileReadonly(path);
+84
View File
@@ -0,0 +1,84 @@
#include "nix/util/nar-cache.hh"
#include "nix/util/file-system.hh"
#include <nlohmann/json.hpp>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
namespace nix {
NarCache::NarCache(std::optional<std::filesystem::path> cacheDir_)
: cacheDir(std::move(cacheDir_))
{
if (cacheDir)
createDirs(*cacheDir);
}
ref<SourceAccessor> NarCache::getOrInsert(const Hash & narHash, std::function<void(Sink &)> populate)
{
// Check in-memory cache first
if (auto * accessor = get(nars, narHash))
return *accessor;
auto cacheAccessor = [&](ref<SourceAccessor> accessor) {
nars.emplace(narHash, accessor);
return accessor;
};
auto getNar = [&]() {
StringSink sink;
populate(sink);
return std::move(sink.s);
};
if (cacheDir) {
auto makeCacheFile = [&](const std::string & ext) {
auto res = *cacheDir / narHash.to_string(HashFormat::Nix32, false);
res += ".";
res += ext;
return res;
};
auto cacheFile = makeCacheFile("nar");
auto listingFile = makeCacheFile("ls");
if (nix::pathExists(cacheFile)) {
try {
return cacheAccessor(makeLazyNarAccessor(
nlohmann::json::parse(nix::readFile(listingFile)).template get<NarListing>(),
seekableGetNarBytes(cacheFile)));
} catch (SystemError &) {
}
try {
return cacheAccessor(makeNarAccessor(nix::readFile(cacheFile)));
} catch (SystemError &) {
}
}
auto nar = getNar();
try {
/* FIXME: do this asynchronously. */
writeFile(cacheFile, nar);
} catch (...) {
ignoreExceptionExceptInterrupt();
}
auto narAccessor = makeNarAccessor(std::move(nar));
try {
nlohmann::json j = narAccessor->getListing();
writeFile(listingFile, j.dump());
} catch (...) {
ignoreExceptionExceptInterrupt();
}
return cacheAccessor(narAccessor);
}
return cacheAccessor(makeNarAccessor(getNar()));
}
} // namespace nix
+3 -5
View File
@@ -55,9 +55,7 @@ void PosixSourceAccessor::readFile(const CanonPath & path, Sink & sink, std::fun
if (!fd)
throw SysError("opening file '%1%'", ap.string());
struct stat st;
if (fstat(fromDescriptorReadOnly(fd.get()), &st) == -1)
throw SysError("statting file");
auto st = nix::fstat(fromDescriptorReadOnly(fd.get()));
sizeCallback(st.st_size);
@@ -87,9 +85,9 @@ bool PosixSourceAccessor::pathExists(const CanonPath & path)
return nix::pathExists(makeAbsPath(path).string());
}
std::optional<struct stat> PosixSourceAccessor::cachedLstat(const CanonPath & path)
std::optional<PosixStat> PosixSourceAccessor::cachedLstat(const CanonPath & path)
{
using Cache = boost::concurrent_flat_map<Path, std::optional<struct stat>>;
using Cache = boost::concurrent_flat_map<Path, std::optional<PosixStat>>;
static Cache cache;
// Note: we convert std::filesystem::path to Path because the
+1 -2
View File
@@ -38,8 +38,7 @@ AutoCloseFD createUnixDomainSocket(const Path & path, mode_t mode)
bind(fdSocket.get(), path);
if (chmod(path.c_str(), mode) == -1)
throw SysError("changing permissions on '%1%'", path);
chmod(path, mode);
if (listen(toSocket(fdSocket.get()), 100) == -1)
throw SysError("cannot listen on socket '%1%'", path);
+1 -4
View File
@@ -51,10 +51,7 @@ void pollFD(int fd, int events)
std::string readFile(int fd)
{
struct stat st;
if (fstat(fd, &st) == -1)
throw SysError("statting file");
auto st = nix::fstat(fd);
return drainFD(fd, true, st.st_size);
}
+4 -4
View File
@@ -70,15 +70,15 @@ void setWriteTime(
};
# if HAVE_LUTIMES
if (lutimes(path.c_str(), times) == -1)
throw SysError("changing modification time of %s", path);
throw SysError("changing modification time of %s", PathFmt{path});
# else
bool isSymlink = optIsSymlink ? *optIsSymlink : std::filesystem::is_symlink(path);
if (!isSymlink) {
if (utimes(path.c_str(), times) == -1)
throw SysError("changing modification time of %s (not a symlink)", path);
throw SysError("changing modification time of %s (not a symlink)", PathFmt{path});
} else {
throw Error("Cannot change modification time of symlink %s", path);
throw Error("Cannot change modification time of symlink %s", PathFmt{path});
}
# endif
#endif
@@ -110,7 +110,7 @@ static void _deletePath(
std::string name(path.filename());
assert(name != "." && name != ".." && !name.empty());
struct stat st;
PosixStat st;
if (fstatat(parentfd, name.c_str(), &st, AT_SYMLINK_NOFOLLOW) == -1) {
if (errno == ENOENT)
return;
+10 -12
View File
@@ -35,18 +35,16 @@ std::filesystem::path getHome()
auto homeDir = getEnv("HOME");
if (homeDir) {
// Only use $HOME if doesn't exist or is owned by the current user.
struct stat st;
int result = stat(homeDir->c_str(), &st);
if (result != 0) {
if (errno != ENOENT) {
warn(
"couldn't stat $HOME ('%s') for reason other than not existing ('%d'), falling back to the one defined in the 'passwd' file",
*homeDir,
errno);
homeDir.reset();
}
} else if (st.st_uid != geteuid()) {
unownedUserHomeDir.swap(homeDir);
try {
auto st = maybeStat(homeDir->c_str());
if (st && st->st_uid != geteuid())
unownedUserHomeDir.swap(homeDir);
} catch (SysError & e) {
warn(
"couldn't stat $HOME ('%s') for reason other than not existing, falling back to the one defined in the 'passwd' file: %s",
*homeDir,
e.what());
homeDir.reset();
}
}
if (!homeDir) {
+1 -1
View File
@@ -154,7 +154,7 @@ struct CmdLsNar : Command, MixLs
if (!fd)
throw NativeSysError("opening NAR file %s", PathFmt(narPath));
auto source = FdSource{fd.get()};
list(makeLazyNarAccessor(source, seekableGetNarBytes(fd.get())), CanonPath{path});
list(makeLazyNarAccessor(parseNarListing(source), seekableGetNarBytes(fd.get())), CanonPath{path});
}
};
+4 -4
View File
@@ -374,10 +374,6 @@ void mainWrapped(int argc, char ** argv)
}
#endif
initNix();
initGC();
flakeSettings.configureEvalSettings(evalSettings);
/* Set the build hook location
For builds we perform a self-invocation, so Nix has to be
@@ -390,6 +386,10 @@ void mainWrapped(int argc, char ** argv)
"__build-remote",
});
initNix();
initGC();
flakeSettings.configureEvalSettings(evalSettings);
#ifdef __linux__
if (isRootUser()) {
try {
+1 -1
View File
@@ -163,7 +163,7 @@ static void update(const StringSet & channelNames)
runProgram(getNixBin("nix-env").string(), false, envArgs);
// Make the channels appear in nix-env.
struct stat st;
PosixStat st;
if (lstat(nixDefExpr.c_str(), &st) == 0) {
if (S_ISLNK(st.st_mode))
// old-skool ~/.nix-defexpr
+38 -4
View File
@@ -5,6 +5,7 @@
#include "nix/expr/eval.hh"
#include "nix/expr/get-drvs.hh"
#include "nix/store/globals.hh"
#include "nix/util/config-global.hh"
#include "nix/store/names.hh"
#include "nix/store/profiles.hh"
#include "nix/store/path-with-outputs.hh"
@@ -34,6 +35,39 @@
using namespace nix;
using std::cout;
/**
* Settings related to Nix user environments.
*/
struct EnvSettings : Config
{
Setting<bool> keepDerivations{
this,
false,
"keep-env-derivations",
R"(
If `false` (default), derivations are not stored in Nix user
environments. That is, the derivations of any build-time-only
dependencies may be garbage-collected.
If `true`, when you add a Nix derivation to a user environment, the
path of the derivation is stored in the user environment. Thus, the
derivation isn't garbage-collected until the user environment
generation is deleted (`nix-env --delete-generations`). To prevent
build-time-only dependencies from being collected, you should also
turn on `keep-outputs`.
The difference between this option and `keep-derivations` is that
this one is "sticky": it applies to any user environment created
while this option was enabled, while `keep-derivations` only applies
at the moment the garbage collector is run.
)",
{"env-keep-derivations"}};
};
EnvSettings envSettings;
static GlobalConfig::Register rSettings(&envSettings);
typedef enum { srcNixExprDrvs, srcNixExprs, srcStorePaths, srcProfile, srcAttrPath, srcUnknown } InstallSourceType;
struct InstallSourceInfo
@@ -546,7 +580,7 @@ installDerivations(Globals & globals, const Strings & args, const Path & profile
if (globals.dryRun)
return;
if (createUserEnv(*globals.state, allElems, profile, settings.envKeepDerivations, lockToken))
if (createUserEnv(*globals.state, allElems, profile, envSettings.keepDerivations, lockToken))
break;
}
}
@@ -657,7 +691,7 @@ static void upgradeDerivations(Globals & globals, const Strings & args, UpgradeT
if (globals.dryRun)
return;
if (createUserEnv(*globals.state, newElems, globals.profile, settings.envKeepDerivations, lockToken))
if (createUserEnv(*globals.state, newElems, globals.profile, envSettings.keepDerivations, lockToken))
break;
}
}
@@ -716,7 +750,7 @@ static void opSetFlag(Globals & globals, Strings opFlags, Strings opArgs)
checkSelectorUse(selectors);
/* Write the new user environment. */
if (createUserEnv(*globals.state, installedElems, globals.profile, settings.envKeepDerivations, lockToken))
if (createUserEnv(*globals.state, installedElems, globals.profile, envSettings.keepDerivations, lockToken))
break;
}
}
@@ -799,7 +833,7 @@ static void uninstallDerivations(Globals & globals, Strings & selectors, Path &
if (globals.dryRun)
return;
if (createUserEnv(*globals.state, workingElems, profile, settings.envKeepDerivations, lockToken))
if (createUserEnv(*globals.state, workingElems, profile, envSettings.keepDerivations, lockToken))
break;
}
}
+5 -12
View File
@@ -15,6 +15,7 @@
#include "nix/store/globals.hh"
#include "nix/store/path-with-outputs.hh"
#include "nix/store/export-import.hh"
#include "nix/util/strings.hh"
#include "man-pages.hh"
@@ -535,17 +536,9 @@ static void opPrintEnv(Strings opFlags, Strings opArgs)
for (auto & i : drv.env)
logger->cout("export %1%; %1%=%2%\n", i.first, escapeShellArgAlways(i.second));
/* Also output the arguments. This doesn't preserve whitespace in
arguments. */
cout << "export _args; _args='";
bool first = true;
for (auto & i : drv.args) {
if (!first)
cout << ' ';
first = false;
cout << escapeShellArgAlways(i);
}
cout << "'\n";
/* Also output the arguments. */
std::string argsStr = concatStringsSep(" ", drv.args);
cout << "export _args; _args=" << escapeShellArgAlways(argsStr) << "\n";
}
static void opReadLog(Strings opFlags, Strings opArgs)
@@ -910,7 +903,7 @@ static void opServe(Strings opFlags, Strings opArgs)
// FIXME: changing options here doesn't work if we're
// building through the daemon.
verbosity = lvlError;
settings.keepLog = false;
settings.getLogFileSettings().keepLog = false;
settings.useSubstitutes = false;
auto options = ServeProto::Serialise<ServeProto::BuildOptions>::read(*store, rconn);
+25 -2
View File
@@ -9,10 +9,33 @@
#include "nix/store/names.hh"
#include "nix/util/executable-path.hh"
#include "nix/store/globals.hh"
#include "nix/util/config-global.hh"
#include "self-exe.hh"
using namespace nix;
/**
* Settings related to upgrading Nix itself.
*/
struct UpgradeSettings : Config
{
/**
* The URL of the file that contains the store paths of the latest Nix release.
*/
Setting<std::string> storePathUrl{
this,
"https://github.com/NixOS/nixpkgs/raw/master/nixos/modules/installer/tools/nix-fallback-paths.nix",
"upgrade-nix-store-path-url",
R"(
Used by `nix upgrade-nix`, the URL of the file that contains the
store paths of the latest Nix release.
)"};
};
UpgradeSettings upgradeSettings;
static GlobalConfig::Register rSettings(&upgradeSettings);
struct CmdUpgradeNix : MixDryRun, StoreCommand
{
std::filesystem::path profileDir;
@@ -31,7 +54,7 @@ struct CmdUpgradeNix : MixDryRun, StoreCommand
.longName = "nix-store-paths-url",
.description = "The URL of the file that contains the store paths of the latest Nix release.",
.labels = {"url"},
.handler = {&(std::string &) settings.upgradeNixStorePathUrl},
.handler = {&(std::string &) upgradeSettings.storePathUrl},
});
}
@@ -156,7 +179,7 @@ struct CmdUpgradeNix : MixDryRun, StoreCommand
Activity act(*logger, lvlInfo, actUnknown, "querying latest Nix version");
// FIXME: use nixos.org?
auto req = FileTransferRequest(parseURL(settings.upgradeNixStorePathUrl.get()));
auto req = FileTransferRequest(parseURL(upgradeSettings.storePathUrl.get()));
auto res = getFileTransfer()->download(req);
auto state = std::make_unique<EvalState>(LookupPath{}, store, fetchSettings, evalSettings);
+2 -2
View File
@@ -47,6 +47,8 @@ perl.pkgs.toPerlModule (
nix-store
bzip2
libsodium
perlPackages.DBI
perlPackages.DBDSQLite
];
# `perlPackages.Test2Harness` is marked broken for Darwin
@@ -64,8 +66,6 @@ perl.pkgs.toPerlModule (
'';
mesonFlags = [
(lib.mesonOption "dbi_path" "${perlPackages.DBI}/${perl.libPrefix}")
(lib.mesonOption "dbd_sqlite_path" "${perlPackages.DBDSQLite}/${perl.libPrefix}")
(lib.mesonEnable "tests" finalAttrs.finalPackage.doCheck)
];
+18 -3
View File
@@ -191,7 +191,10 @@ test "$status" = 1
# Precise number of errors depends on daemon version / goal refactorings
(( "$(<<<"$out" grep -cE '^error:')" >= 2 ))
if isDaemonNewer "2.29pre"; then
if isDaemonNewer "2.31"; then
<<<"$out" grepQuiet -E "error: Cannot build '.*-x4\\.drv'"
<<<"$out" grepQuiet -E "Reason: 1 dependency failed."
elif isDaemonNewer "2.29pre"; then
<<<"$out" grepQuiet -E "error: Cannot build '.*-x4\\.drv'"
<<<"$out" grepQuiet -E "Reason: 1 dependency failed."
<<<"$out" grepQuiet -E "Build failed due to failed dependency"
@@ -224,9 +227,17 @@ if isDaemonNewer "2.34pre" && canUseSandbox; then
mkdir -p "$fifoDir"
mkfifo "$fifoDir/fifo"
chmod a+rw "$fifoDir/fifo"
# When using a separate test store, we need sandbox-paths to access
# the system store (where bash/coreutils live). On NixOS, the test
# uses the system store directly, so this isn't needed (and would
# conflict with input paths).
sandboxPathsArg=()
if ! isTestOnNixOS; then
sandboxPathsArg=(--option sandbox-paths "/nix/store")
fi
out="$(nix flake check ./cancelled-builds --impure -L -j2 \
--option sandbox true \
--option sandbox-paths "${NIX_STORE:-/nix/store}" \
"${sandboxPathsArg[@]}" \
--option sandbox-build-dir /build-tmp \
--option extra-sandbox-paths "/cancelled-builds-fifo=$fifoDir" \
2>&1)" && status=0 || status=$?
@@ -245,10 +256,14 @@ if isDaemonNewer "2.34pre" && canUseSandbox; then
mkdir -p "$fifoDir"
mkfifo "$fifoDir/fifo"
chmod a+rw "$fifoDir/fifo"
sandboxPathsArg=()
if ! isTestOnNixOS; then
sandboxPathsArg=(--option sandbox-paths "/nix/store")
fi
system=$(nix eval --raw --impure --expr builtins.currentSystem)
out="$(nix build --impure -L -j2 \
--option sandbox true \
--option sandbox-paths "${NIX_STORE:-/nix/store}" \
"${sandboxPathsArg[@]}" \
--option sandbox-build-dir /build-tmp \
--option extra-sandbox-paths "/cancelled-builds-fifo=$fifoDir" \
"./cancelled-builds#checks.$system.slow" \
+4 -4
View File
@@ -42,7 +42,7 @@ buildViaSubstitute () {
nix build -f issue-13247.nix "$1" --no-link --max-jobs 0 --substituters "$cache" --no-require-sigs --offline --substitute
}
# Substitue just the first output
# Substitute just the first output
buildViaSubstitute use-a-more-outputs^first
# Should only fetch the output we asked for
@@ -52,10 +52,10 @@ buildViaSubstitute use-a-more-outputs^first
delete
# Failure with 2.28 encountered in CI
requireDaemonNewerThan "2.29"
# Failure with 2.33 encountered in CI
requireDaemonNewerThan "2.34pre"
# Substitue just the first output
# Substitute just the first output
#
# This derivation is the same after normalization, so we should get
# early cut-off, and thus a chance to download just the output we want
+3
View File
@@ -64,6 +64,9 @@ unset XDG_CONFIG_HOME
unset XDG_CONFIG_DIRS
unset XDG_CACHE_HOME
unset GIT_DIR
# Isolate tests from host git config (signing, url rewrites, etc.)
export GIT_CONFIG_SYSTEM=/dev/null
export GIT_CONFIG_GLOBAL=/dev/null
export IMPURE_VAR1=foo
export IMPURE_VAR2=bar
@@ -41,5 +41,8 @@ out=$(nix build --impure --no-link --expr '
builtins.outputOf failingProducer.outPath "out"
' 2>&1) || true
# Store layer needs bugfix
requireDaemonNewerThan "2.34pre"
# The error message must NOT be empty - it should mention the failed derivation
echo "$out" | grepQuiet "failed to obtain derivation of"
+5 -3
View File
@@ -18,9 +18,11 @@ rm -rf "${rootRepo}" "${subRepo}" "$TEST_HOME"/.cache/nix
# submodule is intentionally local and it's all trusted, so we
# disable this restriction. Setting it per repo is not sufficient, as
# the repo-local config does not apply to the commands run from
# outside the repos by Nix.
export XDG_CONFIG_HOME=$TEST_HOME/.config
git config --global protocol.file.allow always
# outside the repos by Nix. We use environment variables to avoid
# attempting to write to a read-only system git config.
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=protocol.file.allow
export GIT_CONFIG_VALUE_0=always
addGitContent() {
echo "lorem ipsum" > "$1"/content
@@ -21,8 +21,9 @@ clearStore
# Submodules can't be fetched locally by default.
# See fetchGitSubmodules.sh
export XDG_CONFIG_HOME=$TEST_HOME/.config
git config --global protocol.file.allow always
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=protocol.file.allow
export GIT_CONFIG_VALUE_0=always
rootRepo=$TEST_ROOT/rootRepo
@@ -0,0 +1,4 @@
warning:
… while evaluating the first argument passed to builtins.toPath
warning: in a future version of Nix this will be an error: cannot coerce a set that is not a package to a single derived path: { outPath = «thunk»; }
@@ -0,0 +1 @@
warning: in a future version of Nix this will be an error: derivation '/nix/store/y1s2fiq89v2h9vkb38w508ir20dwv6v2-test.drv' is not valid
@@ -0,0 +1,5 @@
error: syntax error, unexpected '&&'
at «stdin»:1:1:
1| && true
| ^
2|
@@ -0,0 +1 @@
&& true
Loaded 100 of 163 files, more files were not shown because too many files have changed in this diff. Show more