Commit Graph

22753 Commits

Author SHA1 Message Date
John Ericson
91c706852b Merge pull request #15155 from obsidiansystems/protocol-version-structs
worker-protocol: replace version bit-shifting with structs
2026-02-05 18:09:53 +00:00
John Ericson
80b944a3f6 Merge pull request #15156 from obsidiansystems/findroots-refactor
libstore: move logic of `findRuntimeRoots` to new file
2026-02-05 17:04:47 +00:00
John Ericson
cccc9440d7 worker-protocol: replace version bit-shifting with structs
This commit replaces the `GET_PROTOCOL_MINOR(version)` macros with a proper `WorkerProto::Version` struct. As a bonus, this also fixes some version checks that were incorrectly ignoring the major version number.

Co-authored-by: Amaan Qureshi <git@amaanq.com>
2026-02-05 11:50:07 -05:00
John Ericson
4496a7eead Merge pull request #15101 from obsidiansystems/split-local-store-settings
libstore: split out local build and store related settings under `LocalSettings`
2026-02-05 16:23:59 +00:00
John Ericson
d1ad4b183a Merge pull request #14769 from NixOS/nar-hash-cache
Replace fetchToStore cache by sourcePathToHash cache
2026-02-05 16:23:52 +00:00
Artemis Tosini
766316223c libstore: move logic of findRuntimeRoots to new file
This change is required for implementing the unprivileged garbage collection daemon,
but it may also be useful to reduce code duplication and separate out OS-specific
garbage collector roots implementations in the future.
2026-02-05 11:18:23 -05:00
Eelco Dolstra
de6b5f60cd Replace fetchToStore cache by sourcePathToHash cache
Caching NAR hashes instead of store paths makes the cache more
general, because we can always compute the store path from the NAR
hash, but not the other way around. This is useful for lazy trees,
where we want to compute the NAR hash of an input with caching.
2026-02-05 16:45:50 +01:00
Amaan Qureshi
afd40adc90 libstore: split out local build and store related settings under LocalSettings
The global `Settings` struct contained many settings that only apply to
local builds or the local store (sandbox configuration, GC settings,
build user groups, etc.). This commit extracts these into a dedicated
`LocalSettings` struct in its own header, along with `GCSettings` and
`AutoAllocateUidSettings`.

This improves code organization and prepares for eventually making these
per-store settings in the future. Settings are accessed via
`getLocalSettings()` from the global settings object or through
`LocalStoreConfig::getLocalSettings()` for store-specific access.
2026-02-05 10:38:08 -05:00
Jörg Thalheim
dcc71da7e8 Merge pull request #15148 from Mic92/fix-tests
tests: fix URL literals in functional tests
2026-02-05 01:04:32 +00:00
Sergei Zimmerman
ecda8c2329 Merge pull request #15149 from NixOS/align-up-robustness
libutil: Add overflow check to alignUp
2026-02-05 00:51:20 +00:00
Jörg Thalheim
0da728b1f5 tests: fix URL literals in functional tests 2026-02-05 00:52:02 +01:00
Sergei Zimmerman
ea53914e47 Merge pull request #15147 from NixOS/fix-mingw-read-line
libutil: Fix mingw build
2026-02-04 23:14:52 +00:00
Sergei Zimmerman
d77c131df3 libutil: Add overflow check to alignUp
Old code with size + (size % 8 ? 8 - (size % 8) : 0) also suffered from this.
2026-02-05 02:04:33 +03:00
Sergei Zimmerman
72c2954625 libutil: Fix mingw build
This was broken by b038500b47.
2026-02-05 00:53:23 +03:00
Eelco Dolstra
27d5cc39c8 Merge pull request #14957 from NixOS/invalidate-lstat-cache
SourceAccessor: Allow the lstat cache to be invalidated
2026-02-04 21:44:04 +00:00
John Ericson
25ab7f5850 Merge pull request #15038 from obsidiansystems/drainfd-improvements
More IO portability cleanups
2026-02-04 21:36:18 +00:00
Eelco Dolstra
139d05af6f SourceAccessor: Allow cached information to be invalidated
After lockFlake() creates flake.lock in a PosixSourceAccessor, it
needs to be able to invalidate the lstat cache.
2026-02-04 21:56:17 +01:00
John Ericson
b489c8ea15 More IO portability cleanups
- options structs for `drainFD` (both versions)
- portable `read` wrapper
- portable `GetFileSize` wrapper
- dedup `readFile` and `drainFD` using the above
- Use `drainFD` in `PosixSourceAccessor`, avoiding manual IO
- Remove `fromDescriptorReadOnly` entirely!
2026-02-04 15:49:21 -05:00
John Ericson
a4c0295822 Merge pull request #15060 from NixOS/read-link-at
Support `readLinkAt` and `openFileEnsureBeneathNoSymlinks` on Windows too
2026-02-04 20:40:08 +00:00
Eelco Dolstra
8336e71c19 Merge pull request #15135 from NixOS/dependabot/github_actions/docker/login-action-3.7.0
build(deps): bump docker/login-action from 3.6.0 to 3.7.0
2026-02-04 20:34:18 +00:00
John Ericson
037a19441a Merge pull request #15145 from obsidiansystems/storeconfigcommand
libcmd: add new `StoreConfigCommand` class
2026-02-04 20:09:48 +00:00
Artemis Tosini
124605dffc libcmd: add new StoreConfigCommand class
Useful for commands that need a `StoreConfig` but do not want to open
the store, as a `StoreCommand` would do.

At the same time, make copy commands more clear by making store choice
for `updateProfile` explicit and removing the unused `getDstStore` function
in `StoreCommand`. This was a layer violation, as `StoreCommand` does
not have a concept of a source/destination store distinction.
2026-02-04 14:31:51 -05:00
John Ericson
936f6c6c7d Merge pull request #15144 from obsidiansystems/readline-terminator
libutil: add terminator option to readLine
2026-02-04 17:48:28 +00:00
Artemis Tosini
b038500b47 libutil: add terminator option to readLine
Some APIs use "lines" that end in `\0` instead of `\n`.
2026-02-04 11:58:34 -05:00
John Ericson
a357d77492 Merge pull request #15141 from NixOS/open-new-file-for-write-helper
libutil: Add openNewFileForWrite helper function, wrap callsites
2026-02-04 03:23:40 +00:00
Sergei Zimmerman
47f261cc19 libutil: Add openNewFileForWrite helper function, wrap callsites
This is purely a fix to use CreateFileW in mingw builds. Also adds some
FIXMEs for suspicious symlink following on truncation that can probably
be tightened down without any problems (other than nix-channel), but for
now this is a no-op change other than consistently using O_CLOEXEC, which
is harmless.
2026-02-04 02:10:12 +03:00
Sergei Zimmerman
27435e0036 Merge pull request #15134 from pkpbynum/pb/fix-query-path-info-daemon
Fix: `QueryPathInfo` throws on invalid path error in daemon
2026-02-03 21:34:55 +00:00
John Ericson
0e2fc2a2f1 Merge pull request #15119 from obsidiansystems/canonicalize-pmd-options
libstore: introduce `CanonicalizePathMetadataOptions` for `canonicalisePathMetaData`
2026-02-03 20:00:27 +00:00
John Ericson
6d6cbf78cc Merge pull request #15137 from NixOS/pragma-once
input-cache.hh: Add missing `#pragma once`
2026-02-03 12:17:33 +00:00
Eelco Dolstra
39a9a004e2 input-cache.hh: Add missing #pragma once 2026-02-03 11:51:04 +01:00
dependabot[bot]
5f9483519a build(deps): bump docker/login-action from 3.6.0 to 3.7.0
Bumps [docker/login-action](https://github.com/docker/login-action) from 3.6.0 to 3.7.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](5e57cd1181...c94ce9fb46)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 3.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-03 03:28:21 +00:00
Peter Bynum
b9c77ecafc Fix QueryPathInfo in daemon 2026-02-02 20:52:43 -05:00
Eelco Dolstra
d5ce1a79ec Merge pull request #15107 from bouk/fsync-key-generation
nix-store: fsync generated key files
2026-02-02 16:12:38 +00:00
John Ericson
663d27c9df Merge pull request #7892 from obsidiansystems/systemd-multi-socket
Support systemd socket activation with multiple sockets
2026-02-02 16:09:32 +00:00
John Ericson
4da0b36f83 Merge pull request #15127 from NixOS/s3-binary-cache-store-md5
libstore/s3-binary-cache-store: Add Content-MD5 header as message int…
2026-02-01 02:58:28 +00:00
John Ericson
a2de09c9fa Support systemd socket activation with multiple sockets
We now support `LISTEN_FDS` values greater than 1, per the systemd
socket activation spec.

These changes are by @edolstra, taken from #5265. This is just that PR
*without* the TCP parts, which I gathered are the controversial parts.
Hopefully this remainder is not so controversial.

Review with indentation ignored, because some code has moved inside a
new loop but otherwise is mostly unchanged.
2026-01-31 21:52:37 -05:00
Sergei Zimmerman
2ba65f1f26 libstore/s3-binary-cache-store: Add Content-MD5 header as message integrity check
aws-sdk-cpp used to include a checksum for uploads (CRC64 since ~September 2025).
Content-MD5 [1] should be universally supported by all s3 compatible services, since the SDK used
to include it unconditionally too.

[1]: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html
2026-02-01 00:26:19 +03:00
John Ericson
77b6b01b72 Merge pull request #15118 from obsidiansystems/exit-status-flags
libstore: introduce `ExitStatusFlags` for exit status computation
2026-01-30 18:37:47 +00:00
Amaan Qureshi
5e7195e1a4 libstore: introduce ExitStatusFlags for exit status computation
This commit consolidates the four separate boolean flags
(`permanentFailure`, `timedOut`, `hashMismatch`, & `checkMismatch`) into
a single `ExitStatusFlags` struct with methods for computing exit status
codes and updating from failure status.
2026-01-30 12:52:51 -05:00
Amaan Qureshi
78e8896d22 Move HashMismatch wire protocol back compat logic to better spot
The explicit serializer added in
bfdd124837 is the right place to adjust
values for sake of wire protocol compat. The protocol-agnostic `Worker`
code where it was before is the wrong spot.

(That spot was originally chosen because the back compat logic predates
having an explicit serializer for this data type to use instead.)

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
2026-01-30 12:52:51 -05:00
John Ericson
d84624d23d Merge pull request #15123 from obsidiansystems/gc-test-vm
Support gc-runtime functional tests in VMs
2026-01-30 17:42:19 +00:00
Artemis Tosini
9e8cf9055a tests/gc-functional: fix running on NixOS
This test insisted on placing profiles in NIX_STATE_DIR, but all
packages were removed from the profile immediately after so they did not
act as garbage collector roots. Switch to directly calling nix-build,
allowing the test to run in VMs without NIX_STATE_DIR.
2026-01-30 11:52:34 -05:00
John Ericson
22372d7889 Merge pull request #15122 from obsidiansystems/gc-test
libstore: fix runtime gc on non-standard store paths
2026-01-30 16:48:20 +00:00
Amaan Qureshi
d09f03d742 libstore: introduce CanonicalizePathMetadataOptions for canonicalisePathMetaData
This commit refactors the `canonicalisePathMetaData` function to take an
options struct instead of individual parameters with platform-specific
`#ifdef`s.

The struct contains a `uidRange` field (Unix only) for build user
ownership validation, and an `ignoredAcls` field for ACLs to skip when
removing extended attributes
2026-01-30 11:06:01 -05:00
Artemis Tosini
b026649c62 libstore: fix runtime gc roots on non-standard store paths
Due to a typo in quoteRegexChars, finding runtime garbage collection roots
was failing on paths that contained a dot, or any other regex chars that would
have to be replaced.

When fixing that error, also add tests to make sure gc continues to
work.
2026-01-30 11:05:10 -05:00
John Ericson
d69ca7bf35 Remove obsolete CPP for Windows in nix-store
`LocalStore` and `canonicalisePathMetaData` are defined on Windows by
now, so we don't have to gate their usage like this.
2026-01-29 19:08:46 -05:00
John Ericson
ee6cb7890a Merge pull request #15117 from obsidiansystems/move-impersonate-linux
libstore: add `PersonalityArgs` struct for `setPersonality`
2026-01-29 16:34:12 +00:00
Amaan Qureshi
351b8dd768 libstore: add PersonalityArgs struct for setPersonality
This introduces a `PersonalityArgs` struct to pass named arguments to `setPersonality`. The `impersonateLinux26` setting is now passed from the call site rather than read from settings inside the function.
2026-01-29 10:50:07 -05:00
John Ericson
1713f4c976 Merge pull request #15070 from amaanq/build-result-error-cleanup
build-result: Make `Failure` an alias for `BuildError`
2026-01-29 01:53:14 +00:00
Amaan Qureshi
de88141cdf build-result: Make Failure an alias for BuildError and remove exception parameters from goal 2026-01-28 19:32:41 -05:00