Compare commits

...
Author SHA1 Message Date
John Ericson 456b917535 Store::addToStore bail out on no-op earlier in many cases
`BinaryCacheStore` and `LocalStore` already had this optimization. Now,
all the implementations do.
2025-10-12 22:49:04 -04:00
John Ericson f8095b793d Add a new addToStore variant
This is nicer for the dummy store to implement, and some callers to use.
2025-10-12 22:39:23 -04:00
John Ericson f77094715f Merge pull request #14222 from lovesegfault/curl-based-s3-fix-race
fix(libstore): fix race condition in AWS credential provider caching
2025-10-12 22:07:10 +00:00
Bernardo Meurer Costa f0e1f65260 fix(libstore): fix race condition in AWS credential provider caching
The previous implementation had a check-then-create race condition where
multiple threads could simultaneously:
1. Check the cache and find no provider (line 122)
2. Create their own providers (lines 126-145)
3. Insert into cache (line 161)

This resulted in multiple credential providers being created when
downloading multiple packages in parallel, as each .narinfo download
would trigger provider creation on its own thread.

Fix by using boost::concurrent_flat_map's try_emplace_and_cvisit, which
provides atomic get-or-create semantics:
- f1 callback: Called atomically during insertion, creates the provider
- f2 callback: Called if key exists, returns cached provider
- Other threads are blocked during f1, so no nullptr is ever visible
2025-10-12 20:16:02 +00:00
Sergei Zimmerman 4f585dedbe Merge pull request #14228 from obsidiansystems/fix-windows-build
Fix windows build
2025-10-12 18:07:50 +00:00
John Ericson a01df8de21 Merge pull request #14227 from NixOS/asan-hydra
packaging: Add buildWithSanitizers to hydraJobs
2025-10-12 17:43:54 +00:00
John Ericson 10223fae86 Fix windows build
I forget to add some CPP in b57caaa1a2.

Hopefully, as we relyon RAII more, these explicit resets become
unneeded.
2025-10-12 13:22:35 -04:00
John Ericson 9150ccb89e Fix Windows dev shell (mostly)
gbenchmark still has too-narrow supported systems, however. That needs
to be fixed in Nixpkgs.
2025-10-12 13:16:50 -04:00
Sergei Zimmerman de75a180cb packaging: Add buildWithSanitizers to hydraJobs 2025-10-12 19:38:01 +03:00
Sergei Zimmerman a491173369 packaging: Add withASan,withUBSan options to the scope 2025-10-12 19:16:08 +03:00
Sergei Zimmerman 199b6ff3fb Disable detect_odr_violation for ASan
There's some unfortunate ODR violations that get dianosed with GCC but not Clang
for static inline constexpr variables defined inside the class body:

template<typename T>
struct static_const
{
    static JSON_INLINE_VARIABLE constexpr T value{};
};

This can be ignored pretty much. There is the same problem for std::piecewise_construct:

http://lists.boost.org/Archives/boost/2007/06/123353.php

==2455704==ERROR: AddressSanitizer: odr-violation (0x7efddc460e20):
  [1] size=1 'value' /nix/store/235hvgzcbl06fxy53515q8sr6lljvf68-nlohmann_json-3.11.3/include/nlohmann/detail/meta/cpp_future.hpp:156:45 in /nix/store/pkmljfq97a83dbanr0n64zbm8cyhna33-nix-store-2.33.0pre/lib/libnixstore.so.2.33.0
  [2] size=1 'value' /nix/store/235hvgzcbl06fxy53515q8sr6lljvf68-nlohmann_json-3.11.3/include/nlohmann/detail/meta/cpp_future.hpp:156:45 in /nix/store/gbjpkjj0g8vk20fzlyrwj491gwp6g1qw-nix-util-2.33.0pre/lib/libnixutil.so.2.33.0
2025-10-12 19:16:07 +03:00
Sergei Zimmerman 711e738bf9 meson: Simplify asan-options handling even more
Instead of specifying env variables all the time
we can instead embed the __asan_default_options symbol
in all executables / shared objects. This reduces code
duplication.
2025-10-12 19:16:06 +03:00
Sergei Zimmerman d9cabddd17 Merge pull request #14214 from obsidiansystems/derivation-resolution-goal
Split out `DerivationResolutionGoal`
2025-10-12 11:31:54 +00:00
John Ericson 0da430be35 Split out DerivationResolutionGoal
This prepares the way for fixing a few issues.

Take 2: was landed before in 8f4a739d0f.
2025-10-11 19:52:37 -04:00
John Ericson 07df87652c Make keys of Derivation*Goal more legible
The property that substitution goals come first is still preserved
2025-10-11 19:50:45 -04:00
John Ericson a629ce3dec Use member initializer list for Derivation*Goal::drv 2025-10-11 18:37:04 -04:00
John Ericson ba7bbcd1da Cleanup Derivation*Goal names 2025-10-11 18:30:47 -04:00
Sergei Zimmerman 97e770ad01 Merge pull request #14209 from lovesegfault/curl-based-s3-pieces-fix
fix(libstore): improve http-binary-cache-store S3 compatibility
2025-10-11 21:19:18 +00:00
John Ericson 682cf0b266 Merge pull request #14216 from NixOS/simplify-asan-meson
meson: Move asan-options to common
2025-10-11 18:21:24 +00:00
John Ericson f1cb837888 Merge pull request #14217 from NixOS/remove-libgit2-patches
packaging: Remove no longer necessary libgit2 patches
2025-10-11 18:20:36 +00:00
Sergei Zimmerman 47705139c9 packaging: Remove no longer necessary libgit2 patches
25.05 already has 1.9.0 and we don't support older nixpkgs versions.
2025-10-11 16:30:55 +03:00
Sergei Zimmerman d26a337c09 meson: Move asan-options to common
This way we don't have to duplicate the subdir everywhere.
Less copy-pasta is good.
2025-10-11 16:08:35 +03:00
John Ericson 8064c75694 Merge pull request #14194 from xokdvium/stacktrace-assertion
libutil: Print stack trace on assertion failure
2025-10-10 23:25:16 +00:00
John Ericson d75614a315 Merge pull request #14208 from obsidiansystems/consolidate-builder-dispatch
Consolidate logic choosing where we can/should build a bit
2025-10-10 23:05:40 +00:00
Sergei Zimmerman 46382ade74 libutil: Print stack trace on assertion failure
This change overrides __assert_fail on glibc/musl
to instead call std::terminate that we have a custom
handler for. This ensures that we have more context
to diagnose issues encountered by users in the wild.
2025-10-11 01:35:58 +03:00
John Ericson 9cb686f816 Merge pull request #14210 from NixOS/double-quotes
libstore: Fix double-quoting of paths in logs
2025-10-10 22:10:36 +00:00
Bernardo Meurer Costa f02218873e fix(libstore): improve http-binary-cache-store S3 compatibility
This commit adds two key fixes to http-binary-cache-store.cc to
properly support the new curl-based S3 implementation:

1. **Consistent cache key handling**: Use `getReference().render(withParams=false)`
   for disk cache keys instead of `cacheUri.to_string()`. This ensures cache
   keys are consistent with the S3 implementation and don't include query
   parameters, which matches the behavior expected by Store::queryPathInfo()
   lookups.

2. **S3 query parameter preservation**: When generating file transfer requests
   for S3 URLs, preserve query parameters from the base URL (region, endpoint,
   etc.) when the relative path doesn't have its own query parameters. This
   ensures S3-specific configuration is propagated to all requests.
2025-10-10 22:05:57 +00:00
John Ericson b57caaa1a2 Consolidate logic choosing where we can/should build a bit
I want to separate "policy" from "mechanism".

Now the logic to decide how to build (a policy choice, though with some
hard constraints) is all in derivation building goal, and all in the
same spot. build hook, external builder, or local builder --- the choice
between all three is made in the same spot --- pure policy.

Now, if you want to use the external deriation builder, you simply
provide the `ExternalBuilder` you wish to use, and there is no
additional checking --- pure mechanism. It is the responsibility of the
caller to choose an external builder that works for the derivation in
question.

Also, `checkSystem()` was the only thing throwing `BuildError` from
`startBuilder`. Now that that is gone, we can now remove the
`try...catch` around that.
2025-10-10 17:28:57 -04:00
John EricsonandSergei Zimmerman 2ff59ec3e0 Use std::ranges::find_if for finding external builders
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-10-10 17:27:41 -04:00
Sergei Zimmerman f30cb8667b libstore: Fix double-quoting of paths in logs
std::filesystem::path is already quoted by boost::format with double quotes (").
2025-10-11 00:25:51 +03:00
John Ericson b56dd21c31 Settings::ExternalBuilder::systems make set
Nothing cares about the order, actually.
2025-10-10 17:23:45 -04:00
Sergei Zimmerman 2308aaf192 Merge pull request #14198 from lovesegfault/curl-based-s3-pieces
feat(libstore): add curl-based S3 store implementation
2025-10-10 07:55:44 +00:00
Bernardo Meurer Costa 0855b715a9 feat(libstore): add curl-based S3 store implementation
Add a new S3BinaryCacheStore implementation that inherits from
HttpBinaryCacheStore.

The implementation is activated with NIX_WITH_CURL_S3, keeping the
existing NIX_WITH_S3_SUPPORT (AWS SDK) implementation unchanged.
2025-10-10 00:13:29 +00:00
John Ericson 2fc88ec114 Merge pull request #14204 from NixOS/kill-to-real-path-fetchers
libfetchers: Remove toRealPath in SourceHutInputScheme::getRevFromRef
2025-10-09 22:01:03 +00:00
Sergei Zimmerman 9c8480becb Merge pull request #14203 from roberth/move-eval-cache-open-to-libflake
Move eval cache open to libflake
2025-10-09 21:42:06 +00:00
Sergei Zimmerman c58acff42a libfetchers: Remove toRealPath in SourceHutInputScheme::getRevFromRef
This code had several issues:

1. Not going through the SourceAccessor means that we can only work
   with physical paths.

2. It did not actually check that the file exists. (std::ifstream does not check
   it by default).
2025-10-10 00:26:57 +03:00
Robert Hensing abcceafbce Use const for lock in openEvalCache 2025-10-09 21:25:40 +02:00
Robert Hensing 42c9cbf9ca Use ref<LockedFlake> where non-null 2025-10-09 21:25:40 +02:00
Robert Hensing 0387b7d6db Move openEvalCache to libflake
Most of the eval cache logic is flake-independent and libexpr,
but the loading part is not.
`nix-flake` is the right component for this, as the eval cache
isn't exactly specific to the command line.
2025-10-09 20:40:40 +02:00
Sergei Zimmerman dfafd8bc38 Merge pull request #14199 from getchoo-contrib/getchoo/toml11-fix
packaging: only override `toml11` when necessary
2025-10-09 17:20:35 +00:00
Sergei Zimmerman 44486871e9 Merge pull request #14200 from Mic92/less-ci-builds
only build on push to master
2025-10-09 15:07:06 +00:00
Jörg Thalheim 118acc84ba only build on push to master
we have now merge queues for maintainance branches. We still build it
for master to have our installer beeing updated. In future this part
could go in new workflow instead.
2025-10-09 14:16:14 +01:00
Seth Flynn 0f016f9bf5 packaging: only override toml11 when necessary
v4.4.0 hit Nixpkgs in https://github.com/NixOS/nixpkgs/pull/442682.
Ideally we'd just use that, but this keeps the fallback behavior until
it's more widespread
2025-10-09 03:24:57 -04:00
John Ericson ce38b46e06 Merge pull request #14170 from lovesegfault/curl-based-s3-pieces
feat(libstore/filetransfer): add S3 signing support
2025-10-08 22:35:32 +00:00
Sergei Zimmerman 33d9270109 Merge pull request #14191 from NixOS/fix-14188
libutil: Fix renderAuthorityAndPath unreachable for path:/ URLs
2025-10-08 21:34:28 +00:00
Bernardo Meurer Costa 00c2a57666 feat(libstore/filetransfer): add S3 signing support 2025-10-08 21:31:34 +00:00
Eelco Dolstra d591f17ecb Merge pull request #14189 from NixOS/fix-exportReferencesGraph
exportReferencesGraph: Handle heterogeneous arrays
2025-10-08 21:19:30 +00:00
John Ericson bb1945a090 Merge pull request #14182 from NixOS/simplify-archive-tests
tests: Move invalid nar tests from tests/functional to libutil-tests
2025-10-08 20:46:37 +00:00
Eelco Dolstra 94f410b628 exportReferencesGraph: Handle heterogeneous arrays
This barfed with

   error: [json.exception.type_error.302] type must be string, but is array

on `nix build github:malt3/bazel-env#bazel-env` because it has a `exportReferencesGraph` with a value like `["string",...["string"]]`.
2025-10-08 22:15:33 +02:00
Bernardo Meurer Costa 3c1e2e56ea feat(libstore/filetransfer): add username/password authentication support
Add a `UsernameAuth` struct and optional `usernameAuth` field to
`FileTransferRequest` to support programmatic username/password
authentication.

This uses curl's `CURLOPT_USERNAME`/`CURLOPT_PASSWORD` options, which
works with multiple protocols (HTTP, FTP, etc.) and is not specific to
any particular authentication scheme.

The primary motivation is to enable S3 authentication refactoring where
AWS credentials (access key ID and secret access key) can be passed
through this general-purpose mechanism, reducing the amount of
S3-specific code behind `#if NIX_WITH_CURL_S3` guards.
2025-10-08 20:10:53 +00:00
Eelco Dolstra 925f10d5ea Merge pull request #14087 from NixOS/required-args
Args::Flag: Add `required` attribute
2025-10-08 19:33:22 +00:00
Sergei Zimmerman 1d8dd77e1d libutil: Fix renderAuthorityAndPath unreachable for path:/ URLs
This was mistakenly triggered by path:/ URL, since the `//` would
correspond to 3 empty segments.
2025-10-08 22:14:49 +03:00
Jörg Thalheim 090f7fb05e Merge pull request #14002 from getchoo-contrib/getchoo/dogfood-experimental-installer
Add experimental installer to installer tests
2025-10-08 04:05:46 +00:00
Jörg Thalheim 2ce343716d Merge pull request #14169 from NixOS/dev-shell-no-separate-debug-info
dev-shell: Disable separateDebugInfo
2025-10-08 04:03:43 +00:00
Jörg Thalheim e69e621578 Merge pull request #14180 from NixOS/release-process-drop-mergify
maintainers: Remove mergify note from release-process.md
2025-10-08 03:34:05 +00:00
Jörg Thalheim d2a0a11a8e Merge pull request #14184 from lovesegfault/nix-better-ci
ci: integrate vm_tests into main tests job
2025-10-08 05:23:53 +02:00
Bernardo Meurer Costa a400ea4257 ci: integrate vm_tests into main tests job
This consolidates the separate vm_tests job into the main tests job,
simplifying the CI workflow. VM tests now run as part of the regular
test matrix.
2025-10-08 02:46:56 +00:00
Sergei Zimmerman fc8b784924 Merge pull request #14147 from lovesegfault/nix-multi-ci
ci: test without s3 and with curl-based-s3
2025-10-08 02:38:03 +03:00
Sergei Zimmerman 0619351326 tests: Move invalid nar tests from tests/functional to libutil-tests
Since 242f362567 we have better infrastructure
for this kind of tests.
2025-10-08 02:02:33 +03:00
Sergei Zimmerman c5b88c22fa dev-shell: Disable separateDebugInfo
This breaks gdb pretty-printers inserted into .debug_gdb_scripts section,
because it implies --compress-debug-sections=zlib, -Wa,--compress-debug-sections.
This is very unfortunate, because then gdb can't use pretty printers for
Boost.Unordered (which are very useful, since boost::unoredred_flat_map is
impossible to debug). This seems perfectly fine to disable in the dev-shell for
the time being.

See [1-3] for further references.

With this change I'm able to use boost's pretty-printers out-of-the box:

```
p *importResolutionCache
$2 = boost::concurrent_flat_map with 1 elements = {[{accessor = {p = std::shared_ptr<nix::SourceAccessor> (use count 5, weak count 1) = {
        get() = 0x555555d830a8}}, path = {static root = {static root = <same as static member of an already seen type>, path = "/"},
      path = "/derivation-internal.nix"}}] = {accessor = {p = std::shared_ptr<nix::SourceAccessor> (use count 5, weak count 1) = {
        get() = 0x555555d830a8}}, path = {static root = {static root = <same as static member of an already seen type>, path = "/"},
      path = "/derivation-internal.nix"}}}
```

When combined with a simple `add-auto-load-safe-path ~/code` in .gdbinit

[1]: https://gerrit.lix.systems/c/lix/+/3880
[2]: https://git.lix.systems/lix-project/lix/issues/1003
[3]: https://sourceware.org/pipermail/gdb-patches/2025-October/221398.html
2025-10-08 00:57:34 +03:00
Sergei Zimmerman 75b18a6e47 maintainers: Remove mergify note from release-process.md 2025-10-08 00:51:50 +03:00
Sergei Zimmerman fb117e0cac Merge pull request #14155 from Mic92/backport-action
ci: Switch away from mergify to backport action
2025-10-08 00:32:00 +03:00
Jörg Thalheim 63e8b5f94a ci: Switch away from mergify to backport action
We want to use github native queues.
2025-10-07 23:43:03 +03:00
Eelco Dolstra 559061bf5a Merge pull request #14173 from NixOS/bump-2.33.0
Bump version
2025-10-07 20:42:25 +02:00
Eelco Dolstra b0f567e18b Update mergify.yml 2025-10-07 17:16:57 +02:00
Eelco Dolstra 875f7ec25f Bump version 2025-10-07 17:15:28 +02:00
Bernardo Meurer Costa 8c28283876 ci: test without s3 and with curl-based-s3 2025-10-06 16:24:21 +00:00
Bernardo Meurer Costa 7f22a40e3b build(libstore): assert withAWS xor withCurlS3 2025-10-06 16:22:01 +00:00
Seth Flynn 92d7381826 ci: allow for using the latest build of the experimental installer
Until these repos are potentially merged, this is good for dogfooding
alongside the experimental installer. It also uses the more official
`artifacts.nixos.org` endpoint to install stable releases now

More immediately though, we need a patch for the experimental installer
to really work in CI at all, and that hasn't landed in a tag yet. So,
this lets us use it right from `main`!
2025-10-03 03:37:59 -04:00
Seth Flynn d2293fb458 ci: enable experimental installer tests 2025-10-03 01:26:55 -04:00
Seth Flynn 2cbbb63628 ci: enable use of the experimental installer 2025-10-03 01:26:52 -04:00
Eelco Dolstra eec4dece33 Args::Flag: Add required attribute 2025-09-26 16:01:59 +02:00
111 changed files with 1541 additions and 2040 deletions

No files matched your search

@@ -4,12 +4,22 @@ inputs:
dogfood:
description: "Whether to use Nix installed from the latest artifact from master branch"
required: true # Be explicit about the fact that we are using unreleased artifacts
experimental-installer:
description: "Whether to use the experimental installer to install Nix"
default: false
experimental-installer-version:
description: "Version of the experimental installer to use. If `latest`, the newest artifact from the default branch is used."
# TODO: This should probably be pinned to a release after https://github.com/NixOS/experimental-nix-installer/pull/49 lands in one
default: "latest"
extra_nix_config:
description: "Gets appended to `/etc/nix/nix.conf` if passed."
install_url:
description: "URL of the Nix installer"
required: false
default: "https://releases.nixos.org/nix/nix-2.30.2/install"
tarball_url:
description: "URL of the Nix tarball to use with the experimental installer"
required: false
github_token:
description: "Github token"
required: true
@@ -37,14 +47,74 @@ runs:
gh run download "$RUN_ID" --repo "$DOGFOOD_REPO" -n "$INSTALLER_ARTIFACT" -D "$INSTALLER_DOWNLOAD_DIR"
echo "installer-path=file://$INSTALLER_DOWNLOAD_DIR" >> "$GITHUB_OUTPUT"
TARBALL_PATH="$(find "$INSTALLER_DOWNLOAD_DIR" -name 'nix*.tar.xz' -print | head -n 1)"
echo "tarball-path=file://$TARBALL_PATH" >> "$GITHUB_OUTPUT"
echo "::notice ::Dogfooding Nix installer from master (https://github.com/$DOGFOOD_REPO/actions/runs/$RUN_ID)"
env:
GH_TOKEN: ${{ inputs.github_token }}
DOGFOOD_REPO: "NixOS/nix"
- name: "Gather system info for experimental installer"
shell: bash
if: ${{ inputs.experimental-installer == 'true' }}
run: |
echo "::notice Using experimental installer from $EXPERIMENTAL_INSTALLER_REPO (https://github.com/$EXPERIMENTAL_INSTALLER_REPO)"
if [ "$RUNNER_OS" == "Linux" ]; then
EXPERIMENTAL_INSTALLER_SYSTEM="linux"
echo "EXPERIMENTAL_INSTALLER_SYSTEM=$EXPERIMENTAL_INSTALLER_SYSTEM" >> "$GITHUB_ENV"
elif [ "$RUNNER_OS" == "macOS" ]; then
EXPERIMENTAL_INSTALLER_SYSTEM="darwin"
echo "EXPERIMENTAL_INSTALLER_SYSTEM=$EXPERIMENTAL_INSTALLER_SYSTEM" >> "$GITHUB_ENV"
else
echo "::error ::Unsupported RUNNER_OS: $RUNNER_OS"
exit 1
fi
if [ "$RUNNER_ARCH" == "X64" ]; then
EXPERIMENTAL_INSTALLER_ARCH=x86_64
echo "EXPERIMENTAL_INSTALLER_ARCH=$EXPERIMENTAL_INSTALLER_ARCH" >> "$GITHUB_ENV"
elif [ "$RUNNER_ARCH" == "ARM64" ]; then
EXPERIMENTAL_INSTALLER_ARCH=aarch64
echo "EXPERIMENTAL_INSTALLER_ARCH=$EXPERIMENTAL_INSTALLER_ARCH" >> "$GITHUB_ENV"
else
echo "::error ::Unsupported RUNNER_ARCH: $RUNNER_ARCH"
exit 1
fi
echo "EXPERIMENTAL_INSTALLER_ARTIFACT=nix-installer-$EXPERIMENTAL_INSTALLER_ARCH-$EXPERIMENTAL_INSTALLER_SYSTEM" >> "$GITHUB_ENV"
env:
EXPERIMENTAL_INSTALLER_REPO: "NixOS/experimental-nix-installer"
- name: "Download latest experimental installer"
shell: bash
id: download-latest-experimental-installer
if: ${{ inputs.experimental-installer == 'true' && inputs.experimental-installer-version == 'latest' }}
run: |
RUN_ID=$(gh run list --repo "$EXPERIMENTAL_INSTALLER_REPO" --workflow ci.yml --branch main --status success --json databaseId --jq ".[0].databaseId")
EXPERIMENTAL_INSTALLER_DOWNLOAD_DIR="$GITHUB_WORKSPACE/$EXPERIMENTAL_INSTALLER_ARTIFACT"
mkdir -p "$EXPERIMENTAL_INSTALLER_DOWNLOAD_DIR"
gh run download "$RUN_ID" --repo "$EXPERIMENTAL_INSTALLER_REPO" -n "$EXPERIMENTAL_INSTALLER_ARTIFACT" -D "$EXPERIMENTAL_INSTALLER_DOWNLOAD_DIR"
# Executable permissions are lost in artifacts
find $EXPERIMENTAL_INSTALLER_DOWNLOAD_DIR -type f -exec chmod +x {} +
echo "installer-path=$EXPERIMENTAL_INSTALLER_DOWNLOAD_DIR" >> "$GITHUB_OUTPUT"
env:
GH_TOKEN: ${{ inputs.github_token }}
EXPERIMENTAL_INSTALLER_REPO: "NixOS/experimental-nix-installer"
- uses: cachix/install-nix-action@c134e4c9e34bac6cab09cf239815f9339aaaf84e # v31.5.1
if: ${{ inputs.experimental-installer != 'true' }}
with:
# Ternary operator in GHA: https://www.github.com/actions/runner/issues/409#issuecomment-752775072
install_url: ${{ inputs.dogfood == 'true' && format('{0}/install', steps.download-nix-installer.outputs.installer-path) || inputs.install_url }}
install_options: ${{ inputs.dogfood == 'true' && format('--tarball-url-prefix {0}', steps.download-nix-installer.outputs.installer-path) || '' }}
extra_nix_config: ${{ inputs.extra_nix_config }}
- uses: DeterminateSystems/nix-installer-action@786fff0690178f1234e4e1fe9b536e94f5433196 # v20
if: ${{ inputs.experimental-installer == 'true' }}
with:
diagnostic-endpoint: ""
# TODO: It'd be nice to use `artifacts.nixos.org` for both of these, maybe through an `/experimental-installer/latest` endpoint? or `/commit/<hash>`?
local-root: ${{ inputs.experimental-installer-version == 'latest' && steps.download-latest-experimental-installer.outputs.installer-path || '' }}
source-url: ${{ inputs.experimental-installer-version != 'latest' && 'https://artifacts.nixos.org/experimental-installer/tag/${{ inputs.experimental-installer-version }}/${{ env.EXPERIMENTAL_INSTALLER_ARTIFACT }}' || '' }}
nix-package-url: ${{ inputs.dogfood == 'true' && steps.download-nix-installer.outputs.tarball-path || (inputs.tarball_url || '') }}
extra-conf: ${{ inputs.extra_nix_config }}
+37
View File
@@ -0,0 +1,37 @@
name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for korthout/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-24.04-arm
steps:
- name: Generate GitHub App token
id: generate-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ vars.CI_APP_ID }}
private-key: ${{ secrets.CI_APP_PRIVATE_KEY }}
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
uses: korthout/backport-action@d07416681cab29bf2661702f925f020aaa962997 # v3.4.1
id: backport
with:
# Config README: https://github.com/korthout/backport-action#backport-action
github_token: ${{ steps.generate-token.outputs.token }}
github_workspace: ${{ github.workspace }}
auto_merge_enabled: true
pull_description: |-
Automatic backport to `${target_branch}`, triggered by a label in #${pull_number}.
+63 -24
View File
@@ -4,6 +4,8 @@ on:
pull_request:
merge_group:
push:
branches:
- master
workflow_dispatch:
inputs:
dogfood:
@@ -65,18 +67,42 @@ jobs:
instrumented: false
primary: true
stdenv: stdenv
withAWS: true
withCurlS3: false
# TODO: remove once curl-based-s3 fully lands
- scenario: on ubuntu (no s3)
runs-on: ubuntu-24.04
os: linux
instrumented: false
primary: false
stdenv: stdenv
withAWS: false
withCurlS3: false
# TODO: remove once curl-based-s3 fully lands
- scenario: on ubuntu (curl s3)
runs-on: ubuntu-24.04
os: linux
instrumented: false
primary: false
stdenv: stdenv
withAWS: false
withCurlS3: true
- scenario: on macos
runs-on: macos-14
os: darwin
instrumented: false
primary: true
stdenv: stdenv
withAWS: true
withCurlS3: false
- scenario: on ubuntu (with sanitizers / coverage)
runs-on: ubuntu-24.04
os: linux
instrumented: true
primary: false
stdenv: clangStdenv
withAWS: true
withCurlS3: false
name: tests ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60
@@ -99,7 +125,17 @@ jobs:
run: |
nix build --file ci/gha/tests/wrapper.nix componentTests -L \
--arg withInstrumentation ${{ matrix.instrumented }} \
--argstr stdenv "${{ matrix.stdenv }}"
--argstr stdenv "${{ matrix.stdenv }}" \
${{ format('--arg withAWS {0}', matrix.withAWS) }} \
${{ format('--arg withCurlS3 {0}', matrix.withCurlS3) }}
- name: Run VM tests
run: |
nix build --file ci/gha/tests/wrapper.nix vmTests -L \
--arg withInstrumentation ${{ matrix.instrumented }} \
--argstr stdenv "${{ matrix.stdenv }}" \
${{ format('--arg withAWS {0}', matrix.withAWS) }} \
${{ format('--arg withCurlS3 {0}', matrix.withCurlS3) }}
if: ${{ matrix.os == 'linux' }}
- name: Run flake checks and prepare the installer tarball
run: |
ci/gha/tests/build-checks
@@ -110,6 +146,8 @@ jobs:
nix build --file ci/gha/tests/wrapper.nix codeCoverage.coverageReports -L \
--arg withInstrumentation ${{ matrix.instrumented }} \
--argstr stdenv "${{ matrix.stdenv }}" \
${{ format('--arg withAWS {0}', matrix.withAWS) }} \
${{ format('--arg withCurlS3 {0}', matrix.withCurlS3) }} \
--out-link coverage-reports
cat coverage-reports/index.txt >> $GITHUB_STEP_SUMMARY
if: ${{ matrix.instrumented }}
@@ -135,9 +173,19 @@ jobs:
- scenario: on ubuntu
runs-on: ubuntu-24.04
os: linux
experimental-installer: false
- scenario: on macos
runs-on: macos-14
os: darwin
experimental-installer: false
- scenario: on ubuntu (experimental)
runs-on: ubuntu-24.04
os: linux
experimental-installer: true
- scenario: on macos (experimental)
runs-on: macos-14
os: darwin
experimental-installer: true
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps:
@@ -149,11 +197,22 @@ jobs:
path: out
- name: Looking up the installer tarball URL
id: installer-tarball-url
run: echo "installer-url=file://$GITHUB_WORKSPACE/out" >> "$GITHUB_OUTPUT"
run: |
echo "installer-url=file://$GITHUB_WORKSPACE/out" >> "$GITHUB_OUTPUT"
TARBALL_PATH="$(find "$GITHUB_WORKSPACE/out" -name 'nix*.tar.xz' -print | head -n 1)"
echo "tarball-path=file://$TARBALL_PATH" >> "$GITHUB_OUTPUT"
- uses: cachix/install-nix-action@v31
if: ${{ !matrix.experimental-installer }}
with:
install_url: ${{ format('{0}/install', steps.installer-tarball-url.outputs.installer-url) }}
install_options: ${{ format('--tarball-url-prefix {0}', steps.installer-tarball-url.outputs.installer-url) }}
- uses: ./.github/actions/install-nix-action
if: ${{ matrix.experimental-installer }}
with:
dogfood: false
experimental-installer: true
tarball_url: ${{ steps.installer-tarball-url.outputs.tarball-path }}
github_token: ${{ secrets.GITHUB_TOKEN }}
- run: sudo apt install fish zsh
if: matrix.os == 'linux'
- run: brew install fish
@@ -185,7 +244,7 @@ jobs:
echo "docker=${{ env._DOCKER_SECRETS != '' }}" >> $GITHUB_OUTPUT
docker_push_image:
needs: [tests, vm_tests, check_secrets]
needs: [tests, check_secrets]
permissions:
contents: read
packages: write
@@ -238,28 +297,8 @@ jobs:
docker tag nix:$NIX_VERSION $IMAGE_ID:master
docker push $IMAGE_ID:master
vm_tests:
needs: basic-checks
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/install-nix-action
with:
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config:
experimental-features = nix-command flakes
github_token: ${{ secrets.GITHUB_TOKEN }}
- uses: DeterminateSystems/magic-nix-cache-action@main
- run: |
nix build -L \
.#hydraJobs.tests.functional_user \
.#hydraJobs.tests.githubFlakes \
.#hydraJobs.tests.nix-docker \
.#hydraJobs.tests.tarballFlakes \
;
flake_regressions:
needs: vm_tests
needs: tests
runs-on: ubuntu-24.04
steps:
- name: Checkout nix
-174
View File
@@ -1,174 +0,0 @@
queue_rules:
- name: default
# all required tests need to go here
merge_conditions:
- check-success=tests on macos
- check-success=tests on ubuntu
- check-success=installer test on macos
- check-success=installer test on ubuntu
- check-success=vm_tests
batch_size: 5
pull_request_rules:
- name: merge using the merge queue
conditions:
- base~=master|.+-maintenance
- label~=merge-queue|dependencies
actions:
queue: {}
# The rules below will first create backport pull requests and put those in a merge queue.
- name: backport patches to 2.18
conditions:
- label=backport 2.18-maintenance
actions:
backport:
branches:
- 2.18-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.19
conditions:
- label=backport 2.19-maintenance
actions:
backport:
branches:
- 2.19-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.20
conditions:
- label=backport 2.20-maintenance
actions:
backport:
branches:
- 2.20-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.21
conditions:
- label=backport 2.21-maintenance
actions:
backport:
branches:
- 2.21-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.22
conditions:
- label=backport 2.22-maintenance
actions:
backport:
branches:
- 2.22-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.23
conditions:
- label=backport 2.23-maintenance
actions:
backport:
branches:
- 2.23-maintenance
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.24
conditions:
- label=backport 2.24-maintenance
actions:
backport:
branches:
- "2.24-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.25
conditions:
- label=backport 2.25-maintenance
actions:
backport:
branches:
- "2.25-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.26
conditions:
- label=backport 2.26-maintenance
actions:
backport:
branches:
- "2.26-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.27
conditions:
- label=backport 2.27-maintenance
actions:
backport:
branches:
- "2.27-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.28
conditions:
- label=backport 2.28-maintenance
actions:
backport:
branches:
- "2.28-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.29
conditions:
- label=backport 2.29-maintenance
actions:
backport:
branches:
- "2.29-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.30
conditions:
- label=backport 2.30-maintenance
actions:
backport:
branches:
- "2.30-maintenance"
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.31
conditions:
- label=backport 2.31-maintenance
actions:
backport:
branches:
- "2.31-maintenance"
labels:
- automatic backport
- merge-queue
+1 -1
View File
@@ -1 +1 @@
2.32.0
2.33.0
+40 -13
View File
@@ -12,6 +12,8 @@
componentTestsPrefix ? "",
withSanitizers ? false,
withCoverage ? false,
withAWS ? null,
withCurlS3 ? null,
...
}:
@@ -21,16 +23,6 @@ let
packages' = nixFlake.packages.${system};
stdenv = (getStdenv pkgs);
enableSanitizersLayer = finalAttrs: prevAttrs: {
mesonFlags =
(prevAttrs.mesonFlags or [ ])
++ [ (lib.mesonOption "b_sanitize" "address,undefined") ]
++ (lib.optionals stdenv.cc.isClang [
# https://www.github.com/mesonbuild/meson/issues/764
(lib.mesonBool "b_lundef" false)
]);
};
collectCoverageLayer = finalAttrs: prevAttrs: {
env =
let
@@ -53,24 +45,39 @@ let
'';
};
componentOverrides =
(lib.optional withSanitizers enableSanitizersLayer)
++ (lib.optional withCoverage collectCoverageLayer);
componentOverrides = (lib.optional withCoverage collectCoverageLayer);
in
rec {
nixComponentsInstrumented = nixComponents.overrideScope (
final: prev: {
withASan = withSanitizers;
withUBSan = withSanitizers;
nix-store-tests = prev.nix-store-tests.override { withBenchmarks = true; };
# Boehm is incompatible with ASAN.
nix-expr = prev.nix-expr.override { enableGC = !withSanitizers; };
# Override AWS configuration if specified
nix-store = prev.nix-store.override (
lib.optionalAttrs (withAWS != null) { inherit withAWS; }
// lib.optionalAttrs (withCurlS3 != null) { inherit withCurlS3; }
);
mesonComponentOverrides = lib.composeManyExtensions componentOverrides;
# Unclear how to make Perl bindings work with a dynamically linked ASAN.
nix-perl-bindings = if withSanitizers then null else prev.nix-perl-bindings;
}
);
# Import NixOS tests using the instrumented components
nixosTests = import ../../../tests/nixos {
inherit lib pkgs;
nixComponents = nixComponentsInstrumented;
nixpkgs = nixFlake.inputs.nixpkgs;
inherit (nixFlake.inputs) nixpkgs-23-11;
};
/**
Top-level tests for the flake outputs, as they would be built by hydra.
These tests generally can't be overridden to run with sanitizers.
@@ -221,4 +228,24 @@ rec {
{
inherit coverageProfileDrvs mergedProfdata coverageReports;
};
vmTests = {
}
# FIXME: when the curlS3 implementation is complete, it should also enable these tests.
// lib.optionalAttrs (withAWS == true) {
# S3 binary cache store test only runs when S3 support is enabled
inherit (nixosTests) s3-binary-cache-store;
}
// lib.optionalAttrs (!withSanitizers && !withCoverage) {
# evalNixpkgs uses non-instrumented components from hydraJobs, so only run it
# when not testing with sanitizers to avoid rebuilding nix
inherit (hydraJobs.tests) evalNixpkgs;
# FIXME: CI times out when building vm tests instrumented
inherit (nixosTests)
functional_user
githubFlakes
nix-docker
tarballFlakes
;
};
}
+3
View File
@@ -5,6 +5,8 @@
stdenv ? "stdenv",
componentTestsPrefix ? "",
withInstrumentation ? false,
withAWS ? null,
withCurlS3 ? null,
}@args:
import ./. (
args
@@ -12,5 +14,6 @@ import ./. (
getStdenv = p: p.${stdenv};
withSanitizers = withInstrumentation;
withCoverage = withInstrumentation;
inherit withAWS withCurlS3;
}
)
-1
View File
@@ -15,7 +15,6 @@ pymod = import('python')
python = pymod.find_installation('python3')
nix_env_for_docs = {
'ASAN_OPTIONS' : 'abort_on_error=1:print_summary=1:detect_leaks=0',
'HOME' : '/dummy',
'NIX_CONF_DIR' : '/dummy',
'NIX_SSL_CERT_FILE' : '/dummy/no-ca-bundle.crt',
-1
View File
@@ -142,7 +142,6 @@ release:
$ git pull
$ NEW_VERSION=2.13.0
$ echo $NEW_VERSION > .version
$ ... edit .mergify.yml to add the previous version ...
$ git checkout -b bump-$NEW_VERSION
$ git commit -a -m 'Bump version'
$ git push --set-upstream origin bump-$NEW_VERSION
@@ -0,0 +1,6 @@
extern "C" [[gnu::retain, gnu::weak]] const char * __asan_default_options()
{
// We leak a bunch of memory knowingly on purpose. It's not worthwhile to
// diagnose that memory being leaked for now.
return "abort_on_error=1:print_summary=1:detect_leaks=0:detect_odr_violation=0";
}
@@ -1,7 +1,3 @@
asan_test_options_env = {
'ASAN_OPTIONS' : 'abort_on_error=1:print_summary=1:detect_leaks=0',
}
# Clang gets grumpy about missing libasan symbols if -shared-libasan is not
# passed when building shared libs, at least on Linux
if cxx.get_id() == 'clang' and ('address' in get_option('b_sanitize') or 'undefined' in get_option(
@@ -10,3 +6,6 @@ if cxx.get_id() == 'clang' and ('address' in get_option('b_sanitize') or 'undefi
add_project_link_arguments('-shared-libasan', language : 'cpp')
endif
if 'address' in get_option('b_sanitize')
deps_other += declare_dependency(sources : 'asan-options.cc')
endif
@@ -0,0 +1,32 @@
can_wrap_assert_fail_test_code = '''
#include <cstdlib>
#include <cassert>
int main()
{
assert(0);
}
extern "C" void * __real___assert_fail(const char *, const char *, unsigned int, const char *);
extern "C" void *
__wrap___assert_fail(const char *, const char *, unsigned int, const char *)
{
return __real___assert_fail(nullptr, nullptr, 0, nullptr);
}
'''
wrap_assert_fail_args = [ '-Wl,--wrap=__assert_fail' ]
can_wrap_assert_fail = cxx.links(
can_wrap_assert_fail_test_code,
args : wrap_assert_fail_args,
name : 'linker can wrap __assert_fail',
)
if can_wrap_assert_fail
deps_other += declare_dependency(
sources : 'wrap-assert-fail.cc',
link_args : wrap_assert_fail_args,
)
endif
@@ -0,0 +1,17 @@
#include "nix/util/error.hh"
#include <cstdio>
#include <cstdlib>
#include <cinttypes>
#include <string_view>
extern "C" [[noreturn]] void __attribute__((weak))
__wrap___assert_fail(const char * assertion, const char * file, unsigned int line, const char * function)
{
char buf[512];
int n =
snprintf(buf, sizeof(buf), "Assertion '%s' failed in %s at %s:%" PRIuLEAST32, assertion, function, file, line);
if (n < 0)
nix::panic("Assertion failed and could not format error message");
nix::panic(std::string_view(buf, std::min(static_cast<int>(sizeof(buf)), n)));
}
@@ -44,3 +44,6 @@ endif
# Darwin ld doesn't like "X.Y.Zpre"
nix_soversion = meson.project_version().split('pre')[0]
subdir('assert-fail')
subdir('asan-options')
+32
View File
@@ -204,6 +204,25 @@ let
mesonFlags = [ (lib.mesonBool "b_asneeded" false) ] ++ prevAttrs.mesonFlags or [ ];
};
enableSanitizersLayer =
finalAttrs: prevAttrs:
let
sanitizers = lib.optional scope.withASan "address" ++ lib.optional scope.withUBSan "undefined";
in
{
mesonFlags =
(prevAttrs.mesonFlags or [ ])
++ lib.optionals (lib.length sanitizers > 0) (
[
(lib.mesonOption "b_sanitize" (lib.concatStringsSep "," sanitizers))
]
++ (lib.optionals stdenv.cc.isClang [
# https://www.github.com/mesonbuild/meson/issues/764
(lib.mesonBool "b_lundef" false)
])
);
};
nixDefaultsLayer = finalAttrs: prevAttrs: {
strictDeps = prevAttrs.strictDeps or true;
enableParallelBuilding = true;
@@ -246,6 +265,16 @@ in
inherit filesetToSource;
/**
Whether meson components are built with [AddressSanitizer](https://clang.llvm.org/docs/AddressSanitizer.html).
*/
withASan = false;
/**
Whether meson components are built with [UndefinedBehaviorSanitizer](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html).
*/
withUBSan = false;
/**
A user-provided extension function to apply to each component derivation.
*/
@@ -332,6 +361,7 @@ in
setVersionLayer
mesonLayer
fixupStaticLayer
enableSanitizersLayer
scope.mesonComponentOverrides
];
mkMesonExecutable = mkPackageBuilder [
@@ -342,6 +372,7 @@ in
mesonLayer
mesonBuildLayer
fixupStaticLayer
enableSanitizersLayer
scope.mesonComponentOverrides
];
mkMesonLibrary = mkPackageBuilder [
@@ -353,6 +384,7 @@ in
mesonBuildLayer
mesonLibraryLayer
fixupStaticLayer
enableSanitizersLayer
scope.mesonComponentOverrides
];
+14 -43
View File
@@ -57,15 +57,20 @@ scope: {
prevAttrs.postInstall;
});
toml11 = pkgs.toml11.overrideAttrs rec {
version = "4.4.0";
src = pkgs.fetchFromGitHub {
owner = "ToruNiina";
repo = "toml11";
tag = "v${version}";
hash = "sha256-sgWKYxNT22nw376ttGsTdg0AMzOwp8QH3E8mx0BZJTQ=";
};
};
# TODO: Remove this when https://github.com/NixOS/nixpkgs/pull/442682 is included in a stable release
toml11 =
if lib.versionAtLeast pkgs.toml11.version "4.4.0" then
pkgs.toml11
else
pkgs.toml11.overrideAttrs rec {
version = "4.4.0";
src = pkgs.fetchFromGitHub {
owner = "ToruNiina";
repo = "toml11";
tag = "v${version}";
hash = "sha256-sgWKYxNT22nw376ttGsTdg0AMzOwp8QH3E8mx0BZJTQ=";
};
};
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost =
@@ -84,38 +89,4 @@ scope: {
buildPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.buildPhase;
installPhase = lib.replaceStrings [ "--without-python" ] [ "" ] old.installPhase;
});
libgit2 =
if lib.versionAtLeast pkgs.libgit2.version "1.9.0" then
pkgs.libgit2
else
pkgs.libgit2.overrideAttrs (attrs: {
# libgit2: Nixpkgs 24.11 has < 1.9.0, which needs our patches
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch =
attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
'';
patches =
attrs.patches or [ ]
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
}
+41 -32
View File
@@ -70,6 +70,9 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
# We use this shell with the local checkout, not unpackPhase.
src = null;
# Workaround https://sourceware.org/pipermail/gdb-patches/2025-October/221398.html
# Remove when gdb fix is rolled out everywhere.
separateDebugInfo = false;
env = {
# For `make format`, to work without installing pre-commit
@@ -93,38 +96,44 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
++ map (transformFlag "libcmd") (ignoreCrossFile pkgs.nixComponents2.nix-cmd.mesonFlags);
nativeBuildInputs =
attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents2.nix-util.nativeBuildInputs
++ pkgs.nixComponents2.nix-store.nativeBuildInputs
++ pkgs.nixComponents2.nix-fetchers.nativeBuildInputs
++ pkgs.nixComponents2.nix-expr.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents2.nix-perl-bindings.nativeBuildInputs
++ lib.optionals buildCanExecuteHost pkgs.nixComponents2.nix-manual.externalNativeBuildInputs
++ pkgs.nixComponents2.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents2.nix-external-api-docs.nativeBuildInputs
++ pkgs.nixComponents2.nix-functional-tests.externalNativeBuildInputs
++ lib.optional (
!buildCanExecuteHost
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
) pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.gnused
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
pkgs.buildPackages.shellcheck
pkgs.buildPackages.gdb
]
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
)
++ lib.optional stdenv.hostPlatform.isLinux pkgs.buildPackages.mold-wrapped;
let
inputs =
attrs.nativeBuildInputs or [ ]
++ pkgs.nixComponents2.nix-util.nativeBuildInputs
++ pkgs.nixComponents2.nix-store.nativeBuildInputs
++ pkgs.nixComponents2.nix-fetchers.nativeBuildInputs
++ pkgs.nixComponents2.nix-expr.nativeBuildInputs
++ lib.optionals havePerl pkgs.nixComponents2.nix-perl-bindings.nativeBuildInputs
++ lib.optionals buildCanExecuteHost pkgs.nixComponents2.nix-manual.externalNativeBuildInputs
++ pkgs.nixComponents2.nix-internal-api-docs.nativeBuildInputs
++ pkgs.nixComponents2.nix-external-api-docs.nativeBuildInputs
++ pkgs.nixComponents2.nix-functional-tests.externalNativeBuildInputs
++ lib.optional (
!buildCanExecuteHost
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
&& lib.meta.availableOn stdenv.buildPlatform (stdenv.hostPlatform.emulator pkgs.buildPackages)
) pkgs.buildPackages.mesonEmulatorHook
++ [
pkgs.buildPackages.cmake
pkgs.buildPackages.gnused
pkgs.buildPackages.shellcheck
pkgs.buildPackages.changelog-d
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
pkgs.buildPackages.shellcheck
pkgs.buildPackages.gdb
]
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
)
++ lib.optional stdenv.hostPlatform.isLinux pkgs.buildPackages.mold-wrapped;
in
# FIXME: separateDebugInfo = false doesn't actually prevent -Wa,--compress-debug-sections
# from making its way into NIX_CFLAGS_COMPILE.
lib.filter (p: !lib.hasInfix "separate-debug-info" p) inputs;
buildInputs = [
pkgs.gbenchmark
+21
View File
@@ -158,6 +158,27 @@ in
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
buildWithSanitizers =
let
components = forAllSystems (
system:
let
pkgs = nixpkgsFor.${system}.native;
in
pkgs.nixComponents2.overrideScope (
self: super: {
# Boost coroutines fail with ASAN on darwin.
withASan = !pkgs.stdenv.buildPlatform.isDarwin;
withUBSan = true;
nix-expr = super.nix-expr.override { enableGC = false; };
# Unclear how to make Perl bindings work with a dynamically linked ASAN.
nix-perl-bindings = null;
}
)
);
in
forAllPackages (pkgName: forAllSystems (system: components.${system}.${pkgName}));
buildNoTests = forAllSystems (system: nixpkgsFor.${system}.native.nixComponents2.nix-cli);
# Toggles some settings for better coverage. Windows needs these
@@ -1,282 +0,0 @@
commit 9bacade4a3ef4b6b26e2c02f549eef0e9eb9eaa2
Author: Robert Hensing <robert@roberthensing.nl>
Date: Sun Aug 18 20:20:36 2024 +0200
Add unoptimized git_mempack_write_thin_pack
diff --git a/include/git2/sys/mempack.h b/include/git2/sys/mempack.h
index 17da590a3..3688bdd50 100644
--- a/include/git2/sys/mempack.h
+++ b/include/git2/sys/mempack.h
@@ -44,6 +44,29 @@ GIT_BEGIN_DECL
*/
GIT_EXTERN(int) git_mempack_new(git_odb_backend **out);
+/**
+ * Write a thin packfile with the objects in the memory store.
+ *
+ * A thin packfile is a packfile that does not contain its transitive closure of
+ * references. This is useful for efficiently distributing additions to a
+ * repository over the network, but also finds use in the efficient bulk
+ * addition of objects to a repository, locally.
+ *
+ * This operation performs the (shallow) insert operations into the
+ * `git_packbuilder`, but does not write the packfile to disk;
+ * see `git_packbuilder_write_buf`.
+ *
+ * It also does not reset the memory store; see `git_mempack_reset`.
+ *
+ * @note This function may or may not write trees and blobs that are not
+ * referenced by commits. Currently everything is written, but this
+ * behavior may change in the future as the packer is optimized.
+ *
+ * @param backend The mempack backend
+ * @param pb The packbuilder to use to write the packfile
+ */
+GIT_EXTERN(int) git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb);
+
/**
* Dump all the queued in-memory writes to a packfile.
*
diff --git a/src/libgit2/odb_mempack.c b/src/libgit2/odb_mempack.c
index 6f27f45f8..0b61e2b66 100644
--- a/src/libgit2/odb_mempack.c
+++ b/src/libgit2/odb_mempack.c
@@ -132,6 +132,35 @@ cleanup:
return err;
}
+int git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb)
+{
+ struct memory_packer_db *db = (struct memory_packer_db *)backend;
+ const git_oid *oid;
+ size_t iter = 0;
+ int err = -1;
+
+ /* TODO: Implement the recency heuristics.
+ For this it probably makes sense to only write what's referenced
+ through commits, an option I've carved out for you in the docs.
+ wrt heuristics: ask your favorite LLM to translate https://git-scm.com/docs/pack-heuristics/en
+ to actual normal reference documentation. */
+ while (true) {
+ err = git_oidmap_iterate(NULL, db->objects, &iter, &oid);
+ if (err == GIT_ITEROVER) {
+ err = 0;
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ err = git_packbuilder_insert(pb, oid, NULL);
+ if (err != 0)
+ return err;
+ }
+
+ return 0;
+}
+
int git_mempack_dump(
git_buf *pack,
git_repository *repo,
diff --git a/tests/libgit2/mempack/thinpack.c b/tests/libgit2/mempack/thinpack.c
new file mode 100644
index 000000000..604a4dda2
--- /dev/null
+++ b/tests/libgit2/mempack/thinpack.c
@@ -0,0 +1,196 @@
+#include "clar_libgit2.h"
+#include "git2/indexer.h"
+#include "git2/odb_backend.h"
+#include "git2/tree.h"
+#include "git2/types.h"
+#include "git2/sys/mempack.h"
+#include "git2/sys/odb_backend.h"
+#include "util.h"
+
+static git_repository *_repo;
+static git_odb_backend * _mempack_backend;
+
+void test_mempack_thinpack__initialize(void)
+{
+ git_odb *odb;
+
+ _repo = cl_git_sandbox_init_new("mempack_thinpack_repo");
+
+ cl_git_pass(git_mempack_new(&_mempack_backend));
+ cl_git_pass(git_repository_odb(&odb, _repo));
+ cl_git_pass(git_odb_add_backend(odb, _mempack_backend, 999));
+ git_odb_free(odb);
+}
+
+void _mempack_thinpack__cleanup(void)
+{
+ cl_git_sandbox_cleanup();
+}
+
+/*
+ Generating a packfile for an unchanged repo works and produces an empty packfile.
+ Even if we allow this scenario to be detected, it shouldn't misbehave if the
+ application is unaware of it.
+*/
+void test_mempack_thinpack__empty(void)
+{
+ git_packbuilder *pb;
+ int version;
+ int n;
+ git_buf buf = GIT_BUF_INIT;
+
+ git_packbuilder_new(&pb, _repo);
+
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_assert_in_range(12, buf.size, 1024 /* empty packfile is >0 bytes, but certainly not that big */);
+ cl_assert(buf.ptr[0] == 'P');
+ cl_assert(buf.ptr[1] == 'A');
+ cl_assert(buf.ptr[2] == 'C');
+ cl_assert(buf.ptr[3] == 'K');
+ version = (buf.ptr[4] << 24) | (buf.ptr[5] << 16) | (buf.ptr[6] << 8) | buf.ptr[7];
+ /* Subject to change. https://git-scm.com/docs/pack-format: Git currently accepts version number 2 or 3 but generates version 2 only.*/
+ cl_assert_equal_i(2, version);
+ n = (buf.ptr[8] << 24) | (buf.ptr[9] << 16) | (buf.ptr[10] << 8) | buf.ptr[11];
+ cl_assert_equal_i(0, n);
+ git_buf_dispose(&buf);
+
+ git_packbuilder_free(pb);
+}
+
+#define LIT_LEN(x) x, sizeof(x) - 1
+
+/*
+ Check that git_mempack_write_thin_pack produces a thin packfile.
+*/
+void test_mempack_thinpack__thin(void)
+{
+ /* Outline:
+ - Create tree 1
+ - Flush to packfile A
+ - Create tree 2
+ - Flush to packfile B
+
+ Tree 2 has a new blob and a reference to a blob from tree 1.
+
+ Expectation:
+ - Packfile B is thin and does not contain the objects from packfile A
+ */
+
+
+ git_oid oid_blob_1;
+ git_oid oid_blob_2;
+ git_oid oid_blob_3;
+ git_oid oid_tree_1;
+ git_oid oid_tree_2;
+ git_treebuilder *tb;
+
+ git_packbuilder *pb;
+ git_buf buf = GIT_BUF_INIT;
+ git_indexer *indexer;
+ git_indexer_progress stats;
+ char pack_dir_path[1024];
+
+ char sbuf[1024];
+ const char * repo_path;
+ const char * pack_name_1;
+ const char * pack_name_2;
+ git_str pack_path_1 = GIT_STR_INIT;
+ git_str pack_path_2 = GIT_STR_INIT;
+ git_odb_backend * pack_odb_backend_1;
+ git_odb_backend * pack_odb_backend_2;
+
+
+ cl_assert_in_range(0, snprintf(pack_dir_path, sizeof(pack_dir_path), "%s/objects/pack", git_repository_path(_repo)), sizeof(pack_dir_path));
+
+ /* Create tree 1 */
+
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_2, _repo, LIT_LEN("thinpack blob 2")));
+
+
+ cl_git_pass(git_treebuilder_new(&tb, _repo, NULL));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob2", &oid_blob_2, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_1, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_1 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_1);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+
+ /* Create tree 2 */
+
+ cl_git_pass(git_treebuilder_clear(tb));
+ /* blob 1 won't be used, but we add it anyway to test that just "declaring" an object doesn't
+ necessarily cause its inclusion in the next thin packfile. It must only be included if new. */
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_3, _repo, LIT_LEN("thinpack blob 3")));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob3", &oid_blob_3, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_2, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_2 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_2);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+ git_treebuilder_free(tb);
+
+ /* Assertions */
+
+ assert(pack_name_1);
+ assert(pack_name_2);
+
+ repo_path = git_repository_path(_repo);
+
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_1);
+ git_str_joinpath(&pack_path_1, repo_path, sbuf);
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_2);
+ git_str_joinpath(&pack_path_2, repo_path, sbuf);
+
+ /* If they're the same, something definitely went wrong. */
+ cl_assert(strcmp(pack_name_1, pack_name_2) != 0);
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_1, pack_path_1.ptr));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_1));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_2));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_3));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_1));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_2));
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_2, pack_path_2.ptr));
+ /* blob 1 is already in the packfile 1, so packfile 2 must not include it, in order to be _thin_. */
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_1));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_2));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_3));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_1));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_2));
+
+ pack_odb_backend_1->free(pack_odb_backend_1);
+ pack_odb_backend_2->free(pack_odb_backend_2);
+ free((void *)pack_name_1);
+ free((void *)pack_name_2);
+ git_str_dispose(&pack_path_1);
+ git_str_dispose(&pack_path_2);
+
+}
@@ -1,930 +0,0 @@
commit e9823c5da4fa977c46bcb97167fbdd0d70adb5ff
Author: Robert Hensing <robert@roberthensing.nl>
Date: Mon Aug 26 20:07:04 2024 +0200
Make packbuilder interruptible using progress callback
Forward errors from packbuilder->progress_cb
This allows the callback to terminate long-running operations when
the application is interrupted.
diff --git a/include/git2/pack.h b/include/git2/pack.h
index 0f6bd2ab9..bee72a6c0 100644
--- a/include/git2/pack.h
+++ b/include/git2/pack.h
@@ -247,6 +247,9 @@ typedef int GIT_CALLBACK(git_packbuilder_progress)(
* @param progress_cb Function to call with progress information during
* pack building. Be aware that this is called inline with pack building
* operations, so performance may be affected.
+ * When progress_cb returns an error, the pack building process will be
+ * aborted and the error will be returned from the invoked function.
+ * `pb` must then be freed.
* @param progress_cb_payload Payload for progress callback.
* @return 0 or an error code
*/
diff --git a/src/libgit2/pack-objects.c b/src/libgit2/pack-objects.c
index b2d80cba9..7c331c2d5 100644
--- a/src/libgit2/pack-objects.c
+++ b/src/libgit2/pack-objects.c
@@ -932,6 +932,9 @@ static int report_delta_progress(
{
int ret;
+ if (pb->failure)
+ return pb->failure;
+
if (pb->progress_cb) {
uint64_t current_time = git_time_monotonic();
uint64_t elapsed = current_time - pb->last_progress_report_time;
@@ -943,8 +946,10 @@ static int report_delta_progress(
GIT_PACKBUILDER_DELTAFICATION,
count, pb->nr_objects, pb->progress_cb_payload);
- if (ret)
+ if (ret) {
+ pb->failure = ret;
return git_error_set_after_callback(ret);
+ }
}
}
@@ -976,7 +981,10 @@ static int find_deltas(git_packbuilder *pb, git_pobject **list,
}
pb->nr_deltified += 1;
- report_delta_progress(pb, pb->nr_deltified, false);
+ if ((error = report_delta_progress(pb, pb->nr_deltified, false)) < 0) {
+ GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
+ goto on_error;
+ }
po = *list++;
(*list_size)--;
@@ -1124,6 +1132,10 @@ struct thread_params {
size_t depth;
size_t working;
size_t data_ready;
+
+ /* A pb->progress_cb can stop the packing process by returning an error.
+ When that happens, all threads observe the error and stop voluntarily. */
+ bool stopped;
};
static void *threaded_find_deltas(void *arg)
@@ -1133,7 +1145,12 @@ static void *threaded_find_deltas(void *arg)
while (me->remaining) {
if (find_deltas(me->pb, me->list, &me->remaining,
me->window, me->depth) < 0) {
- ; /* TODO */
+ me->stopped = true;
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
+ me->working = false;
+ git_cond_signal(&me->pb->progress_cond);
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_unlock(me->pb) == 0, NULL);
+ return NULL;
}
GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
@@ -1175,8 +1192,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
pb->nr_threads = git__online_cpus();
if (pb->nr_threads <= 1) {
- find_deltas(pb, list, &list_size, window, depth);
- return 0;
+ return find_deltas(pb, list, &list_size, window, depth);
}
p = git__mallocarray(pb->nr_threads, sizeof(*p));
@@ -1195,6 +1211,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
p[i].depth = depth;
p[i].working = 1;
p[i].data_ready = 0;
+ p[i].stopped = 0;
/* try to split chunks on "path" boundaries */
while (sub_size && sub_size < list_size &&
@@ -1262,7 +1279,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
(!victim || victim->remaining < p[i].remaining))
victim = &p[i];
- if (victim) {
+ if (victim && !target->stopped) {
sub_size = victim->remaining / 2;
list = victim->list + victim->list_size - sub_size;
while (sub_size && list[0]->hash &&
@@ -1286,7 +1303,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
target->list_size = sub_size;
target->remaining = sub_size;
- target->working = 1;
+ target->working = 1; /* even when target->stopped, so that we don't process this thread again */
GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
if (git_mutex_lock(&target->mutex)) {
@@ -1299,7 +1316,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
git_cond_signal(&target->cond);
git_mutex_unlock(&target->mutex);
- if (!sub_size) {
+ if (target->stopped || !sub_size) {
git_thread_join(&target->thread, NULL);
git_cond_free(&target->cond);
git_mutex_free(&target->mutex);
@@ -1308,7 +1325,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
git__free(p);
- return 0;
+ return pb->failure;
}
#else
@@ -1319,6 +1336,7 @@ int git_packbuilder__prepare(git_packbuilder *pb)
{
git_pobject **delta_list;
size_t i, n = 0;
+ int error;
if (pb->nr_objects == 0 || pb->done)
return 0; /* nothing to do */
@@ -1327,8 +1345,10 @@ int git_packbuilder__prepare(git_packbuilder *pb)
* Although we do not report progress during deltafication, we
* at least report that we are in the deltafication stage
*/
- if (pb->progress_cb)
- pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload);
+ if (pb->progress_cb) {
+ if ((error = pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload)) < 0)
+ return git_error_set_after_callback(error);
+ }
delta_list = git__mallocarray(pb->nr_objects, sizeof(*delta_list));
GIT_ERROR_CHECK_ALLOC(delta_list);
@@ -1345,31 +1365,33 @@ int git_packbuilder__prepare(git_packbuilder *pb)
if (n > 1) {
git__tsort((void **)delta_list, n, type_size_sort);
- if (ll_find_deltas(pb, delta_list, n,
+ if ((error = ll_find_deltas(pb, delta_list, n,
GIT_PACK_WINDOW + 1,
- GIT_PACK_DEPTH) < 0) {
+ GIT_PACK_DEPTH)) < 0) {
git__free(delta_list);
- return -1;
+ return error;
}
}
- report_delta_progress(pb, pb->nr_objects, true);
+ error = report_delta_progress(pb, pb->nr_objects, true);
pb->done = true;
git__free(delta_list);
- return 0;
+ return error;
}
-#define PREPARE_PACK if (git_packbuilder__prepare(pb) < 0) { return -1; }
+#define PREPARE_PACK error = git_packbuilder__prepare(pb); if (error < 0) { return error; }
int git_packbuilder_foreach(git_packbuilder *pb, int (*cb)(void *buf, size_t size, void *payload), void *payload)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, cb, payload);
}
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, &write_pack_buf, buf);
diff --git a/src/libgit2/pack-objects.h b/src/libgit2/pack-objects.h
index bbc8b9430..380a28ebe 100644
--- a/src/libgit2/pack-objects.h
+++ b/src/libgit2/pack-objects.h
@@ -100,6 +100,10 @@ struct git_packbuilder {
uint64_t last_progress_report_time;
bool done;
+
+ /* A non-zero error code in failure causes all threads to shut themselves
+ down. Some functions will return this error code. */
+ volatile int failure;
};
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb);
diff --git a/tests/libgit2/pack/cancel.c b/tests/libgit2/pack/cancel.c
new file mode 100644
index 000000000..a0aa9716a
--- /dev/null
+++ b/tests/libgit2/pack/cancel.c
@@ -0,0 +1,240 @@
+#include "clar_libgit2.h"
+#include "futils.h"
+#include "pack.h"
+#include "hash.h"
+#include "iterator.h"
+#include "vector.h"
+#include "posix.h"
+#include "hash.h"
+#include "pack-objects.h"
+
+static git_repository *_repo;
+static git_revwalk *_revwalker;
+static git_packbuilder *_packbuilder;
+static git_indexer *_indexer;
+static git_vector _commits;
+static int _commits_is_initialized;
+static git_indexer_progress _stats;
+
+extern bool git_disable_pack_keep_file_checks;
+
+static void pack_packbuilder_init(const char *sandbox) {
+ _repo = cl_git_sandbox_init(sandbox);
+ /* cl_git_pass(p_chdir(sandbox)); */
+ cl_git_pass(git_revwalk_new(&_revwalker, _repo));
+ cl_git_pass(git_packbuilder_new(&_packbuilder, _repo));
+ cl_git_pass(git_vector_init(&_commits, 0, NULL));
+ _commits_is_initialized = 1;
+ memset(&_stats, 0, sizeof(_stats));
+ p_fsync__cnt = 0;
+}
+
+void test_pack_cancel__initialize(void)
+{
+ pack_packbuilder_init("small.git");
+}
+
+void test_pack_cancel__cleanup(void)
+{
+ git_oid *o;
+ unsigned int i;
+
+ cl_git_pass(git_libgit2_opts(GIT_OPT_ENABLE_FSYNC_GITDIR, 0));
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, false));
+
+ if (_commits_is_initialized) {
+ _commits_is_initialized = 0;
+ git_vector_foreach(&_commits, i, o) {
+ git__free(o);
+ }
+ git_vector_free(&_commits);
+ }
+
+ git_packbuilder_free(_packbuilder);
+ _packbuilder = NULL;
+
+ git_revwalk_free(_revwalker);
+ _revwalker = NULL;
+
+ git_indexer_free(_indexer);
+ _indexer = NULL;
+
+ /* cl_git_pass(p_chdir("..")); */
+ cl_git_sandbox_cleanup();
+ _repo = NULL;
+}
+
+static int seed_packbuilder(void)
+{
+ int error;
+ git_oid oid, *o;
+ unsigned int i;
+
+ git_revwalk_sorting(_revwalker, GIT_SORT_TIME);
+ cl_git_pass(git_revwalk_push_ref(_revwalker, "HEAD"));
+
+ while (git_revwalk_next(&oid, _revwalker) == 0) {
+ o = git__malloc(sizeof(git_oid));
+ cl_assert(o != NULL);
+ git_oid_cpy(o, &oid);
+ cl_git_pass(git_vector_insert(&_commits, o));
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ if((error = git_packbuilder_insert(_packbuilder, o, NULL)) < 0)
+ return error;
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ git_object *obj;
+ cl_git_pass(git_object_lookup(&obj, _repo, o, GIT_OBJECT_COMMIT));
+ error = git_packbuilder_insert_tree(_packbuilder,
+ git_commit_tree_id((git_commit *)obj));
+ git_object_free(obj);
+ if (error < 0)
+ return error;
+ }
+
+ return 0;
+}
+
+static int fail_stage;
+
+static int packbuilder_cancel_after_n_calls_cb(int stage, uint32_t current, uint32_t total, void *payload)
+{
+
+ /* Force the callback to run again on the next opportunity regardless
+ of how fast we're running. */
+ _packbuilder->last_progress_report_time = 0;
+
+ if (stage == fail_stage) {
+ int *calls = (int *)payload;
+ int n = *calls;
+ /* Always decrement, including past zero. This way the error is only
+ triggered once, making sure it is picked up immediately. */
+ --*calls;
+ if (n == 0)
+ return GIT_EUSER;
+ }
+
+ return 0;
+}
+
+static void test_cancel(int n)
+{
+
+ int calls_remaining = n;
+ int err;
+ git_buf buf = GIT_BUF_INIT;
+
+ /* Switch to a small repository, so that `packbuilder_cancel_after_n_calls_cb`
+ can hack the time to call the callback on every opportunity. */
+
+ cl_git_pass(git_packbuilder_set_callbacks(_packbuilder, &packbuilder_cancel_after_n_calls_cb, &calls_remaining));
+ err = seed_packbuilder();
+ if (!err)
+ err = git_packbuilder_write_buf(&buf, _packbuilder);
+
+ cl_assert_equal_i(GIT_EUSER, err);
+}
+void test_pack_cancel__cancel_after_add_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_add_1(void)
+{
+ cl_skip();
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_delta_1(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+#else
+ cl_skip();
+#endif
+}
+
+void test_pack_cancel__cancel_after_delta_1_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+#else
+ cl_skip();
+#endif
+}
+
+static int foreach_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *) payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return 0;
+}
+
+void test_pack_cancel__foreach(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_pass(git_packbuilder_foreach(_packbuilder, foreach_cb, idx));
+ cl_git_pass(git_indexer_commit(idx, &_stats));
+ git_indexer_free(idx);
+}
+
+static int foreach_cancel_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *)payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return (_stats.total_objects > 2) ? -1111 : 0;
+}
+
+void test_pack_cancel__foreach_with_cancel(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_fail_with(
+ git_packbuilder_foreach(_packbuilder, foreach_cancel_cb, idx), -1111);
+ git_indexer_free(idx);
+}
+
+void test_pack_cancel__keep_file_check(void)
+{
+ assert(!git_disable_pack_keep_file_checks);
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, true));
+ assert(git_disable_pack_keep_file_checks);
+}
diff --git a/tests/resources/small.git/HEAD b/tests/resources/small.git/HEAD
new file mode 100644
index 0000000000000000000000000000000000000000..cb089cd89a7d7686d284d8761201649346b5aa1c
GIT binary patch
literal 23
ecmXR)O|w!cN=+-)&qz&7Db~+TEG|hc;sO9;xClW2
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/config b/tests/resources/small.git/config
new file mode 100644
index 0000000000000000000000000000000000000000..07d359d07cf1ed0c0074fdad71ffff5942f0adfa
GIT binary patch
literal 66
zcmaz}&M!)h<>D+#Eyyp<EXgmbOv^9IO)M!(Eh^5;&r`5fFyP`$%gjm5%}+@M@=A(I
MQ@J>k5{uv*03B5png9R*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/description b/tests/resources/small.git/description
new file mode 100644
index 0000000000000000000000000000000000000000..498b267a8c7812490d6479839c5577eaaec79d62
GIT binary patch
literal 73
zcmWH|%S+5nO;IRHEyyp<EXgmbv{pz>$t+PQ$;d2LNXyJgRZve!Elw`VEGWs$&r??@
Q$yWgB0LrH#Y0~2Y0PnOK(EtDd
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/applypatch-msg.sample b/tests/resources/small.git/hooks/applypatch-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..dcbf8167fa503f96ff6a39c68409007eadc9b1f3
GIT binary patch
literal 535
zcmY+AX;Q;542A#a6e8^~FyI8r&I~hf2QJ{GO6(?HuvEG*+#R{4EI%zhfA8r{j%sh$
zHE~E-UtQd8{bq4@*S%jq3@bmxwQDXGv#o!N`o3AHMw3xD)hy0#>&E&zzl%vRffo<B
z)-H|+CWHZ~O*S%cfYx9;02_ohIA<Bg(1SxF-6OCb&_lBkf{t<AM9r;%E(Hf#h{|a@
z9>mqo=v6>_2NRa#TwDdYvTVQyueO*15Nlo%=#DXgC0bhF3vTa`LQGaO9;jeD$OP?~
za$G4Q{z+Q_{5V?5h;a-noM$P{<>Q~j4o7u%#P6^o^16{y*jU=-K8GYD_dUtdj4FSx
zSC0C!DvAnv%S!4d<Yg@O<;m`;oSw)=Fz+hrL<mY{rBr8j4pi^88FX3}jKrYUP)>gk
XB^)11aoGMJPCqWs%IS0YSv(eBT&%T6
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/commit-msg.sample b/tests/resources/small.git/hooks/commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..f3780f92349638ebe32f6baf24c7c3027675d7c9
GIT binary patch
literal 953
zcmaJ<TW`}a6n<`g#Ya}rZCaYOzy?UGG&Tf#VG`?}7@eHtB=MTqneFUS%75oLS;atz
zm&iUo=ey->y@-{3h^^Cx;#d0zEA@DDc$nY4ez&|=%jTg@_HU*ub=!!y$xW09TSjlj
z(`I@QCsM`!9&80$I98wsQ8yK#)Orb<8re8FjkKh630D$QUDwi~(gkX=RunYm$rDjk
zlp%RUSnzA#6yjdG5?T?2DcYKp+v_lts0ljn&bh3J0bD5@N@1UKZ190O6ZeWr-BuZ^
zWRebCX%(%=Xoj#(xYk1Cjtr!=tyBesf@m6}8zY6Ijbz9i9ziI_jG9Mv<Cz(ymp*>R
zDH*e>^ga9IR?2wrSrAVm;eButj4<aWB@zzNl|1Wp@4;}1O?MUF>Y>7(E2?b~jsu>&
zRKCJ7bp#19sqYh627wD%D9R$8=Ml$TNlumDypl~$jBu*G>5fIR^FB0h0Ex&TGZNr>
zL5hs1_K>taRb!|ThN9ns7^@4MXKP+6aGI_UK)T-M#rcP$;kN(Vcf#P)+5GzWa{l@J
z>-E{`$1iiNVYxq27}<DnwLRXQUG0o_hw&da-s5T#H=`Y9D_8=eTZ?cpWatp#a1vs@
z2BjrO)z@aTuI#g#`)oJcnhM7oYLT@~CHX@CNXv4>j;uo%;)r3kJI2xCFF~Ux;$Q%)
wjbk6JlDCM`jU&P+UVOvg`|iYl<7~9k>HHB4I;pdlQ=I-^$DrHaN$@lH1?P!0U;qFB
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/fsmonitor-watchman.sample b/tests/resources/small.git/hooks/fsmonitor-watchman.sample
new file mode 100755
index 0000000000000000000000000000000000000000..41184ebc318c159f51cd1ebe2290559805df89d8
GIT binary patch
literal 4777
zcmbtYYi}F368$Xwipg4lq(BeHMvzvH-4;n7DGJBPqq#tw3aed8+IU5-m)yvL>;Cqh
z8FFRGj$`9CA8ao<GaSz2oMCnz4Rv-gw9b@j_$0GQT1?XYi|;S??Y`Z6`Z;}C6#27N
z8ShS>J?j^$%==FV``-=rhLcPW`McSytRm~mEO7_&_cAVZrf1fFy*ha@8oe%*-aBYE
zcjzZg>LOkgxuUr-XJnHyD;zmPnRaSc#!k_P*d_BttRdc+J6G7za5#+<HG#rlmbrN~
z8DwU-3}VABEwM=0VLP@^Dy6ERR5_J6cmg|GEh*M1EliqCGwe^ZT-iZ$2Yc`4!I#WZ
z5nGGhn7*jeW=2ydsmfAmm#=8AD<<;TI+#z{Q)kW;yE!%GB6f~7EtEMLdM47Qaz*8=
zIObA(VVj-VG{Ax|66d*hi`+bRG>^Y1nkc2Oowk`ya47uUR3Feu?B<phm31&PQB<lt
zb{W(W4wf#Bab%|Q_tKPS?3^o=5)Z8^Vh(#slNI}pO(f^|{U0GZhLnycSaNd&h?CaC
z0XklU6^<ky6rES9T=na$M8P<_)aKMAMo+UDewAu4wF{#&6diFshiudixAoh|&0<BJ
zR>(w;S{(VYzxh}q-=#zP@uxSx{wbyPUMFU;K(06)$o{07&3yI?q{GqMcQ1c_^M<0<
zF4acAV)Il-V(rCTC1(;bsZ*}bl8dmejAk~yb`B}!^0;g^(o9kGUfZfDOvyp@x4OQt
zSgWh6T|3eq;9MFs8-#z+FDM1h(IjRUP|``PxupgJ7CUHOH90gbgl^2~97`?_X{P))
zB*$r1cDlF-%azKND}?Gv`2K8-9v5e`gQoft=j?T<&a13c^!wY_$D`5z-X1g?ty&6-
zQN50{8?bUk9AI->^W@~~nkOghHIC2YN+<JiT_ob7ttND1oh`HH28Y+OV~HedG&uB`
zy}rA*r_xT#bR`Q7-*)3t*(!Hf?jKzyxk=8hdi3L^d<p<uU9q_<4k&xEr4@YWv_vsW
zp(#32bYtA5u|s#1+}h`0kwpz4kUD&+>AXkLQG_2-{Pq3%{`3KUMeG$iIn%%^6*NYb
zn|_BdV#C)n4565Vcc<EWC-nglECZGy!j9I4&;hUCzr(?6TftH=0^@!mI^E@y5HZw8
ztH&kaSNyg=O6riqR^MPOX6oD__Jz@K;*icS)?m$!p{S$n;*EwlO<L!d7;utu(W9D!
zaF!B~xV^2i?wH0s?Lw%Q)(`aPkajs1XojlPv@Y-d5#YFg#KG+!r7A(dJLnkiJMs`p
zV|_=d!upN{TsxH1?sZNdzxeHsmtzTV`1{pykJ_~+^*>X;uT8&z3vSi!HXGbUj2B!R
zdz~&#<?<tHJql=W&((YpzS06y-Z6Cn^H!*9qH?pOrD~(GV=JJ~z{tpLnGK|xp&C1`
zsbn7O86xjF<~G*S1l*;;Bh%6><Up=oKy99?62P^?M&22U6QFRBXLb&u%=Ur<74wRy
zMRG!f{AvZ>fk#L-&k$fLwo$4?>12g@AXOKFekuo#6EHB%gmpD?1eyh%N8s{2wGoTu
z*@6cEZ^ZW!FAF_|JL`NkV7k}0ow|-2jHwbgH0;c@Dq*o?@&c*HnGdyx6^su8Qk%2{
z*ye(dxO*6-&>qn1+zw}tc6;=sOX{4WB=VqjTS^))y1jlX2Q;=e!qMmFA5lC$#;BxC
z=Y%tRpWxb+_uQAvAw7Q{HGV#R$xb&udLCzZ+HN?kTyB};1EJ8UlQ5!>5eGW@)RX0n
zkjj>EF!3=0Gl^8dzv$B^NMGRxJoqN4A`xq-@wCbrx*u2NmIJ1<fUDQ=*^)h6`vzco
z3F+ro$F!F6pc<B;<;isobIgbVGKUBByoQ4#CO({S7g?<Dh0^!7uJ3gxS=6F;+^gQc
zeKi4`4`Fm3p|BU2v{M|-u!#QGCXiM=k=%np0<ZOPQqEjP_nneyOdgEuG9IH&YYPtp
zKB_dvcYCcyhyT#<uhUEL$o~!TUz;cb&|`uSM{Dkv%&W2lcpYL&kv)tUvVv?&>xZ%H
zh;{|4T3(!E9sY#Ni(wUJYs1MmIc9bl)(4Nl3_wD_BWB>i<1S(LX7m*{Q7PU$muMS*
zM!%0EZx-Vw=Zey;erC?SNxF;pY@^A%-krqzfLV2meBp1vWdyArFYn`DD19T)Hw(?n
z)}{NP(Lk(o*?gl#B@pP7^*r|=;PIDT4|F#{2Hzh-AL0Rv$6uT;<CP7qxbYms@WU7}
z%}TsHJ!V_56ZFF{BfI=8jTBxMEATG376pS6a;u1@c?{~sL<N52`U6fuLkz4P@Mb^b
z2`Q48y&!C0&A+WRCJAdmo3u2#?eI=si9Vm47$|`m**x<wKkM=QR&g?C63@P5X@xP8
zi5QG2b-Fdz%S0%1{kKvL%^RTdpK|SU^VJ7JCmKdwp1`;HRoGM7ef^k_s_}2An=cql
za|{IaxQWpdq<ae&x3BOJq+M5QNIk#C{Nv@u>n|WzE4=slK?on@(fZeGhRgQCu56qB
z{+n81Az96qnQjMY*-*r-KV*7;Z#4Q<xfjbcBx_6JAN-HP@bq+eI%HhAB9&vLyOap{
bw<Ywj(b#kdcCk7dCBY;|DBNpPjIa1F6*dXN
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/post-update.sample b/tests/resources/small.git/hooks/post-update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..9f95e41a39cf974958277c8881ac6cce405ebb20
GIT binary patch
literal 246
zcmXZVO?HDY3<Ti4Poaiwjq^yGw9DKf7mz^&Ly=V5q$H=W^Rt|p_r9s#9Ea7VERo!9
zyT9>uJRJJV$M^KdldiMhj?ImK6~FvwJ*L5a){QoM=L5TYHkGO1$UrO3`a>{?Opw|b
zG(#59NQ#jFL9v~vgOVkM@^^(^A}onOE))yWEwhIlk&{ZyseZ^O0b=w8&O=BK{k<5B
k^Q-B@eG}LeHrquz%(SVEp_N)VhYZikCW__82JXfD17`J9Qvd(}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-applypatch.sample b/tests/resources/small.git/hooks/pre-applypatch.sample
new file mode 100755
index 0000000000000000000000000000000000000000..625837e25f91421b8809a097f4a3103dd387ef31
GIT binary patch
literal 481
zcmY+ATTa6;5Jms9iouO45IBJXEg&Jm9@v1LPHMM_ZR|;#6tQ<EeSrA%_2`_rGr1_8
z?aM?yVtIc%-@9SGSk&8x=grP-Lf`7!^=$7xgL=|ysZ}!av6zL~ywui}<2##V6L@!k
zy=p^)V7%Wzs-g`9<Y9}^)&uN}BCrXR_T3@Z2$gSJON2`X=mAs+%@7n-2I}ZrP|TFA
zvJJGDl3HPLP<@!Q!}zXQvey#qEE#a#$vs97i4=A0stF@YQ)k_ZajaoS^dVYBc&37_
zVI(L=X<V335r9~7T<;|HfKF+yM}}LB9d96V)Si;sj(;9Rh$#P>h$71hSXq*MxP;V&
zj0cY7SCL=x4`a46sF)C>94Gk%=3q$W2s;j6iHtB2$R0%gix4oK@&T~=ALd_o*CKxt
I-`Pv{1Bpzc>;M1&
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-commit.sample b/tests/resources/small.git/hooks/pre-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..10b39b2e26981b8f87ea424e735ef87359066dbb
GIT binary patch
literal 1706
zcmZuxU2ohr5PY_N#pZ0-F<{-<Zkx1j&cMNSQ3FN`GzR*R1O+9oPV`BnOj7q@1pV!u
zrF0Hl^i33(yR)-1d-!H%&2|=|^E~_R{N1zNJ-&Zmt-t?iwXv&i+ZN}Km(TX8Q$H4u
zd7(m`|1iDmF5k@xV`p;C4zojASmLc}yN0QDZbhN=ri&CEt=XGuN1IwjGJ#a#`t-kG
zDqY)}7+Ft|;YKwLYbtg$S(-TBO=x3cP1cd}%f4kB!<6Wu-dCwz-)KDMEuM^_Hh*UC
zC`1)|)T<(U6b`+yOH!6p*Ll}@qastwA*dyjsgOf5C=?LTprfORG6O{5L%@S0wyHpj
zu|_A-=NWnYYR5m7kvm6|&T~GzoJ_OKR3sgFUgw?ifho^NQhvK#{6g0=&Fh)%n}#m0
zk1sNmwb_AMDq};OOGw5|;OyX#?yQMMH6yAk(x$3tjFjHE?c$E2XC_xXav8tnIeIG?
zYMI|~MLEVGkuT*>v&v-X^RA+u>k}E$4d&uD7=g_fA8+pNNV=4s0|iD3p<=DTXClTS
zXV23tJ;ECmN@M0j@zUAKEYW@3bv!SeYZ8ZH`YQNTApFVNc;F|9r5p4TqGs=>8E?6y
zi|gY{iM#PG1nL?UE9YCnWTk72kgZPG*Usqw!~Qd3c?~@w2?%eg@~)+VlSs6N5Yf2^
zz;ow<fjf3n`imj7u5lnzt||q9QFM(H@<3EJCK|l5P!$yDfn~U-(5Vu7s+GqxNKyeI
zP=-Oa(KH&gK`NhUa`cLj3B8%qL};DR7dk!`A^h&3-hFB6p($5Ufy^tGir)3et}qK4
zpkPKYWzC+?=&gw-L6S)S=<lfq)%uLUAa%~81Jf9hk)w~U!DItnoSy`m^}#38M}K-o
z!PqisQkV!&z4E*V01ro~qlVK^0BI`pLk6D+)f~*y!hCvwHq8zY9BGh<2s$@b^A<8G
zRaqk}&qZyyv&|0QDFPA%h4TgF&vdlc|JUq*=>F#K#r^&KMq1A`oqVGFpD&-!Pv|Rc
zO3KSqA@h9nSc%bm`0)Amk6*J}@14J*1-219l%%7D!Pl}UK>|lVi0Dfgu2jN3WC!uL
z0ej??b2iSehVgdnWHmZV4kUo*QL#aiIp}U=9x)IXk}JJ7VQ;CI9Rtn5e0VcjbY<bp
zv{}WlG6L;H!EzFKdY>cVt+`x5D+svCGD<sXw4|)E|JX43I1_3P(sI4{wj87bPSrNG
w!SIr>;Z5hm*<gY+X;)Ryx4=nzaab9m`bwE*^s(%u*E3HbUuOR@+&s_z1=MCi2LJ#7
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-merge-commit.sample b/tests/resources/small.git/hooks/pre-merge-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..d8e8ad4bcf8337f23c78a98537ecab4831301202
GIT binary patch
literal 473
zcmaJ-O;f@!5WV+TJd4B0whSt$H%Dh2t`2u6q1!glCNbGU;n%yxdNsG~zR#WA6xIwy
zWEZHoU#u?nykD=Y<HPgeWDkDm^kTof*l(|%^gh!nHrZpo^vhMDjV;E1GD~K7wV*+D
zz9lry9T0cHcm_KhDVXYvQ==FrLTT4u=bEr{U1yl7%thf%wJnv<XQZ`ZbQEezaWdS%
zI;c?h9a)Y!ux<WK8rQd_aA^?61hv_Pf<t7*z1USuL40FxYz<|hybsO?qnN}aMpcuf
z6phFw1%Xx=wUk(m>E$jSEQZ%SQ(}oLgslTvrKK@9Qf#b!hajVFnp9@oIix;NcI9Wk
xjnh0ya!AWet{I7YpD;y6HXyzI*lfSvH=o6*7mJZPkuaYpm>vzZ`wyGEBtOQPo|pgt
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-push.sample b/tests/resources/small.git/hooks/pre-push.sample
new file mode 100755
index 0000000000000000000000000000000000000000..02cbd80c287f959fe33975bb66c56293e3f5b396
GIT binary patch
literal 1431
zcmaJ>U60!~5PUX&#a1@z9B{IIZkjLT0t5kq9#8~D(I5{+8&J~9;#ndUk~-ZT`r|uG
z$#K$$J{TsK<m~Lsu9iP+t-0TZ=sa(K+C6);54X>s*LP1}9!GoZ@4I4myMMG_di|of
z%?llx{O8TS-#^<H#%^V=)RNv>;(OioEmPy%kwWQBA1OMzV{hsQ8XFzS1k!~YQoLa5
zhtP1fA$q6VmMbbAC_9)4I628k*O5J$NR19uHe4QYDK<==I~SQk)Nu%xQ~<Hy8U>KH
z53w=!ke(FGb_PpnZfd*+hnXDTn;2*`u^~;?+5C~cn?bRka7NR%06%e6O91{MAgN6J
zmlO8{Biw4&wr&&(z4p3eln`E}XR9m9bNYZ7Ibrg(4yZIXrfgD7N*AFD7L3YSM#j}%
zo__rOS5fr;@8UM<6cl+cv_$YB$PQ&9dv($eM*))g!_cu!QcSh-mqE9i#QDZT)=o#`
z?8!RtE?w6p?GkGZ-6yt_p~5~4ecu|Sf^)6096%h*q-eNiEA1;Xwg)p~Q&iGSG7-IQ
z9aII&`ps$WOojFA`*bjG<mBv1n0hcYZWN0~(X01-hx(ExqWqaXgSk*@-GMp|K_3!5
z9|O21N3%~izh(4fbp9wzd+!b&7cVwSP5H00)m5ej-(s=Pl#(90UOhn@OA9u+D{i@r
za4*CP0I#<d-)-#xq5q-iY5nIef2s5OuQjcA>kFk|E@sHHuD}W^d`7YJ3YE^zrQnqR
zGoq?;YGKe)93o|_=^f%3U1KYZGPOXRRxK7w`UUbMMa3<86OmVH!EKP$8RCrn9mWX+
zC?9yF!fRVLmud3hF<}x;;sR}f(*r}6Gap3fR6zLHR~kbMgD{98N`L+r&?3p~*0+FX
zcAL%j=(SO}xTJUTvA`&Lf`2mv4koPG9&|<CA~EHbWHMoFPwT(&U=7t0`RoFZPO9Kq
zwwe$i=T|AXY#hD$aZlNMH`wZ%gwilGJ(zeYpOn*F3cy0XKXio^Sj#WXx=PWV`WGaA
B&~*R+
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-rebase.sample b/tests/resources/small.git/hooks/pre-rebase.sample
new file mode 100755
index 0000000000000000000000000000000000000000..193ee3f127387f469894675f0d621b5bb9c275da
GIT binary patch
literal 5006
zcmb7IX>;2+68$XxiXKL@ma;l5d2^5Ba_rPh_DHI-u1#&_upttZXp;no03$20|NFiM
zK#D#xQ>!Z3JkX8T-LDVm!B5j7y_{;JDmmTTef+K1oIiPzeEr+Ai*<2PUgnG4^ZB>p
z_fkAvoR1emuf~ri^K$-px=4#D-v<wZ2Xv&$O_eTJh6d4)=DWL(NBs9G{k<+yMMw0T
z$VH*-+LM)}u&m^`l8~1nt(3Z;R8v(KbY5#i3z+~8h0D}Xvq&3J8BMWDizPNpaeb~9
zBN9bSkthfXzskapf%Zt{*e#}{QaNiaAVZ4{$;;I6<vKNhO@%7P-(;l-x=pPoExHC!
zB(hA#cDdD?s4P=!)=-K{<kHAWKetl-8I8wwO<ihJNs-$dEvr;&S_@6E=mNSJ5;mg#
zyb)MbqKH<one{qrV;ZQ6WL}yLtyi*ekNLf|uC6M!)Cmq7*l?g0d6`MlE49|}>Y9w&
z`bCv#<YfTKtb`!}CyNYd;|(C?vRVQmWOfR9X?FZ#=f$No)^#4>2zVn=YnJyeNey(Y
zRh`9vtLw~A+5zsjp|W0Nsa|29Rm!B>OoG5a+vi;ari8O>KkU!KAWg_fa3btK2x*_@
z0bEc7J;Ubghm}n9bOi(Sv_B66nQ7U)J7f0fO}<cB8i8vG{r39s_>8Wuf*uorcIgEG
zOHc|-V6+HlRhOP}?Cn?@5iwSl43abmBA^2lyL$+cpabCGVES+v^j^FO_}?FIp<qP?
z#_?*YMHIkyZxJxS;h@A)WDJ0bN&+F-#_lFjAf^g_Pb#5YXqYe|dZUpa^zI)VOBXQQ
zAMhT>%En%Ll?Z*7*}TwrZyg5OSZ9rY-`aU~Mc-jjv{Ll)FLMgtB4ujktfQ`Xhqrka
zT=P!A;9w^;Z?PqpLwOLu=cj3L>TdUKw2;DMu)`oVkj}<z_EjO_2uWYuvKG==%Zu?h
zJiMVR^c30RbpW}<+z;jjoNC}YI4f6QC7d-08*4mCB1>#bcDx4tYg=j%D`+i{W~fVM
zVmZ>W9VMyin9c-0KzI_;iZ-g|OyzuG`Yq%(%dvl;ifnVr0;jWE&S`z|rQu=!yHBBO
zx`OJ;oOQ(KKM<$(bC38o>pD0%|HA(E0TRw7qj$fJ_pRN+7Nm>dS<q{AcOz#-;d7_2
zL$z(_nhH{vOzT(}>C(gLg{(`t+5Z=?o+}wXU4tHy+&%F&aRhFebeEhR2R5|<c6J);
zEY(q5F5<n*XP0|=PtPBn$B)V~d$Os-?&8UlaVe_|jdkzoWNsTP-_uyq4$R6o<h`&@
z{loXa{^#TF=NJBYu9qB?hs}x=It_O}ZjX(_wy9@X(lmkRpNi0{8T{O_b_j*JC^_SM
zz3G?1$KCNWF-|`Jbx2cQ-y5LW?Z2eikngTZmsx5C(@({8<l)Ue+gIp##Mosfa~iZN
zZ|NLN9uE6Xaqpwk+@D+f?$tg2JRCY`pwZwbR9OvEn*zYm`ffKIzl4{r{ZgjfpbuX)
z_r0=0y3)T-j$gljPyEJO*6Y<pj7G72aLoqaTpc=#u)*xJcVUm0;k(n;r)^DQNa6Oj
zWvlHEGg~lz`Q_8`yQ9<BZ;ylE1Z}bD<8}<uB9Y5lRB=;JUD0h?_)4H&EhJjvwzJx?
zr<o_vk&75j_5^d$8Z$_Oc1=R-I_DlNZ2@~81oV*J6%o3RuiCz}^VEW>$#Ycbp^w@t
zTl%=f1t=w+WpJzF<|CE@?SCNAz)%9?w33lQ8vrHJqPfH9@}qs*QXOG71W=ylx;wOB
zcx!Bj^)Yy6WX$a^vBkBJ5Cob<oubBW+a#9bX{0bkMTX_2sIrs`HMk@$q{dK5h2-g}
z({`~#gm#G?c#>qlaDx_B0c<3b+8)f84LCrt;e;qxc+7>VbwVK{skNv!wvBiTa^9Iu
zkwP;VK)jH$WJ{`MRwAA9fal!y0dtV;FWg8PTkWU>CwnqD>1ZX2B@;$DlX%C5MI+}{
z9xQVnffR*~v2KAUj*hCd<gRCjR7~6Q(vF%VT7j97iv3j4Z0p%mU`7t061~l7b$!#t
z3*Pveii}aQ?QD9aiX>gul~`bk#mk`o>zk9)<2Uc8?hUZAEvd!`9em)~$Z)zev>w^8
zyAgCP_$&Y)7HSQ84`xG}OeTavaEswwF|8Xpi5iZzZa@hCiv(J-%bfFC&)HLlO+Rhw
zG6g?9eL5&A!SuJnQ6}LxG%tU+@vZ`i+!+Rz6iYvsTdhnPo7lW{m-}{hya@viX4)XZ
zngaw+j;gloB#|UwI@8sOmQpc`h+bicQJnQIB5eifIMQNgD2+oai33m!34~xU|0Azj
zhu$8z+T5^;Pxx@d{N)pzOJLSa^e;aDf$W%N5XcOf!mGC9l9j$Ev2h6N+6ZQC+CJzl
zaM7?S!SrFLS2DASjj(h6y1WN3N?|bmqmyzm!&nLoE|`rKBOc_yDF$a#FsUn!IQf(t
zdC&Us(kQz*7mv<H12p@UD8XaLXdK{>H^j*^MC@>wTDb}g%~sx*ng#>{@lR=XG-Z5_
z#<9*Oh0joMzt;nS)ObAp)347`D=}r-;nV!TbIq&xrGRGsF6fZg+!VkfUei@_&l-M&
zPqQ+Dw)RV}+)I8RuqAxa`Pv8e&!_gXS=e2-un>=Ktn}-;%lLZxaVn?Q>yZCb2R3Wk
z77zr%;Rq&h|2ncqyKYmFI0148JVY7Q$V5p=dWj<MQ<8r+@QLq+UROk&%quICq^O2C
zp(17882jXI)_GaqzAY4AjoB_nh8k*r1mJ>+Qqpu%i|xp2<qF`Tw6&3`h654ceBjZ7
z4>C=WaOb2Wudn^h0EcD%$p9YVU1fnoRV9`(cy(vv6K>FXS!2jY>1GnU--7)4usH&K
zao*&P^@9~YmUe|ZdLW@C>H;!*<TIU}-!Tw#3oqZA*99}b3&uHiGO<{I6!pMnAiQdS
P!fA@Yk4s_DjDP<F98V`a
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-receive.sample b/tests/resources/small.git/hooks/pre-receive.sample
new file mode 100755
index 0000000000000000000000000000000000000000..32b597f8c32399017c3d7d51134d1f6490ad9575
GIT binary patch
literal 601
zcmZ`#+iu%141Kn~f>Vt3>Nw4M*;=?j(TBD#O@XCv0|MEhA;z}kTFRv@`tPHhp=&Yh
zg%Zhg4i7o_k{a5i&f5;tZ==%}^Sn4aD_6%qs<o-wO_Prn;}`SPs_*$C$(7T|$#C3`
zPt%-C8gelZ1GqAP8`ZQmg0{8-S9H{R@D>_XAuJt&EumdH4Yu`UjT<s+s_~uXh}qA8
zg|_HG)%7Pdc&$7*uR0HF@)~vmFjqyD?XZwCbLen^h5t)sm9<90Oa!@Y%8!~rF8G?W
zkzmCF8kMtuuelMHIAlqqnm?72LeGM1J4`w(kX9&%LQn}ForlDLjBoCyvxmo@x3kH^
z^loxLyPiDWPo-cBMnsg2M6}kuPGHEGBqVkC{D&9Kt%xFAsTw4QC1$_=fwBl=3dI+e
zaSxI}JS}=Z(Ec80eF`!Zq3mqapXI|UN!a)t;@4hcu%Eq2xcoW}%!><-+XHTuHss+b
YOmM2;hq8Egm*4=7_P9T{21QBYH*F=mfB*mh
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/prepare-commit-msg.sample b/tests/resources/small.git/hooks/prepare-commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..b1970da1b6d3f42f00069fd17c325de72cda812e
GIT binary patch
literal 1702
zcmb_cTW{Mo6n>t6#i?x6xmZ$SFLf{QfG*3r0L?Pg?px55l8$UTGO3bO;spKi{V3XX
z))weX0X>M9bNMcZ-6yG%>(n}JI2|25dr<ew@wmMG{l(3lx~bQz>}WZBP@ih?JX^+@
zu#5O48P>yRX(m<b*PU*sORp92TCD1dX`%HE+1$w5k<(Ngu7zQ83#MGJR?<<W=d@yL
z#heqwo{FmCg0g#x<~R+PBD#}q(MBn;V$x;%UrJPP3*l%XtlvTWChI2SfJ$9e`YvSj
zRSOQ?NUgSMLI`3vL48YBHzwzVXoe7v0ef|0YHgV$N@?N(-R)rPqRDrK$n(%+OF$`P
zWdjC5N~`#RjV9}aYwQ4_yF5O-$h2`>fDIhYP)doc1&TADZa@ZGpusJ$6G+e$ZMcmC
zoOosDQPS}l{H?YPsq(4;0SGkATa9eeqAaDcj<jNAU+LTSmM1jo(ti~T0B7acJnnTX
zTarYy;Husdxal0!S<ba8=uu&a*SNYtr7|d7$g-q3_JHER2*oBsVW~i~XXdMx%LW~0
zT*960LF<qZ6K&FZ;vGmtyywBU3^Q>q8n2wALbFwU@2i@FAaRV!=uw-nwx1gKn2SvY
z>Ff>;2sg!+Hxfkwv1lsiii=p6WenF=5)6LZc<a$zDCD$GRuwvG4Fr+B#h?#9gTbio
zk#MdxC@WY%zSGN#i}Ts_#q`bf-{)`7CcWeB*7WlIyHjioJJWw&A5VItPUq3^9!r}S
zbykelFV-VFvcr>QaZ=aS_}+-4Y&?!@HWh|<^gJ21!|T@+%On#w6azxPHV}XsRbe*w
zR_TZ2XEsQa1lPK~biYqg@0-RW@5J1@=<87cFzEUABdCoFH2CZo?}l(Z*!OFqUxo>K
z_d`l#4d9|H6;VPT{X?^{VJ>oL|D7K{BJwwqB>`YcPoGk+9hbvHnoQ{EM|kPgD_`wk
zKm4#2xu;-y`RAm!=L_BnLvJ8$AZm8@?)v<%vwvsw8AF2x6!mTT;c72A_~U9nIq0ST
zv)N0!I!^1p=g8-RQfx5)E_Mb_4I2vtQpI30XZ&t<!9D6nI|;V7YR3)l6=S~QhuwLQ
g$e&^kTY-M99*<-Iw@(78*M5W!4}U}|8YyMx3->-9h5!Hn
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/push-to-checkout.sample b/tests/resources/small.git/hooks/push-to-checkout.sample
new file mode 100755
index 0000000000000000000000000000000000000000..a80611e18896f212c390d845e49a3f6d5693b41d
GIT binary patch
literal 2840
zcmai0U31$u5PXh)#YOS7cE^-rw@uolNhe9&aUS|HtvhX>G$45tVUYj>fRdF?|9kfU
zNR~aG=E)WbEbeyq7JTw}ZuHIE2kUtL<<n;$&G!2F^Je|kx2ug=4L5!H^!ogx`7o$&
z%Il(3zAe6<oe$^F=A|}s`8}CDp*M#3M)gC-)LOeDUpYMl3YNy9R)I-T)pE7sy09aj
zJ7%&5PnSB-F#2{jc><WLR{I2izuK%VHc+{hRfXe<^_q)8RjcE(6WX+F2)iAtDtI{x
ztAHVBq)eSp_E^xcV^i_5KLIHA$g{zEjh?rsacu+(Eyvve2~Kmw%;n3g(kWB56j~Js
z<yE5tYUsAR&PY0wgRvM8x!zgLX8SI!eVY&}YZ|>AoeCNptd-NM1aZLhESzC;I`+Ns
zfmNNjdAp^W8#Q*}l>CT7RB9F5(BbI8ly2l~+E};JW|>&d1)=epZ-8vm8ppkbEVn#R
zt30a5A-c(YQR8eM5%;|UAnO>rt!&@x@G@yp+92%w-}%(5P_+P&Wf_zb$f-Qrl5(7z
z2ah(bkE;!DK(&aAMuQ%1TS>ai?wSXCOCSj=_}8x4IbCx^$}9q)<W{Y<9o<WqZo^oV
z3bR;Qa%VT-M~kU@2n{=+=)C!MD`12;@<F*EtPfV3K#g^1%&ggH?4{m<R$WEKcUI4%
zl6{ik6Bo46pmNjdXg5@?hn;SjG$}rr3Gy#-BGgVD$8oD)OcK(oqca)L_kk)UBZ_&6
z*ourb#Yc8l3J+uSda_Y$GY--5Zp3QK9w^?P%E0xb57?fY+Q#+KU4)+R>whwv)SBt|
zg#MX4;;Oau`m=MI9(^&zPbueY@~>3*ixX%mvR5m_1&nAg@ZKvY1E$O}&EtLiG;mhV
z1xhMIm~fGjmf_#{62f`y;09?I7M1W2tWQvz<}i9lR>OpQyUJi45_&*pQus&EkwY<>
zI|ZAx=*3i9a-)g)hXkvO7>UJ5MNgL(Z+-wpXVcgbSgpmFmbf1~DPA(OVGI&FNLeIE
zNH!_aiH$vsif$_j7=T2{cS(!DOI`~bn@)vSd-0d7xL=DF;UNP|tW}4i<qWTSNCe|y
zg9kV&L9g;FhC@tvsVu#V-brqShHy2qZpA!gg{ZTY>h>DvHtu9tY_pbJ6x(6E*hxgC
zzNDao%qlr-IE%YGbS4hF!n!on7#W3$bX-_hbZAaws^nHu#)Dx=WzdbJ>AKzAy@T$x
zSWE^x9+|TEHVEPyaPYa0DOChp?AeHSBBDbZNokQpAY{lE!7geZI=jV)G^2@<iI(N4
zJLJjy@Y<VI$yq;3bVpJICW3D?YDMDl4Oe5pDf{d`i1_3Qx%4uX`$dOz<9e}jm2B-u
z8-?%!7XuiNF2O&MAdl-iw{drG+$F^zT2Z7WRV#c6;IRZEf>l)&91Zb1+`T+oq9wWF
zRV~kGTGce0O~p^6mj{kT5kL(pv>r;Lvd7VDX*P>A^Th`$3cWO<svk?_?FeP@458*2
zA1PqUOdd%VP5&4~ocLK16{1GLll64c=mU81RMFstpnMoLuI7hNIE4N)U%h*#<Fz{C
z9#>0<l7lRrls|W(8=W1O80P~6+U7<4BqC}N4-4GR3w#{O7YmH?JxwJfru2d?e){$5
z@5R+`7Z;1)FW;OkE-(HPisCS;5F4O^Q>L81p4Ysdo3ZP1(SrR-peEdTo;-@bkB((G
zPHYQXUL!@Q$e(OQ;R9r%@Afz+50I7>*^^c&&|E*r-jN)LH=pM4AqMwWxSv|nqjddE
Z4{_hwv8!W(<d3>T<BC%y-6O5i(|^OS%`gA}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/sendemail-validate.sample b/tests/resources/small.git/hooks/sendemail-validate.sample
new file mode 100755
index 0000000000000000000000000000000000000000..effa6f54f2a5c1f0cebe258bf4805904be4b4852
GIT binary patch
literal 2365
zcmb_dU2oe)5PUX&#g-Nv2{5JDX_MA~3%Iq?8g*kpRgpYZIFU=~BI=I5J6e{T{`bz^
zk+$48h#x9Ika!=nv$M0y{clD}-j1x(hDWbnzP?l2k8j?TH{brS+Nf21yPm)Nczma>
zYw`X3V>TCdnSD1ru8&`j=2DIPbCT@SnIgUw>$+lEYP}+x8(BMYnr=iT3*ndq)xzaV
z>I+qjv}vC#8_9M+b1p#uNS0M0)q<p>8!3p_LRQ0MA3M`!2foxzRUjbFY@}O~(ki=S
zqscnq8cU*dY)D$$cqE}n)V0yIk>CNKHCrndOtSP*HbOb;nbwAHSb;R+gs^?^Dve%)
zoW}t(*D}$>O3ab0TS^-;J|u&sb-PkZzo#kn*#xYt(;<xzKW(YtQZ$u23?yV#kyh1~
z@+Idh;EG5jXx8@%<;Y_W8SA=|T;MPQ)TB!!<QcbU)YR4)79eeeg4|vp-8jm%Dl3^I
zRS7+i3>FGuwzSb^g&RDiGcOz9TB;Hu`nJh)$W=C=XCSm2AY=$w3G3P-V#Oo+N*;#2
z4ijJ-pBZ=;T(RTgp_HYrD!uW-dTMfkuqY5jwOy)~gM;#=P^i{!l7`pXTS^s(&^{RU
zydaw}OpS#^D1cXM8?FW+fh`t7D(g;yr6|}fdaNtZBx3hlK~IpkTu3!Qq%R+zAo#<L
zU&m+XIFB0>t}Bs8^3$vHD+-TGT@`F>H1Cc#WAVW;&$S6%fE2d6@kLS0g&ihIM{}0z
z8#XhD>b>3{(BH|Px7}&lJ4%y1v<t$W+!F|W@<gaU4;MqSHKR(wdML<XnCv;zaPrSi
zxVCvek26-bf#Q!H+uAgy_{e_1UZCq>(CihZJx@8MPoGdl*BJGD;usf*iS7%;{Joe;
zNFuBa>*~o&qETDPo~u&~$FxE1xb^x&(CbE`Y3GfsibL2rl+L;>P6j&Y3U>K$mkp*6
zd`Q{<^+^&;GskGjwD-%!boR&i-TC<Uvy02w+l$Nb?B}aL-%ZDpluqd=K_@}z*fyuV
zzAs1Hf?3v$k!X7GTc8Q=r`WJ_Uu=>A9UOR|@=GYb5x#<f&WSMH%mCJU<#=7=qFdL6
zG?Wz&6z&J<@I(B>+dhd7fkaVIR^pol`Mv+rUbmZ43dVL6^S7g3{NsPiG$iy$5EDB%
z6KIgnb$H(n&t3e4E6d4V7w^B?JS}JkG)PM6+X3Co`SQs($O*AA+MG~{S7RJ=cy-l&
z>~%3y`tjfx2>uOu<lDGWewcb=oL@}B@B6FCZ?oxSJWldrm%UfOduahk%C0H>tB_^s
ziwG=e=ch|FQ0IkN91US7rhdQkXhwwt$gU0WEVDjo=IPb+?6PC=s8}J*ua(Ms))`UL
fi$|vMHn?H<rred|1&u#kOoJ`%vx)-*g-ZSfgGvdP
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/update.sample b/tests/resources/small.git/hooks/update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..c3430f6fe27e572a1cebdb2f2bec70add0642a60
GIT binary patch
literal 3707
zcmb_fU2oeq6n(aS#jRq*br9K3y0z;^QgkUc^kKlTrB6js&@yE)mPu8l<0$L?`wmIT
zrsXba))@gJi@e|G+<SfSXe`CeSQ}OG@sr8ZTUlQ{dzM}Q@O-hBi}GeUom`#X%FiYH
zX?m4Rna-0RN2lfK)A3ZuvHcz$L<jUn62D=~vfz{}wIH2VqBLX_O$(JSXeF7H$}q!c
zWY}C&R;eX%X?P{%d;|>_tSE3ettp-hLlsZCxaLX8(nU;bVRB;Ce6@s#eu2|WvLz>-
zvy(&>Gyfp@+BtKnpqWkKi^+v{4jn_pNw_zeuxE<mRXKx8G3;9pl+45&4~hHW!G@wo
za7?X(0B}HbX*ExkDmas*xzV)FxygC8AL?2Z1x-0QJqS@qn8sD7r{bm30@<%eL_gOw
z;~85O$Xw2AS}Qp)5ViRUe3|ir8;&&I<B7Y6^!kkNyYXF4EY(b8_5DsTYn_&?wkdEz
z0y$tADo<&}nGs5kg2-J=0KlEGPb(%<An(pXY{K`qIZCuwiT|2{8JD&5o_~`o6<;dD
zi@J#zCE4={8j%<uy|iutvHu1QKo5Tno-BAF2FwD%%O#UDDum=w!;!PNe-cOFNX4)5
zd>TifiGO|)w}OANj2n2D^K=o3j6P6uOL70#cbA{uzWXDlk1wr9GV1X(2W{RuTvjXV
zCmd<W?kH^?PXjkbF`XZtwu1B+%4@ZvHIwGpJ*8@8`MWAhq^B|Hj1lzj3R8bVuMpNb
zz4Gzk!3T3bY;WEGIww&kq9BYW6EP*q$K|EB-@TH(Fjtz*`HMTO?i+3E;5tdSah&xZ
z+t!x4K7)dpy5wiJhlJz~8qF|r8a&-SV7^I3C@_q=P`yt@_x_F-;PQR)fzP<zNN>8u
zH%V`94=q3)Dk)PHNrnFC(T1)Om6f{Usj;u1R->&XoCYVK2V3ZlgZuF?N}1+33<P7e
z<0yVF?Qoa{l#7q(3&jv=Ab)gpM8A7`p%3InNzSxy)ZER2U0C#9zKpnLY0I?>OER*x
z*9Z=L=zI8CN>A_^jYjt0F$psO$sL=38q5q|SG)qCN6{^>RFh5E&l5GZ$pEahnF&d+
z5c>64t}uJPkf~_!VUj#&N%nC-gUMj%=@B=!V>&}xtj2%@-mOm#rQUSJ3(ccmc+fza
znZ#uxF>N?QN5UrIEd!5RgHEf<eGg}P45aAs(Xs6u!XW9r1I*E6XJ^1movX@xYLuPz
z|7xBN4z@b}#x>W#;(nKYF+D<*rdshJ$X-z2OZ2X;)nn@KSVdVhaA?}@3;6gZxb4<W
z`9sa`0lR_azMX|=t_(FvG@%w2);9P}SG0u&K1(*oX3};~=<<!N*F$UTSxD{V&6mgL
ztw9Ntc2eOF@c!OJytNC4T^#)Ien7-`dI{6s#co~0f^E3J<0Ty)l3xq2u@Y8DXTK>v
zozoWSr{{+!h}zGpumG3H`=AvWpm^9kW;J$Jp^Xl*?8ckr`fqN%c|Z;VC0|cM4vSrk
zH_O8Yvh85nvJp^;``wo8=z0f`FWg?`>gO#y1hjX1{}rTlg9rwIKia8eyGexA3GnuR
z`Rg~XZoW;0pA)vI8=p5!+6sIn#C^FCvR>ffv39h6SCNi9v);%WD;WZ`of_MgwyRWy
z-yY%n*Y>X8<Sf+RyB|Sr2YG@1w~%U$o`(5EDkJ|3$u=eMZA&_wxEsy<Xxh2k^tP?4
RGx&ZHQs^8zuIpu!=pQhfp8Eg*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/info/exclude b/tests/resources/small.git/info/exclude
new file mode 100644
index 0000000000000000000000000000000000000000..a5196d1be8fb59edf8062bef36d3a602e0812139
GIT binary patch
literal 240
zcmXYqK?=e!6h!x)VxWtv*mf_t5?px$aS_{}Hj?C*<cHXeXE&AZhT*-L3ZoI&*l1%Z
zqG?zr3TvQGZ__}H4(u*%p*rI=cU!%ya5ugfGATh66$IJHgu1Gs0-<N;$V+SsdE)?u
zIq;i$f#WE4f$_MWicZjMEob9LWKMR#iwZq54~QgST^6=i%u0lUkJu-_J**QBMq}ZG
Wth_)NDbl|`oQr&HAFQ5h`0jqAIZ*BZ
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e b/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e
new file mode 100644
index 0000000000000000000000000000000000000000..7d80b8d78e0dc55669d831a6638f48ec9fed0982
GIT binary patch
literal 50
zcmV-20L}k+0V^p=O;s>9W-v4`Ff%bx$Vkn}$!Ay}rnY6F$m-Kg*KD_+;Lx#g4|^&N
I02NaX#p`nv=Kufz
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b b/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b
new file mode 100644
index 0000000000000000000000000000000000000000..822bc151862ec3763cf2d3fa2372b93bbd3a4b65
GIT binary patch
literal 30
mcmb<m^geacKghr&@q@?NlP9kSYMj?U<r(;diNWtH+YSKNt_|)0
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4 b/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4
new file mode 100644
index 0000000000000000000000000000000000000000..5adfa88cc6b00bb19f2031b8ab61f6d07f9ccdb8
GIT binary patch
literal 130
zcmV-|0Db>>0i}&W3IZ_@1U=^!a~EV1casc=c+{&un1qQN*i9hD|0|m(2n|iwp*q%W
z%N;b$hu%cM`$TMo*~EnC1BFP&Pfj~;jZVKXQ96s_PhV<-XAROi+@-v8dBLUa`!;GB
k^i<X>XlEv8$>R)1G>9th&t3j;s7J{?^9n<zzF|~BaA?ar-~a#s
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/refs/heads/master b/tests/resources/small.git/refs/heads/master
new file mode 100644
index 0000000000000000000000000000000000000000..4eb36d22298f060fd324155ab854d9d6486fc498
GIT binary patch
literal 41
ucmV~$!4Uu;2m`Rc)5uXl$AMP&AHjriQg~T$i(A>|7U^`%mXoWC24Q^m!3%@{
literal 0
HcmV?d00001
@@ -69,7 +69,7 @@ struct InstallableFlake : InstallableValue
*/
std::vector<ref<eval_cache::AttrCursor>> getCursors(EvalState & state) override;
std::shared_ptr<flake::LockedFlake> getLockedFlake() const;
ref<flake::LockedFlake> getLockedFlake() const;
FlakeRef nixpkgsFlakeRef() const;
};
@@ -87,6 +87,4 @@ static inline FlakeRef defaultNixpkgsFlakeRef()
return FlakeRef::fromAttrs(fetchSettings, {{"type", "indirect"}, {"id", "nixpkgs"}});
}
ref<eval_cache::EvalCache> openEvalCache(EvalState & state, std::shared_ptr<flake::LockedFlake> lockedFlake);
} // namespace nix
+4 -4
View File
@@ -185,16 +185,16 @@ std::vector<ref<eval_cache::AttrCursor>> InstallableFlake::getCursors(EvalState
return res;
}
std::shared_ptr<flake::LockedFlake> InstallableFlake::getLockedFlake() const
ref<flake::LockedFlake> InstallableFlake::getLockedFlake() const
{
if (!_lockedFlake) {
flake::LockFlags lockFlagsApplyConfig = lockFlags;
// FIXME why this side effect?
lockFlagsApplyConfig.applyNixConfig = true;
_lockedFlake =
std::make_shared<flake::LockedFlake>(lockFlake(flakeSettings, *state, flakeRef, lockFlagsApplyConfig));
_lockedFlake = make_ref<flake::LockedFlake>(lockFlake(flakeSettings, *state, flakeRef, lockFlagsApplyConfig));
}
return _lockedFlake;
// _lockedFlake is now non-null but still just a shared_ptr
return ref<flake::LockedFlake>(_lockedFlake);
}
FlakeRef InstallableFlake::nixpkgsFlakeRef() const
+1 -38
View File
@@ -342,8 +342,7 @@ void completeFlakeRefWithFragment(
parseFlakeRef(fetchSettings, expandTilde(flakeRefS), std::filesystem::current_path().string());
auto evalCache = openEvalCache(
*evalState,
std::make_shared<flake::LockedFlake>(lockFlake(flakeSettings, *evalState, flakeRef, lockFlags)));
*evalState, make_ref<flake::LockedFlake>(lockFlake(flakeSettings, *evalState, flakeRef, lockFlags)));
auto root = evalCache->getRoot();
@@ -443,42 +442,6 @@ static StorePath getDeriver(ref<Store> store, const Installable & i, const Store
return *derivers.begin();
}
ref<eval_cache::EvalCache> openEvalCache(EvalState & state, std::shared_ptr<flake::LockedFlake> lockedFlake)
{
auto fingerprint = evalSettings.useEvalCache && evalSettings.pureEval
? lockedFlake->getFingerprint(state.store, state.fetchSettings)
: std::nullopt;
auto rootLoader = [&state, lockedFlake]() {
/* For testing whether the evaluation cache is
complete. */
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
auto vFlake = state.allocValue();
flake::callFlake(state, *lockedFlake, *vFlake);
state.forceAttrs(*vFlake, noPos, "while parsing cached flake data");
auto aOutputs = vFlake->attrs()->get(state.symbols.create("outputs"));
assert(aOutputs);
return aOutputs->value;
};
if (fingerprint) {
auto search = state.evalCaches.find(fingerprint.value());
if (search == state.evalCaches.end()) {
search =
state.evalCaches
.emplace(fingerprint.value(), make_ref<nix::eval_cache::EvalCache>(fingerprint, state, rootLoader))
.first;
}
return search->second;
} else {
return make_ref<nix::eval_cache::EvalCache>(std::nullopt, state, rootLoader);
}
}
Installables SourceExprCommand::parseInstallables(ref<Store> store, std::vector<std::string> ss)
{
Installables result;
-1
View File
@@ -67,7 +67,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'built-path.cc',
-1
View File
@@ -28,7 +28,6 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_expr.cc',
-1
View File
@@ -31,7 +31,6 @@ rapidcheck = dependency('rapidcheck')
deps_public += rapidcheck
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'tests/value/context.cc',
+1 -2
View File
@@ -45,7 +45,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'derived-path.cc',
@@ -83,7 +82,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
},
protocol : 'gtest',
-1
View File
@@ -62,7 +62,6 @@ mkMesonExecutable (finalAttrs: {
mkdir -p "$HOME"
''
+ ''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
-1
View File
@@ -97,7 +97,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
parser_tab = custom_target(
input : 'parser.y',
-1
View File
@@ -32,7 +32,6 @@ add_project_arguments(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_fetchers.cc',
+1 -2
View File
@@ -37,7 +37,6 @@ libgit2 = dependency('libgit2')
deps_private += libgit2
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'access-tokens.cc',
@@ -64,7 +63,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
},
protocol : 'gtest',
-1
View File
@@ -61,7 +61,6 @@ mkMesonExecutable (finalAttrs: {
buildInputs = [ writableTmpDirAsHomeHook ];
}
''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
+7 -9
View File
@@ -548,13 +548,10 @@ struct SourceHutInputScheme : GitArchiveInputScheme
std::string refUri;
if (ref == "HEAD") {
auto file = store->toRealPath(
downloadFile(store, *input.settings, fmt("%s/HEAD", base_url), "source", headers).storePath);
std::ifstream is(file);
std::string line;
getline(is, line);
auto downloadFileResult = downloadFile(store, *input.settings, fmt("%s/HEAD", base_url), "source", headers);
auto contents = nix::ref(store->getFSAccessor(downloadFileResult.storePath))->readFile(CanonPath::root);
auto remoteLine = git::parseLsRemoteLine(line);
auto remoteLine = git::parseLsRemoteLine(getLine(contents).first);
if (!remoteLine) {
throw BadURL("in '%d', couldn't resolve HEAD ref '%d'", input.to_string(), ref);
}
@@ -564,9 +561,10 @@ struct SourceHutInputScheme : GitArchiveInputScheme
}
std::regex refRegex(refUri);
auto file = store->toRealPath(
downloadFile(store, *input.settings, fmt("%s/info/refs", base_url), "source", headers).storePath);
std::ifstream is(file);
auto downloadFileResult =
downloadFile(store, *input.settings, fmt("%s/info/refs", base_url), "source", headers);
auto contents = nix::ref(store->getFSAccessor(downloadFileResult.storePath))->readFile(CanonPath::root);
std::istringstream is(contents);
std::string line;
std::optional<std::string> id;
-1
View File
@@ -32,7 +32,6 @@ libgit2 = dependency('libgit2', version : '>= 1.9')
deps_private += libgit2
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'attrs.cc',
-1
View File
@@ -32,7 +32,6 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_flake.cc',
+1 -2
View File
@@ -34,7 +34,6 @@ gtest = dependency('gtest', main : true)
deps_private += gtest
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'flakeref.cc',
@@ -59,7 +58,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
'NIX_CONFIG' : 'extra-experimental-features = flakes',
'HOME' : meson.current_build_dir() / 'test-home',
-1
View File
@@ -59,7 +59,6 @@ mkMesonExecutable (finalAttrs: {
buildInputs = [ writableTmpDirAsHomeHook ];
}
(''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
export NIX_CONFIG="extra-experimental-features = flakes"
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
+40 -2
View File
@@ -1,6 +1,9 @@
#include "nix/util/terminal.hh"
#include "nix/util/ref.hh"
#include "nix/util/environment-variables.hh"
#include "nix/flake/flake.hh"
#include "nix/expr/eval.hh"
#include "nix/expr/eval-cache.hh"
#include "nix/expr/eval-settings.hh"
#include "nix/flake/lockfile.hh"
#include "nix/expr/primops.hh"
@@ -924,8 +927,6 @@ void callFlake(EvalState & state, const LockedFlake & lockedFlake, Value & vRes)
state.callFunction(*vCallFlake, args, vRes, noPos);
}
} // namespace flake
std::optional<Fingerprint> LockedFlake::getFingerprint(ref<Store> store, const fetchers::Settings & fetchSettings) const
{
if (lockFile.isUnlocked(fetchSettings))
@@ -953,4 +954,41 @@ std::optional<Fingerprint> LockedFlake::getFingerprint(ref<Store> store, const f
Flake::~Flake() {}
ref<eval_cache::EvalCache> openEvalCache(EvalState & state, ref<const LockedFlake> lockedFlake)
{
auto fingerprint = state.settings.useEvalCache && state.settings.pureEval
? lockedFlake->getFingerprint(state.store, state.fetchSettings)
: std::nullopt;
auto rootLoader = [&state, lockedFlake]() {
/* For testing whether the evaluation cache is
complete. */
if (getEnv("NIX_ALLOW_EVAL").value_or("1") == "0")
throw Error("not everything is cached, but evaluation is not allowed");
auto vFlake = state.allocValue();
callFlake(state, *lockedFlake, *vFlake);
state.forceAttrs(*vFlake, noPos, "while parsing cached flake data");
auto aOutputs = vFlake->attrs()->get(state.symbols.create("outputs"));
assert(aOutputs);
return aOutputs->value;
};
if (fingerprint) {
auto search = state.evalCaches.find(fingerprint.value());
if (search == state.evalCaches.end()) {
search = state.evalCaches
.emplace(fingerprint.value(), make_ref<eval_cache::EvalCache>(fingerprint, state, rootLoader))
.first;
}
return search->second;
} else {
return make_ref<eval_cache::EvalCache>(std::nullopt, state, rootLoader);
}
}
} // namespace flake
} // namespace nix
+6
View File
@@ -5,6 +5,7 @@
#include "nix/flake/flakeref.hh"
#include "nix/flake/lockfile.hh"
#include "nix/expr/value.hh"
#include "nix/expr/eval-cache.hh"
namespace nix {
@@ -218,6 +219,11 @@ lockFlake(const Settings & settings, EvalState & state, const FlakeRef & flakeRe
void callFlake(EvalState & state, const LockedFlake & lockedFlake, Value & v);
/**
* Open an evaluation cache for a flake.
*/
ref<eval_cache::EvalCache> openEvalCache(EvalState & state, ref<const LockedFlake> lockedFlake);
} // namespace flake
void emitTreeAttrs(
-1
View File
@@ -29,7 +29,6 @@ nlohmann_json = dependency('nlohmann_json', version : '>= 3.9')
deps_public += nlohmann_json
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
subdir('nix-meson-build-support/generate-header')
-1
View File
@@ -28,7 +28,6 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_main.cc',
-1
View File
@@ -53,7 +53,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'common-args.cc',
-1
View File
@@ -26,7 +26,6 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_store.cc',
-1
View File
@@ -29,7 +29,6 @@ rapidcheck = dependency('rapidcheck')
deps_public += rapidcheck
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'derived-path.cc',
+2 -3
View File
@@ -52,7 +52,6 @@ gtest = dependency('gmock')
deps_private += gtest
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'common-protocol.cc',
@@ -105,7 +104,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
'HOME' : meson.current_build_dir() / 'test-home',
'NIX_REMOTE' : meson.current_build_dir() / 'test-home' / 'store',
@@ -139,7 +138,7 @@ if get_option('benchmarks')
benchmark(
'nix-store-benchmarks',
benchmark_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
},
)
-1
View File
@@ -83,7 +83,6 @@ mkMesonExecutable (finalAttrs: {
}
(
''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${data + "/src/libstore-tests/data"}
export NIX_REMOTE=$HOME/store
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
+127
View File
@@ -15,4 +15,131 @@ TEST(S3BinaryCacheStore, constructConfig)
} // namespace nix
#elif NIX_WITH_CURL_S3
# include "nix/store/http-binary-cache-store.hh"
# include "nix/store/filetransfer.hh"
# include "nix/store/s3-url.hh"
# include <gtest/gtest.h>
namespace nix {
TEST(S3BinaryCacheStore, constructConfig)
{
S3BinaryCacheStoreConfig config{"s3", "foobar", {}};
// The bucket name is stored as the host part of the authority in cacheUri
EXPECT_EQ(
config.cacheUri,
(ParsedURL{
.scheme = "s3",
.authority = ParsedURL::Authority{.host = "foobar"},
}));
}
TEST(S3BinaryCacheStore, constructConfigWithRegion)
{
Store::Config::Params params{{"region", "eu-west-1"}};
S3BinaryCacheStoreConfig config{"s3", "my-bucket", params};
EXPECT_EQ(
config.cacheUri,
(ParsedURL{
.scheme = "s3",
.authority = ParsedURL::Authority{.host = "my-bucket"},
.query = (StringMap) {{"region", "eu-west-1"}},
}));
EXPECT_EQ(config.region.get(), "eu-west-1");
}
TEST(S3BinaryCacheStore, defaultSettings)
{
S3BinaryCacheStoreConfig config{"s3", "test-bucket", {}};
EXPECT_EQ(
config.cacheUri,
(ParsedURL{
.scheme = "s3",
.authority = ParsedURL::Authority{.host = "test-bucket"},
}));
// Check default values
EXPECT_EQ(config.region.get(), "us-east-1");
EXPECT_EQ(config.profile.get(), "default");
EXPECT_EQ(config.scheme.get(), "https");
EXPECT_EQ(config.endpoint.get(), "");
}
/**
* Test that S3BinaryCacheStore properly preserves S3-specific parameters
*/
TEST(S3BinaryCacheStore, s3StoreConfigPreservesParameters)
{
StringMap params;
params["region"] = "eu-west-1";
params["endpoint"] = "custom.s3.com";
S3BinaryCacheStoreConfig config("s3", "test-bucket", params);
// The config should preserve S3-specific parameters
EXPECT_EQ(
config.cacheUri,
(ParsedURL{
.scheme = "s3",
.authority = ParsedURL::Authority{.host = "test-bucket"},
.query = (StringMap) {{"region", "eu-west-1"}, {"endpoint", "custom.s3.com"}},
}));
}
/**
* Test that S3 store scheme is properly registered
*/
TEST(S3BinaryCacheStore, s3SchemeRegistration)
{
auto schemes = S3BinaryCacheStoreConfig::uriSchemes();
EXPECT_TRUE(schemes.count("s3") > 0) << "S3 scheme should be supported";
// Verify HttpBinaryCacheStoreConfig doesn't directly list S3
auto httpSchemes = HttpBinaryCacheStoreConfig::uriSchemes();
EXPECT_FALSE(httpSchemes.count("s3") > 0) << "HTTP store shouldn't directly list S3 scheme";
}
/**
* Test that only S3-specific parameters are preserved in cacheUri,
* while non-S3 store parameters are not propagated to the URL
*/
TEST(S3BinaryCacheStore, parameterFiltering)
{
StringMap params;
params["region"] = "eu-west-1";
params["endpoint"] = "minio.local";
params["want-mass-query"] = "true"; // Non-S3 store parameter
params["priority"] = "10"; // Non-S3 store parameter
S3BinaryCacheStoreConfig config("s3", "test-bucket", params);
// Only S3-specific params should be in cacheUri.query
EXPECT_EQ(
config.cacheUri,
(ParsedURL{
.scheme = "s3",
.authority = ParsedURL::Authority{.host = "test-bucket"},
.query = (StringMap) {{"region", "eu-west-1"}, {"endpoint", "minio.local"}},
}));
// But the non-S3 params should still be set on the config
EXPECT_EQ(config.wantMassQuery.get(), true);
EXPECT_EQ(config.priority.get(), 10);
// And all params (S3 and non-S3) should be returned by getReference()
auto ref = config.getReference();
EXPECT_EQ(ref.params["region"], "eu-west-1");
EXPECT_EQ(ref.params["endpoint"], "minio.local");
EXPECT_EQ(ref.params["want-mass-query"], "true");
EXPECT_EQ(ref.params["priority"], "10");
}
} // namespace nix
#endif
+63 -50
View File
@@ -24,6 +24,22 @@ namespace nix {
namespace {
// Global credential provider cache using boost's concurrent map
// Key: profile name (empty string for default profile)
using CredentialProviderCache =
boost::concurrent_flat_map<std::string, std::shared_ptr<Aws::Crt::Auth::ICredentialsProvider>>;
static CredentialProviderCache credentialProviderCache;
/**
* Clear all cached credential providers.
* Called automatically by CrtWrapper destructor during static destruction.
*/
static void clearAwsCredentialsCache()
{
credentialProviderCache.clear();
}
static void initAwsCrt()
{
struct CrtWrapper
@@ -95,13 +111,6 @@ static AwsCredentials getCredentialsFromProvider(std::shared_ptr<Aws::Crt::Auth:
return fut.get(); // This will throw if set_exception was called
}
// Global credential provider cache using boost's concurrent map
// Key: profile name (empty string for default profile)
using CredentialProviderCache =
boost::concurrent_flat_map<std::string, std::shared_ptr<Aws::Crt::Auth::ICredentialsProvider>>;
static CredentialProviderCache credentialProviderCache;
} // anonymous namespace
AwsCredentials getAwsCredentials(const std::string & profile)
@@ -109,48 +118,57 @@ AwsCredentials getAwsCredentials(const std::string & profile)
// Get or create credential provider with caching
std::shared_ptr<Aws::Crt::Auth::ICredentialsProvider> provider;
// Try to find existing provider
credentialProviderCache.visit(profile, [&](const auto & pair) { provider = pair.second; });
// Use try_emplace_and_cvisit for atomic get-or-create
// This prevents race conditions where multiple threads create providers
credentialProviderCache.try_emplace_and_cvisit(
profile,
nullptr, // Placeholder - will be replaced in f1 before any thread can see it
[&](auto & kv) {
// f1: Called atomically during insertion with non-const reference
// Other threads are blocked until we finish, so nullptr is never visible
debug(
"[pid=%d] creating new AWS credential provider for profile '%s'",
getpid(),
profile.empty() ? "(default)" : profile.c_str());
if (!provider) {
// Create new provider if not found
debug(
"[pid=%d] creating new AWS credential provider for profile '%s'",
getpid(),
profile.empty() ? "(default)" : profile.c_str());
try {
initAwsCrt();
try {
initAwsCrt();
if (profile.empty()) {
Aws::Crt::Auth::CredentialsProviderChainDefaultConfig config;
config.Bootstrap = Aws::Crt::ApiHandle::GetOrCreateStaticDefaultClientBootstrap();
kv.second = Aws::Crt::Auth::CredentialsProvider::CreateCredentialsProviderChainDefault(config);
} else {
Aws::Crt::Auth::CredentialsProviderProfileConfig config;
config.Bootstrap = Aws::Crt::ApiHandle::GetOrCreateStaticDefaultClientBootstrap();
// This is safe because the underlying C library will copy this string
// c.f. https://github.com/awslabs/aws-c-auth/blob/main/source/credentials_provider_profile.c#L220
config.ProfileNameOverride = Aws::Crt::ByteCursorFromCString(profile.c_str());
kv.second = Aws::Crt::Auth::CredentialsProvider::CreateCredentialsProviderProfile(config);
}
if (profile.empty()) {
Aws::Crt::Auth::CredentialsProviderChainDefaultConfig config;
config.Bootstrap = Aws::Crt::ApiHandle::GetOrCreateStaticDefaultClientBootstrap();
provider = Aws::Crt::Auth::CredentialsProvider::CreateCredentialsProviderChainDefault(config);
} else {
Aws::Crt::Auth::CredentialsProviderProfileConfig config;
config.Bootstrap = Aws::Crt::ApiHandle::GetOrCreateStaticDefaultClientBootstrap();
// This is safe because the underlying C library will copy this string
// c.f. https://github.com/awslabs/aws-c-auth/blob/main/source/credentials_provider_profile.c#L220
config.ProfileNameOverride = Aws::Crt::ByteCursorFromCString(profile.c_str());
provider = Aws::Crt::Auth::CredentialsProvider::CreateCredentialsProviderProfile(config);
if (!kv.second) {
throw AwsAuthError(
"Failed to create AWS credentials provider for %s",
profile.empty() ? "default profile" : fmt("profile '%s'", profile));
}
provider = kv.second;
} catch (Error & e) {
// Exception during creation - remove the entry to allow retry
credentialProviderCache.erase(profile);
e.addTrace({}, "for AWS profile: %s", profile.empty() ? "(default)" : profile);
throw;
} catch (...) {
// Non-Error exception - still need to clean up
credentialProviderCache.erase(profile);
throw;
}
} catch (Error & e) {
e.addTrace(
{},
"while creating AWS credentials provider for %s",
profile.empty() ? "default profile" : fmt("profile '%s'", profile));
throw;
}
if (!provider) {
throw AwsAuthError(
"Failed to create AWS credentials provider for %s",
profile.empty() ? "default profile" : fmt("profile '%s'", profile));
}
// Insert into cache (try_emplace is thread-safe and won't overwrite if another thread added it)
credentialProviderCache.try_emplace(profile, provider);
}
},
[&](const auto & kv) {
// f2: Called if key already exists (const reference)
provider = kv.second;
});
return getCredentialsFromProvider(provider);
}
@@ -160,11 +178,6 @@ void invalidateAwsCredentials(const std::string & profile)
credentialProviderCache.erase(profile);
}
void clearAwsCredentialsCache()
{
credentialProviderCache.clear();
}
AwsCredentials preResolveAwsCredentials(const ParsedS3URL & s3Url)
{
std::string profile = s3Url.profile.value_or("");
+2 -2
View File
@@ -296,8 +296,8 @@ void BinaryCacheStore::addToStore(
const ValidPathInfo & info, Source & narSource, RepairFlag repair, CheckSigsFlag checkSigs)
{
if (!repair && isValidPath(info.path)) {
// FIXME: copyNAR -> null sink
narSource.drain();
NullFileSystemObjectSink s;
parseDump(s, narSource);
return;
}
+27
View File
@@ -0,0 +1,27 @@
#include "nix/util/json-utils.hh"
#include "nix/store/build/derivation-builder.hh"
namespace nlohmann {
using namespace nix;
ExternalBuilder adl_serializer<ExternalBuilder>::from_json(const json & json)
{
auto obj = getObject(json);
return {
.systems = valueAt(obj, "systems"),
.program = valueAt(obj, "program"),
.args = valueAt(obj, "args"),
};
}
void adl_serializer<ExternalBuilder>::to_json(json & json, const ExternalBuilder & eb)
{
json = {
{"systems", eb.systems},
{"program", eb.program},
{"args", eb.args},
};
}
} // namespace nlohmann
+86 -172
View File
@@ -1,4 +1,5 @@
#include "nix/store/build/derivation-building-goal.hh"
#include "nix/store/build/derivation-resolution-goal.hh"
#include "nix/store/build/derivation-env-desugar.hh"
#include "nix/store/build/derivation-trampoline-goal.hh"
#ifndef _WIN32 // TODO enable build hook on Windows
@@ -27,22 +28,21 @@
namespace nix {
DerivationBuildingGoal::DerivationBuildingGoal(
const StorePath & drvPath, const Derivation & drv_, Worker & worker, BuildMode buildMode)
const StorePath & drvPath, const Derivation & drv, Worker & worker, BuildMode buildMode)
: Goal(worker, gaveUpOnSubstitution())
, drvPath(drvPath)
, drv{std::make_unique<Derivation>(drv)}
, buildMode(buildMode)
{
drv = std::make_unique<Derivation>(drv_);
try {
drvOptions =
std::make_unique<DerivationOptions>(DerivationOptions::fromStructuredAttrs(drv->env, drv->structuredAttrs));
std::make_unique<DerivationOptions>(DerivationOptions::fromStructuredAttrs(drv.env, drv.structuredAttrs));
} catch (Error & e) {
e.addTrace({}, "while parsing derivation '%s'", worker.store.printStorePath(drvPath));
throw;
}
name = fmt("building of '%s' from in-memory derivation", worker.store.printStorePath(drvPath));
name = fmt("building derivation '%s'", worker.store.printStorePath(drvPath));
trace("created");
/* Prevent the .chroot directory from being
@@ -67,11 +67,7 @@ DerivationBuildingGoal::~DerivationBuildingGoal()
std::string DerivationBuildingGoal::key()
{
/* Ensure that derivations get built in order of their name,
i.e. a derivation named "aardvark" always comes before
"baboon". And substitution goals always happen before
derivation goals (due to "bd$"). */
return "bd$" + std::string(drvPath.name()) + "$" + worker.store.printStorePath(drvPath);
return "dd$" + std::string(drvPath.name()) + "$" + worker.store.printStorePath(drvPath);
}
void DerivationBuildingGoal::killChild()
@@ -93,18 +89,6 @@ void DerivationBuildingGoal::timedOut(Error && ex)
[[maybe_unused]] Done _ = doneFailure({BuildResult::Failure::TimedOut, std::move(ex)});
}
/**
* Used for `inputGoals` local variable below
*/
struct value_comparison
{
template<typename T>
bool operator()(const ref<T> & lhs, const ref<T> & rhs) const
{
return *lhs < *rhs;
}
};
std::string showKnownOutputs(const StoreDirConfig & store, const Derivation & drv)
{
std::string msg;
@@ -129,46 +113,6 @@ Goal::Co DerivationBuildingGoal::gaveUpOnSubstitution()
{
Goals waitees;
std::map<ref<const SingleDerivedPath>, GoalPtr, value_comparison> inputGoals;
{
std::function<void(ref<const SingleDerivedPath>, const DerivedPathMap<StringSet>::ChildNode &)>
addWaiteeDerivedPath;
addWaiteeDerivedPath = [&](ref<const SingleDerivedPath> inputDrv,
const DerivedPathMap<StringSet>::ChildNode & inputNode) {
if (!inputNode.value.empty()) {
auto g = worker.makeGoal(
DerivedPath::Built{
.drvPath = inputDrv,
.outputs = inputNode.value,
},
buildMode == bmRepair ? bmRepair : bmNormal);
inputGoals.insert_or_assign(inputDrv, g);
waitees.insert(std::move(g));
}
for (const auto & [outputName, childNode] : inputNode.childMap)
addWaiteeDerivedPath(
make_ref<SingleDerivedPath>(SingleDerivedPath::Built{inputDrv, outputName}), childNode);
};
for (const auto & [inputDrvPath, inputNode] : drv->inputDrvs.map) {
/* Ensure that pure, non-fixed-output derivations don't
depend on impure derivations. */
if (experimentalFeatureSettings.isEnabled(Xp::ImpureDerivations) && !drv->type().isImpure()
&& !drv->type().isFixed()) {
auto inputDrv = worker.evalStore.readDerivation(inputDrvPath);
if (inputDrv.type().isImpure())
throw Error(
"pure derivation '%s' depends on impure derivation '%s'",
worker.store.printStorePath(drvPath),
worker.store.printStorePath(inputDrvPath));
}
addWaiteeDerivedPath(makeConstantStorePathRef(inputDrvPath), inputNode);
}
}
/* Copy the input sources from the eval store to the build
store.
@@ -213,88 +157,22 @@ Goal::Co DerivationBuildingGoal::gaveUpOnSubstitution()
/* Determine the full set of input paths. */
/* First, the input derivations. */
{
auto & fullDrv = *drv;
auto resolutionGoal = worker.makeDerivationResolutionGoal(drvPath, *drv, buildMode);
{
Goals waitees{resolutionGoal};
co_await await(std::move(waitees));
}
if (nrFailed != 0) {
co_return doneFailure({BuildResult::Failure::DependencyFailed, "resolution failed"});
}
auto drvType = fullDrv.type();
bool resolveDrv =
std::visit(
overloaded{
[&](const DerivationType::InputAddressed & ia) {
/* must resolve if deferred. */
return ia.deferred;
},
[&](const DerivationType::ContentAddressed & ca) {
return !fullDrv.inputDrvs.map.empty()
&& (ca.fixed
/* Can optionally resolve if fixed, which is good
for avoiding unnecessary rebuilds. */
? experimentalFeatureSettings.isEnabled(Xp::CaDerivations)
/* Must resolve if floating and there are any inputs
drvs. */
: true);
},
[&](const DerivationType::Impure &) { return true; }},
drvType.raw)
/* no inputs are outputs of dynamic derivations */
|| std::ranges::any_of(fullDrv.inputDrvs.map.begin(), fullDrv.inputDrvs.map.end(), [](auto & pair) {
return !pair.second.childMap.empty();
});
if (resolutionGoal->resolvedDrv) {
auto & [pathResolved, drvResolved] = *resolutionGoal->resolvedDrv;
if (resolveDrv && !fullDrv.inputDrvs.map.empty()) {
experimentalFeatureSettings.require(Xp::CaDerivations);
/* We are be able to resolve this derivation based on the
now-known results of dependencies. If so, we become a
stub goal aliasing that resolved derivation goal. */
std::optional attempt = fullDrv.tryResolve(
worker.store,
[&](ref<const SingleDerivedPath> drvPath, const std::string & outputName) -> std::optional<StorePath> {
auto mEntry = get(inputGoals, drvPath);
if (!mEntry)
return std::nullopt;
auto & buildResult = (*mEntry)->buildResult;
return std::visit(
overloaded{
[](const BuildResult::Failure &) -> std::optional<StorePath> { return std::nullopt; },
[&](const BuildResult::Success & success) -> std::optional<StorePath> {
auto i = get(success.builtOutputs, outputName);
if (!i)
return std::nullopt;
return i->outPath;
},
},
buildResult.inner);
});
if (!attempt) {
/* TODO (impure derivations-induced tech debt) (see below):
The above attempt should have found it, but because we manage
inputDrvOutputs statefully, sometimes it gets out of sync with
the real source of truth (store). So we query the store
directly if there's a problem. */
attempt = fullDrv.tryResolve(worker.store, &worker.evalStore);
}
assert(attempt);
Derivation drvResolved{std::move(*attempt)};
auto pathResolved = writeDerivation(worker.store, drvResolved);
auto msg =
fmt("resolved derivation: '%s' -> '%s'",
worker.store.printStorePath(drvPath),
worker.store.printStorePath(pathResolved));
act = std::make_unique<Activity>(
*logger,
lvlInfo,
actBuildWaiting,
msg,
Logger::Fields{
worker.store.printStorePath(drvPath),
worker.store.printStorePath(pathResolved),
});
/* Store the resolved derivation, as part of the record of
what we're actually building */
writeDerivation(worker.store, drvResolved);
/* TODO https://github.com/NixOS/nix/issues/13247 we should
let the calling goal do this, so it has a change to pass
@@ -383,7 +261,7 @@ Goal::Co DerivationBuildingGoal::gaveUpOnSubstitution()
/* If we get this far, we know no dynamic drvs inputs */
for (auto & [depDrvPath, depNode] : fullDrv.inputDrvs.map) {
for (auto & [depDrvPath, depNode] : drv->inputDrvs.map) {
for (auto & outputName : depNode.value) {
/* Don't need to worry about `inputGoals`, because
impure derivations are always resolved above. Can
@@ -491,6 +369,8 @@ Goal::Co DerivationBuildingGoal::tryToBuild()
bool useHook;
const ExternalBuilder * externalBuilder = nullptr;
while (true) {
trace("trying to build");
@@ -584,7 +464,44 @@ Goal::Co DerivationBuildingGoal::tryToBuild()
co_await waitForAWhile();
continue;
case rpDecline:
/* We should do it ourselves. */
/* We should do it ourselves.
Now that we've decided we can't / won't do a remote build, check
that we can in fact build locally. First see if there is an
external builder for a "semi-local build". If there is, prefer to
use that. If there is not, then check if we can do a "true" local
build. */
externalBuilder = settings.findExternalDerivationBuilderIfSupported(*drv);
if (!externalBuilder && !drvOptions->canBuildLocally(worker.store, *drv)) {
auto msg =
fmt("Cannot build '%s'.\n"
"Reason: " ANSI_RED "required system or feature not available" ANSI_NORMAL
"\n"
"Required system: '%s' with features {%s}\n"
"Current system: '%s' with features {%s}",
Magenta(worker.store.printStorePath(drvPath)),
Magenta(drv->platform),
concatStringsSep(", ", drvOptions->getRequiredSystemFeatures(*drv)),
Magenta(settings.thisSystem),
concatStringsSep<StringSet>(", ", worker.store.Store::config.systemFeatures));
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their hardware -
// we should tell them to run the command to install Darwin 2
if (drv->platform == "x86_64-darwin" && settings.thisSystem == "aarch64-darwin")
msg += fmt(
"\nNote: run `%s` to run programs for x86_64-darwin",
Magenta(
"/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
#ifndef _WIN32 // TODO enable `DerivationBuilder` on Windows
builder.reset();
#endif
outputLocks.unlock();
worker.permanentFailure = true;
co_return doneFailure({BuildResult::Failure::InputRejected, std::move(msg)});
}
useHook = false;
break;
}
@@ -771,36 +688,35 @@ Goal::Co DerivationBuildingGoal::tryToBuild()
co_return doneFailure(std::move(e));
}
DerivationBuilderParams params{
.drvPath = drvPath,
.buildResult = buildResult,
.drv = *drv,
.drvOptions = *drvOptions,
.inputPaths = inputPaths,
.initialOutputs = initialOutputs,
.buildMode = buildMode,
.defaultPathsInChroot = std::move(defaultPathsInChroot),
.systemFeatures = worker.store.config.systemFeatures.get(),
.desugaredEnv = std::move(desugaredEnv),
};
/* If we have to wait and retry (see below), then `builder` will
already be created, so we don't need to create it again. */
builder = makeDerivationBuilder(
*localStoreP,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, builder),
DerivationBuilderParams{
.drvPath = drvPath,
.buildResult = buildResult,
.drv = *drv,
.drvOptions = *drvOptions,
.inputPaths = inputPaths,
.initialOutputs = initialOutputs,
.buildMode = buildMode,
.defaultPathsInChroot = std::move(defaultPathsInChroot),
.systemFeatures = worker.store.config.systemFeatures.get(),
.desugaredEnv = std::move(desugaredEnv),
});
builder = externalBuilder ? makeExternalDerivationBuilder(
*localStoreP,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, builder),
std::move(params),
*externalBuilder)
: makeDerivationBuilder(
*localStoreP,
std::make_unique<DerivationBuildingGoalCallbacks>(*this, builder),
std::move(params));
}
std::optional<Descriptor> builderOutOpt;
try {
/* Okay, we have to build. */
builderOutOpt = builder->startBuild();
} catch (BuildError & e) {
builder.reset();
outputLocks.unlock();
worker.permanentFailure = true;
co_return doneFailure(std::move(e)); // InputRejected
}
if (!builderOutOpt) {
if (auto builderOutOpt = builder->startBuild()) {
builderOut = *std::move(builderOutOpt);
} else {
if (!actLock)
actLock = std::make_unique<Activity>(
*logger,
@@ -809,9 +725,7 @@ Goal::Co DerivationBuildingGoal::tryToBuild()
fmt("waiting for a free build user ID for '%s'", Magenta(worker.store.printStorePath(drvPath))));
co_await waitForAWhile();
continue;
} else {
builderOut = *std::move(builderOutOpt);
};
}
break;
}
+3 -21
View File
@@ -33,6 +33,7 @@ DerivationGoal::DerivationGoal(
: Goal(worker, haveDerivation())
, drvPath(drvPath)
, wantedOutput(wantedOutput)
, drv{std::make_unique<Derivation>(drv)}
, outputHash{[&] {
auto outputHashes = staticOutputHashes(worker.evalStore, drv);
if (auto * mOutputHash = get(outputHashes, wantedOutput))
@@ -41,11 +42,8 @@ DerivationGoal::DerivationGoal(
}()}
, buildMode(buildMode)
{
this->drv = std::make_unique<Derivation>(drv);
name =
fmt("building of '%s' from in-memory derivation",
DerivedPath::Built{makeConstantStorePathRef(drvPath), drv.outputNames()}.to_string(worker.store));
name = fmt("getting output '%s' from derivation '%s'", wantedOutput, worker.store.printStorePath(drvPath));
trace("created");
mcExpectedBuilds = std::make_unique<MaintainCount<uint64_t>>(worker.expectedBuilds);
@@ -54,11 +52,7 @@ DerivationGoal::DerivationGoal(
std::string DerivationGoal::key()
{
/* Ensure that derivations get built in order of their name,
i.e. a derivation named "aardvark" always comes before
"baboon". And substitution goals always happen before
derivation goals (due to "b$"). */
return "b$" + std::string(drvPath.name()) + "$" + SingleDerivedPath::Built{
return "db$" + std::string(drvPath.name()) + "$" + SingleDerivedPath::Built{
.drvPath = makeConstantStorePathRef(drvPath),
.output = wantedOutput,
}.to_string(worker.store);
@@ -189,18 +183,6 @@ Goal::Co DerivationGoal::haveDerivation()
co_return amDone(g->exitCode, g->ex);
}
/**
* Used for `inputGoals` local variable below
*/
struct value_comparison
{
template<typename T>
bool operator()(const ref<T> & lhs, const ref<T> & rhs) const
{
return *lhs < *rhs;
}
};
Goal::Co DerivationGoal::repairClosure()
{
assert(!drv->type().isImpure());
@@ -0,0 +1,191 @@
#include "nix/store/build/derivation-resolution-goal.hh"
#include "nix/store/build/worker.hh"
#include "nix/util/util.hh"
#include <nlohmann/json.hpp>
namespace nix {
DerivationResolutionGoal::DerivationResolutionGoal(
const StorePath & drvPath, const Derivation & drv, Worker & worker, BuildMode buildMode)
: Goal(worker, resolveDerivation())
, drvPath(drvPath)
, drv{std::make_unique<Derivation>(drv)}
, buildMode{buildMode}
{
name = fmt("resolving derivation '%s'", worker.store.printStorePath(drvPath));
trace("created");
}
std::string DerivationResolutionGoal::key()
{
return "dc$" + std::string(drvPath.name()) + "$" + worker.store.printStorePath(drvPath);
}
/**
* Used for `inputGoals` local variable below
*/
struct value_comparison
{
template<typename T>
bool operator()(const ref<T> & lhs, const ref<T> & rhs) const
{
return *lhs < *rhs;
}
};
Goal::Co DerivationResolutionGoal::resolveDerivation()
{
Goals waitees;
std::map<ref<const SingleDerivedPath>, GoalPtr, value_comparison> inputGoals;
{
std::function<void(ref<const SingleDerivedPath>, const DerivedPathMap<StringSet>::ChildNode &)>
addWaiteeDerivedPath;
addWaiteeDerivedPath = [&](ref<const SingleDerivedPath> inputDrv,
const DerivedPathMap<StringSet>::ChildNode & inputNode) {
if (!inputNode.value.empty()) {
auto g = worker.makeGoal(
DerivedPath::Built{
.drvPath = inputDrv,
.outputs = inputNode.value,
},
buildMode == bmRepair ? bmRepair : bmNormal);
inputGoals.insert_or_assign(inputDrv, g);
waitees.insert(std::move(g));
}
for (const auto & [outputName, childNode] : inputNode.childMap)
addWaiteeDerivedPath(
make_ref<SingleDerivedPath>(SingleDerivedPath::Built{inputDrv, outputName}), childNode);
};
for (const auto & [inputDrvPath, inputNode] : drv->inputDrvs.map) {
/* Ensure that pure, non-fixed-output derivations don't
depend on impure derivations. */
if (experimentalFeatureSettings.isEnabled(Xp::ImpureDerivations) && !drv->type().isImpure()
&& !drv->type().isFixed()) {
auto inputDrv = worker.evalStore.readDerivation(inputDrvPath);
if (inputDrv.type().isImpure())
throw Error(
"pure derivation '%s' depends on impure derivation '%s'",
worker.store.printStorePath(drvPath),
worker.store.printStorePath(inputDrvPath));
}
addWaiteeDerivedPath(makeConstantStorePathRef(inputDrvPath), inputNode);
}
}
co_await await(std::move(waitees));
trace("all inputs realised");
if (nrFailed != 0) {
auto msg =
fmt("Cannot build '%s'.\n"
"Reason: " ANSI_RED "%d %s failed" ANSI_NORMAL ".",
Magenta(worker.store.printStorePath(drvPath)),
nrFailed,
nrFailed == 1 ? "dependency" : "dependencies");
msg += showKnownOutputs(worker.store, *drv);
co_return amDone(ecFailed, {BuildError(BuildResult::Failure::DependencyFailed, msg)});
}
/* Gather information necessary for computing the closure and/or
running the build hook. */
/* Determine the full set of input paths. */
/* First, the input derivations. */
{
auto & fullDrv = *drv;
auto drvType = fullDrv.type();
bool resolveDrv =
std::visit(
overloaded{
[&](const DerivationType::InputAddressed & ia) {
/* must resolve if deferred. */
return ia.deferred;
},
[&](const DerivationType::ContentAddressed & ca) {
return !fullDrv.inputDrvs.map.empty()
&& (ca.fixed
/* Can optionally resolve if fixed, which is good
for avoiding unnecessary rebuilds. */
? experimentalFeatureSettings.isEnabled(Xp::CaDerivations)
/* Must resolve if floating and there are any inputs
drvs. */
: true);
},
[&](const DerivationType::Impure &) { return true; }},
drvType.raw)
/* no inputs are outputs of dynamic derivations */
|| std::ranges::any_of(fullDrv.inputDrvs.map.begin(), fullDrv.inputDrvs.map.end(), [](auto & pair) {
return !pair.second.childMap.empty();
});
if (resolveDrv && !fullDrv.inputDrvs.map.empty()) {
experimentalFeatureSettings.require(Xp::CaDerivations);
/* We are be able to resolve this derivation based on the
now-known results of dependencies. If so, we become a
stub goal aliasing that resolved derivation goal. */
std::optional attempt = fullDrv.tryResolve(
worker.store,
[&](ref<const SingleDerivedPath> drvPath, const std::string & outputName) -> std::optional<StorePath> {
auto mEntry = get(inputGoals, drvPath);
if (!mEntry)
return std::nullopt;
auto & buildResult = (*mEntry)->buildResult;
return std::visit(
overloaded{
[](const BuildResult::Failure &) -> std::optional<StorePath> { return std::nullopt; },
[&](const BuildResult::Success & success) -> std::optional<StorePath> {
auto i = get(success.builtOutputs, outputName);
if (!i)
return std::nullopt;
return i->outPath;
},
},
buildResult.inner);
});
if (!attempt) {
/* TODO (impure derivations-induced tech debt) (see below):
The above attempt should have found it, but because we manage
inputDrvOutputs statefully, sometimes it gets out of sync with
the real source of truth (store). So we query the store
directly if there's a problem. */
attempt = fullDrv.tryResolve(worker.store, &worker.evalStore);
}
assert(attempt);
auto pathResolved = writeDerivation(worker.store, *attempt, NoRepair, /*readOnly =*/true);
auto msg =
fmt("resolved derivation: '%s' -> '%s'",
worker.store.printStorePath(drvPath),
worker.store.printStorePath(pathResolved));
act = std::make_unique<Activity>(
*logger,
lvlInfo,
actBuildWaiting,
msg,
Logger::Fields{
worker.store.printStorePath(drvPath),
worker.store.printStorePath(pathResolved),
});
resolvedDrv =
std::make_unique<std::pair<StorePath, BasicDerivation>>(std::move(pathResolved), *std::move(attempt));
}
}
co_return amDone(ecSuccess, std::nullopt);
}
} // namespace nix
@@ -31,7 +31,7 @@ DerivationTrampolineGoal::DerivationTrampolineGoal(
void DerivationTrampolineGoal::commonInit()
{
name =
fmt("outer obtaining drv from '%s' and then building outputs %s",
fmt("obtaining derivation from '%s' and then building outputs %s",
drvReq->to_string(worker.store),
std::visit(
overloaded{
@@ -58,18 +58,12 @@ static StorePath pathPartOfReq(const SingleDerivedPath & req)
std::string DerivationTrampolineGoal::key()
{
/* Ensure that derivations get built in order of their name,
i.e. a derivation named "aardvark" always comes before "baboon". And
substitution goals, derivation goals, and derivation building goals always happen before
derivation goals (due to "bt$"). */
return "bt$" + std::string(pathPartOfReq(*drvReq).name()) + "$" + DerivedPath::Built{
return "da$" + std::string(pathPartOfReq(*drvReq).name()) + "$" + DerivedPath::Built{
.drvPath = drvReq,
.outputs = wantedOutputs,
}.to_string(worker.store);
}
void DerivationTrampolineGoal::timedOut(Error && ex) {}
Goal::Co DerivationTrampolineGoal::init()
{
trace("need to load derivation from file");
@@ -153,8 +153,6 @@ Goal::Co DrvOutputSubstitutionGoal::realisationFetched(
std::string DrvOutputSubstitutionGoal::key()
{
/* "a$" ensures substitution goals happen before derivation
goals. */
return "a$" + std::string(id.to_string());
}
+9
View File
@@ -4,6 +4,7 @@
#include "nix/store/build/substitution-goal.hh"
#include "nix/store/build/drv-output-substitution-goal.hh"
#include "nix/store/build/derivation-goal.hh"
#include "nix/store/build/derivation-resolution-goal.hh"
#include "nix/store/build/derivation-building-goal.hh"
#include "nix/store/build/derivation-trampoline-goal.hh"
#ifndef _WIN32 // TODO Enable building on Windows
@@ -80,6 +81,12 @@ std::shared_ptr<DerivationGoal> Worker::makeDerivationGoal(
return initGoalIfNeeded(derivationGoals[drvPath][wantedOutput], drvPath, drv, wantedOutput, *this, buildMode);
}
std::shared_ptr<DerivationResolutionGoal>
Worker::makeDerivationResolutionGoal(const StorePath & drvPath, const Derivation & drv, BuildMode buildMode)
{
return initGoalIfNeeded(derivationResolutionGoals[drvPath], drvPath, drv, *this, buildMode);
}
std::shared_ptr<DerivationBuildingGoal>
Worker::makeDerivationBuildingGoal(const StorePath & drvPath, const Derivation & drv, BuildMode buildMode)
{
@@ -158,6 +165,8 @@ void Worker::removeGoal(GoalPtr goal)
nix::removeGoal(drvGoal, derivationTrampolineGoals.map);
else if (auto drvGoal = std::dynamic_pointer_cast<DerivationGoal>(goal))
nix::removeGoal(drvGoal, derivationGoals);
else if (auto drvResolutionGoal = std::dynamic_pointer_cast<DerivationResolutionGoal>(goal))
nix::removeGoal(drvResolutionGoal, derivationResolutionGoals);
else if (auto drvBuildingGoal = std::dynamic_pointer_cast<DerivationBuildingGoal>(goal))
nix::removeGoal(drvBuildingGoal, derivationBuildingGoals);
else if (auto subGoal = std::dynamic_pointer_cast<PathSubstitutionGoal>(goal))
+14 -6
View File
@@ -99,6 +99,17 @@ DerivationOptions DerivationOptions::fromStructuredAttrs(
return fromStructuredAttrs(env, parsed ? &*parsed : nullptr);
}
static void flatten(const nlohmann::json & value, StringSet & res)
{
if (value.is_array())
for (auto & v : value)
flatten(v, res);
else if (value.is_string())
res.insert(value);
else
throw Error("'exportReferencesGraph' value is not an array or a string");
}
DerivationOptions
DerivationOptions::fromStructuredAttrs(const StringMap & env, const StructuredAttrs * parsed, bool shouldWarn)
{
@@ -219,12 +230,9 @@ DerivationOptions::fromStructuredAttrs(const StringMap & env, const StructuredAt
if (!e || !e->is_object())
return ret;
for (auto & [key, value] : getObject(*e)) {
if (value.is_array())
ret.insert_or_assign(key, value);
else if (value.is_string())
ret.insert_or_assign(key, StringSet{value});
else
throw Error("'exportReferencesGraph' value is not an array or a string");
StringSet ss;
flatten(value, ss);
ret.insert_or_assign(key, std::move(ss));
}
} else {
auto s = getOr(env, "exportReferencesGraph", "");
+14 -9
View File
@@ -157,8 +157,13 @@ struct DummyStoreImpl : DummyStore
unsupported("queryPathFromHashPart");
}
void addToStore(const ValidPathInfo & info, Source & source, RepairFlag repair, CheckSigsFlag checkSigs) override
void
addToStore(const ValidPathInfo & info, const SourcePath & path, RepairFlag repair, CheckSigsFlag checkSigs) override
{
if (!repair && isValidPath(info.path)) {
return;
}
if (config->readOnly)
unsupported("addToStore");
@@ -168,19 +173,19 @@ struct DummyStoreImpl : DummyStore
if (checkSigs)
throw Error("checking signatures is not supported for '%s' store", config->getHumanReadableURI());
auto temp = make_ref<MemorySourceAccessor>();
MemorySink tempSink{*temp};
parseDump(tempSink, source);
auto path = info.path;
auto accessor = make_ref<MemorySourceAccessor>();
{
MemorySink tempSink{*accessor};
copyRecursive(*path.accessor, path.path, tempSink, CanonPath::root);
}
auto accessor = make_ref<MemorySourceAccessor>(std::move(*temp));
contents.insert(
{path,
{info.path,
PathInfoAndContents{
std::move(info),
info,
accessor,
}});
wholeStoreView->addObject(path.to_string(), accessor);
wholeStoreView->addObject(info.path.to_string(), accessor);
}
StorePath addToStoreFromDump(
+73 -6
View File
@@ -9,9 +9,14 @@
#include "nix/util/signals.hh"
#include "store-config-private.hh"
#include <optional>
#if NIX_WITH_S3_SUPPORT
# include <aws/core/client/ClientConfiguration.h>
#endif
#if NIX_WITH_CURL_S3
# include "nix/store/aws-creds.hh"
# include "nix/store/s3-url.hh"
#endif
#ifdef __linux__
# include "nix/util/linux-namespaces.hh"
@@ -426,6 +431,24 @@ struct curlFileTransfer : public FileTransfer
curl_easy_setopt(req, CURLOPT_ERRORBUFFER, errbuf);
errbuf[0] = 0;
// Set up username/password authentication if provided
if (request.usernameAuth) {
curl_easy_setopt(req, CURLOPT_USERNAME, request.usernameAuth->username.c_str());
if (request.usernameAuth->password) {
curl_easy_setopt(req, CURLOPT_PASSWORD, request.usernameAuth->password->c_str());
}
}
#if NIX_WITH_CURL_S3
// Set up AWS SigV4 signing if this is an S3 request
// Note: AWS SigV4 support guaranteed available (curl >= 7.75.0 checked at build time)
// The username/password (access key ID and secret key) are set via the general
// usernameAuth mechanism above.
if (request.awsSigV4Provider) {
curl_easy_setopt(req, CURLOPT_AWS_SIGV4, request.awsSigV4Provider->c_str());
}
#endif
result.data.clear();
result.bodySize = 0;
}
@@ -800,7 +823,11 @@ struct curlFileTransfer : public FileTransfer
void enqueueItem(std::shared_ptr<TransferItem> item)
{
if (item->request.data && item->request.uri.scheme() != "http" && item->request.uri.scheme() != "https")
if (item->request.data && item->request.uri.scheme() != "http" && item->request.uri.scheme() != "https"
#if NIX_WITH_CURL_S3
&& item->request.uri.scheme() != "s3"
#endif
)
throw nix::Error("uploading to '%s' is not supported", item->request.uri.to_string());
{
@@ -818,9 +845,15 @@ struct curlFileTransfer : public FileTransfer
{
/* Ugly hack to support s3:// URIs. */
if (request.uri.scheme() == "s3") {
#if NIX_WITH_CURL_S3
// New curl-based S3 implementation
auto modifiedRequest = request;
modifiedRequest.setupForS3();
enqueueItem(std::make_shared<TransferItem>(*this, std::move(modifiedRequest), std::move(callback)));
#elif NIX_WITH_S3_SUPPORT
// Old AWS SDK-based implementation
// FIXME: do this on a worker thread
try {
#if NIX_WITH_S3_SUPPORT
auto parsed = ParsedS3URL::parse(request.uri.parsed());
std::string profile = parsed.profile.value_or("");
@@ -838,13 +871,12 @@ struct curlFileTransfer : public FileTransfer
res.data = std::move(*s3Res.data);
res.urls.push_back(request.uri.to_string());
callback(std::move(res));
#else
throw nix::Error(
"cannot download '%s' because Nix is not built with S3 support", request.uri.to_string());
#endif
} catch (...) {
callback.rethrow();
}
#else
throw nix::Error("cannot download '%s' because Nix is not built with S3 support", request.uri.to_string());
#endif
return;
}
@@ -872,6 +904,41 @@ ref<FileTransfer> makeFileTransfer()
return makeCurlFileTransfer();
}
#if NIX_WITH_CURL_S3
void FileTransferRequest::setupForS3()
{
auto parsedS3 = ParsedS3URL::parse(uri.parsed());
// Update the request URI to use HTTPS
uri = parsedS3.toHttpsUrl();
// This gets used later in a curl setopt
awsSigV4Provider = "aws:amz:" + parsedS3.region.value_or("us-east-1") + ":s3";
// check if the request already has pre-resolved credentials
std::optional<std::string> sessionToken;
if (usernameAuth) {
debug("Using pre-resolved AWS credentials from parent process");
sessionToken = preResolvedAwsSessionToken;
} else {
std::string profile = parsedS3.profile.value_or("");
try {
auto creds = getAwsCredentials(profile);
usernameAuth = UsernameAuth{
.username = creds.accessKeyId,
.password = creds.secretAccessKey,
};
sessionToken = creds.sessionToken;
} catch (const AwsAuthError & e) {
warn("AWS authentication failed for S3 request %s: %s", uri, e.what());
// Invalidate the cached credentials so next request will retry
invalidateAwsCredentials(profile);
// Continue without authentication - might be a public bucket
return;
}
}
if (sessionToken)
headers.emplace_back("x-amz-security-token", *sessionToken);
}
#endif
std::future<FileTransferResult> FileTransfer::enqueueFileTransfer(const FileTransferRequest & request)
{
auto promise = std::make_shared<std::promise<FileTransferResult>>();
+9 -2
View File
@@ -258,6 +258,15 @@ Path Settings::getDefaultSSLCertFile()
return "";
}
const ExternalBuilder * Settings::findExternalDerivationBuilderIfSupported(const Derivation & drv)
{
if (auto it = std::ranges::find_if(
externalBuilders.get(), [&](const auto & handler) { return handler.systems.contains(drv.platform); });
it != externalBuilders.get().end())
return &*it;
return nullptr;
}
std::string nixVersion = PACKAGE_VERSION;
NLOHMANN_JSON_SERIALIZE_ENUM(
@@ -379,8 +388,6 @@ unsigned int MaxBuildJobsSetting::parse(const std::string & str) const
}
}
NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE(Settings::ExternalBuilder, systems, program, args);
template<>
Settings::ExternalBuilders BaseSetting<Settings::ExternalBuilders>::parse(const std::string & str) const
{
+16 -4
View File
@@ -78,7 +78,11 @@ public:
void init() override
{
// FIXME: do this lazily?
if (auto cacheInfo = diskCache->upToDateCacheExists(config->cacheUri.to_string())) {
// For consistent cache key handling, use the reference without parameters
// This matches what's used in Store::queryPathInfo() lookups
auto cacheKey = config->getReference().render(/*withParams=*/false);
if (auto cacheInfo = diskCache->upToDateCacheExists(cacheKey)) {
config->wantMassQuery.setDefault(cacheInfo->wantMassQuery);
config->priority.setDefault(cacheInfo->priority);
} else {
@@ -87,8 +91,7 @@ public:
} catch (UploadToHTTP &) {
throw Error("'%s' does not appear to be a binary cache", config->cacheUri.to_string());
}
diskCache->createCache(
config->cacheUri.to_string(), config->storeDir, config->wantMassQuery, config->priority);
diskCache->createCache(cacheKey, config->storeDir, config->wantMassQuery, config->priority);
}
}
@@ -184,7 +187,16 @@ protected:
field which is
`nar/15f99rdaf26k39knmzry4xd0d97wp6yfpnfk1z9avakis7ipb9yg.nar?hash=zphkqn2wg8mnvbkixnl2aadkbn0rcnfj`
(note the query param) and that gets passed here. */
return FileTransferRequest(parseURLRelative(path, cacheUriWithTrailingSlash));
auto result = parseURLRelative(path, cacheUriWithTrailingSlash);
/* For S3 URLs, preserve query parameters from the base URL when the
relative path doesn't have its own query parameters. This is needed
to preserve S3-specific parameters like endpoint and region. */
if (config->cacheUri.scheme == "s3" && result.query.empty()) {
result.query = config->cacheUri.query;
}
return FileTransferRequest(result);
}
void getFile(const std::string & path, Sink & sink) override
@@ -57,12 +57,6 @@ AwsCredentials getAwsCredentials(const std::string & profile = "");
*/
void invalidateAwsCredentials(const std::string & profile);
/**
* Clear all cached credential providers.
* Typically called during application cleanup.
*/
void clearAwsCredentialsCache();
/**
* Pre-resolve AWS credentials for S3 URLs.
* Used to cache credentials in parent process before forking.
@@ -1,12 +1,15 @@
#pragma once
///@file
#include <nlohmann/json_fwd.hpp>
#include "nix/store/build-result.hh"
#include "nix/store/derivation-options.hh"
#include "nix/store/build/derivation-building-misc.hh"
#include "nix/store/derivations.hh"
#include "nix/store/parsed-derivations.hh"
#include "nix/util/processes.hh"
#include "nix/util/json-impls.hh"
#include "nix/store/restricted-store.hh"
#include "nix/store/build/derivation-env-desugar.hh"
@@ -179,9 +182,28 @@ struct DerivationBuilder : RestrictionContext
virtual bool killChild() = 0;
};
struct ExternalBuilder
{
StringSet systems;
Path program;
std::vector<std::string> args;
};
#ifndef _WIN32 // TODO enable `DerivationBuilder` on Windows
std::unique_ptr<DerivationBuilder> makeDerivationBuilder(
LocalStore & store, std::unique_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params);
/**
* @param handler Must be chosen such that it supports the given
* derivation.
*/
std::unique_ptr<DerivationBuilder> makeExternalDerivationBuilder(
LocalStore & store,
std::unique_ptr<DerivationBuilderCallbacks> miscMethods,
DerivationBuilderParams params,
const ExternalBuilder & handler);
#endif
} // namespace nix
JSON_IMPL(nix::ExternalBuilder)
@@ -0,0 +1,82 @@
#pragma once
///@file
#include "nix/store/derivations.hh"
#include "nix/store/derivation-options.hh"
#include "nix/store/build/derivation-building-misc.hh"
#include "nix/store/store-api.hh"
#include "nix/store/build/goal.hh"
namespace nix {
struct BuilderFailureError;
/**
* A goal for resolving a derivation. Resolving a derivation (@see
* `Derivation::tryResolve`) simplifies its inputs, replacing
* `inputDrvs` with `inputSrcs`.
*
* Conceptually, we resolve all derivations. For input-addressed
* derivations (that don't transtively depend on content-addressed
* derivations), however, we don't actually use the resolved derivation,
* because the output paths would appear invalid (if we tried to verify
* them), since they are computed from the original, unresolved inputs.
*
* That said, if we ever made the new flavor of input-addressing as described
* in issue #9259, then the input-addressing would be based on the resolved
* inputs, and we like the CA case *would* use the output of this goal.
*
* (The point of this discussion is not to randomly stuff information on
* a yet-unimplemented feature (issue #9259) in the codebase, but
* rather, to illustrate that there is no inherent tension between
* explicit derivation resolution and input-addressing in general. That
* tension only exists with the type of input-addressing we've
* historically used.)
*/
struct DerivationResolutionGoal : public Goal
{
DerivationResolutionGoal(
const StorePath & drvPath, const Derivation & drv, Worker & worker, BuildMode buildMode = bmNormal);
/**
* If the derivation needed to be resolved, this is resulting
* resolved derivations and its path.
*/
std::unique_ptr<std::pair<StorePath, BasicDerivation>> resolvedDrv;
void timedOut(Error && ex) override {}
private:
/**
* The path of the derivation.
*/
StorePath drvPath;
/**
* The derivation stored at drvPath.
*/
std::unique_ptr<Derivation> drv;
/**
* The remainder is state held during the build.
*/
BuildMode buildMode;
std::unique_ptr<Activity> act;
std::string key() override;
/**
* The states.
*/
Co resolveDerivation();
JobCategory jobCategory() const override
{
return JobCategory::Administration;
};
};
} // namespace nix
@@ -109,7 +109,7 @@ struct DerivationTrampolineGoal : public Goal
virtual ~DerivationTrampolineGoal();
void timedOut(Error && ex) override;
void timedOut(Error && ex) override {}
std::string key() override;
@@ -456,6 +456,18 @@ public:
*/
virtual void timedOut(Error && ex) = 0;
/**
* Used for comparisons. The order matters a bit for scheduling. We
* want:
*
* 1. Substitution
* 2. Derivation administrativia
* 3. Actual building
*
* Also, ensure that derivations get processed in order of their
* name, i.e. a derivation named "aardvark" always comes before
* "baboon".
*/
virtual std::string key() = 0;
/**
@@ -58,10 +58,6 @@ public:
unreachable();
};
/**
* We prepend "a$" to the key name to ensure substitution goals
* happen before derivation goals.
*/
std::string key() override
{
return "a$" + std::string(storePath.name()) + "$" + worker.store.printStorePath(storePath);
@@ -16,6 +16,7 @@ namespace nix {
/* Forward definition. */
struct DerivationTrampolineGoal;
struct DerivationGoal;
struct DerivationResolutionGoal;
struct DerivationBuildingGoal;
struct PathSubstitutionGoal;
class DrvOutputSubstitutionGoal;
@@ -111,6 +112,7 @@ private:
DerivedPathMap<std::map<OutputsSpec, std::weak_ptr<DerivationTrampolineGoal>>> derivationTrampolineGoals;
std::map<StorePath, std::map<OutputName, std::weak_ptr<DerivationGoal>>> derivationGoals;
std::map<StorePath, std::weak_ptr<DerivationResolutionGoal>> derivationResolutionGoals;
std::map<StorePath, std::weak_ptr<DerivationBuildingGoal>> derivationBuildingGoals;
std::map<StorePath, std::weak_ptr<PathSubstitutionGoal>> substitutionGoals;
std::map<DrvOutput, std::weak_ptr<DrvOutputSubstitutionGoal>> drvOutputSubstitutionGoals;
@@ -224,7 +226,13 @@ public:
BuildMode buildMode = bmNormal);
/**
* @ref DerivationBuildingGoal "derivation goal"
* @ref DerivationResolutionGoal "derivation resolution goal"
*/
std::shared_ptr<DerivationResolutionGoal>
makeDerivationResolutionGoal(const StorePath & drvPath, const Derivation & drv, BuildMode buildMode = bmNormal);
/**
* @ref DerivationBuildingGoal "derivation building goal"
*/
std::shared_ptr<DerivationBuildingGoal>
makeDerivationBuildingGoal(const StorePath & drvPath, const Derivation & drv, BuildMode buildMode = bmNormal);
@@ -11,6 +11,11 @@
#include "nix/util/serialise.hh"
#include "nix/util/url.hh"
#include "nix/store/config.hh"
#if NIX_WITH_CURL_S3
# include "nix/store/aws-creds.hh"
#endif
namespace nix {
struct FileTransferSettings : Config
@@ -77,6 +82,17 @@ extern FileTransferSettings fileTransferSettings;
extern const unsigned int RETRY_TIME_MS_DEFAULT;
/**
* Username and optional password for HTTP basic authentication.
* These are used with curl's CURLOPT_USERNAME and CURLOPT_PASSWORD options
* for various protocols including HTTP, FTP, and others.
*/
struct UsernameAuth
{
std::string username;
std::optional<std::string> password;
};
struct FileTransferRequest
{
ValidURL uri;
@@ -92,6 +108,18 @@ struct FileTransferRequest
std::optional<std::string> data;
std::string mimeType;
std::function<void(std::string_view data)> dataCallback;
/**
* Optional username and password for HTTP basic authentication.
* When provided, these credentials will be used with curl's CURLOPT_USERNAME/PASSWORD option.
*/
std::optional<UsernameAuth> usernameAuth;
#if NIX_WITH_CURL_S3
/**
* Pre-resolved AWS session token for S3 requests.
* When provided along with usernameAuth, this will be used instead of fetching fresh credentials.
*/
std::optional<std::string> preResolvedAwsSessionToken;
#endif
FileTransferRequest(ValidURL uri)
: uri(std::move(uri))
@@ -103,6 +131,13 @@ struct FileTransferRequest
{
return data ? "upload" : "download";
}
#if NIX_WITH_CURL_S3
private:
friend struct curlFileTransfer;
void setupForS3();
std::optional<std::string> awsSigV4Provider;
#endif
};
struct FileTransferResult
+6 -7
View File
@@ -1373,13 +1373,6 @@ public:
Set it to 1 to warn on all paths.
)"};
struct ExternalBuilder
{
std::vector<std::string> systems;
Path program;
std::vector<std::string> args;
};
using ExternalBuilders = std::vector<ExternalBuilder>;
Setting<ExternalBuilders> externalBuilders{
@@ -1443,6 +1436,12 @@ public:
// Current system: 'aarch64-darwin' with features {apple-virt, benchmark, big-parallel, nixos-test}
// Xp::ExternalBuilders
};
/**
* Finds the first external derivation builder that supports this
* derivation, or else returns a null pointer.
*/
const ExternalBuilder * findExternalDerivationBuilderIfSupported(const Derivation & drv);
};
// FIXME: don't use a global variable.
@@ -18,6 +18,7 @@ headers = [ config_pub_h ] + files(
'build/derivation-building-misc.hh',
'build/derivation-env-desugar.hh',
'build/derivation-goal.hh',
'build/derivation-resolution-goal.hh',
'build/derivation-trampoline-goal.hh',
'build/drv-output-substitution-goal.hh',
'build/goal.hh',
@@ -134,4 +134,79 @@ struct S3BinaryCacheStore : virtual BinaryCacheStore
} // namespace nix
#elif NIX_WITH_CURL_S3
# include "nix/store/http-binary-cache-store.hh"
namespace nix {
struct S3BinaryCacheStoreConfig : HttpBinaryCacheStoreConfig
{
using HttpBinaryCacheStoreConfig::HttpBinaryCacheStoreConfig;
S3BinaryCacheStoreConfig(std::string_view uriScheme, std::string_view bucketName, const Params & params);
const Setting<std::string> profile{
this,
"default",
"profile",
R"(
The name of the AWS configuration profile to use. By default
Nix uses the `default` profile.
)"};
public:
const Setting<std::string> region{
this,
"us-east-1",
"region",
R"(
The region of the S3 bucket. If your bucket is not in
`us-east-1`, you should always explicitly specify the region
parameter.
)"};
const Setting<std::string> scheme{
this,
"https",
"scheme",
R"(
The scheme used for S3 requests, `https` (default) or `http`. This
option allows you to disable HTTPS for binary caches which don't
support it.
> **Note**
>
> HTTPS should be used if the cache might contain sensitive
> information.
)"};
const Setting<std::string> endpoint{
this,
"",
"endpoint",
R"(
The S3 endpoint to use. When empty (default), uses AWS S3 with
region-specific endpoints (e.g., s3.us-east-1.amazonaws.com).
For S3-compatible services such as MinIO, set this to your service's endpoint.
> **Note**
>
> Custom endpoints must support HTTPS and use path-based
> addressing instead of virtual host based addressing.
)"};
static const std::string name()
{
return "S3 Binary Cache Store";
}
static StringSet uriSchemes();
static std::string doc();
};
} // namespace nix
#endif
+17 -2
View File
@@ -511,13 +511,28 @@ public:
virtual void querySubstitutablePathInfos(const StorePathCAMap & paths, SubstitutablePathInfos & infos);
/**
* Import a path into the store.
* Import a path into the store, via NAR stream.
*
* One of these two must be overridden, as the defaults will
* infinitely mutually recur.
*/
virtual void addToStore(
const ValidPathInfo & info,
Source & narSource,
RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs) = 0;
CheckSigsFlag checkSigs = CheckSigs);
/**
* Import a path into the store, via arbitrary `SourcePath`.
*
* One of these two must be overridden, as the defaults will
* infinitely mutually recur.
*/
virtual void addToStore(
const ValidPathInfo & info,
const SourcePath & path,
RepairFlag repair = NoRepair,
CheckSigsFlag checkSigs = CheckSigs);
/**
* A list of paths infos along with a source providing the content
+6
View File
@@ -145,6 +145,12 @@ void LegacySSHStore::queryPathInfoUncached(
void LegacySSHStore::addToStore(const ValidPathInfo & info, Source & source, RepairFlag repair, CheckSigsFlag checkSigs)
{
if (!repair && isValidPath(info.path)) {
NullFileSystemObjectSink s;
parseDump(s, source);
return;
}
debug("adding path '%s' to remote host '%s'", printStorePath(info.path), config->authority.host);
auto conn(connections->get());
+3 -3
View File
@@ -1383,7 +1383,7 @@ bool LocalStore::verifyStore(bool checkContents, RepairFlag repair)
for (auto & link : DirectoryIterator{linksDir}) {
checkInterrupt();
auto name = link.path().filename();
printMsg(lvlTalkative, "checking contents of '%s'", name);
printMsg(lvlTalkative, "checking contents of %s", name);
PosixSourceAccessor accessor;
std::string hash = hashPath(
PosixSourceAccessor::createAtRoot(link.path()),
@@ -1391,10 +1391,10 @@ bool LocalStore::verifyStore(bool checkContents, RepairFlag repair)
HashAlgorithm::SHA256)
.first.to_string(HashFormat::Nix32, false);
if (hash != name.string()) {
printError("link '%s' was modified! expected hash '%s', got '%s'", link.path(), name, hash);
printError("link %s was modified! expected hash %s, got '%s'", link.path(), name, hash);
if (repair) {
std::filesystem::remove(link.path());
printInfo("removed link '%s'", link.path());
printInfo("removed link %s", link.path());
} else {
errors = true;
}
+2 -1
View File
@@ -292,16 +292,17 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'aws-creds.cc',
'binary-cache-store.cc',
'build-result.cc',
'build/derivation-builder.cc',
'build/derivation-building-goal.cc',
'build/derivation-check.cc',
'build/derivation-env-desugar.cc',
'build/derivation-goal.cc',
'build/derivation-resolution-goal.cc',
'build/derivation-trampoline-goal.cc',
'build/drv-output-substitution-goal.cc',
'build/entry-points.cc',
+6 -6
View File
@@ -202,7 +202,7 @@ void LocalStore::optimisePath_(
full. When that happens, it's fine to ignore it: we
just effectively disable deduplication of this
file. */
printInfo("cannot link '%s' to '%s': %s", linkPath, path, strerror(errno));
printInfo("cannot link %s to '%s': %s", linkPath, path, strerror(errno));
return;
}
@@ -216,11 +216,11 @@ void LocalStore::optimisePath_(
auto stLink = lstat(linkPath.string());
if (st.st_ino == stLink.st_ino) {
debug("'%1%' is already linked to '%2%'", path, linkPath);
debug("'%1%' is already linked to %2%", path, linkPath);
return;
}
printMsg(lvlTalkative, "linking '%1%' to '%2%'", path, linkPath);
printMsg(lvlTalkative, "linking '%1%' to %2%", path, linkPath);
/* Make the containing directory writable, but only if it's not
the store itself (we don't want or need to mess with its
@@ -245,7 +245,7 @@ void LocalStore::optimisePath_(
systems). This is likely to happen with empty files.
Just shrug and ignore. */
if (st.st_size)
printInfo("'%1%' has maximum number of links", linkPath);
printInfo("%1% has maximum number of links", linkPath);
return;
}
throw;
@@ -256,13 +256,13 @@ void LocalStore::optimisePath_(
std::filesystem::rename(tempLink, path);
} catch (std::filesystem::filesystem_error & e) {
std::filesystem::remove(tempLink);
printError("unable to unlink '%1%'", tempLink);
printError("unable to unlink %1%", tempLink);
if (e.code() == std::errc::too_many_links) {
/* Some filesystems generate too many links on the rename,
rather than on the original link. (Probably it
temporarily increases the st_nlink field before
decreasing it again.) */
debug("'%s' has reached maximum number of links", linkPath);
debug("%s has reached maximum number of links", linkPath);
return;
}
throw;
+3
View File
@@ -34,6 +34,9 @@ let
inherit (lib) fileset;
in
assert lib.assertMsg (!withAWS || !withCurlS3)
"withAWS and withCurlS3 are mutually exclusive - cannot enable both S3 implementations simultaneously";
mkMesonLibrary (finalAttrs: {
pname = "nix-store";
inherit version;
+6
View File
@@ -424,6 +424,12 @@ StorePath RemoteStore::addToStoreFromDump(
void RemoteStore::addToStore(const ValidPathInfo & info, Source & source, RepairFlag repair, CheckSigsFlag checkSigs)
{
if (!repair && isValidPath(info.path)) {
NullFileSystemObjectSink s;
parseDump(s, source);
return;
}
auto conn(getConnection());
conn->to << WorkerProto::Op::AddToStoreNar << printStorePath(info.path)
+46
View File
@@ -589,4 +589,50 @@ static RegisterStoreImplementation<S3BinaryCacheStoreImpl::Config> regS3BinaryCa
} // namespace nix
#elif NIX_WITH_CURL_S3
# include <cassert>
# include "nix/store/s3-binary-cache-store.hh"
# include "nix/store/http-binary-cache-store.hh"
# include "nix/store/store-registration.hh"
namespace nix {
StringSet S3BinaryCacheStoreConfig::uriSchemes()
{
return {"s3"};
}
S3BinaryCacheStoreConfig::S3BinaryCacheStoreConfig(
std::string_view scheme, std::string_view _cacheUri, const Params & params)
: StoreConfig(params)
, HttpBinaryCacheStoreConfig(scheme, _cacheUri, params)
{
// For S3 stores, preserve S3-specific query parameters as part of the URL
// These are needed for region specification and other S3-specific settings
assert(cacheUri.query.empty());
// Only copy S3-specific parameters to the URL query
static const std::set<std::string> s3Params = {"region", "endpoint", "profile", "scheme"};
for (const auto & [key, value] : params) {
if (s3Params.contains(key)) {
cacheUri.query[key] = value;
}
}
}
std::string S3BinaryCacheStoreConfig::doc()
{
return R"(
**Store URL format**: `s3://bucket-name`
This store allows reading and writing a binary cache stored in an AWS S3 bucket.
)";
}
static RegisterStoreImplementation<S3BinaryCacheStoreConfig> registerS3BinaryCacheStore;
} // namespace nix
#endif
+25 -1
View File
@@ -1,6 +1,6 @@
#include "nix/util/logging.hh"
#include "nix/util/signature/local-keys.hh"
#include "nix/util/source-accessor.hh"
#include "nix/util/memory-source-accessor.hh"
#include "nix/store/globals.hh"
#include "nix/store/derived-path.hh"
#include "nix/store/realisation.hh"
@@ -84,6 +84,30 @@ StorePath Store::followLinksToStorePath(std::string_view path) const
return toStorePath(followLinksToStore(path)).first;
}
void Store::addToStore(const ValidPathInfo & info, Source & narSource, RepairFlag repair, CheckSigsFlag checkSigs)
{
if (!repair && isValidPath(info.path)) {
NullFileSystemObjectSink s;
parseDump(s, narSource);
return;
}
auto temp = make_ref<MemorySourceAccessor>();
MemorySink tempSink{*temp};
parseDump(tempSink, narSource);
addToStore(info, {temp}, repair, checkSigs);
}
void Store::addToStore(const ValidPathInfo & info, const SourcePath & path, RepairFlag repair, CheckSigsFlag checkSigs)
{
if (!repair && isValidPath(info.path)) {
return;
}
auto sink = sourceToSink([&](Source & source) { addToStore(info, source, repair, checkSigs); });
dumpPath(path, *sink, FileSerialisationMethod::NixArchive);
}
StorePath Store::addToStore(
std::string_view name,
const SourcePath & path,
@@ -229,12 +229,6 @@ protected:
return acquireUserLock(1, false);
}
/**
* Throw an exception if we can't do this derivation because of
* missing system features.
*/
virtual void checkSystem();
/**
* Return the paths that should be made available in the sandbox.
* This includes:
@@ -672,33 +666,6 @@ static bool checkNotWorldWritable(std::filesystem::path path)
return true;
}
void DerivationBuilderImpl::checkSystem()
{
/* Right platform? */
if (!drvOptions.canBuildLocally(store, drv)) {
auto msg =
fmt("Cannot build '%s'.\n"
"Reason: " ANSI_RED "required system or feature not available" ANSI_NORMAL
"\n"
"Required system: '%s' with features {%s}\n"
"Current system: '%s' with features {%s}",
Magenta(store.printStorePath(drvPath)),
Magenta(drv.platform),
concatStringsSep(", ", drvOptions.getRequiredSystemFeatures(drv)),
Magenta(settings.thisSystem),
concatStringsSep<StringSet>(", ", store.Store::config.systemFeatures));
// since aarch64-darwin has Rosetta 2, this user can actually run x86_64-darwin on their hardware - we should
// tell them to run the command to install Darwin 2
if (drv.platform == "x86_64-darwin" && settings.thisSystem == "aarch64-darwin")
msg +=
fmt("\nNote: run `%s` to run programs for x86_64-darwin",
Magenta("/usr/sbin/softwareupdate --install-rosetta && launchctl stop org.nixos.nix-daemon"));
throw BuildError(BuildResult::Failure::InputRejected, msg);
}
}
std::optional<Descriptor> DerivationBuilderImpl::startBuild()
{
if (useBuildUsers()) {
@@ -709,8 +676,6 @@ std::optional<Descriptor> DerivationBuilderImpl::startBuild()
return std::nullopt;
}
checkSystem();
/* Make sure that no other processes are executing under the
sandbox uids. This must be done before any chownToBuilder()
calls. */
@@ -1922,9 +1887,6 @@ namespace nix {
std::unique_ptr<DerivationBuilder> makeDerivationBuilder(
LocalStore & store, std::unique_ptr<DerivationBuilderCallbacks> miscMethods, DerivationBuilderParams params)
{
if (auto builder = ExternalDerivationBuilder::newIfSupported(store, miscMethods, params))
return builder;
bool useSandbox = false;
/* Are we doing a sandboxed build? */
@@ -2,31 +2,19 @@ namespace nix {
struct ExternalDerivationBuilder : DerivationBuilderImpl
{
Settings::ExternalBuilder externalBuilder;
ExternalBuilder externalBuilder;
ExternalDerivationBuilder(
LocalStore & store,
std::unique_ptr<DerivationBuilderCallbacks> miscMethods,
DerivationBuilderParams params,
Settings::ExternalBuilder externalBuilder)
ExternalBuilder externalBuilder)
: DerivationBuilderImpl(store, std::move(miscMethods), std::move(params))
, externalBuilder(std::move(externalBuilder))
{
experimentalFeatureSettings.require(Xp::ExternalBuilders);
}
static std::unique_ptr<ExternalDerivationBuilder> newIfSupported(
LocalStore & store, std::unique_ptr<DerivationBuilderCallbacks> & miscMethods, DerivationBuilderParams & params)
{
for (auto & handler : settings.externalBuilders.get()) {
for (auto & system : handler.systems)
if (params.drv.platform == system)
return std::make_unique<ExternalDerivationBuilder>(
store, std::move(miscMethods), std::move(params), handler);
}
return {};
}
Path tmpDirInSandbox() override
{
/* In a sandbox, for determinism, always use the same temporary
@@ -40,8 +28,6 @@ struct ExternalDerivationBuilder : DerivationBuilderImpl
createDir(tmpDir, 0700);
}
void checkSystem() override {}
void startChild() override
{
if (drvOptions.getRequiredSystemFeatures(drv).count("recursive-nix"))
@@ -120,4 +106,13 @@ struct ExternalDerivationBuilder : DerivationBuilderImpl
}
};
std::unique_ptr<DerivationBuilder> makeExternalDerivationBuilder(
LocalStore & store,
std::unique_ptr<DerivationBuilderCallbacks> miscMethods,
DerivationBuilderParams params,
const ExternalBuilder & handler)
{
return std::make_unique<ExternalDerivationBuilder>(store, std::move(miscMethods), std::move(params), handler);
}
} // namespace nix
-1
View File
@@ -32,7 +32,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_util.cc',
-1
View File
@@ -27,7 +27,6 @@ rapidcheck = dependency('rapidcheck')
deps_public += rapidcheck
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'hash.cc',
+15 -1
View File
@@ -42,6 +42,20 @@ INSTANTIATE_TEST_SUITE_P(
NarTest,
InvalidNarTest,
::testing::Values(
std::pair{"invalid-tag-instead-of-contents", "bad archive: expected tag 'contents', got 'AAAAAAAA'"}));
std::pair{"invalid-tag-instead-of-contents", "bad archive: expected tag 'contents', got 'AAAAAAAA'"},
// Unpacking a NAR with a NUL character in a file name should fail.
std::pair{"nul-character", "bad archive: NAR contains invalid file name 'f"},
// Likewise for a '.' filename.
std::pair{"dot", "bad archive: NAR contains invalid file name '.'"},
// Likewise for a '..' filename.
std::pair{"dotdot", "bad archive: NAR contains invalid file name '..'"},
// Likewise for a filename containing a slash.
std::pair{"slash", "bad archive: NAR contains invalid file name 'x/y'"},
// Likewise for an empty filename.
std::pair{"empty", "bad archive: NAR contains invalid file name ''"},
// Test that the 'executable' field cannot come before the 'contents' field.
std::pair{"executable-after-contents", "bad archive: expected tag ')', got 'executable'"},
// Test that the 'name' field cannot come before the 'node' field in a directory entry.
std::pair{"name-after-node", "bad archive: expected tag 'name'"}));
} // namespace nix
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+1 -2
View File
@@ -42,7 +42,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'archive.cc',
@@ -98,7 +97,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
},
protocol : 'gtest',
-1
View File
@@ -61,7 +61,6 @@ mkMesonExecutable (finalAttrs: {
mkdir -p "$HOME"
''
+ ''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
+13
View File
@@ -868,6 +868,12 @@ TEST_P(ParsedURLPathSegmentsTest, segmentsAreCorrect)
EXPECT_EQ(encodeUrlPath(segments), testCase.path);
}
TEST_P(ParsedURLPathSegmentsTest, to_string)
{
const auto & testCase = GetParam();
EXPECT_EQ(testCase.url, parseURL(testCase.url).to_string());
}
INSTANTIATE_TEST_SUITE_P(
ParsedURL,
ParsedURLPathSegmentsTest,
@@ -886,6 +892,13 @@ INSTANTIATE_TEST_SUITE_P(
.skipEmpty = false,
.description = "empty_authority_empty_path",
},
ParsedURLPathSegmentsTestCase{
.url = "path:/",
.segments = {"", ""},
.path = "/",
.skipEmpty = false,
.description = "empty_authority_root_path",
},
ParsedURLPathSegmentsTestCase{
.url = "scheme:///",
.segments = {"", ""},
+21 -1
View File
@@ -318,6 +318,7 @@ void RootArgs::parseCmdline(const Strings & _cmdline, bool allowShebang)
} catch (SystemError &) {
}
}
for (auto pos = cmdline.begin(); pos != cmdline.end();) {
auto arg = *pos;
@@ -354,6 +355,9 @@ void RootArgs::parseCmdline(const Strings & _cmdline, bool allowShebang)
processArgs(pendingArgs, true);
if (!completions)
checkArgs();
initialFlagsProcessed();
/* Now that we are done parsing, make sure that any experimental
@@ -384,7 +388,7 @@ bool Args::processFlag(Strings::iterator & pos, Strings::iterator end)
auto & rootArgs = getRoot();
auto process = [&](const std::string & name, const Flag & flag) -> bool {
auto process = [&](const std::string & name, Flag & flag) -> bool {
++pos;
if (auto & f = flag.experimentalFeature)
@@ -413,6 +417,7 @@ bool Args::processFlag(Strings::iterator & pos, Strings::iterator end)
}
if (!anyCompleted)
flag.handler.fun(std::move(args));
flag.timesUsed++;
return true;
};
@@ -504,6 +509,14 @@ bool Args::processArgs(const Strings & args, bool finish)
return res;
}
void Args::checkArgs()
{
for (auto & [name, flag] : longFlags) {
if (flag->required && flag->timesUsed == 0)
throw UsageError("required argument '--%s' is missing", name);
}
}
nlohmann::json Args::toJSON()
{
auto flags = nlohmann::json::object();
@@ -643,6 +656,13 @@ bool MultiCommand::processArgs(const Strings & args, bool finish)
return Args::processArgs(args, finish);
}
void MultiCommand::checkArgs()
{
Args::checkArgs();
if (command)
command->second->checkArgs();
}
nlohmann::json MultiCommand::toJSON()
{
auto cmds = nlohmann::json::object();
+8
View File
@@ -202,8 +202,12 @@ public:
Strings labels;
Handler handler;
CompleterClosure completer;
bool required = false;
std::optional<ExperimentalFeature> experimentalFeature;
// FIXME: this should be private, but that breaks designated initializers.
size_t timesUsed = 0;
};
protected:
@@ -283,6 +287,8 @@ protected:
StringSet hiddenCategories;
virtual void checkArgs();
/**
* Called after all command line flags before the first non-flag
* argument (if any) have been processed.
@@ -428,6 +434,8 @@ public:
protected:
std::string commandName = "";
bool aliasUsed = false;
void checkArgs() override;
};
Strings argvToStrings(int argc, char ** argv);
-1
View File
@@ -118,7 +118,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = [ config_priv_h ] + files(
'archive.cc',
+1 -1
View File
@@ -350,7 +350,7 @@ std::string ParsedURL::renderAuthorityAndPath() const
must either be empty or begin with a slash ("/") character. */
assert(path.empty() || path.front().empty());
res += authority->to_string();
} else if (std::ranges::equal(std::views::take(path, 2), std::views::repeat("", 2))) {
} else if (std::ranges::equal(std::views::take(path, 3), std::views::repeat("", 3))) {
/* If a URI does not contain an authority component, then the path cannot begin
with two slash characters ("//") */
unreachable();
Loaded 100 of 111 files, more files were not shown because too many files have changed in this diff. Show more