Compare commits

..
Author SHA1 Message Date
Robert Hensing 97bd6ebfed Add hydraJobs.tests.nix-serve 2024-09-18 18:32:46 +02:00
Robert Hensing cf5592c66a TMP: flake: Update nixpkgs to release-24.05 (3a458f7, 2024-09-18)
This includes https://github.com/NixOS/nixpkgs/pull/342817

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/c3d4ac725177c030b1e289015989da2ad9d56af0' (2024-08-15)
  → 'github:NixOS/nixpkgs/3a458f7c763ca62c6bf454b8d828bd86b7250671' (2024-09-18)
2024-09-18 18:24:35 +02:00
Eelco Dolstra 0ed2ab0533 Merge pull request #11527 from Mic92/macos-test-fix
Fix macOS tests with meson
2024-09-18 15:07:22 +02:00
Jörg Thalheim 2b7642632e devShell: increase priority of clang-tools
Before we would get the unwrapped version of clang-tools from clang
itself, which doesn't quite work.
2024-09-18 14:29:26 +02:00
Nikodem Rabuliński 8105307f0f Always initialize curl in parent process on darwin
Because of an objc quirk[1], calling curl_global_init for the first time
after fork() will always result in a crash.
Up until now the solution has been to set
OBJC_DISABLE_INITIALIZE_FORK_SAFETY for every nix process to ignore
that error.
This is less than ideal because we were setting it in package.nix,
which meant that running nix tests locally would fail because
that variable was not set.
Instead of working around that error we address it at the core -
by calling curl_global_init inside initLibStore, which should mean
curl will already have been initialized by the time we try to do so in
a forked process.

[1] https://github.com/apple-oss-distributions/objc4/blob/01edf1705fbc3ff78a423cd21e03dfc21eb4d780/runtime/objc-initialize.mm#L614-L636

(cherry-picked and adapted from https://git.lix.systems/lix-project/lix/commit/c7d97802e4f59b8621e67cf62275d6a7fde8fe62)
2024-09-18 14:29:26 +02:00
Valentin Gagarin ec2b01585f Merge pull request #11526 from quatquatt/update-multiple-inputs
docs: Provide an example of updating multiple inputs
2024-09-18 13:34:50 +02:00
quatquatt 9e335ee3b9 docs: Provide an example of updating multiple inputs with nix flake update
docs: Provide an example of updating multiple inputs with `nix flake update`
2024-09-17 22:50:21 -04:00
Jörg Thalheim a20659f4fa unitests: fix tmpdir when running with meson on macOS 2024-09-17 19:15:01 +02:00
Jörg Thalheim 98db531df2 libstore-support: check that we can create the store 2024-09-17 18:46:50 +02:00
John Ericson 00013c7348 Merge pull request #11522 from tweag/fix-meson-build-in-sandbox
Fix meson build on macOS in sandbox
2024-09-17 12:04:16 -04:00
Yuriy Taraday c7c3a7f667 Fix meson build on macOS in sandbox
Workaround at src/libstore/meson.build#L429-L434 by @Ericson2314 from
https://github.com/NixOS/nix/pull/11302 erroneously used `macos` instead
of `darwin` to distinguish macOS, while meson docs list only `darwin`:
https://mesonbuild.com/Reference-tables.html#operating-system-names.

Original thread: https://github.com/NixOS/nix/issues/2503#issuecomment-2353184049
2024-09-17 16:15:39 +02:00
John Ericson 95f2b2beab Merge pull request #8766 from brianmcgee/fix/base64-decode-error-message
base64Decode: clearer error message when an invalid character is detected
2024-09-17 09:59:15 -04:00
Eelco Dolstra 3fb6605d3d Merge pull request #11514 from DeterminateSystems/fix-derivation-add
nix derivation add: Remove reference to "installable"
2024-09-17 13:12:12 +02:00
Eelco Dolstra 65f138f669 nix derivation add: Remove reference to "installable"
It doesn't operate on an installable, so don't mention it.
2024-09-17 12:37:33 +02:00
Eelco Dolstra 8321fe7c2b Merge pull request #11512 from NixOS/dependabot/github_actions/cachix/install-nix-action-V28
Bump cachix/install-nix-action from V27 to 28
2024-09-17 12:29:11 +02:00
dependabot[bot] afa6cc7271 Bump cachix/install-nix-action from V27 to 28
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from V27 to 28. This release includes the previously tagged commit.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Commits](https://github.com/cachix/install-nix-action/compare/V27...V28)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-16 22:59:12 +00:00
Robert Hensing 799abea0c4 Merge pull request #11330 from NixOS/packfile
libgit2, GitRepo: Write (thin) packfiles
2024-09-16 14:42:59 +02:00
Robert Hensing 459d02672c fix Windows build 2024-09-16 13:42:46 +02:00
Robert Hensing 976f539f7d Make Repo::flush interruptible 2024-09-16 13:42:46 +02:00
Eelco Dolstra 176334d8be Merge pull request #10612 from edolstra/no-flake-substitution
Input::fetchToStore(): Don't try to substitute
2024-09-16 12:47:50 +02:00
Eelco Dolstra 2c42a9dbaa Merge pull request #11485 from abathur/sequoia_migration_errexit
sequoia-nixbld-user-migration: halt on error
2024-09-13 17:12:28 +02:00
Eelco Dolstra 11452ce674 Merge pull request #11490 from bryanhonof/bryanhonof.add-mixenvironment-to-run
Add MixEnvironment to CmdRun
2024-09-12 20:29:16 +02:00
Robert Hensing bbef37b4a6 Merge pull request #11423 from DeterminateSystems/ignore-stale-submodules
Git fetcher: Ignore .gitmodules entries that are not submodules
2024-09-12 18:14:18 +02:00
Eelco Dolstra bb1ce014be Merge pull request #11492 from DeterminateSystems/canon-nars
Make the NAR parser stricter and add some tests
2024-09-12 18:14:06 +02:00
Bryan Honof 2226f9864e feat(run): inherit from MixEnvironment 2024-09-12 17:47:48 +02:00
Eelco Dolstra 5737d31d4e Test the case hack a bit more 2024-09-12 17:27:41 +02:00
Eelco Dolstra 7aa3e7e3a5 Make the NAR parser much stricter wrt field order
We really want to enforce a canonical representation since NAR
hashing/signing/deduplication depends on that.
2024-09-12 15:57:46 +02:00
Eelco Dolstra 27ec0def74 Typo 2024-09-12 15:33:55 +02:00
Eelco Dolstra 69bf9947c7 Put 'names' in the right scope 2024-09-12 15:29:54 +02:00
Eelco Dolstra 4de9587e50 Improve badArchive() 2024-09-12 15:27:55 +02:00
Eelco Dolstra 421aa1add1 Add tests for invalid file names in NARs
Note: in general, we rely on the OS to tell us if a name is invalid or
if two names normalize in the same way. But for security, we do want
to make sure that we catch '.', '..', slashes and NUL characters. (NUL
characters aren't really a security issue, but since they would be
truncated when we pass them to the OS, it would be canonicity problem.)
2024-09-12 15:02:30 +02:00
Eelco Dolstra 12fd65d179 Disable subflakes test
Relative path flakes ("subflakes") are basically fundamentally
broken, since they produce lock file entries like

  "locked": {
    "lastModified": 1,
    "narHash": "sha256-/2tW9SKjQbRLzfcJs5SHijli6l3+iPr1235zylGynK8=",
    "path": "./flakeC",
    "type": "path"
  },

that don't specify what "./flakeC" is relative to. They *sometimes*
worked by accident because the `narHash` field allowed
`fetchToStore()` to get the store path of the subflake *if* it
happened to exist in the local store or in a substituter.

Subflakes are properly fixed in #10089 (which adds a "parent" field to
the lock file). Rather than come up with some crazy hack to make them
work in the interim, let's just disable the only test that depends on
the broken behaviour for now.
2024-09-11 21:58:32 +02:00
Eelco Dolstra 30aa45a373 Formatting 2024-09-11 20:35:04 +02:00
Travis A. Everett 48477d4a3e doc: add admonitions for macOS 15 Sequoia update (#11487)
The impending release of macOS 15 Sequoia will break many existing nix
installs on macOS, which may lead to an increased number of people who
are looking to try to reinstall Nix without noticing the open/pinned
issue (#10892) that explains the problem and outlines how to migrate
existing installs.

These admonitions are a short-term measure until we are over the hump
and support volumes dwindle.
2024-09-11 17:50:47 +00:00
John Ericson db7c868d24 Merge pull request #11378 from Mic92/nix-dir-errors
builtins.readDir: fix nix error trace on filesystem errors
2024-09-11 13:10:28 -04:00
John Ericson 193dc49097 tweak unpack channel built-in, std::filesystem::path for tarball 2024-09-11 12:29:49 -04:00
Eelco Dolstra 1c2508f176 Merge pull request #11466 from DeterminateSystems/git-mtime-warning
Git fetcher: Don't update mtime of ref file if fetching by rev
2024-09-11 17:34:46 +02:00
Eelco Dolstra e557096cef Add release note 2024-09-11 17:32:16 +02:00
Eelco Dolstra 3fbd71701a Add test 2024-09-11 17:27:39 +02:00
Eelco Dolstra aa41de7d82 Merge remote-tracking branch 'origin/master' into no-flake-substitution 2024-09-11 17:10:55 +02:00
Travis A. Everett 51a01aa6c5 sequoia-nixbld-user-migration: halt on error
Addressing user feedback about a case where actions the script takes
may fail without a specific permission if run over SSH.
2024-09-11 08:56:53 -05:00
Eelco Dolstra 46339db18d Merge pull request #11471 from zimbatm/hacking-md
doc: add HACKING.md symlink
2024-09-11 14:54:28 +02:00
Eelco Dolstra 9a7c0d8933 Merge pull request #11473 from Artturin/keepfailedreadable
Fix making the build directory kept by `keep-failed` readable
2024-09-11 14:52:50 +02:00
Eelco Dolstra 894da59186 Merge pull request #11447 from DeterminateSystems/nix-flake-metadata-chroot-store
nix flake {metadata,archive}: Fix chroot stores
2024-09-11 14:51:56 +02:00
Noam Yorav-Raphael 38bfbb297c Use envvars NIX_CACHE_HOME, NIX_CONFIG_HOME, NIX_DATA_HOME, NIX_STATE_HOME if defined (#11351) 2024-09-11 10:36:46 +00:00
Jörg ThalheimandJohn Ericson 04ce0e648a add release notes for filesystem fixes
Update doc/manual/rl-next/filesystem-errors.md

Co-authored-by: John Ericson <git@JohnEricson.me>
2024-09-11 08:23:18 +02:00
tomberek c60e1be62c Merge pull request #11438 from zimbatm/nix-fmt-pwd
fix(nix fmt): remove the default "." argument
2024-09-10 23:44:16 -04:00
tomberek b30b3b27f8 Merge pull request #11476 from abathur/sequoia_migration_undebug
sequoia-nixbld-user-migration: disable trace mode
2024-09-10 23:35:23 -04:00
Travis A. Everett f2e7e996da sequoia-nixbld-user-migration: disable trace mode
Was hoping to leave this enabled for a little while as core community
members test this script out, but Apple's aggressive release timeline
for macOS 15 Sequoia has caught us off-guard here.

It's probably not ideal for a general audience if the script spews all
of this output--and people can still force bash to run in trace mode
if we really need to debug a problem.
2024-09-10 20:54:09 -05:00
Artturin ebebe626ff Fix making the build directory kept by keep-failed readable
Caused by 1d3696f0fb

Without this fix the kept build directory is readable only by root

```
$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5
drwx------ root root 60 B Wed Sep 11 00:09:48 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/

$ sudo ls -ld /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build
drwxr-xr-x nixbld1 nixbld 80 B Wed Sep 11 00:09:58 2024  /comp-temp/nix-build-openssh-static-x86_64-unknown-linux-musl-9.8p1.drv-5/build/
```
2024-09-11 00:23:14 +03:00
zimbatm c4766d7b8b fix(nix fmt): remove the default "." argument
When `nix fmt` is called without an argument, Nix appends the "." argument before calling the formatter. The comment in the code is:
> Format the current flake out of the box

This also happens when formatting sub-folders.

This means that the formatter is now unable to distinguish, as an interface, whether the "." argument is coming from the flake or the user's intent to format the current folder. This decision should be up to the formatter.

Treefmt, for example, will automatically look up the project's root and format all the files. This is the desired behaviour. But because the "." argument is passed, it cannot function as expected.
2024-09-10 22:18:56 +02:00
Eelco Dolstra b9d3cdfbd2 Merge pull request #11472 from Mic92/darwin-fix
add missing filesystem include (x86_64-darwin fix)
2024-09-10 17:33:09 +02:00
Jörg Thalheim 1ca1439b1f add missing filesystem include (x86_64-darwin fix) 2024-09-10 16:45:39 +02:00
zimbatm be480971c2 doc: add HACKING.md symlink
As a hacker, I should be able to checkout the repo, and find relevant
information on how to develop in the project somewhere in the top-level.
Either in the README.md, or CONTRIBUTING.md or HACKING.md files.

This PR symlinks the HACKING.md into the right place in the manual.
2024-09-10 15:26:38 +02:00
Eelco Dolstra 9c0e968843 Merge commit from fork
Fix unsafe NAR unpacking
2024-09-10 12:42:55 +02:00
Tom Bereknyei c55b285cf9 tests: test was re-named 2024-09-09 22:15:45 -04:00
Eelco Dolstra b80b091bac Git fetcher: Don't update mtime of ref file if fetching by rev
This fixes the warning

  $ nix eval --store /tmp/nix --expr 'builtins.fetchTree { type = "git"; url = "https://github.com/DeterminateSystems/attic"; ref = "fixups-for-magic-nix-cache"; rev = "635753a2069d4b8228e846dc5c09ad361c75cd1a"; }'
  warning: could not update mtime for file '/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache': error: changing modification time of '"/home/eelco/.cache/nix/gitv3/09788h9zgba5lbfkaa6ija2dvi004jwsqjf5ln21i2njs07cz766/refs/heads/fixups-for-magic-nix-cache"': No such file or directory

When we're fetching by rev, that file doesn't necessarily exist, and we
don't care about it anyway.
2024-09-09 19:52:21 +02:00
Eelco Dolstra c5a4dfa660 nix flake {metadata,archive}: Fix chroot stores
Fixes

  $ nix flake metadata --store /tmp/nix nixpkgs
  error: path '/tmp/nix/nix/store/65xpqkz92d9j7k5ric4z8lzhiigxsfbg-source/flake.nix' is not in the Nix store

This has been broken since 598deb2b23.
2024-09-09 15:41:38 +02:00
Eelco Dolstra 5ca2f58798 Improve use-case-hack description slightly 2024-09-09 14:29:05 +02:00
Eelco Dolstra 4cfa59fdb3 Typo 2024-09-09 14:11:35 +02:00
Eelco Dolstra 4c7a6ffee7 Merge pull request #11380 from roberth/eco-friendly-progress-bar
progress-bar: Only write when truly updated
2024-09-09 13:58:42 +02:00
Eelco Dolstra 5e337ee60d Merge pull request #11412 from parkerhoyes/daemon-cgroup
Move daemon process into sub-cgroup
2024-09-09 13:49:31 +02:00
Robert Hensing 69e925640f Merge pull request #11432 from fricklerhandwerk/docs-defexpr
docs: small fixups on the default expression
2024-09-08 12:31:23 +02:00
Robert Hensing c955563b64 fix: Avoid deadlock in ProgressBar::redraw() 2024-09-08 11:44:24 +02:00
Robert Hensing e10ea78f93 refact: Inline ProgressBar::draw(state, newOutput), inline local output 2024-09-08 01:23:01 +02:00
Robert Hensing 047d9643b5 refact: Extract ProgressBar::redraw(newOutput) 2024-09-08 01:23:01 +02:00
Robert Hensing 9df5236c46 progress-bar: Only write when truly updated 2024-09-08 01:04:14 +02:00
Robert Hensing 9ae7140beb Merge pull request #11436 from fricklerhandwerk/reword-nix-expr-gloss
docs: reword glossary entry on Nix expression
2024-09-07 22:47:32 +01:00
Robert Hensing 791f943761 Merge pull request #11437 from fricklerhandwerk/default-nix-path-docs
docs: explain provenance of default `nix-path` values
2024-09-07 22:38:50 +01:00
Eelco Dolstra 52ba3cc5ea Test that deserializing regular files / symlinks is exclusive 2024-09-06 16:28:09 +02:00
Eelco Dolstra f15aef2a79 Merge pull request #11413 from cole-h/fixup-daemon-service
fixup: use the real bindir for systemd unit's bindir
2024-09-06 15:56:44 +02:00
Eelco Dolstra 644a034783 Merge pull request #11440 from fricklerhandwerk/fix-docs-build-instructions
update instructions to build the manual
2024-09-06 15:48:52 +02:00
Eelco Dolstra bb1af0645e Merge pull request #11439 from Mic92/revert-warning
Revert "fix: Error on malformed URI query parameter"
2024-09-06 15:48:18 +02:00
Valentin Gagarin 92be107c8e update instructions to build the manual 2024-09-06 10:33:12 +02:00
Eelco Dolstra 9fcb588dd8 RestoreSink::createDirectory(): Use append()
On macOS, `mkdir("x/')` behaves differently than `mkdir("x")` if `x` is
a dangling symlink (the formed succeed while the latter fails). So make
sure we always strip the trailing slash.
2024-09-05 22:21:53 +02:00
Eelco Dolstra 21dcbd7e83 Fix test on macOS 2024-09-05 20:55:24 +02:00
Eelco Dolstra 7a765a6aaf Test that deserializing NARs with names with equal Unicode normal forms fails on macOS
The test is based on the one by @puckipedia but with the file names
swapped to make them sorted.
2024-09-05 20:37:26 +02:00
Eelco Dolstra 3557587381 Detect NAR directory entries that collide with another path after case-hacking
The test was made by @puckipedia.
2024-09-05 19:26:10 +02:00
Eelco Dolstra 77c090cdbd More tests 2024-09-05 16:54:12 +02:00
Eelco Dolstra da1ad28912 Test that nix-store --restore fails if the output already exists
This restores the behaviour from before the std::filesystem
refactorings.
2024-09-05 16:48:43 +02:00
Eelco Dolstra 83d5b32803 Add test case for NARs with duplicate directory entries
This test was made by @puckipedia.
2024-09-05 16:41:15 +02:00
Jörg Thalheim 5a5a010120 Revert "fix: Error on malformed URI query parameter"
This reverts commit c9f45677b5.

This now triggers on simple cases like `nix build .#nix`.
Reverting for now.
2024-09-05 15:18:16 +02:00
Jörg Thalheim a81083d080 Revert "Update src/libutil/url.cc"
This reverts commit 9b1cefe27e.
2024-09-05 15:18:16 +02:00
Jörg Thalheim 70c52d72f4 builtins.unpackChannel: wrap filesystem errors and sanitize channelName
Otherwise these errors are not caught correctly
2024-09-05 14:02:00 +02:00
Jörg Thalheim 05a1ffe236 repl: wrap std::filesystem error into SysError
/tmp/ecstatic-euler-mAFGV7
% /home/joerg/git/nix/build/subprojects/nix/nix repl
Nix 2.25.0
Type :? for help.

after doing rm /tmp/ecstatic-euler-mAFGV7 this will result in:

nix-repl> :lf .
error: cannot determine current working directory: No such file or directory

Before it would make the repl crash

/tmp/clever-hermann-MCm7A9
% /home/joerg/git/nix/build/subprojects/nix/nix repl
Nix 2.25.0
Type :? for help.
nix-repl> :lf .
error: filesystem error: cannot get current path: No such file or directory
2024-09-05 14:00:39 +02:00
Jörg Thalheim 22ba4dc78d builtins.readDir: fix nix error trace on filesystem errors
Before:

nix-env % ./src/nix/nix eval --impure --expr 'let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f' --show-trace
error: filesystem error: directory iterator cannot open directory: No such file or directory [/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo]

After:

error:
       … while calling the 'readDir' builtin
         at «string»:1:9:
            1| let f = builtins.readDir "/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo"; in f
             |         ^

       error: reading directory '/nix/store/hs3yxdq9knimwdm51gvbs4dvncz46f9d-hello-2.12.1/foo': No such file or directory
2024-09-05 14:00:39 +02:00
Valentin Gagarin a1cc362d9d fix broken link (#11435) 2024-09-05 08:34:07 +00:00
tomberek 5bfe198ad5 Merge pull request #11433 from abathur/fix_sequoia_migration_gid
use existing GID in sequoia migration script
2024-09-05 00:20:56 -04:00
Valentin Gagarin 17655ecfef docs: explain provenance of default nix-path values
this should make it more obvious how things are related to each other, and also
hopefully expose the historical context without having to say on every
corner that these details are accounting for legacy decisions.
2024-09-05 04:35:06 +02:00
Valentin Gagarin 48249e001a docs: reword glossary entry on Nix expression
this makes it less cumbersome to read and puts the statements in
meaningful order.
2024-09-05 04:13:43 +02:00
Valentin Gagarin 38d9d536a8 docs: small fixups on the default expression
- highlighted example
- linked definitions to the glossary (this is a shorter read)
- fixed some artefact
2024-09-05 03:32:30 +02:00
Travis A. Everett ef3d3c5682 use existing GID in sequoia migration script
I hardcoded the wrong GID (30001 instead of 30000), but it's
better to just pick up the GID from the existing group.
2024-09-04 20:26:50 -05:00
Eelco Dolstra 495d32e1b8 NAR parser: Fix check for duplicate / incorrectly sorted entries
"prevName" was always empty because it was declared in the wrong scope.
2024-09-04 21:43:59 +02:00
Parker Hoyes bd6ae2f3b9 Use getCurrentCgroup() in getMaxCPU() 2024-09-04 19:10:31 +00:00
Eelco Dolstra 668d63d8dd Merge pull request #11424 from kjeremy/lix-1462
Pull fut.get() out of the lock
2024-09-04 20:52:38 +02:00
Parker Hoyes 03484641a1 Simplify getRootCgroup()
Static local initializers are atomic in C++.
2024-09-04 18:11:16 +00:00
Jeremy KolbandEelco Dolstra 8152c5c828 Remote nullptr
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-09-04 12:55:32 -04:00
Jeremy Kolb 46f6505865 Pull fut.get() out of the lock
This is https://gerrit.lix.systems/c/lix/+/1462 by @jade_

see: https://git.lix.systems/lix-project/lix/issues/366
see: https://gerrit.lix.systems/c/lix/+/1462
2024-09-04 10:14:51 -04:00
Eelco Dolstra 9d24080090 Git fetcher: Ignore .gitmodules entries that are not submodules
Fixes #10739.
2024-09-04 14:46:33 +02:00
Eelco Dolstra 357cec93b6 Merge pull request #11414 from parkerhoyes/cgroups-delegation
Enable cgroups delegation for systemd
2024-09-04 13:28:10 +02:00
tomberek a6e6da3b0c Merge pull request #10919 from abathur/macos_sequoia_fixes
install-darwin: fix _nixbld uids for macOS sequoia
2024-09-03 19:55:55 -04:00
Cole Helbling 9e79061bac fixup: use the real bindir for systemd unit's bindir
Prior to this commit, the unit contained this line:

     ExecStart=@share/nix-daemon nix-daemon --daemon

which caused systemd to complain:

     Failed to restart nix-daemon.service: Unit nix-daemon.service has a bad unit file setting.
     See system logs and 'systemctl status nix-daemon.service' for details.

and had this in the unit output:

     Sep 03 13:34:59 scadrial systemd[1]: /etc/systemd/system/nix-daemon.service:10: Neither a valid executable name nor an absolute path: share/nix-daemon
     Sep 03 13:34:59 scadrial systemd[1]: nix-daemon.service: Unit configuration has fatal error, unit will not be started.

(Notice how it's trying to execute `share/nix-daemon`, which is unlikely
to exist.)

Now with this commit, the path to the daemon binary is properly set:

     ExecStart=@/nix/store/lcbx6d8gzznf3z3c8lsv9jy3j6c67x6r-nix-2.25.0pre20240903_dirty/bin/nix-daemon nix-daemon --daemon
2024-09-03 13:34:37 -07:00
Parker Hoyes 62a99049c4 Enable cgroups delegation for systemd 2024-09-03 19:07:18 +00:00
Parker Hoyes 4c88deef38 Add tests for daemon with cgroups 2024-09-03 17:27:56 +00:00
Parker Hoyes 46b3188045 Move daemon process into sub-cgroup
The daemon process is now moved into a new sub-cgroup called nix-daemon when the
daemon starts. This is necessary to abide by the no-processes-in-inner-nodes
rule, because the service cgroup becomes an inner node when the child cgroups
for the build are created (see LocalDerivationGoal::startBuilder()).

See #9675
2024-09-03 17:18:01 +00:00
Eelco Dolstra 1073a8effa Merge pull request #11410 from DeterminateSystems/upload-release-cleanup
maintainers/upload-release.pl: Delete temporary directories when we're done
2024-09-03 17:53:29 +02:00
Eelco Dolstra 02bb633a58 maintainers/upload-release.pl: Delete temporary directories when we're done 2024-09-03 16:49:57 +02:00
Eelco Dolstra 784a8436a0 Merge pull request #11402 from DeterminateSystems/fix-max-substitution-jobs
Respect max-substitution-jobs again
2024-09-03 13:35:34 +02:00
Robert Hensing 9a9f5ae5f2 Merge pull request #11405 from SuperSandro2000/patch-2
Fix link anchor
2024-09-03 10:53:45 +02:00
Jeremy Kerfs 4decd6f8b3 add removal of ~root/.cache/nix to uninstall instructions (#11407) 2024-09-03 08:36:00 +00:00
Sandro 13100eaa4f Fix link anchor 2024-09-02 23:25:44 +02:00
Eelco Dolstra b7acd1c414 "unsigned" -> size_t
Slight cleanup.
2024-09-02 17:28:55 +02:00
Eelco Dolstra a33cb8af56 Respect max-substitution-jobs again
This broke in #11005. Any number of PathSubstitutionGoals would
be woken up by a single build slot becoming available. If there
are a lot of substitution goals active, this could lead to us
running out of file descriptors (especially on macOS where the
default limit is 256).
2024-09-02 17:28:11 +02:00
Eelco Dolstra ef1ac0d117 Merge pull request #11390 from alyssais/key
Don't refer to public keys as secret keys in error
2024-09-02 14:20:04 +02:00
Alyssa Ross 9cc550d652 Don't refer to public keys as secret keys in error
This constructor is used for public keys as well.
2024-08-31 16:00:11 +02:00
Robert Hensing b89eca9aec Merge pull request #11349 from bryanhonof/bryanhonof.check-query-for-equals
Warn on malformed URI query parameter
2024-08-28 21:56:00 +02:00
Bryan HonofandRobert Hensing 9b1cefe27e Update src/libutil/url.cc
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-08-28 18:48:18 +02:00
Robert Hensing c1fe3546ed libgit2: Add libgit2-packbuilder-callback-interruptible.patch 2024-08-28 17:49:22 +02:00
Robert Hensing 57c48304bb fixup: Release odb 2024-08-28 17:49:22 +02:00
Robert Hensing fb8d3ed150 fixup: sync -> flush
The latter is not used for memory synchronization things.
2024-08-28 17:49:22 +02:00
Robert Hensing 97ff2ed455 Sync tarball cache within tarball cache Activity 2024-08-28 17:49:22 +02:00
Robert Hensing d0f8a92363 Make tarball cache more interruptible 2024-08-28 17:49:22 +02:00
Robert Hensing 5dd6c4f062 libgit2, GitRepo: Write thin packfiles
libgit2 didn't write thin ones, hence the patch.

This should improve performance on systems with weak I/O in ~/.cache,
especially in terms of operations per second, or where system calls
are slower. (macOS, VMs?)
2024-08-28 17:49:21 +02:00
Jan Hrcek cb4b9be458 Fix few duplicate word occurrences (#11381) 2024-08-28 07:01:56 +00:00
Seggy UmbohandValentin Gagarin 8e63dc4455 Fix diskutil deleteVolume command in Uninstall guide (#11371)
Co-authored-by: Valentin Gagarin <valentin@gagarin.work>
2024-08-28 00:23:27 +00:00
Robert Hensing 076b6f7bb1 Merge pull request #11302 from obsidiansystems/meson-misc-2
Misc Meson progress
2024-08-27 19:32:03 +02:00
c7ec33605e Meson misc things
Meson-ify a few things, scripts, completions, etc. Should make our Meson
build complete except for docs.

Co-Authored-By: Qyriad <qyriad@qyriad.me>
Co-Authored-By: eldritch horrors <pennae@lix.systems>
2024-08-27 10:19:55 -04:00
John Ericson 8af73f0a74 Merge pull request #11379 from nix-windows/nix-collect-garbage-std-filesystem-path
More `std::filesystem` for `nix-collect-garbage`
2024-08-26 18:27:55 -04:00
John Ericson 2a1a26f8fc Merge pull request #11362 from siddhantk232/path-in-exec
More `std::filesystem::path` in end executables
2024-08-26 17:58:29 -04:00
John EricsonandsiddhantCodes 8bce63f30a More std::filesystem for nix-collect-garbage
Co-Authored-By: siddhantCodes <siddhantk232@gmail.com>
2024-08-26 17:29:23 -04:00
John EricsonandsiddhantCodes a97a08411c More support for std::filepath in libnixutil
We're not replacing `Path` in exposed definitions in many cases, but
just adding alternatives. This will allow us to "top down" change `Path`
to `std::fileysystem::path`, and then we can remove the `Path`-using
utilities which will become unused.

Also add some test files which we forgot to include in the libutil unit
tests `meson.build`.

Co-Authored-By: siddhantCodes <siddhantk232@gmail.com>
2024-08-26 17:23:56 -04:00
John Ericson dbabfc92d4 Make sure we have an execvpe on Windows too
Necessary to fix a build (that was already broken in other ways) after
PR #11021.
2024-08-26 15:45:55 -04:00
Emily 11cf29b15c install-darwin: increment base UID by 1 (#15) 2024-08-26 11:59:58 -05:00
Robert Hensing 88998fae74 Merge pull request #11304 from hercules-ci/repl-doc-functor
`:doc`: support `__functor`
2024-08-26 17:20:37 +02:00
tomberek 96a2ddab27 Merge pull request #10910 from hercules-ci/c-api-nix_clear_err
C API: `nix_clear_err`
2024-08-26 10:50:26 -04:00
tomberek 3b6e24f66c Merge pull request #11346 from mjgallag/nix-profile-daemon-set-u
nix-daemon.sh profile script: operate under `set -u` in bash
2024-08-26 10:49:31 -04:00
tomberek 440de80d34 Merge pull request #11021 from hercules-ci/issue-11010
Fix SSH invocation when local SHELL misbehaves
2024-08-26 10:40:51 -04:00
Robert Hensing 77ddcbe12e getDoc: Explain why we partially apply __functor 2024-08-26 16:15:13 +02:00
tomberek 7b53636150 Merge pull request #11321 from Mic92/mergify-automerge
replace backport github action with mergify
2024-08-26 10:07:03 -04:00
tomberek e285a6a8bf Merge pull request #11372 from emilazy/push-xryryzwxnxsn
sequoia-nixbld-user-migration: increment base UID by 1
2024-08-26 09:53:13 -04:00
tomberek 1a281e5ddc Merge pull request #11361 from hercules-ci/packaging-expose-libs
packaging: Expose libs
2024-08-26 09:52:27 -04:00
Robert Hensing 8bd835b7bc Merge pull request #11278 from Mic92/close-fds
libutil: rename and optimize closeMostFDs
2024-08-26 15:51:34 +02:00
Emily 93a8b06070 sequoia-nixbld-user-migration: increment base UID by 1 2024-08-26 05:27:10 +01:00
tomberek a30eb6de1b Merge pull request #11366 from Mic92/homeless-shelter
Revert: Make $HOME=/proc/homeless-shelter instead of /homeless-shelter
2024-08-25 05:07:41 -04:00
Jörg Thalheim 90560eeccc Revert "Merge pull request #11350 from noamraph/homeless-shelter-to-proc-changelog"
This reverts commit fa49d2e356, reversing
changes made to af26fe3934.
2024-08-25 09:08:27 +02:00
Jörg Thalheim 096bec8eb2 Revert "Merge pull request #11300 from noamraph/homeless-shelter-to-proc"
This reverts commit 43e82c9446, reversing
changes made to d79b9bdec0.

Since /proc/homeless-shelter returns a different errno than /homeless-shelter (ENOENT vs EACCES), we need to revert this change.
Software depends on this error code i.e. cargo and therefore breaks.
2024-08-25 09:08:17 +02:00
Bryan Honof c9f45677b5 fix: Error on malformed URI query parameter
Signed-off-by: Bryan Honof <bryanhonof@gmail.com>
2024-08-23 22:04:37 +02:00
Robert Hensing 85f1aa6b3d Merge pull request #11356 from ehmry/nix_store_copy_closure
Add nix_store_copy_closure to libstore-c
2024-08-23 20:57:14 +02:00
Robert Hensing 2a14a20bb3 packaging: Expose libs
This exposes the libraries in a way that is easily replicated in
Nixpkgs, without having to create a package set within a package
set.
2024-08-23 20:47:22 +02:00
Emery Hemingway 5d28a00322 Add nix_store_copy_closure to libstore-c 2024-08-23 12:49:39 +03:00
Robert Hensing 0bcc83d0e7 Merge pull request #11348 from DeterminateSystems/remove-makeMutable
Remove dead makeMutable() function
2024-08-23 11:08:28 +02:00
Eelco Dolstra 277412e9e9 Merge pull request #11355 from Aleksanaa/ci-badge
README: update CI badge
2024-08-22 17:48:15 +02:00
Eelco Dolstra a66fa0850d Merge pull request #11354 from DeterminateSystems/speed-up-ci
Speed up GHA CI
2024-08-22 17:46:44 +02:00
Eelco Dolstra 1facc3e35e Merge pull request #7126 from squalus/fsync-store-paths
Add fsync-store-paths option
2024-08-22 17:45:11 +02:00
aleksana 168bf9c3cd README: update CI badge
The `Test` workflow was renamed to `CI` in
https://github.com/NixOS/nix/commit/9aa486c4be2801a25847b9c75012054d04e792d0.
It still seems to be showing the status it was last running on the
master branch. This information is misleading and should be corrected.
2024-08-22 23:16:14 +08:00
Eelco Dolstra 71e7188e07 Add release note 2024-08-22 15:48:36 +02:00
Eelco Dolstra 9ff0b55d4e Add a VM test for fsync-store-paths
Based on https://github.com/squalus/nix-durability-tests/blob/master/flake.nix.
2024-08-22 15:34:19 +02:00
Eelco Dolstra 21a164aa03 Fix hang
Signed-off-by: Eelco Dolstra <edolstra@gmail.com>
2024-08-22 15:26:34 +02:00
Eelco Dolstra 915db74dbf Merge pull request #11343 from DeterminateSystems/no-framedsink-threads
withFramedSink(): Don't use a thread to monitor the other side
2024-08-22 14:23:19 +02:00
Eelco Dolstra 3af73bae5c Remove checks for nixComponents
We are currently building Nix twice in the main GHA CI job, which is
frequently timing out. Obviously, we want this to be fast, so only do
the main build for now.
2024-08-22 13:48:46 +02:00
Eelco Dolstra 528e490418 Disable the static build in CI
GHA builds are now frequently timing out, so let's not do this. If
wanted, it could be done in a separate job.
2024-08-22 13:47:33 +02:00
Eelco Dolstra fac756fed4 Add FIXME 2024-08-21 21:08:26 +02:00
Eelco Dolstra 270c8469d7 select() -> poll() for Windows compat 2024-08-21 20:54:02 +02:00
Eelco DolstraandJohn Ericson ce7cf4a2d3 Update src/libutil/serialise.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-08-21 19:50:24 +02:00
Eelco Dolstra e8752ca57a Add FIXME 2024-08-21 17:05:36 +02:00
Eelco Dolstra efbf499635 Remove redundant " 2024-08-21 16:50:18 +02:00
Eelco Dolstra 3fe1b60c57 Remove dead makeMutable() function
This code wasn't being compiled because the #if conditional didn't
trigger. Anyway, Nix 0.15 is 15 years old so we don't need to keep
this around.
2024-08-21 16:45:46 +02:00
Eelco Dolstra e049d38290 Merge remote-tracking branch 'origin/master' into fsync-store-paths 2024-08-21 16:37:21 +02:00
03b258bf97 libutil: rename and optimize closeMostFDs
this is only used to close non-stdio files in derivation sandboxes. we
may as well encode that in its name, drop the unnecessary integer set,
and use close_range to deal with the actual closing of files. not only
is this clearer, it also makes sandbox setup on linux fast by 1ms each

(cherry-picked and adapted from
https://git.lix.systems/lix-project/lix/commit/c7d97802e4f59b8621e67cf62275d6a7fde8fe62)

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2024-08-21 06:45:31 +02:00
Travis A. Everett 75567423fb install-darwin: move nixbld gid to match first UID 2024-08-20 09:20:24 -05:00
Travis A. Everett df36ff0d1e install-darwin: fix _nixbld uids for macOS sequoia
Starting in macOS 15 Sequoia, macOS daemon UIDs are encroaching on our
default UIDs of 301-332. This commit relocates our range up to avoid
clashing with the current UIDs of 301-304 and buy us a little time
while still leaving headroom for people installing more than 32 users.
2024-08-20 09:20:24 -05:00
Eelco Dolstra 39daa4a0d3 withFramedSink(): Don't use a thread to monitor the other side
Since withFramedSink() is now used a lot more than in the past (for
every addToStore() variant), we were creating a lot of threads, e.g.

  nix flake show --no-eval-cache --all-systems github:NixOS/nix/afdd12be5e19c0001ff3297dea544301108d298

would create 46418 threads. While threads on Linux are cheap, this is
still substantial overhead.

So instead, just poll from FramedSink before every write whether there
are pending messages from the daemon. This could slightly increase the
latency on log messages from the daemon, but not on exceptions (which
were only synchronously checked from FramedSink anyway).

This speeds up the command above from 19.2s to 17.5s on my machine (a
9% speedup).
2024-08-19 18:15:15 +02:00
Robert Hensing 2f0db04da0 tests.installer: Load profile with -o unset
Tested with an ubuntu case. Might need revision depending on what hydra
thinks of the rest.
2024-08-19 16:41:40 +02:00
Jörg Thalheim 80f20fa4cb replace backport github action with mergify
The current backport action cannot automerge because
the github action bot does not trigger github CI actions.
Mergify instead does not have this limitation and can also
use a merge queue.

On top we have now a declarative configuration to allow
contributers to add new tests to required without having access
to the github org.

An example pull request and backport can be seen here:

https://github.com/Mic92/nix-1/pull/4

and here:

https://github.com/Mic92/nix-1/pull/5

To complete the setup the mergify app must be enabled for this repository.
It's already installed in the nixos organization for nixos-hardware and
other repositories.
2024-08-17 09:21:46 +02:00
Robert Hensing a03bb4455c Fix SSH invocation when local SHELL misbehaves
Setting it to /bin/sh will make it more predictable when users have
their favorite shell in SHELL, which might not behave as expected.
For instance, a bad rc file could send something to stdout before
our LocalCommand gets to write "started".

This may help https://github.com/NixOS/nix/issues/11010
2024-08-16 15:53:30 +02:00
Robert Hensing c4192a6617 Add nix::execvpe 2024-08-16 15:53:30 +02:00
Michael Gallagher 30af4a9e27 nix-daemon.sh profile script: operate under set -u in bash
see https://github.com/NixOS/nix/commit/d459d3307c7b1b10f6489ed048fff192e7834928
2024-08-15 17:56:05 -07:00
Robert Hensing 72a4d1f52d Add :doc support for __functor 2024-08-15 13:04:34 +02:00
Robert Hensing 6068e32aa7 refactor: Extract EvalState::addCallDepth 2024-08-15 13:04:34 +02:00
Robert Hensing 61381c9964 C API: Make nix_err_msg treat NIX_OK as having no message
The documentation "solved" this by specifying a precondition, but
let's just make it more robust, and not leak irrelevant messages
that might linger.
We don't clear the message when clearing the status, in order to
keep clearing fast; see last_err field doc.
2024-06-14 16:36:23 +02:00
Robert Hensing 2dc7598779 C API: Add nix_clear_err 2024-06-14 16:30:34 +02:00
Eelco Dolstra ff107d9d03 Input::fetchToStore(): Don't try to substitute
Having a narHash doesn't mean that we have the other attributes
returned by the fetcher (such as lastModified or rev). For instance,

   $ nix flake metadata github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f
   Last modified: 2024-01-15 10:51:22

but

   $ nix flake metadata github:NixOS/patchelf/7c2f768bf9601268a4e71c2ebe91e2011918a70f?narHash=sha256-PPXqKY2hJng4DBVE0I4xshv/vGLUskL7jl53roB8UdU%3D
   (does not print a "Last modified")

The latter only happens if the store path already exists or is
substitutable, which made this impure behaviour unpredictable.

Fixes #10601.
2024-04-26 16:41:33 +02:00
Brian McGee dc3ccf02bf base64Decode: clearer error message when an invalid character is detected
Output the offending string in its entirety to provide context.

Closes #8479
2023-08-01 08:55:00 +01:00
squalus 5987fb7459 Add fsync-store-paths option
- Add recursiveSync function to flush a directory tree to disk

- Add AutoCloseFD::startFsync to initiate an asynchronous fsync
  without waiting for the result

- Initiate an asynchronous fsync while extracting NAR files

- Implement the fsync-store-paths option in LocalStore
2022-12-20 12:03:35 -08:00
208 changed files with 3638 additions and 957 deletions

No files matched your search

-32
View File
@@ -1,32 +0,0 @@
name: Backport
on:
pull_request_target:
types: [closed, labeled]
permissions:
contents: read
jobs:
backport:
name: Backport Pull Request
permissions:
# for zeebe-io/backport-action
contents: write
pull-requests: write
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
# required to find all branches
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v3.0.2
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
github_workspace: ${{ github.workspace }}
pull_description: |-
Automatic backport to `${target_branch}`, triggered by a label in #${pull_number}.
# should be kept in sync with `uses`
version: v0.0.5
+4 -4
View File
@@ -20,7 +20,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V27
- uses: cachix/install-nix-action@V28
with:
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: "sandbox = true"
@@ -89,7 +89,7 @@ jobs:
with:
fetch-depth: 0
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
- uses: cachix/install-nix-action@V28
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- uses: cachix/cachix-action@v15
@@ -112,7 +112,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
- uses: cachix/install-nix-action@V27
- uses: cachix/install-nix-action@V28
with:
install_url: '${{needs.installer.outputs.installerURL}}'
install_options: "--tarball-url-prefix https://${{ env.CACHIX_NAME }}.cachix.org/serve"
@@ -142,7 +142,7 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: cachix/install-nix-action@V27
- uses: cachix/install-nix-action@V28
with:
install_url: https://releases.nixos.org/nix/nix-2.20.3/install
- run: echo CACHIX_NAME="$(echo $GITHUB_REPOSITORY-install-tests | tr "[A-Z]/" "[a-z]-")" >> $GITHUB_ENV
+92
View File
@@ -0,0 +1,92 @@
queue_rules:
- name: default
# all required tests need to go here
merge_conditions:
- check-success=installer
- check-success=installer_test (macos-latest)
- check-success=installer_test (ubuntu-latest)
- check-success=tests (macos-latest)
- check-success=tests (ubuntu-latest)
- check-success=vm_tests
merge_method: rebase
batch_size: 5
pull_request_rules:
- name: merge using the merge queue
conditions:
- base=master
- label~=merge-queue|dependencies
actions:
queue: {}
# The rules below will first create backport pull requests and put those in a merge queue.
- name: backport patches to 2.18
conditions:
- label=backport 2.18-maintenance
actions:
backport:
branches:
- 2.18-maintenance
labels:
- merge-queue
- name: backport patches to 2.19
conditions:
- label=backport 2.19-maintenance
actions:
backport:
branches:
- 2.19-maintenance
labels:
- merge-queue
- name: backport patches to 2.20
conditions:
- label=backport 2.20-maintenance
actions:
backport:
branches:
- 2.20-maintenance
labels:
- merge-queue
- name: backport patches to 2.21
conditions:
- label=backport 2.21-maintenance
actions:
backport:
branches:
- 2.21-maintenance
labels:
- merge-queue
- name: backport patches to 2.22
conditions:
- label=backport 2.22-maintenance
actions:
backport:
branches:
- 2.22-maintenance
labels:
- merge-queue
- name: backport patches to 2.23
conditions:
- label=backport 2.23-maintenance
actions:
backport:
branches:
- 2.23-maintenance
labels:
- merge-queue
- name: backport patches to 2.24
conditions:
- label=backport 2.24-maintenance
actions:
backport:
branches:
- "2.24-maintenance"
labels:
- merge-queue
Symlink
+1
View File
@@ -0,0 +1 @@
doc/manual/src/development/building.md
+1 -1
View File
@@ -1,7 +1,7 @@
# Nix
[![Open Collective supporters](https://opencollective.com/nixos/tiers/supporter/badge.svg?label=Supporters&color=brightgreen)](https://opencollective.com/nixos)
[![Test](https://github.com/NixOS/nix/workflows/Test/badge.svg)](https://github.com/NixOS/nix/actions)
[![CI](https://github.com/NixOS/nix/workflows/CI/badge.svg)](https://github.com/NixOS/nix/actions/workflows/ci.yml)
Nix is a powerful package manager for Linux and other Unix systems that makes package
management reliable and reproducible. Please refer to the [Nix manual](https://nix.dev/reference/nix-manual)
+3
View File
@@ -10,6 +10,7 @@ foreach dep : deps_public_subproject
endforeach
requires_public += deps_public
extra_pkg_config_variables = get_variable('extra_pkg_config_variables', {})
import('pkgconfig').generate(
this_library,
filebase : meson.project_name(),
@@ -20,6 +21,7 @@ import('pkgconfig').generate(
requires : requires_public,
requires_private : requires_private,
libraries_private : libraries_private,
variables : extra_pkg_config_variables,
)
meson.override_dependency(meson.project_name(), declare_dependency(
@@ -27,4 +29,5 @@ meson.override_dependency(meson.project_name(), declare_dependency(
link_with : this_library,
compile_args : ['-std=c++2a'],
dependencies : deps_public_subproject + deps_public,
variables : extra_pkg_config_variables,
))
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: Use envvars NIX_CACHE_HOME, NIX_CONFIG_HOME, NIX_DATA_HOME, NIX_STATE_HOME if defined
prs: [11351]
---
Added new environment variables:
- `NIX_CACHE_HOME`
- `NIX_CONFIG_HOME`
- `NIX_DATA_HOME`
- `NIX_STATE_HOME`
Each, if defined, takes precedence over the corresponding [XDG environment variable](@docroot@/command-ref/env-common.md#xdg-base-directories).
This provides more fine-grained control over where Nix looks for files, and allows to have a stand-alone Nix environment, which only uses files in a specific directory, and doesn't interfere with the user environment.
+14
View File
@@ -0,0 +1,14 @@
---
synopsis: wrap filesystem exceptions more correctly
issues: []
prs: [11378]
---
With the switch to `std::filesystem` in different places, Nix started to throw `std::filesystem::filesystem_error` in many places instead of its own exceptions.
This lead to no longer generating error traces, for example when listing a non-existing directory, and can also lead to crashes inside the Nix REPL.
This version catches these types of exception correctly and wrap them into Nix's own exeception type.
Author: [**@Mic92**](https://github.com/Mic92)
+9
View File
@@ -0,0 +1,9 @@
---
synopsis: Add setting `fsync-store-paths`
issues: [1218]
prs: [7126]
---
Nix now has a setting `fsync-store-paths` that ensures that new store paths are durably written to disk before they are registered as "valid" in Nix's database. This can prevent Nix store corruption if the system crashes or there is a power loss. This setting defaults to `false`.
Author: [**@squalus**](https://github.com/squalus)
@@ -1,9 +0,0 @@
---
synopsis: On linux, set $HOME=/proc/homeless-shelter instead of /homeless-shelter
issues: [8313, 11295]
prs: [11300]
---
When building, $HOME is set to a non-existing directory. Previously it was always set to `/homeless-shelter`. Before a build, Nix verifies that it doesn't exist. In some scenarios (specifically when using the Linux sandbox with a single-user installation), it is possible to create the `/homeless-shelter` directory, and some tools will create it, resulting in a build error.
Now, on Linux, $HOME is set to `/proc/homeless-shelter`. This directory can never be created, since `/proc` is a virtual filesystem. This resolves the issue.
@@ -0,0 +1,17 @@
---
synopsis: Removing the default argument passed to the `nix fmt` formatter
issues: []
prs: [11438]
---
The underlying formatter no longer receives the ". " default argument when `nix fmt` is called with no arguments.
This change was necessary as the formatter wasn't able to distinguish between
a user wanting to format the current folder with `nix fmt .` or the generic
`nix fmt`.
The default behaviour is now the responsibility of the formatter itself, and
allows tools such as treefmt to format the whole tree instead of only the
current directory and below.
Author: [**@zimbatm**](https://github.com/zimbatm)
@@ -0,0 +1,8 @@
---
synopsis: Flakes are no longer substituted
prs: [10612]
---
Nix will no longer attempt to substitute the source code of flakes from a binary cache. This functionality was broken because it could lead to different evaluation results depending on whether the flake was available in the binary cache, or even depending on whether the flake was already in the local store.
Author: [**@edolstra**](https://github.com/edolstra)
+14 -1
View File
@@ -138,6 +138,19 @@ The following environment variables are used to determine locations of various s
- [`XDG_STATE_HOME`]{#env-XDG_STATE_HOME} (default `~/.local/state`)
- [`XDG_CACHE_HOME`]{#env-XDG_CACHE_HOME} (default `~/.cache`)
[XDG Base Directory Specification]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
In addition, setting the following environment variables overrides the XDG base directories:
- [`NIX_CONFIG_HOME`]{#env-NIX_CONFIG_HOME} (default `$XDG_CONFIG_HOME/nix`)
- [`NIX_STATE_HOME`]{#env-NIX_STATE_HOME} (default `$XDG_STATE_HOME/nix`)
- [`NIX_CACHE_HOME`]{#env-NIX_CACHE_HOME} (default `$XDG_CACHE_HOME/nix`)
When [`use-xdg-base-directories`] is enabled, the configuration directory is:
1. `$NIX_CONFIG_HOME`, if it is defined
2. Otherwise, `$XDG_CONFIG_HOME/nix`, if `XDG_CONFIG_HOME` is defined
3. Otherwise, `~/.config/nix`.
Likewise for the state and cache directories.
@@ -1,6 +1,6 @@
## Default Nix expression
The source for the default [Nix expressions](@docroot@/language/index.md) used by [`nix-env`]:
The source for the [Nix expressions](@docroot@/glossary.md#gloss-nix-expression) used by [`nix-env`] by default:
- `~/.nix-defexpr`
- `$XDG_STATE_HOME/nix/defexpr` if [`use-xdg-base-directories`] is set to `true`.
@@ -18,24 +18,25 @@ Then, the resulting expression is interpreted like this:
- If the expression is an attribute set, it is used as the default Nix expression.
- If the expression is a function, an empty set is passed as argument and the return value is used as the default Nix expression.
For example, if the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
```nix
{
foo = import ~/.nix-defexpr/foo.nix;
bar = import ~/.nix-defexpr/bar.nix;
}
```
> **Example**
>
> If the default expression contains two files, `foo.nix` and `bar.nix`, then the default Nix expression will be equivalent to
>
> ```nix
> {
> foo = import ~/.nix-defexpr/foo.nix;
> bar = import ~/.nix-defexpr/bar.nix;
> }
> ```
The file [`manifest.nix`](@docroot@/command-ref/files/manifest.nix.md) is always ignored.
The command [`nix-channel`] places a symlink to the user's current [channels profile](@docroot@/command-ref/files/channels.md) in this directory.
The command [`nix-channel`] places a symlink to the current user's [channels] in this directory, the [user channel link](#user-channel-link).
This makes all subscribed channels available as attributes in the default expression.
## User channel link
A symlink that ensures that [`nix-env`] can find your channels:
A symlink that ensures that [`nix-env`] can find the current user's [channels]:
- `~/.nix-defexpr/channels`
- `$XDG_STATE_HOME/defexpr/channels` if [`use-xdg-base-directories`] is set to `true`.
@@ -45,8 +46,9 @@ This symlink points to:
- `$XDG_STATE_HOME/profiles/channels` for regular users
- `$NIX_STATE_DIR/profiles/per-user/root/channels` for `root`
In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.[`nix-env`]: ../nix-env.md
In a multi-user installation, you may also have `~/.nix-defexpr/channels_root`, which links to the channels of the root user.
[`nix-env`]: @docroot@/command-ref/nix-env.md
[`nix-channel`]: @docroot@/command-ref/nix-channel.md
[`nix-env`]: @docroot@/command-ref/nix-env.md
[`use-xdg-base-directories`]: @docroot@/command-ref/conf-file.md#conf-use-xdg-base-directories
[channels]: @docroot@/command-ref/files/channels.md
+2 -2
View File
@@ -13,13 +13,13 @@ Incremental refactorings of the documentation build setup to make it faster or e
Build the manual from scratch:
```console
nix-build $(nix-instantiate)'!doc'
nix-build -E '(import ./.).packages.${builtins.currentSystem}.nix.doc'
```
or
```console
nix build .#^doc
nix build .#nix^doc
```
and open `./result-doc/share/doc/nix/manual/index.html`.
+1 -1
View File
@@ -90,7 +90,7 @@ This representation is extensible and preserves the ordering:
## Self-describing values
As described in the previous section, it's crucial that schemas can be extended with with new fields without breaking compatibility.
As described in the previous section, it's crucial that schemas can be extended with new fields without breaking compatibility.
However, that should *not* mean we use the presence/absence of fields to indicate optional information *within* a version of the schema.
Instead, always include the field, and use `null` to indicate the "nothing" case.
+12 -7
View File
@@ -119,7 +119,7 @@
A store object consists of a [file system object], [references][reference] to other store objects, and other metadata.
It can be referred to by a [store path].
See [Store Object](@docroot@/store/index.md#store-object) for details.
See [Store Object](@docroot@/store/store-object.md) for details.
[store object]: #gloss-store-object
@@ -182,13 +182,18 @@
- [Nix expression]{#gloss-nix-expression}
1. Commonly, a high-level description of software packages and compositions
thereof. Deploying software using Nix entails writing Nix
expressions for your packages. Nix expressions specify [derivations][derivation],
which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
A syntactically valid use of the [Nix language].
2. A syntactically valid use of the [Nix language]. For example, the contents of a `.nix` file form an expression.
> **Example**
>
> The contents of a `.nix` file form a Nix expression.
Nix expressions specify [derivations][derivation], which are [instantiated][instantiate] into the Nix store as [store derivations][store derivation].
These derivations can then be [realised][realise] to produce [outputs][output].
> **Example**
>
> Building and deploying software using Nix entails writing Nix expressions as a high-level description of packages and compositions thereof.
- [reference]{#gloss-reference}
+8
View File
@@ -14,6 +14,14 @@ This option requires either:
* Linux running systemd, with SELinux disabled
* MacOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
```console
$ bash <(curl -L https://nixos.org/nix/install) --daemon
```
@@ -1,5 +1,13 @@
# Installing a Binary Distribution
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
To install the latest version Nix, run the following command:
```console
+12 -2
View File
@@ -19,7 +19,7 @@ If you are on Linux with systemd:
Remove files created by Nix:
```console
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile
sudo rm -rf /etc/nix /etc/profile.d/nix.sh /etc/tmpfiles.d/nix-daemon.conf /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix
```
Remove build users and their group:
@@ -43,6 +43,14 @@ which you may remove.
### macOS
> **Updating to macOS 15 Sequoia**
>
> If you recently updated to macOS 15 Sequoia and are getting
> ```console
> error: the user '_nixbld1' in the group 'nixbld' does not exist
> ```
> when running Nix commands, refer to GitHub issue [NixOS/nix#10892](https://github.com/NixOS/nix/issues/10892) for instructions to fix your installation without reinstalling.
1. If system-wide shell initialisation files haven't been altered since installing Nix, use the backups made by the installer:
```console
@@ -133,7 +141,9 @@ which you may remove.
diskutil list
```
If you _do_ find a "Nix Store" volume, delete it by running `diskutil deleteVolume` with the store volume's `diskXsY` identifier.
If you _do_ find a "Nix Store" volume, delete it by running `diskutil apfs deleteVolume` with the store volume's `diskXsY` identifier.
If you get an error that the volume is in use by the kernel, reboot and immediately delete the volume before starting any other process.
> **Note**
>
+1 -2
View File
@@ -264,8 +264,7 @@ The [`builder`](#attr-builder) is executed as follows:
- `PATH` is set to `/path-not-set` to prevent shells from
initialising it to their built-in default value.
- `HOME` is set to `/proc/homeless-shelter` on Linux and `/homeless-shelter`
on OSX, to prevent programs from
- `HOME` is set to `/homeless-shelter` to prevent programs from
using `/etc/passwd` or the like to find the user's home
directory, which could cause impurity. Usually, when `HOME` is
set, it is used as the location of the home directory, even if
+1 -1
View File
@@ -82,7 +82,7 @@ where
- if `type` = `"source:" ...`:
the the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
the hash of the [Nix Archive (NAR)] serialization of the [file system object](@docroot@/store/file-system-object.md) of the store object.
- if `type` = `"output:" id`:
+1 -1
View File
@@ -85,7 +85,7 @@
- Store object info JSON format now uses `null` rather than omitting fields [#9995](https://github.com/NixOS/nix/pull/9995)
The [store object info JSON format](@docroot@/protocols/json/store-object-info.md), used for e.g. `nix path-info`, no longer omits fields to indicate absent information, but instead includes the fields with a `null` value.
For example, `"ca": null` is used to to indicate a store object that isn't content-addressed rather than omitting the `ca` field entirely.
For example, `"ca": null` is used to indicate a store object that isn't content-addressed rather than omitting the `ca` field entirely.
This makes records of this sort more self-describing, and easier to consume programmatically.
We will follow this design principle going forward;
Generated
+4 -4
View File
@@ -80,16 +80,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1723688146,
"narHash": "sha256-sqLwJcHYeWLOeP/XoLwAtYjr01TISlkOfz+NG82pbdg=",
"lastModified": 1726674819,
"narHash": "sha256-qyfnYJsZq8cBBat6oGjHnu5MaWbNw47Ph31fX0iWPIU=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "c3d4ac725177c030b1e289015989da2ad9d56af0",
"rev": "3a458f7c763ca62c6bf454b8d828bd86b7250671",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-24.05",
"ref": "release-24.05",
"repo": "nixpkgs",
"type": "github"
}
+14 -4
View File
@@ -1,7 +1,7 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-24.05";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/release-24.05";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
inputs.flake-compat = { url = "github:edolstra/flake-compat"; flake = false; };
@@ -139,6 +139,12 @@
nix = final.nixComponents.nix;
nix-serve =
prev.nix-serve.override {
# undo potential version pinning
nix = final.nix;
};
# See https://github.com/NixOS/nixpkgs/pull/214409
# Remove when fixed in this flake's nixpkgs
pre-commit =
@@ -189,13 +195,15 @@
# system, we should reenable this.
#perlBindings = self.hydraJobs.perlBindings.${system};
}
/*
# Add "passthru" tests
// flatMapAttrs ({
"" = nixpkgsFor.${system}.native;
} // lib.optionalAttrs (! nixpkgsFor.${system}.native.stdenv.hostPlatform.isDarwin) {
# TODO: enable static builds for darwin, blocked on:
# https://github.com/NixOS/nixpkgs/issues/320448
"static-" = nixpkgsFor.${system}.static;
# TODO: disabled to speed up GHA CI.
#"static-" = nixpkgsFor.${system}.static;
})
(nixpkgsPrefix: nixpkgs:
flatMapAttrs nixpkgs.nixComponents
@@ -209,6 +217,7 @@
"${nixpkgsPrefix}nix-functional-tests" = nixpkgs.nixComponents.nix-functional-tests;
}
)
*/
// devFlake.checks.${system} or {}
);
@@ -292,6 +301,7 @@
devShells = let
makeShell = pkgs: stdenv: (pkgs.nix.override { inherit stdenv; forDevShell = true; }).overrideAttrs (attrs:
let
buildCanExecuteHost = stdenv.buildPlatform.canExecute stdenv.hostPlatform;
modular = devFlake.getSystem stdenv.buildPlatform.system;
transformFlag = prefix: flag:
assert builtins.isString flag;
@@ -349,7 +359,7 @@
++ pkgs.nixComponents.nix-external-api-docs.nativeBuildInputs
++ pkgs.nixComponents.nix-functional-tests.baseNativeBuildInputs
++ lib.optional
(!stdenv.buildPlatform.canExecute stdenv.hostPlatform
(!buildCanExecuteHost
# Hack around https://github.com/nixos/nixpkgs/commit/bf7ad8cfbfa102a90463433e2c5027573b462479
&& !(stdenv.hostPlatform.isWindows && stdenv.buildPlatform.isDarwin)
&& stdenv.hostPlatform.emulatorAvailable pkgs.buildPackages
@@ -366,7 +376,7 @@
# TODO: Remove the darwin check once
# https://github.com/NixOS/nixpkgs/pull/291814 is available
++ lib.optional (stdenv.cc.isClang && !stdenv.buildPlatform.isDarwin) pkgs.buildPackages.bear
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) pkgs.buildPackages.clang-tools;
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (lib.hiPrio pkgs.buildPackages.clang-tools);
buildInputs = attrs.buildInputs or []
++ [
+4 -1
View File
@@ -112,7 +112,7 @@ sub copyManual {
system("xz -d < '$manualNar' | nix-store --restore $tmpDir/manual.tmp") == 0
or die "unable to unpack $manualNar\n";
rename("$tmpDir/manual.tmp/share/doc/nix/manual", "$tmpDir/manual") or die;
system("rm -rf '$tmpDir/manual.tmp'") == 0 or die;
File::Path::remove_tree("$tmpDir/manual.tmp", {safe => 1});
}
system("aws s3 sync '$tmpDir/manual' s3://$releasesBucketName/$releaseDir/manual") == 0
@@ -281,3 +281,6 @@ system("git remote update origin") == 0 or die;
system("git tag --force --sign $version $nixRev -m 'Tagging release $version'") == 0 or die;
system("git push --tags") == 0 or die;
system("git push --force-with-lease origin $nixRev:refs/heads/latest-release") == 0 or die if $isLatest;
File::Path::remove_tree($narCache, {safe => 1});
File::Path::remove_tree($tmpDir, {safe => 1});
+3 -1
View File
@@ -3,7 +3,9 @@
project('nix-dev-shell', 'cpp',
version : files('.version'),
subproject_dir : 'src',
default_options : [
'localstatedir=/nix/var',
]
)
# Internal Libraries
+8
View File
@@ -0,0 +1,8 @@
configure_file(
input : 'completion.sh',
output : 'nix',
install : true,
install_dir : get_option('datadir') / 'bash-completion' / 'completions',
install_mode : 'rw-r--r--',
copy : true,
)
+8
View File
@@ -0,0 +1,8 @@
configure_file(
input : 'completion.fish',
output : 'nix.fish',
install : true,
install_dir : get_option('datadir') / 'fish' / 'vendor_completions.d',
install_mode : 'rw-r--r--',
copy : true,
)
@@ -2,11 +2,6 @@
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>EnvironmentVariables</key>
<dict>
<key>OBJC_DISABLE_INITIALIZE_FORK_SAFETY</key>
<string>YES</string>
</dict>
<key>Label</key>
<string>org.nixos.nix-daemon</string>
<key>KeepAlive</key>
+5
View File
@@ -0,0 +1,5 @@
subdir('bash')
subdir('fish')
subdir('zsh')
subdir('systemd')
+25
View File
@@ -0,0 +1,25 @@
foreach config : [ 'nix-daemon.socket', 'nix-daemon.service' ]
configure_file(
input : config + '.in',
output : config,
install : true,
install_dir : get_option('prefix') / 'lib/systemd/system',
install_mode : 'rw-r--r--',
configuration : {
'storedir' : store_dir,
'localstatedir' : localstatedir,
'bindir' : bindir,
},
)
endforeach
configure_file(
input : 'nix-daemon.conf.in',
output : 'nix-daemon.conf',
install : true,
install_dir : get_option('prefix') / 'lib/tmpfiles.d',
install_mode : 'rw-r--r--',
configuration : {
'localstatedir' : localstatedir,
},
)
+1
View File
@@ -11,6 +11,7 @@ ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
KillMode=process
LimitNOFILE=1048576
TasksMax=1048576
Delegate=yes
[Install]
WantedBy=multi-user.target
+10
View File
@@ -0,0 +1,10 @@
foreach script : [ [ 'completion.zsh', '_nix' ], [ 'run-help-nix' ] ]
configure_file(
input : script[0],
output : script.get(1, script[0]),
install : true,
install_dir : get_option('datadir') / 'zsh/site-functions',
install_mode : 'rw-r--r--',
copy : true,
)
endforeach
+6 -6
View File
@@ -27,6 +27,7 @@
, mdbook
, mdbook-linkcheck
, mercurial
, nix-perl-bindings
, openssh
, openssl
, pkg-config
@@ -60,7 +61,7 @@
# Run the functional tests as part of the build.
, doInstallCheck ? test-client != null || __forDefaults.canRunInstalled
# Check test coverage of Nix. Probably want to use with with at least
# Check test coverage of Nix. Probably want to use with at least
# one of `doCHeck` or `doInstallCheck` enabled.
, withCoverageChecks ? false
@@ -325,11 +326,6 @@ in {
preInstallCheck =
lib.optionalString (! doBuild) ''
mkdir -p src/nix-channel
''
# See https://github.com/NixOS/nix/issues/2523
# Occurs often in tests since https://github.com/NixOS/nix/pull/9900
+ lib.optionalString stdenv.hostPlatform.isDarwin ''
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
'';
separateDebugInfo = !stdenv.hostPlatform.isStatic;
@@ -339,6 +335,10 @@ in {
hardeningDisable = lib.optional stdenv.hostPlatform.isStatic "pie";
passthru = lib.optionalAttrs (stdenv.buildPlatform.canExecute stdenv.hostPlatform) {
perl-bindings = nix-perl-bindings;
};
meta = {
platforms = lib.platforms.unix ++ lib.platforms.windows;
mainProgram = "nix";
+23
View File
@@ -115,6 +115,29 @@ scope: {
version = inputs.libgit2.lastModifiedDate;
cmakeFlags = attrs.cmakeFlags or []
++ [ "-DUSE_SSH=exec" ];
nativeBuildInputs = attrs.nativeBuildInputs or []
# gitMinimal does not build on Windows. See packbuilder patch.
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# Needed for `git apply`; see `prePatch`
pkgs.buildPackages.gitMinimal
];
# Only `git apply` can handle git binary patches
prePatch = attrs.prePatch or ""
+ lib.optionalString (!stdenv.hostPlatform.isWindows) ''
patch() {
git apply
}
'';
patches = attrs.patches or []
++ [
./patches/libgit2-mempack-thin-packfile.patch
]
# gitMinimal does not build on Windows, but fortunately this patch only
# impacts interruptibility
++ lib.optionals (!stdenv.hostPlatform.isWindows) [
# binary patch; see `prePatch`
./patches/libgit2-packbuilder-callback-interruptible.patch
];
});
busybox-sandbox-shell = pkgs.busybox-sandbox-shell or (pkgs.busybox.override {
+37 -1
View File
@@ -76,7 +76,7 @@
] ++ lib.optionals (stdenv.buildPlatform.canExecute stdenv.hostPlatform) [
nix-perl-bindings
];
}).overrideAttrs (_: {
}).overrideAttrs (finalAttrs: prevAttrs: {
doCheck = true;
doInstallCheck = true;
@@ -90,4 +90,40 @@
installCheckInputs = [
nix-functional-tests
];
passthru = prevAttrs.passthru // {
/**
These are the libraries that are part of the Nix project. They are used
by the Nix CLI and other tools.
If you need to use these libraries in your project, we recommend to use
the `-c` C API libraries exclusively, if possible.
We also recommend that you build the complete package to ensure that the unit tests pass.
You could do this in CI, or by passing it in an unused environment variable. e.g in a `mkDerivation` call:
```nix
buildInputs = [ nix.libs.nix-util-c nix.libs.nix-store-c ];
# Make sure the nix libs we use are ok
unusedInputsForTests = [ nix ];
disallowedReferences = nix.all;
```
*/
libs = {
inherit
nix-util
nix-util-c
nix-store
nix-store-c
nix-fetchers
nix-expr
nix-expr-c
nix-flake
nix-main
nix-main-c
;
};
}
// lib.optionalAttrs (stdenv.buildPlatform.canExecute stdenv.hostPlatform) {
perl-bindings = nix-perl-bindings;
};
})
@@ -0,0 +1,282 @@
commit 9bacade4a3ef4b6b26e2c02f549eef0e9eb9eaa2
Author: Robert Hensing <robert@roberthensing.nl>
Date: Sun Aug 18 20:20:36 2024 +0200
Add unoptimized git_mempack_write_thin_pack
diff --git a/include/git2/sys/mempack.h b/include/git2/sys/mempack.h
index 17da590a3..3688bdd50 100644
--- a/include/git2/sys/mempack.h
+++ b/include/git2/sys/mempack.h
@@ -44,6 +44,29 @@ GIT_BEGIN_DECL
*/
GIT_EXTERN(int) git_mempack_new(git_odb_backend **out);
+/**
+ * Write a thin packfile with the objects in the memory store.
+ *
+ * A thin packfile is a packfile that does not contain its transitive closure of
+ * references. This is useful for efficiently distributing additions to a
+ * repository over the network, but also finds use in the efficient bulk
+ * addition of objects to a repository, locally.
+ *
+ * This operation performs the (shallow) insert operations into the
+ * `git_packbuilder`, but does not write the packfile to disk;
+ * see `git_packbuilder_write_buf`.
+ *
+ * It also does not reset the memory store; see `git_mempack_reset`.
+ *
+ * @note This function may or may not write trees and blobs that are not
+ * referenced by commits. Currently everything is written, but this
+ * behavior may change in the future as the packer is optimized.
+ *
+ * @param backend The mempack backend
+ * @param pb The packbuilder to use to write the packfile
+ */
+GIT_EXTERN(int) git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb);
+
/**
* Dump all the queued in-memory writes to a packfile.
*
diff --git a/src/libgit2/odb_mempack.c b/src/libgit2/odb_mempack.c
index 6f27f45f8..0b61e2b66 100644
--- a/src/libgit2/odb_mempack.c
+++ b/src/libgit2/odb_mempack.c
@@ -132,6 +132,35 @@ cleanup:
return err;
}
+int git_mempack_write_thin_pack(git_odb_backend *backend, git_packbuilder *pb)
+{
+ struct memory_packer_db *db = (struct memory_packer_db *)backend;
+ const git_oid *oid;
+ size_t iter = 0;
+ int err = -1;
+
+ /* TODO: Implement the recency heuristics.
+ For this it probably makes sense to only write what's referenced
+ through commits, an option I've carved out for you in the docs.
+ wrt heuristics: ask your favorite LLM to translate https://git-scm.com/docs/pack-heuristics/en
+ to actual normal reference documentation. */
+ while (true) {
+ err = git_oidmap_iterate(NULL, db->objects, &iter, &oid);
+ if (err == GIT_ITEROVER) {
+ err = 0;
+ break;
+ }
+ if (err != 0)
+ return err;
+
+ err = git_packbuilder_insert(pb, oid, NULL);
+ if (err != 0)
+ return err;
+ }
+
+ return 0;
+}
+
int git_mempack_dump(
git_buf *pack,
git_repository *repo,
diff --git a/tests/libgit2/mempack/thinpack.c b/tests/libgit2/mempack/thinpack.c
new file mode 100644
index 000000000..604a4dda2
--- /dev/null
+++ b/tests/libgit2/mempack/thinpack.c
@@ -0,0 +1,196 @@
+#include "clar_libgit2.h"
+#include "git2/indexer.h"
+#include "git2/odb_backend.h"
+#include "git2/tree.h"
+#include "git2/types.h"
+#include "git2/sys/mempack.h"
+#include "git2/sys/odb_backend.h"
+#include "util.h"
+
+static git_repository *_repo;
+static git_odb_backend * _mempack_backend;
+
+void test_mempack_thinpack__initialize(void)
+{
+ git_odb *odb;
+
+ _repo = cl_git_sandbox_init_new("mempack_thinpack_repo");
+
+ cl_git_pass(git_mempack_new(&_mempack_backend));
+ cl_git_pass(git_repository_odb(&odb, _repo));
+ cl_git_pass(git_odb_add_backend(odb, _mempack_backend, 999));
+ git_odb_free(odb);
+}
+
+void _mempack_thinpack__cleanup(void)
+{
+ cl_git_sandbox_cleanup();
+}
+
+/*
+ Generating a packfile for an unchanged repo works and produces an empty packfile.
+ Even if we allow this scenario to be detected, it shouldn't misbehave if the
+ application is unaware of it.
+*/
+void test_mempack_thinpack__empty(void)
+{
+ git_packbuilder *pb;
+ int version;
+ int n;
+ git_buf buf = GIT_BUF_INIT;
+
+ git_packbuilder_new(&pb, _repo);
+
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_assert_in_range(12, buf.size, 1024 /* empty packfile is >0 bytes, but certainly not that big */);
+ cl_assert(buf.ptr[0] == 'P');
+ cl_assert(buf.ptr[1] == 'A');
+ cl_assert(buf.ptr[2] == 'C');
+ cl_assert(buf.ptr[3] == 'K');
+ version = (buf.ptr[4] << 24) | (buf.ptr[5] << 16) | (buf.ptr[6] << 8) | buf.ptr[7];
+ /* Subject to change. https://git-scm.com/docs/pack-format: Git currently accepts version number 2 or 3 but generates version 2 only.*/
+ cl_assert_equal_i(2, version);
+ n = (buf.ptr[8] << 24) | (buf.ptr[9] << 16) | (buf.ptr[10] << 8) | buf.ptr[11];
+ cl_assert_equal_i(0, n);
+ git_buf_dispose(&buf);
+
+ git_packbuilder_free(pb);
+}
+
+#define LIT_LEN(x) x, sizeof(x) - 1
+
+/*
+ Check that git_mempack_write_thin_pack produces a thin packfile.
+*/
+void test_mempack_thinpack__thin(void)
+{
+ /* Outline:
+ - Create tree 1
+ - Flush to packfile A
+ - Create tree 2
+ - Flush to packfile B
+
+ Tree 2 has a new blob and a reference to a blob from tree 1.
+
+ Expectation:
+ - Packfile B is thin and does not contain the objects from packfile A
+ */
+
+
+ git_oid oid_blob_1;
+ git_oid oid_blob_2;
+ git_oid oid_blob_3;
+ git_oid oid_tree_1;
+ git_oid oid_tree_2;
+ git_treebuilder *tb;
+
+ git_packbuilder *pb;
+ git_buf buf = GIT_BUF_INIT;
+ git_indexer *indexer;
+ git_indexer_progress stats;
+ char pack_dir_path[1024];
+
+ char sbuf[1024];
+ const char * repo_path;
+ const char * pack_name_1;
+ const char * pack_name_2;
+ git_str pack_path_1 = GIT_STR_INIT;
+ git_str pack_path_2 = GIT_STR_INIT;
+ git_odb_backend * pack_odb_backend_1;
+ git_odb_backend * pack_odb_backend_2;
+
+
+ cl_assert_in_range(0, snprintf(pack_dir_path, sizeof(pack_dir_path), "%s/objects/pack", git_repository_path(_repo)), sizeof(pack_dir_path));
+
+ /* Create tree 1 */
+
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_2, _repo, LIT_LEN("thinpack blob 2")));
+
+
+ cl_git_pass(git_treebuilder_new(&tb, _repo, NULL));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob2", &oid_blob_2, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_1, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_1 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_1);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+
+ /* Create tree 2 */
+
+ cl_git_pass(git_treebuilder_clear(tb));
+ /* blob 1 won't be used, but we add it anyway to test that just "declaring" an object doesn't
+ necessarily cause its inclusion in the next thin packfile. It must only be included if new. */
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_1, _repo, LIT_LEN("thinpack blob 1")));
+ cl_git_pass(git_blob_create_from_buffer(&oid_blob_3, _repo, LIT_LEN("thinpack blob 3")));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob1", &oid_blob_1, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_insert(NULL, tb, "blob3", &oid_blob_3, GIT_FILEMODE_BLOB));
+ cl_git_pass(git_treebuilder_write(&oid_tree_2, tb));
+
+ /* Flush */
+
+ cl_git_pass(git_packbuilder_new(&pb, _repo));
+ cl_git_pass(git_mempack_write_thin_pack(_mempack_backend, pb));
+ cl_git_pass(git_packbuilder_write_buf(&buf, pb));
+ cl_git_pass(git_indexer_new(&indexer, pack_dir_path, 0, NULL, NULL));
+ cl_git_pass(git_indexer_append(indexer, buf.ptr, buf.size, &stats));
+ cl_git_pass(git_indexer_commit(indexer, &stats));
+ pack_name_2 = strdup(git_indexer_name(indexer));
+ cl_assert(pack_name_2);
+ git_buf_dispose(&buf);
+ git_mempack_reset(_mempack_backend);
+ git_indexer_free(indexer);
+ git_packbuilder_free(pb);
+ git_treebuilder_free(tb);
+
+ /* Assertions */
+
+ assert(pack_name_1);
+ assert(pack_name_2);
+
+ repo_path = git_repository_path(_repo);
+
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_1);
+ git_str_joinpath(&pack_path_1, repo_path, sbuf);
+ snprintf(sbuf, sizeof(sbuf), "objects/pack/pack-%s.pack", pack_name_2);
+ git_str_joinpath(&pack_path_2, repo_path, sbuf);
+
+ /* If they're the same, something definitely went wrong. */
+ cl_assert(strcmp(pack_name_1, pack_name_2) != 0);
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_1, pack_path_1.ptr));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_1));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_2));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_blob_3));
+ cl_assert(pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_1));
+ cl_assert(!pack_odb_backend_1->exists(pack_odb_backend_1, &oid_tree_2));
+
+ cl_git_pass(git_odb_backend_one_pack(&pack_odb_backend_2, pack_path_2.ptr));
+ /* blob 1 is already in the packfile 1, so packfile 2 must not include it, in order to be _thin_. */
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_1));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_2));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_blob_3));
+ cl_assert(!pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_1));
+ cl_assert(pack_odb_backend_2->exists(pack_odb_backend_2, &oid_tree_2));
+
+ pack_odb_backend_1->free(pack_odb_backend_1);
+ pack_odb_backend_2->free(pack_odb_backend_2);
+ free((void *)pack_name_1);
+ free((void *)pack_name_2);
+ git_str_dispose(&pack_path_1);
+ git_str_dispose(&pack_path_2);
+
+}
@@ -0,0 +1,930 @@
commit e9823c5da4fa977c46bcb97167fbdd0d70adb5ff
Author: Robert Hensing <robert@roberthensing.nl>
Date: Mon Aug 26 20:07:04 2024 +0200
Make packbuilder interruptible using progress callback
Forward errors from packbuilder->progress_cb
This allows the callback to terminate long-running operations when
the application is interrupted.
diff --git a/include/git2/pack.h b/include/git2/pack.h
index 0f6bd2ab9..bee72a6c0 100644
--- a/include/git2/pack.h
+++ b/include/git2/pack.h
@@ -247,6 +247,9 @@ typedef int GIT_CALLBACK(git_packbuilder_progress)(
* @param progress_cb Function to call with progress information during
* pack building. Be aware that this is called inline with pack building
* operations, so performance may be affected.
+ * When progress_cb returns an error, the pack building process will be
+ * aborted and the error will be returned from the invoked function.
+ * `pb` must then be freed.
* @param progress_cb_payload Payload for progress callback.
* @return 0 or an error code
*/
diff --git a/src/libgit2/pack-objects.c b/src/libgit2/pack-objects.c
index b2d80cba9..7c331c2d5 100644
--- a/src/libgit2/pack-objects.c
+++ b/src/libgit2/pack-objects.c
@@ -932,6 +932,9 @@ static int report_delta_progress(
{
int ret;
+ if (pb->failure)
+ return pb->failure;
+
if (pb->progress_cb) {
uint64_t current_time = git_time_monotonic();
uint64_t elapsed = current_time - pb->last_progress_report_time;
@@ -943,8 +946,10 @@ static int report_delta_progress(
GIT_PACKBUILDER_DELTAFICATION,
count, pb->nr_objects, pb->progress_cb_payload);
- if (ret)
+ if (ret) {
+ pb->failure = ret;
return git_error_set_after_callback(ret);
+ }
}
}
@@ -976,7 +981,10 @@ static int find_deltas(git_packbuilder *pb, git_pobject **list,
}
pb->nr_deltified += 1;
- report_delta_progress(pb, pb->nr_deltified, false);
+ if ((error = report_delta_progress(pb, pb->nr_deltified, false)) < 0) {
+ GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
+ goto on_error;
+ }
po = *list++;
(*list_size)--;
@@ -1124,6 +1132,10 @@ struct thread_params {
size_t depth;
size_t working;
size_t data_ready;
+
+ /* A pb->progress_cb can stop the packing process by returning an error.
+ When that happens, all threads observe the error and stop voluntarily. */
+ bool stopped;
};
static void *threaded_find_deltas(void *arg)
@@ -1133,7 +1145,12 @@ static void *threaded_find_deltas(void *arg)
while (me->remaining) {
if (find_deltas(me->pb, me->list, &me->remaining,
me->window, me->depth) < 0) {
- ; /* TODO */
+ me->stopped = true;
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
+ me->working = false;
+ git_cond_signal(&me->pb->progress_cond);
+ GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_unlock(me->pb) == 0, NULL);
+ return NULL;
}
GIT_ASSERT_WITH_RETVAL(git_packbuilder__progress_lock(me->pb) == 0, NULL);
@@ -1175,8 +1192,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
pb->nr_threads = git__online_cpus();
if (pb->nr_threads <= 1) {
- find_deltas(pb, list, &list_size, window, depth);
- return 0;
+ return find_deltas(pb, list, &list_size, window, depth);
}
p = git__mallocarray(pb->nr_threads, sizeof(*p));
@@ -1195,6 +1211,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
p[i].depth = depth;
p[i].working = 1;
p[i].data_ready = 0;
+ p[i].stopped = 0;
/* try to split chunks on "path" boundaries */
while (sub_size && sub_size < list_size &&
@@ -1262,7 +1279,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
(!victim || victim->remaining < p[i].remaining))
victim = &p[i];
- if (victim) {
+ if (victim && !target->stopped) {
sub_size = victim->remaining / 2;
list = victim->list + victim->list_size - sub_size;
while (sub_size && list[0]->hash &&
@@ -1286,7 +1303,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
target->list_size = sub_size;
target->remaining = sub_size;
- target->working = 1;
+ target->working = 1; /* even when target->stopped, so that we don't process this thread again */
GIT_ASSERT(git_packbuilder__progress_unlock(pb) == 0);
if (git_mutex_lock(&target->mutex)) {
@@ -1299,7 +1316,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
git_cond_signal(&target->cond);
git_mutex_unlock(&target->mutex);
- if (!sub_size) {
+ if (target->stopped || !sub_size) {
git_thread_join(&target->thread, NULL);
git_cond_free(&target->cond);
git_mutex_free(&target->mutex);
@@ -1308,7 +1325,7 @@ static int ll_find_deltas(git_packbuilder *pb, git_pobject **list,
}
git__free(p);
- return 0;
+ return pb->failure;
}
#else
@@ -1319,6 +1336,7 @@ int git_packbuilder__prepare(git_packbuilder *pb)
{
git_pobject **delta_list;
size_t i, n = 0;
+ int error;
if (pb->nr_objects == 0 || pb->done)
return 0; /* nothing to do */
@@ -1327,8 +1345,10 @@ int git_packbuilder__prepare(git_packbuilder *pb)
* Although we do not report progress during deltafication, we
* at least report that we are in the deltafication stage
*/
- if (pb->progress_cb)
- pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload);
+ if (pb->progress_cb) {
+ if ((error = pb->progress_cb(GIT_PACKBUILDER_DELTAFICATION, 0, pb->nr_objects, pb->progress_cb_payload)) < 0)
+ return git_error_set_after_callback(error);
+ }
delta_list = git__mallocarray(pb->nr_objects, sizeof(*delta_list));
GIT_ERROR_CHECK_ALLOC(delta_list);
@@ -1345,31 +1365,33 @@ int git_packbuilder__prepare(git_packbuilder *pb)
if (n > 1) {
git__tsort((void **)delta_list, n, type_size_sort);
- if (ll_find_deltas(pb, delta_list, n,
+ if ((error = ll_find_deltas(pb, delta_list, n,
GIT_PACK_WINDOW + 1,
- GIT_PACK_DEPTH) < 0) {
+ GIT_PACK_DEPTH)) < 0) {
git__free(delta_list);
- return -1;
+ return error;
}
}
- report_delta_progress(pb, pb->nr_objects, true);
+ error = report_delta_progress(pb, pb->nr_objects, true);
pb->done = true;
git__free(delta_list);
- return 0;
+ return error;
}
-#define PREPARE_PACK if (git_packbuilder__prepare(pb) < 0) { return -1; }
+#define PREPARE_PACK error = git_packbuilder__prepare(pb); if (error < 0) { return error; }
int git_packbuilder_foreach(git_packbuilder *pb, int (*cb)(void *buf, size_t size, void *payload), void *payload)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, cb, payload);
}
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb)
{
+ int error;
PREPARE_PACK;
return write_pack(pb, &write_pack_buf, buf);
diff --git a/src/libgit2/pack-objects.h b/src/libgit2/pack-objects.h
index bbc8b9430..380a28ebe 100644
--- a/src/libgit2/pack-objects.h
+++ b/src/libgit2/pack-objects.h
@@ -100,6 +100,10 @@ struct git_packbuilder {
uint64_t last_progress_report_time;
bool done;
+
+ /* A non-zero error code in failure causes all threads to shut themselves
+ down. Some functions will return this error code. */
+ volatile int failure;
};
int git_packbuilder__write_buf(git_str *buf, git_packbuilder *pb);
diff --git a/tests/libgit2/pack/cancel.c b/tests/libgit2/pack/cancel.c
new file mode 100644
index 000000000..a0aa9716a
--- /dev/null
+++ b/tests/libgit2/pack/cancel.c
@@ -0,0 +1,240 @@
+#include "clar_libgit2.h"
+#include "futils.h"
+#include "pack.h"
+#include "hash.h"
+#include "iterator.h"
+#include "vector.h"
+#include "posix.h"
+#include "hash.h"
+#include "pack-objects.h"
+
+static git_repository *_repo;
+static git_revwalk *_revwalker;
+static git_packbuilder *_packbuilder;
+static git_indexer *_indexer;
+static git_vector _commits;
+static int _commits_is_initialized;
+static git_indexer_progress _stats;
+
+extern bool git_disable_pack_keep_file_checks;
+
+static void pack_packbuilder_init(const char *sandbox) {
+ _repo = cl_git_sandbox_init(sandbox);
+ /* cl_git_pass(p_chdir(sandbox)); */
+ cl_git_pass(git_revwalk_new(&_revwalker, _repo));
+ cl_git_pass(git_packbuilder_new(&_packbuilder, _repo));
+ cl_git_pass(git_vector_init(&_commits, 0, NULL));
+ _commits_is_initialized = 1;
+ memset(&_stats, 0, sizeof(_stats));
+ p_fsync__cnt = 0;
+}
+
+void test_pack_cancel__initialize(void)
+{
+ pack_packbuilder_init("small.git");
+}
+
+void test_pack_cancel__cleanup(void)
+{
+ git_oid *o;
+ unsigned int i;
+
+ cl_git_pass(git_libgit2_opts(GIT_OPT_ENABLE_FSYNC_GITDIR, 0));
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, false));
+
+ if (_commits_is_initialized) {
+ _commits_is_initialized = 0;
+ git_vector_foreach(&_commits, i, o) {
+ git__free(o);
+ }
+ git_vector_free(&_commits);
+ }
+
+ git_packbuilder_free(_packbuilder);
+ _packbuilder = NULL;
+
+ git_revwalk_free(_revwalker);
+ _revwalker = NULL;
+
+ git_indexer_free(_indexer);
+ _indexer = NULL;
+
+ /* cl_git_pass(p_chdir("..")); */
+ cl_git_sandbox_cleanup();
+ _repo = NULL;
+}
+
+static int seed_packbuilder(void)
+{
+ int error;
+ git_oid oid, *o;
+ unsigned int i;
+
+ git_revwalk_sorting(_revwalker, GIT_SORT_TIME);
+ cl_git_pass(git_revwalk_push_ref(_revwalker, "HEAD"));
+
+ while (git_revwalk_next(&oid, _revwalker) == 0) {
+ o = git__malloc(sizeof(git_oid));
+ cl_assert(o != NULL);
+ git_oid_cpy(o, &oid);
+ cl_git_pass(git_vector_insert(&_commits, o));
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ if((error = git_packbuilder_insert(_packbuilder, o, NULL)) < 0)
+ return error;
+ }
+
+ git_vector_foreach(&_commits, i, o) {
+ git_object *obj;
+ cl_git_pass(git_object_lookup(&obj, _repo, o, GIT_OBJECT_COMMIT));
+ error = git_packbuilder_insert_tree(_packbuilder,
+ git_commit_tree_id((git_commit *)obj));
+ git_object_free(obj);
+ if (error < 0)
+ return error;
+ }
+
+ return 0;
+}
+
+static int fail_stage;
+
+static int packbuilder_cancel_after_n_calls_cb(int stage, uint32_t current, uint32_t total, void *payload)
+{
+
+ /* Force the callback to run again on the next opportunity regardless
+ of how fast we're running. */
+ _packbuilder->last_progress_report_time = 0;
+
+ if (stage == fail_stage) {
+ int *calls = (int *)payload;
+ int n = *calls;
+ /* Always decrement, including past zero. This way the error is only
+ triggered once, making sure it is picked up immediately. */
+ --*calls;
+ if (n == 0)
+ return GIT_EUSER;
+ }
+
+ return 0;
+}
+
+static void test_cancel(int n)
+{
+
+ int calls_remaining = n;
+ int err;
+ git_buf buf = GIT_BUF_INIT;
+
+ /* Switch to a small repository, so that `packbuilder_cancel_after_n_calls_cb`
+ can hack the time to call the callback on every opportunity. */
+
+ cl_git_pass(git_packbuilder_set_callbacks(_packbuilder, &packbuilder_cancel_after_n_calls_cb, &calls_remaining));
+ err = seed_packbuilder();
+ if (!err)
+ err = git_packbuilder_write_buf(&buf, _packbuilder);
+
+ cl_assert_equal_i(GIT_EUSER, err);
+}
+void test_pack_cancel__cancel_after_add_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_add_1(void)
+{
+ cl_skip();
+ fail_stage = GIT_PACKBUILDER_ADDING_OBJECTS;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+}
+
+void test_pack_cancel__cancel_after_delta_1(void)
+{
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+}
+
+void test_pack_cancel__cancel_after_delta_0_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(0);
+#else
+ cl_skip();
+#endif
+}
+
+void test_pack_cancel__cancel_after_delta_1_threaded(void)
+{
+#ifdef GIT_THREADS
+ git_packbuilder_set_threads(_packbuilder, 8);
+ fail_stage = GIT_PACKBUILDER_DELTAFICATION;
+ test_cancel(1);
+#else
+ cl_skip();
+#endif
+}
+
+static int foreach_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *) payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return 0;
+}
+
+void test_pack_cancel__foreach(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_pass(git_packbuilder_foreach(_packbuilder, foreach_cb, idx));
+ cl_git_pass(git_indexer_commit(idx, &_stats));
+ git_indexer_free(idx);
+}
+
+static int foreach_cancel_cb(void *buf, size_t len, void *payload)
+{
+ git_indexer *idx = (git_indexer *)payload;
+ cl_git_pass(git_indexer_append(idx, buf, len, &_stats));
+ return (_stats.total_objects > 2) ? -1111 : 0;
+}
+
+void test_pack_cancel__foreach_with_cancel(void)
+{
+ git_indexer *idx;
+
+ seed_packbuilder();
+
+#ifdef GIT_EXPERIMENTAL_SHA256
+ cl_git_pass(git_indexer_new(&idx, ".", GIT_OID_SHA1, NULL));
+#else
+ cl_git_pass(git_indexer_new(&idx, ".", 0, NULL, NULL));
+#endif
+
+ cl_git_fail_with(
+ git_packbuilder_foreach(_packbuilder, foreach_cancel_cb, idx), -1111);
+ git_indexer_free(idx);
+}
+
+void test_pack_cancel__keep_file_check(void)
+{
+ assert(!git_disable_pack_keep_file_checks);
+ cl_git_pass(git_libgit2_opts(GIT_OPT_DISABLE_PACK_KEEP_FILE_CHECKS, true));
+ assert(git_disable_pack_keep_file_checks);
+}
diff --git a/tests/resources/small.git/HEAD b/tests/resources/small.git/HEAD
new file mode 100644
index 0000000000000000000000000000000000000000..cb089cd89a7d7686d284d8761201649346b5aa1c
GIT binary patch
literal 23
ecmXR)O|w!cN=+-)&qz&7Db~+TEG|hc;sO9;xClW2
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/config b/tests/resources/small.git/config
new file mode 100644
index 0000000000000000000000000000000000000000..07d359d07cf1ed0c0074fdad71ffff5942f0adfa
GIT binary patch
literal 66
zcmaz}&M!)h<>D+#Eyyp<EXgmbOv^9IO)M!(Eh^5;&r`5fFyP`$%gjm5%}+@M@=A(I
MQ@J>k5{uv*03B5png9R*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/description b/tests/resources/small.git/description
new file mode 100644
index 0000000000000000000000000000000000000000..498b267a8c7812490d6479839c5577eaaec79d62
GIT binary patch
literal 73
zcmWH|%S+5nO;IRHEyyp<EXgmbv{pz>$t+PQ$;d2LNXyJgRZve!Elw`VEGWs$&r??@
Q$yWgB0LrH#Y0~2Y0PnOK(EtDd
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/applypatch-msg.sample b/tests/resources/small.git/hooks/applypatch-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..dcbf8167fa503f96ff6a39c68409007eadc9b1f3
GIT binary patch
literal 535
zcmY+AX;Q;542A#a6e8^~FyI8r&I~hf2QJ{GO6(?HuvEG*+#R{4EI%zhfA8r{j%sh$
zHE~E-UtQd8{bq4@*S%jq3@bmxwQDXGv#o!N`o3AHMw3xD)hy0#>&E&zzl%vRffo<B
z)-H|+CWHZ~O*S%cfYx9;02_ohIA<Bg(1SxF-6OCb&_lBkf{t<AM9r;%E(Hf#h{|a@
z9>mqo=v6>_2NRa#TwDdYvTVQyueO*15Nlo%=#DXgC0bhF3vTa`LQGaO9;jeD$OP?~
za$G4Q{z+Q_{5V?5h;a-noM$P{<>Q~j4o7u%#P6^o^16{y*jU=-K8GYD_dUtdj4FSx
zSC0C!DvAnv%S!4d<Yg@O<;m`;oSw)=Fz+hrL<mY{rBr8j4pi^88FX3}jKrYUP)>gk
XB^)11aoGMJPCqWs%IS0YSv(eBT&%T6
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/commit-msg.sample b/tests/resources/small.git/hooks/commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..f3780f92349638ebe32f6baf24c7c3027675d7c9
GIT binary patch
literal 953
zcmaJ<TW`}a6n<`g#Ya}rZCaYOzy?UGG&Tf#VG`?}7@eHtB=MTqneFUS%75oLS;atz
zm&iUo=ey->y@-{3h^^Cx;#d0zEA@DDc$nY4ez&|=%jTg@_HU*ub=!!y$xW09TSjlj
z(`I@QCsM`!9&80$I98wsQ8yK#)Orb<8re8FjkKh630D$QUDwi~(gkX=RunYm$rDjk
zlp%RUSnzA#6yjdG5?T?2DcYKp+v_lts0ljn&bh3J0bD5@N@1UKZ190O6ZeWr-BuZ^
zWRebCX%(%=Xoj#(xYk1Cjtr!=tyBesf@m6}8zY6Ijbz9i9ziI_jG9Mv<Cz(ymp*>R
zDH*e>^ga9IR?2wrSrAVm;eButj4<aWB@zzNl|1Wp@4;}1O?MUF>Y>7(E2?b~jsu>&
zRKCJ7bp#19sqYh627wD%D9R$8=Ml$TNlumDypl~$jBu*G>5fIR^FB0h0Ex&TGZNr>
zL5hs1_K>taRb!|ThN9ns7^@4MXKP+6aGI_UK)T-M#rcP$;kN(Vcf#P)+5GzWa{l@J
z>-E{`$1iiNVYxq27}<DnwLRXQUG0o_hw&da-s5T#H=`Y9D_8=eTZ?cpWatp#a1vs@
z2BjrO)z@aTuI#g#`)oJcnhM7oYLT@~CHX@CNXv4>j;uo%;)r3kJI2xCFF~Ux;$Q%)
wjbk6JlDCM`jU&P+UVOvg`|iYl<7~9k>HHB4I;pdlQ=I-^$DrHaN$@lH1?P!0U;qFB
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/fsmonitor-watchman.sample b/tests/resources/small.git/hooks/fsmonitor-watchman.sample
new file mode 100755
index 0000000000000000000000000000000000000000..41184ebc318c159f51cd1ebe2290559805df89d8
GIT binary patch
literal 4777
zcmbtYYi}F368$Xwipg4lq(BeHMvzvH-4;n7DGJBPqq#tw3aed8+IU5-m)yvL>;Cqh
z8FFRGj$`9CA8ao<GaSz2oMCnz4Rv-gw9b@j_$0GQT1?XYi|;S??Y`Z6`Z;}C6#27N
z8ShS>J?j^$%==FV``-=rhLcPW`McSytRm~mEO7_&_cAVZrf1fFy*ha@8oe%*-aBYE
zcjzZg>LOkgxuUr-XJnHyD;zmPnRaSc#!k_P*d_BttRdc+J6G7za5#+<HG#rlmbrN~
z8DwU-3}VABEwM=0VLP@^Dy6ERR5_J6cmg|GEh*M1EliqCGwe^ZT-iZ$2Yc`4!I#WZ
z5nGGhn7*jeW=2ydsmfAmm#=8AD<<;TI+#z{Q)kW;yE!%GB6f~7EtEMLdM47Qaz*8=
zIObA(VVj-VG{Ax|66d*hi`+bRG>^Y1nkc2Oowk`ya47uUR3Feu?B<phm31&PQB<lt
zb{W(W4wf#Bab%|Q_tKPS?3^o=5)Z8^Vh(#slNI}pO(f^|{U0GZhLnycSaNd&h?CaC
z0XklU6^<ky6rES9T=na$M8P<_)aKMAMo+UDewAu4wF{#&6diFshiudixAoh|&0<BJ
zR>(w;S{(VYzxh}q-=#zP@uxSx{wbyPUMFU;K(06)$o{07&3yI?q{GqMcQ1c_^M<0<
zF4acAV)Il-V(rCTC1(;bsZ*}bl8dmejAk~yb`B}!^0;g^(o9kGUfZfDOvyp@x4OQt
zSgWh6T|3eq;9MFs8-#z+FDM1h(IjRUP|``PxupgJ7CUHOH90gbgl^2~97`?_X{P))
zB*$r1cDlF-%azKND}?Gv`2K8-9v5e`gQoft=j?T<&a13c^!wY_$D`5z-X1g?ty&6-
zQN50{8?bUk9AI->^W@~~nkOghHIC2YN+<JiT_ob7ttND1oh`HH28Y+OV~HedG&uB`
zy}rA*r_xT#bR`Q7-*)3t*(!Hf?jKzyxk=8hdi3L^d<p<uU9q_<4k&xEr4@YWv_vsW
zp(#32bYtA5u|s#1+}h`0kwpz4kUD&+>AXkLQG_2-{Pq3%{`3KUMeG$iIn%%^6*NYb
zn|_BdV#C)n4565Vcc<EWC-nglECZGy!j9I4&;hUCzr(?6TftH=0^@!mI^E@y5HZw8
ztH&kaSNyg=O6riqR^MPOX6oD__Jz@K;*icS)?m$!p{S$n;*EwlO<L!d7;utu(W9D!
zaF!B~xV^2i?wH0s?Lw%Q)(`aPkajs1XojlPv@Y-d5#YFg#KG+!r7A(dJLnkiJMs`p
zV|_=d!upN{TsxH1?sZNdzxeHsmtzTV`1{pykJ_~+^*>X;uT8&z3vSi!HXGbUj2B!R
zdz~&#<?<tHJql=W&((YpzS06y-Z6Cn^H!*9qH?pOrD~(GV=JJ~z{tpLnGK|xp&C1`
zsbn7O86xjF<~G*S1l*;;Bh%6><Up=oKy99?62P^?M&22U6QFRBXLb&u%=Ur<74wRy
zMRG!f{AvZ>fk#L-&k$fLwo$4?>12g@AXOKFekuo#6EHB%gmpD?1eyh%N8s{2wGoTu
z*@6cEZ^ZW!FAF_|JL`NkV7k}0ow|-2jHwbgH0;c@Dq*o?@&c*HnGdyx6^su8Qk%2{
z*ye(dxO*6-&>qn1+zw}tc6;=sOX{4WB=VqjTS^))y1jlX2Q;=e!qMmFA5lC$#;BxC
z=Y%tRpWxb+_uQAvAw7Q{HGV#R$xb&udLCzZ+HN?kTyB};1EJ8UlQ5!>5eGW@)RX0n
zkjj>EF!3=0Gl^8dzv$B^NMGRxJoqN4A`xq-@wCbrx*u2NmIJ1<fUDQ=*^)h6`vzco
z3F+ro$F!F6pc<B;<;isobIgbVGKUBByoQ4#CO({S7g?<Dh0^!7uJ3gxS=6F;+^gQc
zeKi4`4`Fm3p|BU2v{M|-u!#QGCXiM=k=%np0<ZOPQqEjP_nneyOdgEuG9IH&YYPtp
zKB_dvcYCcyhyT#<uhUEL$o~!TUz;cb&|`uSM{Dkv%&W2lcpYL&kv)tUvVv?&>xZ%H
zh;{|4T3(!E9sY#Ni(wUJYs1MmIc9bl)(4Nl3_wD_BWB>i<1S(LX7m*{Q7PU$muMS*
zM!%0EZx-Vw=Zey;erC?SNxF;pY@^A%-krqzfLV2meBp1vWdyArFYn`DD19T)Hw(?n
z)}{NP(Lk(o*?gl#B@pP7^*r|=;PIDT4|F#{2Hzh-AL0Rv$6uT;<CP7qxbYms@WU7}
z%}TsHJ!V_56ZFF{BfI=8jTBxMEATG376pS6a;u1@c?{~sL<N52`U6fuLkz4P@Mb^b
z2`Q48y&!C0&A+WRCJAdmo3u2#?eI=si9Vm47$|`m**x<wKkM=QR&g?C63@P5X@xP8
zi5QG2b-Fdz%S0%1{kKvL%^RTdpK|SU^VJ7JCmKdwp1`;HRoGM7ef^k_s_}2An=cql
za|{IaxQWpdq<ae&x3BOJq+M5QNIk#C{Nv@u>n|WzE4=slK?on@(fZeGhRgQCu56qB
z{+n81Az96qnQjMY*-*r-KV*7;Z#4Q<xfjbcBx_6JAN-HP@bq+eI%HhAB9&vLyOap{
bw<Ywj(b#kdcCk7dCBY;|DBNpPjIa1F6*dXN
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/post-update.sample b/tests/resources/small.git/hooks/post-update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..9f95e41a39cf974958277c8881ac6cce405ebb20
GIT binary patch
literal 246
zcmXZVO?HDY3<Ti4Poaiwjq^yGw9DKf7mz^&Ly=V5q$H=W^Rt|p_r9s#9Ea7VERo!9
zyT9>uJRJJV$M^KdldiMhj?ImK6~FvwJ*L5a){QoM=L5TYHkGO1$UrO3`a>{?Opw|b
zG(#59NQ#jFL9v~vgOVkM@^^(^A}onOE))yWEwhIlk&{ZyseZ^O0b=w8&O=BK{k<5B
k^Q-B@eG}LeHrquz%(SVEp_N)VhYZikCW__82JXfD17`J9Qvd(}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-applypatch.sample b/tests/resources/small.git/hooks/pre-applypatch.sample
new file mode 100755
index 0000000000000000000000000000000000000000..625837e25f91421b8809a097f4a3103dd387ef31
GIT binary patch
literal 481
zcmY+ATTa6;5Jms9iouO45IBJXEg&Jm9@v1LPHMM_ZR|;#6tQ<EeSrA%_2`_rGr1_8
z?aM?yVtIc%-@9SGSk&8x=grP-Lf`7!^=$7xgL=|ysZ}!av6zL~ywui}<2##V6L@!k
zy=p^)V7%Wzs-g`9<Y9}^)&uN}BCrXR_T3@Z2$gSJON2`X=mAs+%@7n-2I}ZrP|TFA
zvJJGDl3HPLP<@!Q!}zXQvey#qEE#a#$vs97i4=A0stF@YQ)k_ZajaoS^dVYBc&37_
zVI(L=X<V335r9~7T<;|HfKF+yM}}LB9d96V)Si;sj(;9Rh$#P>h$71hSXq*MxP;V&
zj0cY7SCL=x4`a46sF)C>94Gk%=3q$W2s;j6iHtB2$R0%gix4oK@&T~=ALd_o*CKxt
I-`Pv{1Bpzc>;M1&
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-commit.sample b/tests/resources/small.git/hooks/pre-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..10b39b2e26981b8f87ea424e735ef87359066dbb
GIT binary patch
literal 1706
zcmZuxU2ohr5PY_N#pZ0-F<{-<Zkx1j&cMNSQ3FN`GzR*R1O+9oPV`BnOj7q@1pV!u
zrF0Hl^i33(yR)-1d-!H%&2|=|^E~_R{N1zNJ-&Zmt-t?iwXv&i+ZN}Km(TX8Q$H4u
zd7(m`|1iDmF5k@xV`p;C4zojASmLc}yN0QDZbhN=ri&CEt=XGuN1IwjGJ#a#`t-kG
zDqY)}7+Ft|;YKwLYbtg$S(-TBO=x3cP1cd}%f4kB!<6Wu-dCwz-)KDMEuM^_Hh*UC
zC`1)|)T<(U6b`+yOH!6p*Ll}@qastwA*dyjsgOf5C=?LTprfORG6O{5L%@S0wyHpj
zu|_A-=NWnYYR5m7kvm6|&T~GzoJ_OKR3sgFUgw?ifho^NQhvK#{6g0=&Fh)%n}#m0
zk1sNmwb_AMDq};OOGw5|;OyX#?yQMMH6yAk(x$3tjFjHE?c$E2XC_xXav8tnIeIG?
zYMI|~MLEVGkuT*>v&v-X^RA+u>k}E$4d&uD7=g_fA8+pNNV=4s0|iD3p<=DTXClTS
zXV23tJ;ECmN@M0j@zUAKEYW@3bv!SeYZ8ZH`YQNTApFVNc;F|9r5p4TqGs=>8E?6y
zi|gY{iM#PG1nL?UE9YCnWTk72kgZPG*Usqw!~Qd3c?~@w2?%eg@~)+VlSs6N5Yf2^
zz;ow<fjf3n`imj7u5lnzt||q9QFM(H@<3EJCK|l5P!$yDfn~U-(5Vu7s+GqxNKyeI
zP=-Oa(KH&gK`NhUa`cLj3B8%qL};DR7dk!`A^h&3-hFB6p($5Ufy^tGir)3et}qK4
zpkPKYWzC+?=&gw-L6S)S=<lfq)%uLUAa%~81Jf9hk)w~U!DItnoSy`m^}#38M}K-o
z!PqisQkV!&z4E*V01ro~qlVK^0BI`pLk6D+)f~*y!hCvwHq8zY9BGh<2s$@b^A<8G
zRaqk}&qZyyv&|0QDFPA%h4TgF&vdlc|JUq*=>F#K#r^&KMq1A`oqVGFpD&-!Pv|Rc
zO3KSqA@h9nSc%bm`0)Amk6*J}@14J*1-219l%%7D!Pl}UK>|lVi0Dfgu2jN3WC!uL
z0ej??b2iSehVgdnWHmZV4kUo*QL#aiIp}U=9x)IXk}JJ7VQ;CI9Rtn5e0VcjbY<bp
zv{}WlG6L;H!EzFKdY>cVt+`x5D+svCGD<sXw4|)E|JX43I1_3P(sI4{wj87bPSrNG
w!SIr>;Z5hm*<gY+X;)Ryx4=nzaab9m`bwE*^s(%u*E3HbUuOR@+&s_z1=MCi2LJ#7
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-merge-commit.sample b/tests/resources/small.git/hooks/pre-merge-commit.sample
new file mode 100755
index 0000000000000000000000000000000000000000..d8e8ad4bcf8337f23c78a98537ecab4831301202
GIT binary patch
literal 473
zcmaJ-O;f@!5WV+TJd4B0whSt$H%Dh2t`2u6q1!glCNbGU;n%yxdNsG~zR#WA6xIwy
zWEZHoU#u?nykD=Y<HPgeWDkDm^kTof*l(|%^gh!nHrZpo^vhMDjV;E1GD~K7wV*+D
zz9lry9T0cHcm_KhDVXYvQ==FrLTT4u=bEr{U1yl7%thf%wJnv<XQZ`ZbQEezaWdS%
zI;c?h9a)Y!ux<WK8rQd_aA^?61hv_Pf<t7*z1USuL40FxYz<|hybsO?qnN}aMpcuf
z6phFw1%Xx=wUk(m>E$jSEQZ%SQ(}oLgslTvrKK@9Qf#b!hajVFnp9@oIix;NcI9Wk
xjnh0ya!AWet{I7YpD;y6HXyzI*lfSvH=o6*7mJZPkuaYpm>vzZ`wyGEBtOQPo|pgt
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-push.sample b/tests/resources/small.git/hooks/pre-push.sample
new file mode 100755
index 0000000000000000000000000000000000000000..02cbd80c287f959fe33975bb66c56293e3f5b396
GIT binary patch
literal 1431
zcmaJ>U60!~5PUX&#a1@z9B{IIZkjLT0t5kq9#8~D(I5{+8&J~9;#ndUk~-ZT`r|uG
z$#K$$J{TsK<m~Lsu9iP+t-0TZ=sa(K+C6);54X>s*LP1}9!GoZ@4I4myMMG_di|of
z%?llx{O8TS-#^<H#%^V=)RNv>;(OioEmPy%kwWQBA1OMzV{hsQ8XFzS1k!~YQoLa5
zhtP1fA$q6VmMbbAC_9)4I628k*O5J$NR19uHe4QYDK<==I~SQk)Nu%xQ~<Hy8U>KH
z53w=!ke(FGb_PpnZfd*+hnXDTn;2*`u^~;?+5C~cn?bRka7NR%06%e6O91{MAgN6J
zmlO8{Biw4&wr&&(z4p3eln`E}XR9m9bNYZ7Ibrg(4yZIXrfgD7N*AFD7L3YSM#j}%
zo__rOS5fr;@8UM<6cl+cv_$YB$PQ&9dv($eM*))g!_cu!QcSh-mqE9i#QDZT)=o#`
z?8!RtE?w6p?GkGZ-6yt_p~5~4ecu|Sf^)6096%h*q-eNiEA1;Xwg)p~Q&iGSG7-IQ
z9aII&`ps$WOojFA`*bjG<mBv1n0hcYZWN0~(X01-hx(ExqWqaXgSk*@-GMp|K_3!5
z9|O21N3%~izh(4fbp9wzd+!b&7cVwSP5H00)m5ej-(s=Pl#(90UOhn@OA9u+D{i@r
za4*CP0I#<d-)-#xq5q-iY5nIef2s5OuQjcA>kFk|E@sHHuD}W^d`7YJ3YE^zrQnqR
zGoq?;YGKe)93o|_=^f%3U1KYZGPOXRRxK7w`UUbMMa3<86OmVH!EKP$8RCrn9mWX+
zC?9yF!fRVLmud3hF<}x;;sR}f(*r}6Gap3fR6zLHR~kbMgD{98N`L+r&?3p~*0+FX
zcAL%j=(SO}xTJUTvA`&Lf`2mv4koPG9&|<CA~EHbWHMoFPwT(&U=7t0`RoFZPO9Kq
zwwe$i=T|AXY#hD$aZlNMH`wZ%gwilGJ(zeYpOn*F3cy0XKXio^Sj#WXx=PWV`WGaA
B&~*R+
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-rebase.sample b/tests/resources/small.git/hooks/pre-rebase.sample
new file mode 100755
index 0000000000000000000000000000000000000000..193ee3f127387f469894675f0d621b5bb9c275da
GIT binary patch
literal 5006
zcmb7IX>;2+68$XxiXKL@ma;l5d2^5Ba_rPh_DHI-u1#&_upttZXp;no03$20|NFiM
zK#D#xQ>!Z3JkX8T-LDVm!B5j7y_{;JDmmTTef+K1oIiPzeEr+Ai*<2PUgnG4^ZB>p
z_fkAvoR1emuf~ri^K$-px=4#D-v<wZ2Xv&$O_eTJh6d4)=DWL(NBs9G{k<+yMMw0T
z$VH*-+LM)}u&m^`l8~1nt(3Z;R8v(KbY5#i3z+~8h0D}Xvq&3J8BMWDizPNpaeb~9
zBN9bSkthfXzskapf%Zt{*e#}{QaNiaAVZ4{$;;I6<vKNhO@%7P-(;l-x=pPoExHC!
zB(hA#cDdD?s4P=!)=-K{<kHAWKetl-8I8wwO<ihJNs-$dEvr;&S_@6E=mNSJ5;mg#
zyb)MbqKH<one{qrV;ZQ6WL}yLtyi*ekNLf|uC6M!)Cmq7*l?g0d6`MlE49|}>Y9w&
z`bCv#<YfTKtb`!}CyNYd;|(C?vRVQmWOfR9X?FZ#=f$No)^#4>2zVn=YnJyeNey(Y
zRh`9vtLw~A+5zsjp|W0Nsa|29Rm!B>OoG5a+vi;ari8O>KkU!KAWg_fa3btK2x*_@
z0bEc7J;Ubghm}n9bOi(Sv_B66nQ7U)J7f0fO}<cB8i8vG{r39s_>8Wuf*uorcIgEG
zOHc|-V6+HlRhOP}?Cn?@5iwSl43abmBA^2lyL$+cpabCGVES+v^j^FO_}?FIp<qP?
z#_?*YMHIkyZxJxS;h@A)WDJ0bN&+F-#_lFjAf^g_Pb#5YXqYe|dZUpa^zI)VOBXQQ
zAMhT>%En%Ll?Z*7*}TwrZyg5OSZ9rY-`aU~Mc-jjv{Ll)FLMgtB4ujktfQ`Xhqrka
zT=P!A;9w^;Z?PqpLwOLu=cj3L>TdUKw2;DMu)`oVkj}<z_EjO_2uWYuvKG==%Zu?h
zJiMVR^c30RbpW}<+z;jjoNC}YI4f6QC7d-08*4mCB1>#bcDx4tYg=j%D`+i{W~fVM
zVmZ>W9VMyin9c-0KzI_;iZ-g|OyzuG`Yq%(%dvl;ifnVr0;jWE&S`z|rQu=!yHBBO
zx`OJ;oOQ(KKM<$(bC38o>pD0%|HA(E0TRw7qj$fJ_pRN+7Nm>dS<q{AcOz#-;d7_2
zL$z(_nhH{vOzT(}>C(gLg{(`t+5Z=?o+}wXU4tHy+&%F&aRhFebeEhR2R5|<c6J);
zEY(q5F5<n*XP0|=PtPBn$B)V~d$Os-?&8UlaVe_|jdkzoWNsTP-_uyq4$R6o<h`&@
z{loXa{^#TF=NJBYu9qB?hs}x=It_O}ZjX(_wy9@X(lmkRpNi0{8T{O_b_j*JC^_SM
zz3G?1$KCNWF-|`Jbx2cQ-y5LW?Z2eikngTZmsx5C(@({8<l)Ue+gIp##Mosfa~iZN
zZ|NLN9uE6Xaqpwk+@D+f?$tg2JRCY`pwZwbR9OvEn*zYm`ffKIzl4{r{ZgjfpbuX)
z_r0=0y3)T-j$gljPyEJO*6Y<pj7G72aLoqaTpc=#u)*xJcVUm0;k(n;r)^DQNa6Oj
zWvlHEGg~lz`Q_8`yQ9<BZ;ylE1Z}bD<8}<uB9Y5lRB=;JUD0h?_)4H&EhJjvwzJx?
zr<o_vk&75j_5^d$8Z$_Oc1=R-I_DlNZ2@~81oV*J6%o3RuiCz}^VEW>$#Ycbp^w@t
zTl%=f1t=w+WpJzF<|CE@?SCNAz)%9?w33lQ8vrHJqPfH9@}qs*QXOG71W=ylx;wOB
zcx!Bj^)Yy6WX$a^vBkBJ5Cob<oubBW+a#9bX{0bkMTX_2sIrs`HMk@$q{dK5h2-g}
z({`~#gm#G?c#>qlaDx_B0c<3b+8)f84LCrt;e;qxc+7>VbwVK{skNv!wvBiTa^9Iu
zkwP;VK)jH$WJ{`MRwAA9fal!y0dtV;FWg8PTkWU>CwnqD>1ZX2B@;$DlX%C5MI+}{
z9xQVnffR*~v2KAUj*hCd<gRCjR7~6Q(vF%VT7j97iv3j4Z0p%mU`7t061~l7b$!#t
z3*Pveii}aQ?QD9aiX>gul~`bk#mk`o>zk9)<2Uc8?hUZAEvd!`9em)~$Z)zev>w^8
zyAgCP_$&Y)7HSQ84`xG}OeTavaEswwF|8Xpi5iZzZa@hCiv(J-%bfFC&)HLlO+Rhw
zG6g?9eL5&A!SuJnQ6}LxG%tU+@vZ`i+!+Rz6iYvsTdhnPo7lW{m-}{hya@viX4)XZ
zngaw+j;gloB#|UwI@8sOmQpc`h+bicQJnQIB5eifIMQNgD2+oai33m!34~xU|0Azj
zhu$8z+T5^;Pxx@d{N)pzOJLSa^e;aDf$W%N5XcOf!mGC9l9j$Ev2h6N+6ZQC+CJzl
zaM7?S!SrFLS2DASjj(h6y1WN3N?|bmqmyzm!&nLoE|`rKBOc_yDF$a#FsUn!IQf(t
zdC&Us(kQz*7mv<H12p@UD8XaLXdK{>H^j*^MC@>wTDb}g%~sx*ng#>{@lR=XG-Z5_
z#<9*Oh0joMzt;nS)ObAp)347`D=}r-;nV!TbIq&xrGRGsF6fZg+!VkfUei@_&l-M&
zPqQ+Dw)RV}+)I8RuqAxa`Pv8e&!_gXS=e2-un>=Ktn}-;%lLZxaVn?Q>yZCb2R3Wk
z77zr%;Rq&h|2ncqyKYmFI0148JVY7Q$V5p=dWj<MQ<8r+@QLq+UROk&%quICq^O2C
zp(17882jXI)_GaqzAY4AjoB_nh8k*r1mJ>+Qqpu%i|xp2<qF`Tw6&3`h654ceBjZ7
z4>C=WaOb2Wudn^h0EcD%$p9YVU1fnoRV9`(cy(vv6K>FXS!2jY>1GnU--7)4usH&K
zao*&P^@9~YmUe|ZdLW@C>H;!*<TIU}-!Tw#3oqZA*99}b3&uHiGO<{I6!pMnAiQdS
P!fA@Yk4s_DjDP<F98V`a
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/pre-receive.sample b/tests/resources/small.git/hooks/pre-receive.sample
new file mode 100755
index 0000000000000000000000000000000000000000..32b597f8c32399017c3d7d51134d1f6490ad9575
GIT binary patch
literal 601
zcmZ`#+iu%141Kn~f>Vt3>Nw4M*;=?j(TBD#O@XCv0|MEhA;z}kTFRv@`tPHhp=&Yh
zg%Zhg4i7o_k{a5i&f5;tZ==%}^Sn4aD_6%qs<o-wO_Prn;}`SPs_*$C$(7T|$#C3`
zPt%-C8gelZ1GqAP8`ZQmg0{8-S9H{R@D>_XAuJt&EumdH4Yu`UjT<s+s_~uXh}qA8
zg|_HG)%7Pdc&$7*uR0HF@)~vmFjqyD?XZwCbLen^h5t)sm9<90Oa!@Y%8!~rF8G?W
zkzmCF8kMtuuelMHIAlqqnm?72LeGM1J4`w(kX9&%LQn}ForlDLjBoCyvxmo@x3kH^
z^loxLyPiDWPo-cBMnsg2M6}kuPGHEGBqVkC{D&9Kt%xFAsTw4QC1$_=fwBl=3dI+e
zaSxI}JS}=Z(Ec80eF`!Zq3mqapXI|UN!a)t;@4hcu%Eq2xcoW}%!><-+XHTuHss+b
YOmM2;hq8Egm*4=7_P9T{21QBYH*F=mfB*mh
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/prepare-commit-msg.sample b/tests/resources/small.git/hooks/prepare-commit-msg.sample
new file mode 100755
index 0000000000000000000000000000000000000000..b1970da1b6d3f42f00069fd17c325de72cda812e
GIT binary patch
literal 1702
zcmb_cTW{Mo6n>t6#i?x6xmZ$SFLf{QfG*3r0L?Pg?px55l8$UTGO3bO;spKi{V3XX
z))weX0X>M9bNMcZ-6yG%>(n}JI2|25dr<ew@wmMG{l(3lx~bQz>}WZBP@ih?JX^+@
zu#5O48P>yRX(m<b*PU*sORp92TCD1dX`%HE+1$w5k<(Ngu7zQ83#MGJR?<<W=d@yL
z#heqwo{FmCg0g#x<~R+PBD#}q(MBn;V$x;%UrJPP3*l%XtlvTWChI2SfJ$9e`YvSj
zRSOQ?NUgSMLI`3vL48YBHzwzVXoe7v0ef|0YHgV$N@?N(-R)rPqRDrK$n(%+OF$`P
zWdjC5N~`#RjV9}aYwQ4_yF5O-$h2`>fDIhYP)doc1&TADZa@ZGpusJ$6G+e$ZMcmC
zoOosDQPS}l{H?YPsq(4;0SGkATa9eeqAaDcj<jNAU+LTSmM1jo(ti~T0B7acJnnTX
zTarYy;Husdxal0!S<ba8=uu&a*SNYtr7|d7$g-q3_JHER2*oBsVW~i~XXdMx%LW~0
zT*960LF<qZ6K&FZ;vGmtyywBU3^Q>q8n2wALbFwU@2i@FAaRV!=uw-nwx1gKn2SvY
z>Ff>;2sg!+Hxfkwv1lsiii=p6WenF=5)6LZc<a$zDCD$GRuwvG4Fr+B#h?#9gTbio
zk#MdxC@WY%zSGN#i}Ts_#q`bf-{)`7CcWeB*7WlIyHjioJJWw&A5VItPUq3^9!r}S
zbykelFV-VFvcr>QaZ=aS_}+-4Y&?!@HWh|<^gJ21!|T@+%On#w6azxPHV}XsRbe*w
zR_TZ2XEsQa1lPK~biYqg@0-RW@5J1@=<87cFzEUABdCoFH2CZo?}l(Z*!OFqUxo>K
z_d`l#4d9|H6;VPT{X?^{VJ>oL|D7K{BJwwqB>`YcPoGk+9hbvHnoQ{EM|kPgD_`wk
zKm4#2xu;-y`RAm!=L_BnLvJ8$AZm8@?)v<%vwvsw8AF2x6!mTT;c72A_~U9nIq0ST
zv)N0!I!^1p=g8-RQfx5)E_Mb_4I2vtQpI30XZ&t<!9D6nI|;V7YR3)l6=S~QhuwLQ
g$e&^kTY-M99*<-Iw@(78*M5W!4}U}|8YyMx3->-9h5!Hn
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/push-to-checkout.sample b/tests/resources/small.git/hooks/push-to-checkout.sample
new file mode 100755
index 0000000000000000000000000000000000000000..a80611e18896f212c390d845e49a3f6d5693b41d
GIT binary patch
literal 2840
zcmai0U31$u5PXh)#YOS7cE^-rw@uolNhe9&aUS|HtvhX>G$45tVUYj>fRdF?|9kfU
zNR~aG=E)WbEbeyq7JTw}ZuHIE2kUtL<<n;$&G!2F^Je|kx2ug=4L5!H^!ogx`7o$&
z%Il(3zAe6<oe$^F=A|}s`8}CDp*M#3M)gC-)LOeDUpYMl3YNy9R)I-T)pE7sy09aj
zJ7%&5PnSB-F#2{jc><WLR{I2izuK%VHc+{hRfXe<^_q)8RjcE(6WX+F2)iAtDtI{x
ztAHVBq)eSp_E^xcV^i_5KLIHA$g{zEjh?rsacu+(Eyvve2~Kmw%;n3g(kWB56j~Js
z<yE5tYUsAR&PY0wgRvM8x!zgLX8SI!eVY&}YZ|>AoeCNptd-NM1aZLhESzC;I`+Ns
zfmNNjdAp^W8#Q*}l>CT7RB9F5(BbI8ly2l~+E};JW|>&d1)=epZ-8vm8ppkbEVn#R
zt30a5A-c(YQR8eM5%;|UAnO>rt!&@x@G@yp+92%w-}%(5P_+P&Wf_zb$f-Qrl5(7z
z2ah(bkE;!DK(&aAMuQ%1TS>ai?wSXCOCSj=_}8x4IbCx^$}9q)<W{Y<9o<WqZo^oV
z3bR;Qa%VT-M~kU@2n{=+=)C!MD`12;@<F*EtPfV3K#g^1%&ggH?4{m<R$WEKcUI4%
zl6{ik6Bo46pmNjdXg5@?hn;SjG$}rr3Gy#-BGgVD$8oD)OcK(oqca)L_kk)UBZ_&6
z*ourb#Yc8l3J+uSda_Y$GY--5Zp3QK9w^?P%E0xb57?fY+Q#+KU4)+R>whwv)SBt|
zg#MX4;;Oau`m=MI9(^&zPbueY@~>3*ixX%mvR5m_1&nAg@ZKvY1E$O}&EtLiG;mhV
z1xhMIm~fGjmf_#{62f`y;09?I7M1W2tWQvz<}i9lR>OpQyUJi45_&*pQus&EkwY<>
zI|ZAx=*3i9a-)g)hXkvO7>UJ5MNgL(Z+-wpXVcgbSgpmFmbf1~DPA(OVGI&FNLeIE
zNH!_aiH$vsif$_j7=T2{cS(!DOI`~bn@)vSd-0d7xL=DF;UNP|tW}4i<qWTSNCe|y
zg9kV&L9g;FhC@tvsVu#V-brqShHy2qZpA!gg{ZTY>h>DvHtu9tY_pbJ6x(6E*hxgC
zzNDao%qlr-IE%YGbS4hF!n!on7#W3$bX-_hbZAaws^nHu#)Dx=WzdbJ>AKzAy@T$x
zSWE^x9+|TEHVEPyaPYa0DOChp?AeHSBBDbZNokQpAY{lE!7geZI=jV)G^2@<iI(N4
zJLJjy@Y<VI$yq;3bVpJICW3D?YDMDl4Oe5pDf{d`i1_3Qx%4uX`$dOz<9e}jm2B-u
z8-?%!7XuiNF2O&MAdl-iw{drG+$F^zT2Z7WRV#c6;IRZEf>l)&91Zb1+`T+oq9wWF
zRV~kGTGce0O~p^6mj{kT5kL(pv>r;Lvd7VDX*P>A^Th`$3cWO<svk?_?FeP@458*2
zA1PqUOdd%VP5&4~ocLK16{1GLll64c=mU81RMFstpnMoLuI7hNIE4N)U%h*#<Fz{C
z9#>0<l7lRrls|W(8=W1O80P~6+U7<4BqC}N4-4GR3w#{O7YmH?JxwJfru2d?e){$5
z@5R+`7Z;1)FW;OkE-(HPisCS;5F4O^Q>L81p4Ysdo3ZP1(SrR-peEdTo;-@bkB((G
zPHYQXUL!@Q$e(OQ;R9r%@Afz+50I7>*^^c&&|E*r-jN)LH=pM4AqMwWxSv|nqjddE
Z4{_hwv8!W(<d3>T<BC%y-6O5i(|^OS%`gA}
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/sendemail-validate.sample b/tests/resources/small.git/hooks/sendemail-validate.sample
new file mode 100755
index 0000000000000000000000000000000000000000..effa6f54f2a5c1f0cebe258bf4805904be4b4852
GIT binary patch
literal 2365
zcmb_dU2oe)5PUX&#g-Nv2{5JDX_MA~3%Iq?8g*kpRgpYZIFU=~BI=I5J6e{T{`bz^
zk+$48h#x9Ika!=nv$M0y{clD}-j1x(hDWbnzP?l2k8j?TH{brS+Nf21yPm)Nczma>
zYw`X3V>TCdnSD1ru8&`j=2DIPbCT@SnIgUw>$+lEYP}+x8(BMYnr=iT3*ndq)xzaV
z>I+qjv}vC#8_9M+b1p#uNS0M0)q<p>8!3p_LRQ0MA3M`!2foxzRUjbFY@}O~(ki=S
zqscnq8cU*dY)D$$cqE}n)V0yIk>CNKHCrndOtSP*HbOb;nbwAHSb;R+gs^?^Dve%)
zoW}t(*D}$>O3ab0TS^-;J|u&sb-PkZzo#kn*#xYt(;<xzKW(YtQZ$u23?yV#kyh1~
z@+Idh;EG5jXx8@%<;Y_W8SA=|T;MPQ)TB!!<QcbU)YR4)79eeeg4|vp-8jm%Dl3^I
zRS7+i3>FGuwzSb^g&RDiGcOz9TB;Hu`nJh)$W=C=XCSm2AY=$w3G3P-V#Oo+N*;#2
z4ijJ-pBZ=;T(RTgp_HYrD!uW-dTMfkuqY5jwOy)~gM;#=P^i{!l7`pXTS^s(&^{RU
zydaw}OpS#^D1cXM8?FW+fh`t7D(g;yr6|}fdaNtZBx3hlK~IpkTu3!Qq%R+zAo#<L
zU&m+XIFB0>t}Bs8^3$vHD+-TGT@`F>H1Cc#WAVW;&$S6%fE2d6@kLS0g&ihIM{}0z
z8#XhD>b>3{(BH|Px7}&lJ4%y1v<t$W+!F|W@<gaU4;MqSHKR(wdML<XnCv;zaPrSi
zxVCvek26-bf#Q!H+uAgy_{e_1UZCq>(CihZJx@8MPoGdl*BJGD;usf*iS7%;{Joe;
zNFuBa>*~o&qETDPo~u&~$FxE1xb^x&(CbE`Y3GfsibL2rl+L;>P6j&Y3U>K$mkp*6
zd`Q{<^+^&;GskGjwD-%!boR&i-TC<Uvy02w+l$Nb?B}aL-%ZDpluqd=K_@}z*fyuV
zzAs1Hf?3v$k!X7GTc8Q=r`WJ_Uu=>A9UOR|@=GYb5x#<f&WSMH%mCJU<#=7=qFdL6
zG?Wz&6z&J<@I(B>+dhd7fkaVIR^pol`Mv+rUbmZ43dVL6^S7g3{NsPiG$iy$5EDB%
z6KIgnb$H(n&t3e4E6d4V7w^B?JS}JkG)PM6+X3Co`SQs($O*AA+MG~{S7RJ=cy-l&
z>~%3y`tjfx2>uOu<lDGWewcb=oL@}B@B6FCZ?oxSJWldrm%UfOduahk%C0H>tB_^s
ziwG=e=ch|FQ0IkN91US7rhdQkXhwwt$gU0WEVDjo=IPb+?6PC=s8}J*ua(Ms))`UL
fi$|vMHn?H<rred|1&u#kOoJ`%vx)-*g-ZSfgGvdP
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/hooks/update.sample b/tests/resources/small.git/hooks/update.sample
new file mode 100755
index 0000000000000000000000000000000000000000..c3430f6fe27e572a1cebdb2f2bec70add0642a60
GIT binary patch
literal 3707
zcmb_fU2oeq6n(aS#jRq*br9K3y0z;^QgkUc^kKlTrB6js&@yE)mPu8l<0$L?`wmIT
zrsXba))@gJi@e|G+<SfSXe`CeSQ}OG@sr8ZTUlQ{dzM}Q@O-hBi}GeUom`#X%FiYH
zX?m4Rna-0RN2lfK)A3ZuvHcz$L<jUn62D=~vfz{}wIH2VqBLX_O$(JSXeF7H$}q!c
zWY}C&R;eX%X?P{%d;|>_tSE3ettp-hLlsZCxaLX8(nU;bVRB;Ce6@s#eu2|WvLz>-
zvy(&>Gyfp@+BtKnpqWkKi^+v{4jn_pNw_zeuxE<mRXKx8G3;9pl+45&4~hHW!G@wo
za7?X(0B}HbX*ExkDmas*xzV)FxygC8AL?2Z1x-0QJqS@qn8sD7r{bm30@<%eL_gOw
z;~85O$Xw2AS}Qp)5ViRUe3|ir8;&&I<B7Y6^!kkNyYXF4EY(b8_5DsTYn_&?wkdEz
z0y$tADo<&}nGs5kg2-J=0KlEGPb(%<An(pXY{K`qIZCuwiT|2{8JD&5o_~`o6<;dD
zi@J#zCE4={8j%<uy|iutvHu1QKo5Tno-BAF2FwD%%O#UDDum=w!;!PNe-cOFNX4)5
zd>TifiGO|)w}OANj2n2D^K=o3j6P6uOL70#cbA{uzWXDlk1wr9GV1X(2W{RuTvjXV
zCmd<W?kH^?PXjkbF`XZtwu1B+%4@ZvHIwGpJ*8@8`MWAhq^B|Hj1lzj3R8bVuMpNb
zz4Gzk!3T3bY;WEGIww&kq9BYW6EP*q$K|EB-@TH(Fjtz*`HMTO?i+3E;5tdSah&xZ
z+t!x4K7)dpy5wiJhlJz~8qF|r8a&-SV7^I3C@_q=P`yt@_x_F-;PQR)fzP<zNN>8u
zH%V`94=q3)Dk)PHNrnFC(T1)Om6f{Usj;u1R->&XoCYVK2V3ZlgZuF?N}1+33<P7e
z<0yVF?Qoa{l#7q(3&jv=Ab)gpM8A7`p%3InNzSxy)ZER2U0C#9zKpnLY0I?>OER*x
z*9Z=L=zI8CN>A_^jYjt0F$psO$sL=38q5q|SG)qCN6{^>RFh5E&l5GZ$pEahnF&d+
z5c>64t}uJPkf~_!VUj#&N%nC-gUMj%=@B=!V>&}xtj2%@-mOm#rQUSJ3(ccmc+fza
znZ#uxF>N?QN5UrIEd!5RgHEf<eGg}P45aAs(Xs6u!XW9r1I*E6XJ^1movX@xYLuPz
z|7xBN4z@b}#x>W#;(nKYF+D<*rdshJ$X-z2OZ2X;)nn@KSVdVhaA?}@3;6gZxb4<W
z`9sa`0lR_azMX|=t_(FvG@%w2);9P}SG0u&K1(*oX3};~=<<!N*F$UTSxD{V&6mgL
ztw9Ntc2eOF@c!OJytNC4T^#)Ien7-`dI{6s#co~0f^E3J<0Ty)l3xq2u@Y8DXTK>v
zozoWSr{{+!h}zGpumG3H`=AvWpm^9kW;J$Jp^Xl*?8ckr`fqN%c|Z;VC0|cM4vSrk
zH_O8Yvh85nvJp^;``wo8=z0f`FWg?`>gO#y1hjX1{}rTlg9rwIKia8eyGexA3GnuR
z`Rg~XZoW;0pA)vI8=p5!+6sIn#C^FCvR>ffv39h6SCNi9v);%WD;WZ`of_MgwyRWy
z-yY%n*Y>X8<Sf+RyB|Sr2YG@1w~%U$o`(5EDkJ|3$u=eMZA&_wxEsy<Xxh2k^tP?4
RGx&ZHQs^8zuIpu!=pQhfp8Eg*
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/info/exclude b/tests/resources/small.git/info/exclude
new file mode 100644
index 0000000000000000000000000000000000000000..a5196d1be8fb59edf8062bef36d3a602e0812139
GIT binary patch
literal 240
zcmXYqK?=e!6h!x)VxWtv*mf_t5?px$aS_{}Hj?C*<cHXeXE&AZhT*-L3ZoI&*l1%Z
zqG?zr3TvQGZ__}H4(u*%p*rI=cU!%ya5ugfGATh66$IJHgu1Gs0-<N;$V+SsdE)?u
zIq;i$f#WE4f$_MWicZjMEob9LWKMR#iwZq54~QgST^6=i%u0lUkJu-_J**QBMq}ZG
Wth_)NDbl|`oQr&HAFQ5h`0jqAIZ*BZ
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e b/tests/resources/small.git/objects/88/f9a6eaa2cf008b9bc92847178621f21fa99f3e
new file mode 100644
index 0000000000000000000000000000000000000000..7d80b8d78e0dc55669d831a6638f48ec9fed0982
GIT binary patch
literal 50
zcmV-20L}k+0V^p=O;s>9W-v4`Ff%bx$Vkn}$!Ay}rnY6F$m-Kg*KD_+;Lx#g4|^&N
I02NaX#p`nv=Kufz
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b b/tests/resources/small.git/objects/af/5626b4a114abcb82d63db7c8082c3c4756e51b
new file mode 100644
index 0000000000000000000000000000000000000000..822bc151862ec3763cf2d3fa2372b93bbd3a4b65
GIT binary patch
literal 30
mcmb<m^geacKghr&@q@?NlP9kSYMj?U<r(;diNWtH+YSKNt_|)0
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4 b/tests/resources/small.git/objects/c6/97d4f7a6eac8d7b131673c340bd3cc5bac14d4
new file mode 100644
index 0000000000000000000000000000000000000000..5adfa88cc6b00bb19f2031b8ab61f6d07f9ccdb8
GIT binary patch
literal 130
zcmV-|0Db>>0i}&W3IZ_@1U=^!a~EV1casc=c+{&un1qQN*i9hD|0|m(2n|iwp*q%W
z%N;b$hu%cM`$TMo*~EnC1BFP&Pfj~;jZVKXQ96s_PhV<-XAROi+@-v8dBLUa`!;GB
k^i<X>XlEv8$>R)1G>9th&t3j;s7J{?^9n<zzF|~BaA?ar-~a#s
literal 0
HcmV?d00001
diff --git a/tests/resources/small.git/refs/heads/master b/tests/resources/small.git/refs/heads/master
new file mode 100644
index 0000000000000000000000000000000000000000..4eb36d22298f060fd324155ab854d9d6486fc498
GIT binary patch
literal 41
ucmV~$!4Uu;2m`Rc)5uXl$AMP&AHjriQg~T$i(A>|7U^`%mXoWC24Q^m!3%@{
literal 0
HcmV?d00001
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
((NEW_NIX_FIRST_BUILD_UID=301))
((NEW_NIX_FIRST_BUILD_UID=351))
id_unavailable(){
dscl . list /Users UniqueID | grep -E '\b'"$1"'\b' >/dev/null
+11 -1
View File
@@ -4,7 +4,17 @@ set -eu
set -o pipefail
# System specific settings
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-301}"
# Notes:
# - up to macOS Big Sur we used the same GID/UIDs as Linux (30000:30001-32)
# - we changed UID to 301 because Big Sur updates failed into recovery mode
# we're targeting the 200-400 UID range for role users mentioned in the
# usage note for sysadminctl
# - we changed UID to 351 because Sequoia now uses UIDs 300-304 for its own
# daemon users
# - we changed GID to 350 alongside above just because it hides the nixbld
# group from the Users & Groups settings panel :)
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-351}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-350}"
export NIX_BUILD_USER_NAME_TEMPLATE="_nixbld%d"
readonly NIX_DAEMON_DEST=/Library/LaunchDaemons/org.nixos.nix-daemon.plist
+2 -4
View File
@@ -23,10 +23,10 @@ readonly RED='\033[31m'
# installer allows overriding build user count to speed up installation
# as creating each user takes non-trivial amount of time on macos
readonly NIX_USER_COUNT=${NIX_USER_COUNT:-32}
readonly NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
readonly NIX_BUILD_GROUP_NAME="nixbld"
# each system specific installer must set these:
# NIX_FIRST_BUILD_UID
# NIX_BUILD_GROUP_ID
# NIX_BUILD_USER_NAME_TEMPLATE
# Please don't change this. We don't support it, because the
# default shell profile that comes with Nix doesn't support it.
@@ -530,9 +530,7 @@ It seems the build group $NIX_BUILD_GROUP_NAME already exists, but
with the UID $primary_group_id. This script can't really handle
that right now, so I'm going to give up.
You can fix this by editing this script and changing the
NIX_BUILD_GROUP_ID variable near the top to from $NIX_BUILD_GROUP_ID
to $primary_group_id and re-run.
You can export NIX_BUILD_GROUP_ID=$primary_group_id and re-run.
EOF
else
row " Exists" "Yes"
+1
View File
@@ -5,6 +5,7 @@ set -o pipefail
# System specific settings
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
export NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
readonly SERVICE_SRC=/lib/systemd/system/nix-daemon.service
+29
View File
@@ -0,0 +1,29 @@
# configures `scripts/nix-profile.sh.in` (and copies the original to the build directory).
# this is only needed for tests, but running it unconditionally does not hurt enough to care.
configure_file(
input : 'nix-profile.sh.in',
output : 'nix-profile.sh',
configuration : {
'localstatedir': localstatedir,
}
)
# https://github.com/mesonbuild/meson/issues/860
configure_file(
input : 'nix-profile.sh.in',
output : 'nix-profile.sh.in',
copy : true,
)
foreach rc : [ '.sh', '.fish', '-daemon.sh', '-daemon.fish' ]
configure_file(
input : 'nix-profile' + rc + '.in',
output : 'nix' + rc,
install : true,
install_dir : get_option('profile-dir'),
install_mode : 'rw-r--r--',
configuration : {
'localstatedir': localstatedir,
},
)
endforeach
+1 -1
View File
@@ -52,7 +52,7 @@ elif [ -e /etc/pki/tls/certs/ca-bundle.crt ]; then # Fedora, CentOS
else
# Fall back to what is in the nix profiles, favouring whatever is defined last.
check_nix_profiles() {
if [ -n "$ZSH_VERSION" ]; then
if [ -n "${ZSH_VERSION:-}" ]; then
# Zsh by default doesn't split words in unquoted parameter expansion.
# Set local_options for these options to be reverted at the end of the function
# and shwordsplit to force splitting words in $NIX_PROFILES below.
+24 -20
View File
@@ -3,29 +3,33 @@ if [ -n "$HOME" ] && [ -n "$USER" ]; then
# Set up the per-user profile.
NIX_LINK="$HOME/.nix-profile"
if [ -n "${XDG_STATE_HOME-}" ]; then
NIX_LINK_NEW="$XDG_STATE_HOME/nix/profile"
if [ -n "$NIX_STATE_HOME" ]; then
NIX_LINK="$NIX_STATE_HOME/profile"
else
NIX_LINK_NEW="$HOME/.local/state/nix/profile"
fi
if [ -e "$NIX_LINK_NEW" ]; then
if [ -t 2 ] && [ -e "$NIX_LINK" ]; then
warning="\033[1;35mwarning:\033[0m"
printf "$warning Both %s and legacy %s exist; using the former.\n" "$NIX_LINK_NEW" "$NIX_LINK" 1>&2
if [ "$(realpath "$NIX_LINK")" = "$(realpath "$NIX_LINK_NEW")" ]; then
printf " Since the profiles match, you can safely delete either of them.\n" 1>&2
else
# This should be an exceptionally rare occasion: the only way to get it would be to
# 1. Update to newer Nix;
# 2. Remove .nix-profile;
# 3. Set the $NIX_LINK_NEW to something other than the default user profile;
# 4. Roll back to older Nix.
# If someone did all that, they can probably figure out how to migrate the profile.
printf "$warning Profiles do not match. You should manually migrate from %s to %s.\n" "$NIX_LINK" "$NIX_LINK_NEW" 1>&2
NIX_LINK="$HOME/.nix-profile"
if [ -n "${XDG_STATE_HOME-}" ]; then
NIX_LINK_NEW="$XDG_STATE_HOME/nix/profile"
else
NIX_LINK_NEW="$HOME/.local/state/nix/profile"
fi
if [ -e "$NIX_LINK_NEW" ]; then
if [ -t 2 ] && [ -e "$NIX_LINK" ]; then
warning="\033[1;35mwarning:\033[0m"
printf "$warning Both %s and legacy %s exist; using the former.\n" "$NIX_LINK_NEW" "$NIX_LINK" 1>&2
if [ "$(realpath "$NIX_LINK")" = "$(realpath "$NIX_LINK_NEW")" ]; then
printf " Since the profiles match, you can safely delete either of them.\n" 1>&2
else
# This should be an exceptionally rare occasion: the only way to get it would be to
# 1. Update to newer Nix;
# 2. Remove .nix-profile;
# 3. Set the $NIX_LINK_NEW to something other than the default user profile;
# 4. Roll back to older Nix.
# If someone did all that, they can probably figure out how to migrate the profile.
printf "$warning Profiles do not match. You should manually migrate from %s to %s.\n" "$NIX_LINK" "$NIX_LINK_NEW" 1>&2
fi
fi
NIX_LINK="$NIX_LINK_NEW"
fi
NIX_LINK="$NIX_LINK_NEW"
fi
# Set up environment.
+11 -5
View File
@@ -1,8 +1,8 @@
#!/usr/bin/env bash
set -x
set -eo pipefail
((NEW_NIX_FIRST_BUILD_UID=350))
((NEW_NIX_FIRST_BUILD_UID=351))
((TEMP_NIX_FIRST_BUILD_UID=31000))
nix_user_n() {
@@ -17,18 +17,23 @@ any_nixbld(){
dscl . list /Users UniqueID | grep -E '\b_nixbld' >/dev/null
}
dsclattr() {
dscl . -read "$1" | awk "/$2/ { print \$2 }"
}
re_create_nixbld_user(){
local name uid
name="$1"
uid="$2"
gid="$3"
sudo /usr/bin/dscl . -create "/Users/$name" "UniqueID" "$uid"
sudo /usr/bin/dscl . -create "/Users/$name" "IsHidden" "1"
sudo /usr/bin/dscl . -create "/Users/$name" "NFSHomeDirectory" "/var/empty"
sudo /usr/bin/dscl . -create "/Users/$name" "RealName" "Nix build user $name"
sudo /usr/bin/dscl . -create "/Users/$name" "UserShell" "/sbin/nologin"
sudo /usr/bin/dscl . -create "/Users/$name" "PrimaryGroupID" "30001"
sudo /usr/bin/dscl . -create "/Users/$name" "PrimaryGroupID" "$gid"
}
hit_id_cap(){
@@ -68,11 +73,12 @@ temporarily_move_existing_nixbld_uids(){
}
change_nixbld_uids(){
local name next_id user_n
local existing_gid name next_id user_n
((next_id=NEW_NIX_FIRST_BUILD_UID))
((user_n=1))
name="$(nix_user_n "$user_n")"
existing_gid="$(dsclattr "/Groups/nixbld" "PrimaryGroupID")"
# we know that we have *some* nixbld users, but macOS may have
# already clobbered the first few users if this system has been
@@ -91,7 +97,7 @@ change_nixbld_uids(){
fi
done
re_create_nixbld_user "$name" "$next_id"
re_create_nixbld_user "$name" "$next_id" "$existing_gid"
echo " $name was missing; created with uid: $next_id"
((user_n++))
+10 -7
View File
@@ -18,6 +18,8 @@
namespace nix {
namespace fs { using namespace std::filesystem; }
fetchers::Settings fetchSettings;
static GlobalConfig::Register rFetchSettings(&fetchSettings);
@@ -32,7 +34,8 @@ EvalSettings evalSettings {
// FIXME `parseFlakeRef` should take a `std::string_view`.
auto flakeRef = parseFlakeRef(fetchSettings, std::string { rest }, {}, true, false);
debug("fetching flake search path element '%s''", rest);
return flakeRef.resolve(store).lazyFetch(store).first;
auto storePath = flakeRef.resolve(store).fetchTree(store).first;
return store->toRealPath(storePath);
},
},
},
@@ -118,8 +121,8 @@ MixEvalArgs::MixEvalArgs()
.category = category,
.labels = {"original-ref", "resolved-ref"},
.handler = {[&](std::string _from, std::string _to) {
auto from = parseFlakeRef(fetchSettings, _from, absPath("."));
auto to = parseFlakeRef(fetchSettings, _to, absPath("."));
auto from = parseFlakeRef(fetchSettings, _from, fs::current_path().string());
auto to = parseFlakeRef(fetchSettings, _to, fs::current_path().string());
fetchers::Attrs extraAttrs;
if (to.subdir != "") extraAttrs["dir"] = to.subdir;
fetchers::overrideRegistry(from.input, to.input, extraAttrs);
@@ -173,15 +176,15 @@ SourcePath lookupFileArg(EvalState & state, std::string_view s, const Path * bas
state.store,
state.fetchSettings,
EvalSettings::resolvePseudoUrl(s));
state.registerAccessor(accessor);
return SourcePath(accessor);
auto storePath = fetchToStore(*state.store, SourcePath(accessor), FetchMode::Copy);
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
else if (hasPrefix(s, "flake:")) {
experimentalFeatureSettings.require(Xp::Flakes);
auto flakeRef = parseFlakeRef(fetchSettings, std::string(s.substr(6)), {}, true, false);
auto accessor = flakeRef.resolve(state.store).lazyFetch(state.store).first;
return SourcePath(accessor);
auto storePath = flakeRef.resolve(state.store).fetchTree(state.store).first;
return state.rootPath(CanonPath(state.store->toRealPath(storePath)));
}
else if (s.size() > 2 && s.at(0) == '<' && s.at(s.size() - 1) == '>') {
+3 -1
View File
@@ -31,6 +31,8 @@
namespace nix {
namespace fs { using namespace std::filesystem; }
void completeFlakeInputPath(
AddCompletions & completions,
ref<EvalState> evalState,
@@ -341,7 +343,7 @@ void completeFlakeRefWithFragment(
auto flakeRefS = std::string(prefix.substr(0, hash));
// TODO: ideally this would use the command base directory instead of assuming ".".
auto flakeRef = parseFlakeRef(fetchSettings, expandTilde(flakeRefS), absPath("."));
auto flakeRef = parseFlakeRef(fetchSettings, expandTilde(flakeRefS), fs::current_path().string());
auto evalCache = openEvalCache(*evalState,
std::make_shared<flake::LockedFlake>(lockFlake(
+11 -3
View File
@@ -2,6 +2,7 @@
#include <cstdlib>
#include <cstring>
#include "error.hh"
#include "repl-interacter.hh"
#include "repl.hh"
@@ -134,7 +135,7 @@ NixRepl::NixRepl(const LookupPath & lookupPath, nix::ref<Store> store, ref<EvalS
, getValues(getValues)
, staticEnv(new StaticEnv(nullptr, state->staticBaseEnv.get()))
, runNixPtr{runNix}
, interacter(make_unique<ReadlineLikeInteracter>(getDataDir() + "/nix/repl-history"))
, interacter(make_unique<ReadlineLikeInteracter>(getDataDir() + "/repl-history"))
{
}
@@ -622,7 +623,7 @@ ProcessLineResult NixRepl::processLine(std::string line)
// When missing, trigger the normal exception
// e.g. :doc builtins.foo
// behaves like
// nix-repl> builtins.foo
// nix-repl> builtins.foo<tab>
// error: attribute 'foo' missing
evalString(arg, v);
assert(false);
@@ -720,7 +721,14 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
if (flakeRefS.empty())
throw Error("cannot use ':load-flake' without a path specified. (Use '.' for the current working directory.)");
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, absPath("."), true);
std::filesystem::path cwd;
try {
cwd = std::filesystem::current_path();
} catch (std::filesystem::filesystem_error & e) {
throw SysError("cannot determine current working directory");
}
auto flakeRef = parseFlakeRef(fetchSettings, flakeRefS, cwd.string(), true);
if (evalSettings.pureEval && !flakeRef.input.isLocked())
throw Error("cannot use ':load-flake' on locked flake reference '%s' (use --impure to override)", flakeRefS);
+1 -1
View File
@@ -69,7 +69,7 @@ struct AttrDb
{
auto state(_state->lock());
Path cacheDir = getCacheDir() + "/nix/eval-cache-v5";
Path cacheDir = getCacheDir() + "/eval-cache-v5";
createDirs(cacheDir);
Path dbPath = cacheDir + "/" + fingerprint.to_string(HashFormat::Base16, false) + ".sqlite";
+8
View File
@@ -4,6 +4,7 @@
#include "print.hh"
#include "eval.hh"
#include "eval-error.hh"
#include "eval-settings.hh"
namespace nix {
@@ -138,5 +139,12 @@ inline void EvalState::forceList(Value & v, const PosIdx pos, std::string_view e
}
}
[[gnu::always_inline]]
inline CallDepth EvalState::addCallDepth(const PosIdx pos) {
if (callDepth > settings.maxCallDepth)
error<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
return CallDepth(callDepth);
};
}
+1 -1
View File
@@ -99,7 +99,7 @@ const std::string & EvalSettings::getCurrentSystem() const
Path getNixDefExpr()
{
return settings.useXDGBaseDirectories
? getStateDir() + "/nix/defexpr"
? getStateDir() + "/defexpr"
: getHome() + "/.nix-defexpr";
}
+11 -3
View File
@@ -7,7 +7,6 @@
namespace nix {
class Store;
struct SourcePath;
struct EvalSettings : Config
{
@@ -23,7 +22,7 @@ struct EvalSettings : Config
* @todo Return (`std::optional` of) `SourceAccssor` or something
* more structured instead of mere `std::string`?
*/
using LookupPathHook = std::optional<SourcePath>(ref<Store> store, std::string_view);
using LookupPathHook = std::optional<std::string>(ref<Store> store, std::string_view);
/**
* Map from "scheme" to a `LookupPathHook`.
@@ -88,10 +87,19 @@ struct EvalSettings : Config
If the respective paths are accessible, the default values are:
- `$HOME/.nix-defexpr/channels`
The [user channel link](@docroot@/command-ref/files/default-nix-expression.md#user-channel-link), pointing to the current state of [channels](@docroot@/command-ref/files/channels.md) for the current user.
- `nixpkgs=$NIX_STATE_DIR/profiles/per-user/root/channels/nixpkgs`
The current state of the `nixpkgs` channel for the `root` user.
- `$NIX_STATE_DIR/profiles/per-user/root/channels`
See [`NIX_STATE_DIR`](@docroot@/command-ref/env-common.md#env-NIX_STATE_DIR) for details.
The current state of all channels for the `root` user.
These files are set up by the [Nix installer](@docroot@/installation/installing-binary.md).
See [`NIX_STATE_DIR`](@docroot@/command-ref/env-common.md#env-NIX_STATE_DIR) for details on the environment variable.
> **Note**
>
+36 -68
View File
@@ -315,7 +315,6 @@ EvalState::EvalState(
{
corepkgsFS->setPathDisplay("<nix", ">");
internalFS->setPathDisplay("«nix-internal»", "");
rootFS->setPathDisplay("/", "");
countCalls = getEnv("NIX_COUNT_CALLS").value_or("0") != "0";
@@ -617,6 +616,25 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
strdup(ss.data()),
};
}
if (isFunctor(v)) {
try {
Value & functor = *v.attrs()->find(sFunctor)->value;
Value * vp = &v;
Value partiallyApplied;
// The first paramater is not user-provided, and may be
// handled by code that is opaque to the user, like lib.const = x: y: y;
// So preferably we show docs that are relevant to the
// "partially applied" function returned by e.g. `const`.
// We apply the first argument:
callFunction(functor, 1, &vp, partiallyApplied, noPos);
auto _level = addCallDepth(noPos);
return getDoc(partiallyApplied);
}
catch (Error & e) {
e.addTrace(nullptr, "while partially calling '%1%' to retrieve documentation", "__functor");
throw;
}
}
return {};
}
@@ -860,11 +878,6 @@ void Value::mkPath(const SourcePath & path)
mkPath(&*path.accessor, makeImmutableString(path.path.abs()));
}
void EvalState::registerAccessor(const ref<SourceAccessor> accessor)
{
sourceAccessors.push_back(accessor);
}
inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
{
@@ -1477,26 +1490,9 @@ void ExprLambda::eval(EvalState & state, Env & env, Value & v)
v.mkLambda(&env, this);
}
namespace {
/** Increments a count on construction and decrements on destruction.
*/
class CallDepth {
size_t & count;
public:
CallDepth(size_t & count) : count(count) {
++count;
}
~CallDepth() {
--count;
}
};
};
void EvalState::callFunction(Value & fun, size_t nrArgs, Value * * args, Value & vRes, const PosIdx pos)
{
if (callDepth > settings.maxCallDepth)
error<EvalError>("stack overflow; max-call-depth exceeded").atPos(pos).debugThrow();
CallDepth _level(callDepth);
auto _level = addCallDepth(pos);
auto trace = settings.traceFunctionCalls
? std::make_unique<FunctionCallTrace>(positions[pos])
@@ -1979,22 +1975,10 @@ void EvalState::concatLists(Value & v, size_t nrLists, Value * const * lists, co
v.mkList(list);
}
// FIXME limit recursion
Value * resolveOutPath(EvalState & state, Value * v, const PosIdx pos)
{
state.forceValue(*v, pos);
if (v->type() != nAttrs)
return v;
auto found = v->attrs()->find(state.sOutPath);
if (found != v->attrs()->end())
return resolveOutPath(state, found->value, pos);
return v;
}
void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
{
NixStringContext context;
std::shared_ptr<SourceAccessor> accessor;
std::vector<BackedStringView> s;
size_t sSize = 0;
NixInt n{0};
@@ -2028,9 +2012,8 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
Value * vTmpP = values.data();
for (auto & [i_pos, i] : *es) {
Value & vTmp0 = *vTmpP++;
i->eval(state, env, vTmp0);
Value & vTmp = *resolveOutPath(state, &vTmp0, i_pos);
Value & vTmp = *vTmpP++;
i->eval(state, env, vTmp);
/* If the first element is a path, then the result will also
be a path, we don't copy anything (yet - that's done later,
@@ -2038,9 +2021,6 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
and none of the strings are allowed to have contexts. */
if (first) {
firstType = vTmp.type();
if (firstType == nPath) {
accessor = vTmp.path().accessor;
}
}
if (firstType == nInt) {
@@ -2087,7 +2067,7 @@ void ExprConcatStrings::eval(EvalState & state, Env & env, Value & v)
else if (firstType == nPath) {
if (!context.empty())
state.error<EvalError>("a string that refers to a store path cannot be appended to a path").atPos(pos).withFrame(env, *this).debugThrow();
v.mkPath({ref(accessor), CanonPath(str())});
v.mkPath(state.rootPath(CanonPath(canonPath(str()))));
} else
v.mkStringMove(c_str(), context);
}
@@ -2337,8 +2317,6 @@ BackedStringView EvalState::coerceToString(
v.payload.path.path
: copyToStore
? store->printStorePath(copyPathToStore(context, v.path()))
: v.path().accessor->toStringReturnsStorePath()
? store->printStorePath(copyPathToStore(context, SourcePath(v.path().accessor, CanonPath::root))) + v.path().path.absOrEmpty()
: std::string(v.path().path.abs());
}
@@ -2451,14 +2429,10 @@ SourcePath EvalState::coerceToPath(const PosIdx pos, Value & v, NixStringContext
if (v.type() == nPath)
return v.path();
/* Similarly, handle outPath and __toString where the result may be a path
/* Similarly, handle __toString where the result may be a path
value. */
if (v.type() == nAttrs) {
auto i = v.attrs()->find(sOutPath);
if (i != v.attrs()->end()) {
return coerceToPath(pos, *i->value, context, errorCtx);
}
i = v.attrs()->find(sToString);
auto i = v.attrs()->find(sToString);
if (i != v.attrs()->end()) {
Value v1;
callFunction(*i->value, v, v1, pos);
@@ -3081,14 +3055,12 @@ SourcePath EvalState::findFile(const LookupPath & lookupPath, const std::string_
if (!suffixOpt) continue;
auto suffix = *suffixOpt;
auto rOpt = resolveLookupPathPath(
i.path,
true);
auto rOpt = resolveLookupPathPath(i.path);
if (!rOpt) continue;
auto r = *rOpt;
auto res = r / CanonPath(suffix);
if (res.pathExists()) return res;
Path res = suffix == "" ? r : concatStrings(r, "/", suffix);
if (pathExists(res)) return rootPath(CanonPath(canonPath(res)));
}
if (hasPrefix(path, "nix/"))
@@ -3103,30 +3075,26 @@ SourcePath EvalState::findFile(const LookupPath & lookupPath, const std::string_
}
std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Path & value0, bool initAccessControl)
std::optional<std::string> EvalState::resolveLookupPathPath(const LookupPath::Path & value0, bool initAccessControl)
{
auto & value = value0.s;
auto i = lookupPathResolved.find(value);
if (i != lookupPathResolved.end()) return i->second;
auto finish = [&](SourcePath res) {
auto finish = [&](std::string res) {
debug("resolved search path element '%s' to '%s'", value, res);
lookupPathResolved.emplace(value, res);
return res;
};
std::optional<SourcePath> res;
if (EvalSettings::isPseudoUrl(value)) {
try {
auto accessor = fetchers::downloadTarball(
store,
fetchSettings,
EvalSettings::resolvePseudoUrl(value));
// Traditional search path lookups use the absolute path space for
// historical consistency.
auto storePath = fetchToStore(*store, SourcePath(accessor), FetchMode::Copy);
res.emplace(rootPath(CanonPath(store->toRealPath(storePath))));
return finish(store->toRealPath(storePath));
} catch (Error & e) {
logWarning({
.msg = HintFmt("Nix search path entry '%1%' cannot be downloaded, ignoring", value)
@@ -3145,22 +3113,22 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
}
{
auto path = rootPath(value);
auto path = absPath(value);
/* Allow access to paths in the search path. */
if (initAccessControl) {
allowPath(path.path.abs());
if (store->isInStore(path.path.abs())) {
allowPath(path);
if (store->isInStore(path)) {
try {
StorePathSet closure;
store->computeFSClosure(store->toStorePath(path.path.abs()).first, closure);
store->computeFSClosure(store->toStorePath(path).first, closure);
for (auto & p : closure)
allowPath(p);
} catch (InvalidPath &) { }
}
}
if (path.pathExists())
if (pathExists(path))
return finish(std::move(path));
else {
logWarning({
+28 -7
View File
@@ -41,6 +41,21 @@ namespace eval_cache {
class EvalCache;
}
/**
* Increments a count on construction and decrements on destruction.
*/
class CallDepth {
size_t & count;
public:
CallDepth(size_t & count) : count(count) {
++count;
}
~CallDepth() {
--count;
}
};
/**
* Function that implements a primop.
*/
@@ -249,9 +264,6 @@ public:
const SourcePath callFlakeInternal;
/* A collection of InputAccessors, just to keep them alive. */
std::list<ref<SourceAccessor>> sourceAccessors;
/**
* Store used to materialise .drv files.
*/
@@ -342,7 +354,7 @@ private:
LookupPath lookupPath;
std::map<std::string, std::optional<SourcePath>> lookupPathResolved;
std::map<std::string, std::optional<std::string>> lookupPathResolved;
/**
* Cache used by prim_match().
@@ -384,8 +396,6 @@ public:
*/
SourcePath rootPath(PathView path);
void registerAccessor(ref<SourceAccessor> accessor);
/**
* Allow access to a path.
*/
@@ -451,7 +461,7 @@ public:
*
* If it is not found, return `std::nullopt`
*/
std::optional<SourcePath> resolveLookupPathPath(
std::optional<std::string> resolveLookupPathPath(
const LookupPath::Path & elem,
bool initAccessControl = false);
@@ -630,6 +640,12 @@ public:
const char * doc;
};
/**
* Retrieve the documentation for a value. This will evaluate the value if
* it is a thunk, and it will partially apply __functor if applicable.
*
* @param v The value to get the documentation for.
*/
std::optional<Doc> getDoc(Value & v);
private:
@@ -654,6 +670,11 @@ private:
public:
/**
* Check that the call depth is within limits, and increment it, until the returned object is destroyed.
*/
inline CallDepth addCallDepth(const PosIdx pos);
/**
* Do a deep equality test between two values. That is, list
* elements and attributes are compared recursively.
+1 -1
View File
@@ -45,7 +45,7 @@ void ExprString::show(const SymbolTable & symbols, std::ostream & str) const
void ExprPath::show(const SymbolTable & symbols, std::ostream & str) const
{
str << path;
str << s;
}
void ExprVar::show(const SymbolTable & symbols, std::ostream & str) const
+4 -7
View File
@@ -131,15 +131,12 @@ struct ExprString : Expr
struct ExprPath : Expr
{
const SourcePath path;
ref<SourceAccessor> accessor;
std::string s;
Value v;
ExprPath(SourcePath && path)
: path(path)
ExprPath(ref<SourceAccessor> accessor, std::string s) : accessor(accessor), s(std::move(s))
{
v.mkPath(
&*path.accessor,
// TODO: GC_STRDUP
strdup(path.path.abs().c_str()));
v.mkPath(&*accessor, this->s.c_str());
}
Value * maybeThunk(EvalState & state, Env & env) override;
COMMON_METHODS
+10 -23
View File
@@ -136,10 +136,6 @@ static Expr * makeCall(PosIdx pos, Expr * fn, Expr * arg) {
std::vector<std::pair<nix::AttrName, nix::PosIdx>> * inheritAttrs;
std::vector<std::pair<nix::PosIdx, nix::Expr *>> * string_parts;
std::vector<std::pair<nix::PosIdx, std::variant<nix::Expr *, nix::StringToken>>> * ind_string_parts;
struct {
nix::Expr * e;
bool appendSlash;
} pathStart;
}
%type <e> start expr expr_function expr_if expr_op
@@ -153,8 +149,7 @@ static Expr * makeCall(PosIdx pos, Expr * fn, Expr * arg) {
%type <inheritAttrs> attrs
%type <string_parts> string_parts_interpolated
%type <ind_string_parts> ind_string_parts
%type <pathStart> path_start
%type <e> string_parts string_attr
%type <e> path_start string_parts string_attr
%type <id> attr
%token <id> ID
%token <str> STR IND_STR
@@ -300,11 +295,9 @@ expr_simple
$$ = state->stripIndentation(CUR_POS, std::move(*$2));
delete $2;
}
| path_start PATH_END { $$ = $1.e; }
| path_start PATH_END
| path_start string_parts_interpolated PATH_END {
if ($1.appendSlash)
$2->insert($2->begin(), {noPos, new ExprString("/")});
$2->insert($2->begin(), {state->at(@1), $1.e});
$2->insert($2->begin(), {state->at(@1), $1});
$$ = new ExprConcatStrings(CUR_POS, false, $2);
}
| SPATH {
@@ -357,17 +350,11 @@ string_parts_interpolated
path_start
: PATH {
std::string_view path({$1.p, $1.l});
$$ = {
.e = new ExprPath(
/* Absolute paths are always interpreted relative to the
root filesystem accessor, rather than the accessor of the
current Nix expression. */
hasPrefix(path, "/")
? SourcePath{state->rootFS, CanonPath(path)}
: SourcePath{state->basePath.accessor, CanonPath(path, state->basePath.path)}),
.appendSlash = hasSuffix(path, "/")
};
Path path(absPath(std::string_view{$1.p, $1.l}, state->basePath.path.abs()));
/* add back in the trailing '/' to the first segment */
if ($1.p[$1.l-1] == '/' && $1.l > 1)
path += "/";
$$ = new ExprPath(ref<SourceAccessor>(state->rootFS), std::move(path));
}
| HPATH {
if (state->settings.pureEval) {
@@ -376,8 +363,8 @@ path_start
std::string_view($1.p, $1.l)
);
}
CanonPath path(getHome() + std::string($1.p + 1, $1.l - 1));
$$ = {.e = new ExprPath(SourcePath{state->rootFS, std::move(path)}), .appendSlash = true};
Path path(getHome() + std::string($1.p + 1, $1.l - 1));
$$ = new ExprPath(ref<SourceAccessor>(state->rootFS), std::move(path));
}
;
+5 -5
View File
@@ -178,6 +178,7 @@ static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * v
auto path = realisePath(state, pos, vPath, std::nullopt);
auto path2 = path.path.abs();
// FIXME
auto isValidDerivationInStore = [&]() -> std::optional<StorePath> {
if (!state.store->isStorePath(path2))
return std::nullopt;
@@ -215,7 +216,9 @@ static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * v
state.forceFunction(**state.vImportedDrvToDerivation, pos, "while evaluating imported-drv-to-derivation.nix.gen.hh");
v.mkApp(*state.vImportedDrvToDerivation, w);
state.forceAttrs(v, pos, "while calling imported-drv-to-derivation.nix.gen.hh");
} else {
}
else {
if (!vScope)
state.evalFile(path, v);
else {
@@ -1697,10 +1700,7 @@ static std::string_view legacyBaseNameOf(std::string_view path)
static void prim_baseNameOf(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
NixStringContext context;
if (v.type() == nPath)
v.mkString(baseNameOf(v.path().path.abs()), context);
else
v.mkString(legacyBaseNameOf(*state.coerceToString(pos, *args[0], context,
v.mkString(legacyBaseNameOf(*state.coerceToString(pos, *args[0], context,
"while evaluating the first argument passed to builtins.baseNameOf",
false, false)), context);
}
+9 -49
View File
@@ -19,20 +19,21 @@ namespace nix {
void emitTreeAttrs(
EvalState & state,
const StorePath & storePath,
const fetchers::Input & input,
Value & v,
std::function<void(Value &)> setOutPath,
bool emptyRevFallback,
bool forceDirty)
{
auto attrs = state.buildBindings(100);
setOutPath(attrs.alloc(state.sOutPath));
state.mkStorePathString(storePath, attrs.alloc(state.sOutPath));
// FIXME: support arbitrary input attributes.
if (auto narHash = input.getNarHash())
attrs.alloc("narHash").mkString(narHash->to_string(HashFormat::SRI, true));
auto narHash = input.getNarHash();
assert(narHash);
attrs.alloc("narHash").mkString(narHash->to_string(HashFormat::SRI, true));
if (input.getType() == "git")
attrs.alloc("submodules").mkBool(
@@ -71,28 +72,10 @@ void emitTreeAttrs(
v.mkAttrs(attrs);
}
void emitTreeAttrs(
EvalState & state,
const SourcePath & storePath,
const fetchers::Input & input,
Value & v,
bool emptyRevFallback,
bool forceDirty)
{
emitTreeAttrs(state, input, v,
[&](Value & vOutPath) {
state.registerAccessor(storePath.accessor);
vOutPath.mkPath(storePath);
},
emptyRevFallback,
forceDirty);
}
struct FetchTreeParams {
bool emptyRevFallback = false;
bool allowNameArgument = false;
bool isFetchGit = false;
bool returnPath = true; // whether to return a SourcePath instead of a StorePath
};
static void fetchTree(
@@ -129,9 +112,7 @@ static void fetchTree(
for (auto & attr : *args[0]->attrs()) {
if (attr.name == state.sType) continue;
state.forceValue(*attr.value, attr.pos);
if (attr.value->type() == nPath || attr.value->type() == nString) {
auto s = state.coerceToString(attr.pos, *attr.value, context, "", false, false).toOwned();
attrs.emplace(state.symbols[attr.name],
@@ -212,31 +193,11 @@ static void fetchTree(
state.checkURI(input.toURLString());
if (params.returnPath) {
// Clang16+: change to `auto [accessor, input2] =`
auto pair = input.getAccessor(state.store);
auto & accessor = pair.first;
auto & input2 = pair.second;
auto [storePath, input2] = input.fetchToStore(state.store);
emitTreeAttrs(state, input2, v,
[&](Value & vOutPath) {
state.registerAccessor(accessor);
vOutPath.mkPath(SourcePath { accessor, CanonPath::root });
},
params.emptyRevFallback, false);
} else {
auto pair = input.fetchToStore(state.store);
auto & storePath = pair.first;
auto & input2 = pair.second;
state.allowPath(storePath);
state.allowPath(storePath);
emitTreeAttrs(state, input2, v,
[&](Value & vOutPath) {
state.mkStorePathString(storePath, vOutPath);
},
params.emptyRevFallback, false);
}
emitTreeAttrs(state, storePath, input2, v, params.emptyRevFallback, false);
}
static void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args, Value & v)
@@ -653,8 +614,7 @@ static void prim_fetchGit(EvalState & state, const PosIdx pos, Value * * args, V
FetchTreeParams {
.emptyRevFallback = true,
.allowNameArgument = true,
.isFetchGit = true,
.returnPath = false,
.isFetchGit = true
});
}
+1 -1
View File
@@ -36,7 +36,7 @@ struct CacheImpl : Cache
{
auto state(_state.lock());
auto dbPath = getCacheDir() + "/nix/fetcher-cache-v2.sqlite";
auto dbPath = getCacheDir() + "/fetcher-cache-v2.sqlite";
createDirs(dirOf(dbPath));
state->db = SQLite(dbPath);
+2 -24
View File
@@ -167,39 +167,17 @@ bool Input::contains(const Input & other) const
std::pair<StorePath, Input> Input::fetchToStore(ref<Store> store) const
{
// TODO: lazy-trees gets rid of this. Why?
#if 0
if (!scheme)
throw Error("cannot fetch unsupported input '%s'", attrsToJSON(toAttrs()));
/* The tree may already be in the Nix store, or it could be
substituted (which is often faster than fetching from the
original source). So check that. */
if (getNarHash()) {
try {
auto storePath = computeStorePath(*store);
store->ensurePath(storePath);
debug("using substituted/cached input '%s' in '%s'",
to_string(), store->printStorePath(storePath));
return {std::move(storePath), *this};
} catch (Error & e) {
debug("substitution of input '%s' failed: %s", to_string(), e.what());
}
}
#endif
auto [storePath, input] = [&]() -> std::pair<StorePath, Input> {
try {
auto [accessor, final] = getAccessorUnchecked(store);
auto storePath = nix::fetchToStore(*store, SourcePath(accessor), FetchMode::Copy, final.getName());
// TODO: do we really want to throw this out?
// auto narHash = store->queryPathInfo(storePath)->narHash;
// final.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
auto narHash = store->queryPathInfo(storePath)->narHash;
final.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
scheme->checkLocks(*this, final);
+139 -8
View File
@@ -13,13 +13,17 @@
#include <git2/describe.h>
#include <git2/errors.h>
#include <git2/global.h>
#include <git2/indexer.h>
#include <git2/object.h>
#include <git2/odb.h>
#include <git2/refs.h>
#include <git2/remote.h>
#include <git2/repository.h>
#include <git2/revparse.h>
#include <git2/status.h>
#include <git2/submodule.h>
#include <git2/sys/odb_backend.h>
#include <git2/sys/mempack.h>
#include <git2/tree.h>
#include <iostream>
@@ -76,6 +80,9 @@ typedef std::unique_ptr<git_status_list, Deleter<git_status_list_free>> StatusLi
typedef std::unique_ptr<git_remote, Deleter<git_remote_free>> Remote;
typedef std::unique_ptr<git_config, Deleter<git_config_free>> GitConfig;
typedef std::unique_ptr<git_config_iterator, Deleter<git_config_iterator_free>> ConfigIterator;
typedef std::unique_ptr<git_odb, Deleter<git_odb_free>> ObjectDb;
typedef std::unique_ptr<git_packbuilder, Deleter<git_packbuilder_free>> PackBuilder;
typedef std::unique_ptr<git_indexer, Deleter<git_indexer_free>> Indexer;
// A helper to ensure that we don't leak objects returned by libgit2.
template<typename T>
@@ -159,11 +166,60 @@ static Object peelToTreeOrBlob(git_object * obj)
return peelObject<Object>(obj, GIT_OBJECT_TREE);
}
struct PackBuilderContext {
std::exception_ptr exception;
void handleException(const char * activity, int errCode)
{
switch (errCode) {
case GIT_OK:
break;
case GIT_EUSER:
if (!exception)
panic("PackBuilderContext::handleException: user error, but exception was not set");
std::rethrow_exception(exception);
default:
throw Error("%s: %i, %s", Uncolored(activity), errCode, git_error_last()->message);
}
}
};
extern "C" {
/**
* A `git_packbuilder_progress` implementation that aborts the pack building if needed.
*/
static int packBuilderProgressCheckInterrupt(int stage, uint32_t current, uint32_t total, void *payload)
{
PackBuilderContext & args = * (PackBuilderContext *) payload;
try {
checkInterrupt();
return GIT_OK;
} catch (const std::exception & e) {
args.exception = std::current_exception();
return GIT_EUSER;
}
};
static git_packbuilder_progress PACKBUILDER_PROGRESS_CHECK_INTERRUPT = &packBuilderProgressCheckInterrupt;
} // extern "C"
struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
{
/** Location of the repository on disk. */
std::filesystem::path path;
/**
* libgit2 repository. Note that new objects are not written to disk,
* because we are using a mempack backend. For writing to disk, see
* `flush()`, which is also called by `GitFileSystemObjectSink::sync()`.
*/
Repository repo;
/**
* In-memory object store for efficient batched writing to packfiles.
* Owned by `repo`.
*/
git_odb_backend * mempack_backend;
GitRepoImpl(std::filesystem::path _path, bool create, bool bare)
: path(std::move(_path))
@@ -177,6 +233,17 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
if (git_repository_init(Setter(repo), path.string().c_str(), bare))
throw Error("creating Git repository '%s': %s", path, git_error_last()->message);
}
ObjectDb odb;
if (git_repository_odb(Setter(odb), repo.get()))
throw Error("getting Git object database: %s", git_error_last()->message);
// mempack_backend will be owned by the repository, so we are not expected to free it ourselves.
if (git_mempack_new(&mempack_backend))
throw Error("creating mempack backend: %s", git_error_last()->message);
if (git_odb_add_backend(odb.get(), mempack_backend, 999))
throw Error("adding mempack backend to Git object database: %s", git_error_last()->message);
}
operator git_repository * ()
@@ -184,6 +251,62 @@ struct GitRepoImpl : GitRepo, std::enable_shared_from_this<GitRepoImpl>
return repo.get();
}
void flush() override {
checkInterrupt();
git_buf buf = GIT_BUF_INIT;
Finally _disposeBuf { [&] { git_buf_dispose(&buf); } };
PackBuilder packBuilder;
PackBuilderContext packBuilderContext;
git_packbuilder_new(Setter(packBuilder), *this);
git_packbuilder_set_callbacks(packBuilder.get(), PACKBUILDER_PROGRESS_CHECK_INTERRUPT, &packBuilderContext);
git_packbuilder_set_threads(packBuilder.get(), 0 /* autodetect */);
packBuilderContext.handleException(
"preparing packfile",
git_mempack_write_thin_pack(mempack_backend, packBuilder.get())
);
checkInterrupt();
packBuilderContext.handleException(
"writing packfile",
git_packbuilder_write_buf(&buf, packBuilder.get())
);
checkInterrupt();
std::string repo_path = std::string(git_repository_path(repo.get()));
while (!repo_path.empty() && repo_path.back() == '/')
repo_path.pop_back();
std::string pack_dir_path = repo_path + "/objects/pack";
// TODO (performance): could the indexing be done in a separate thread?
// we'd need a more streaming variation of
// git_packbuilder_write_buf, or incur the cost of
// copying parts of the buffer to a separate thread.
// (synchronously on the git_packbuilder_write_buf thread)
Indexer indexer;
git_indexer_progress stats;
if (git_indexer_new(Setter(indexer), pack_dir_path.c_str(), 0, nullptr, nullptr))
throw Error("creating git packfile indexer: %s", git_error_last()->message);
// TODO: provide index callback for checkInterrupt() termination
// though this is about an order of magnitude faster than the packbuilder
// expect up to 1 sec latency due to uninterruptible git_indexer_append.
constexpr size_t chunkSize = 128 * 1024;
for (size_t offset = 0; offset < buf.size; offset += chunkSize) {
if (git_indexer_append(indexer.get(), buf.ptr + offset, std::min(chunkSize, buf.size - offset), &stats))
throw Error("appending to git packfile index: %s", git_error_last()->message);
checkInterrupt();
}
if (git_indexer_commit(indexer.get(), &stats))
throw Error("committing git packfile index: %s", git_error_last()->message);
if (git_mempack_reset(mempack_backend))
throw Error("resetting git mempack backend: %s", git_error_last()->message);
checkInterrupt();
}
uint64_t getRevCount(const Hash & rev) override
{
std::unordered_set<git_oid> done;
@@ -601,12 +724,16 @@ struct GitSourceAccessor : SourceAccessor
return readBlob(path, true);
}
Hash getSubmoduleRev(const CanonPath & path)
/**
* If `path` exists and is a submodule, return its
* revision. Otherwise return nothing.
*/
std::optional<Hash> getSubmoduleRev(const CanonPath & path)
{
auto entry = need(path);
auto entry = lookup(path);
if (git_tree_entry_type(entry) != GIT_OBJECT_COMMIT)
throw Error("'%s' is not a submodule", showPath(path));
if (!entry || git_tree_entry_type(entry) != GIT_OBJECT_COMMIT)
return std::nullopt;
return toHash(*git_tree_entry_id(entry));
}
@@ -1002,12 +1129,14 @@ struct GitFileSystemObjectSinkImpl : GitFileSystemObjectSink
git_tree_entry_filemode(entry));
}
Hash sync() override
Hash flush() override
{
updateBuilders({});
auto [oid, _name] = popBuilder();
repo->flush();
return toHash(oid);
}
};
@@ -1074,8 +1203,10 @@ std::vector<std::tuple<GitRepoImpl::Submodule, Hash>> GitRepoImpl::getSubmodules
auto rawAccessor = getRawAccessor(rev);
for (auto & submodule : parseSubmodules(pathTemp)) {
auto rev = rawAccessor->getSubmoduleRev(submodule.path);
result.push_back({std::move(submodule), rev});
/* Filter out .gitmodules entries that don't exist or are not
submodules. */
if (auto rev = rawAccessor->getSubmoduleRev(submodule.path))
result.push_back({std::move(submodule), *rev});
}
return result;
@@ -1083,7 +1214,7 @@ std::vector<std::tuple<GitRepoImpl::Submodule, Hash>> GitRepoImpl::getSubmodules
ref<GitRepo> getTarballCache()
{
static auto repoDir = std::filesystem::path(getCacheDir()) / "nix" / "tarball-cache";
static auto repoDir = std::filesystem::path(getCacheDir()) / "tarball-cache";
return GitRepo::openRepo(repoDir, true, true);
}
+7 -1
View File
@@ -7,12 +7,16 @@ namespace nix {
namespace fetchers { struct PublicKey; }
/**
* A sink that writes into a Git repository. Note that nothing may be written
* until `flush()` is called.
*/
struct GitFileSystemObjectSink : ExtendedFileSystemObjectSink
{
/**
* Flush builder and return a final Git hash.
*/
virtual Hash sync() = 0;
virtual Hash flush() = 0;
};
struct GitRepo
@@ -80,6 +84,8 @@ struct GitRepo
virtual ref<GitFileSystemObjectSink> getFileSystemObjectSink() = 0;
virtual void flush() = 0;
virtual void fetch(
const std::string & url,
const std::string & refspec,
+4 -3
View File
@@ -44,7 +44,7 @@ bool isCacheFileWithinTtl(time_t now, const struct stat & st)
Path getCachePath(std::string_view key, bool shallow)
{
return getCacheDir()
+ "/nix/gitv3/"
+ "/gitv3/"
+ hashString(HashAlgorithm::SHA256, key).to_string(HashFormat::Nix32, false)
+ (shallow ? "-shallow" : "");
}
@@ -584,9 +584,10 @@ struct GitInputScheme : InputScheme
}
try {
setWriteTime(localRefFile, now, now);
if (!input.getRev())
setWriteTime(localRefFile, now, now);
} catch (Error & e) {
warn("could not update mtime for file '%s': %s", localRefFile, e.msg());
warn("could not update mtime for file '%s': %s", localRefFile, e.info().msg);
}
if (!originalRef && !storeCachedHead(repoInfo.url, ref))
warn("could not update cached head '%s' for '%s'", ref, repoInfo.url);
+2 -1
View File
@@ -261,11 +261,12 @@ struct GitArchiveInputScheme : InputScheme
auto tarballCache = getTarballCache();
auto parseSink = tarballCache->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
auto tree = parseSink->flush();
act.reset();
TarballInfo tarballInfo {
.treeHash = tarballCache->dereferenceSingletonDirectory(parseSink->sync()),
.treeHash = tarballCache->dereferenceSingletonDirectory(tree),
.lastModified = lastModified
};
+1 -1
View File
@@ -263,7 +263,7 @@ struct MercurialInputScheme : InputScheme
return makeResult(res->value, res->storePath);
}
Path cacheDir = fmt("%s/nix/hg/%s", getCacheDir(), hashString(HashAlgorithm::SHA256, actualUrl).to_string(HashFormat::Nix32, false));
Path cacheDir = fmt("%s/hg/%s", getCacheDir(), hashString(HashAlgorithm::SHA256, actualUrl).to_string(HashFormat::Nix32, false));
/* If this is a commit hash that we already have, we don't
have to pull again. */
+2 -2
View File
@@ -116,7 +116,7 @@ static std::shared_ptr<Registry> getSystemRegistry(const Settings & settings)
Path getUserRegistryPath()
{
return getConfigDir() + "/nix/registry.json";
return getConfigDir() + "/registry.json";
}
std::shared_ptr<Registry> getUserRegistry(const Settings & settings)
@@ -159,7 +159,7 @@ static std::shared_ptr<Registry> getGlobalRegistry(const Settings & settings, re
if (!hasPrefix(path, "/")) {
auto storePath = downloadFile(store, path, "flake-registry.json").storePath;
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>())
store2->addPermRoot(storePath, getCacheDir() + "/nix/flake-registry.json");
store2->addPermRoot(storePath, getCacheDir() + "/flake-registry.json");
path = store->toRealPath(storePath);
}
+2 -1
View File
@@ -170,6 +170,7 @@ static DownloadTarballResult downloadTarball_(
auto tarballCache = getTarballCache();
auto parseSink = tarballCache->getFileSystemObjectSink();
auto lastModified = unpackTarfileToSink(archive, *parseSink);
auto tree = parseSink->flush();
act.reset();
@@ -184,7 +185,7 @@ static DownloadTarballResult downloadTarball_(
} else {
infoAttrs.insert_or_assign("etag", res->etag);
infoAttrs.insert_or_assign("treeHash",
tarballCache->dereferenceSingletonDirectory(parseSink->sync()).gitRev());
tarballCache->dereferenceSingletonDirectory(tree).gitRev());
infoAttrs.insert_or_assign("lastModified", uint64_t(lastModified));
if (res->immutableUrl)
infoAttrs.insert_or_assign("immutableUrl", *res->immutableUrl);
+1 -1
View File
@@ -12,7 +12,7 @@ typedef std::map<std::string, std::map<std::string, bool>> TrustedList;
Path trustedListPath()
{
return getDataDir() + "/nix/trusted-settings.json";
return getDataDir() + "/trusted-settings.json";
}
static TrustedList readTrustedList()
+89 -38
View File
@@ -19,17 +19,62 @@ using namespace flake;
namespace flake {
static FlakeRef maybeResolve(
typedef std::pair<StorePath, FlakeRef> FetchedFlake;
typedef std::vector<std::pair<FlakeRef, FetchedFlake>> FlakeCache;
static std::optional<FetchedFlake> lookupInFlakeCache(
const FlakeCache & flakeCache,
const FlakeRef & flakeRef)
{
// FIXME: inefficient.
for (auto & i : flakeCache) {
if (flakeRef == i.first) {
debug("mapping '%s' to previously seen input '%s' -> '%s",
flakeRef, i.first, i.second.second);
return i.second;
}
}
return std::nullopt;
}
static std::tuple<StorePath, FlakeRef, FlakeRef> fetchOrSubstituteTree(
EvalState & state,
const FlakeRef & originalRef,
bool useRegistries)
bool allowLookup,
FlakeCache & flakeCache)
{
auto fetched = lookupInFlakeCache(flakeCache, originalRef);
FlakeRef resolvedRef = originalRef;
if (!fetched) {
if (originalRef.input.isDirect()) {
if (!useRegistries)
fetched.emplace(originalRef.fetchTree(state.store));
} else {
if (allowLookup) {
resolvedRef = originalRef.resolve(state.store);
auto fetchedResolved = lookupInFlakeCache(flakeCache, originalRef);
if (!fetchedResolved) fetchedResolved.emplace(resolvedRef.fetchTree(state.store));
flakeCache.push_back({resolvedRef, *fetchedResolved});
fetched.emplace(*fetchedResolved);
}
else {
throw Error("'%s' is an indirect flake reference, but registry lookups are not allowed", originalRef);
return originalRef.resolve(state.store);
} else
return originalRef;
}
}
flakeCache.push_back({originalRef, *fetched});
}
auto [storePath, lockedRef] = *fetched;
debug("got tree '%s' from '%s'",
state.store->printStorePath(storePath), lockedRef);
state.allowPath(storePath);
assert(!originalRef.input.getNarHash() || storePath == originalRef.input.computeStorePath(*state.store));
return {std::move(storePath), resolvedRef, lockedRef};
}
static void forceTrivialValue(EvalState & state, Value & value, const PosIdx pos)
@@ -271,19 +316,24 @@ static Flake getFlake(
EvalState & state,
const FlakeRef & originalRef,
bool allowLookup,
FlakeCache & flakeCache,
InputPath lockRootPath)
{
auto resolvedRef = maybeResolve(state, originalRef, allowLookup);
auto [accessor, lockedRef] = resolvedRef.lazyFetch(state.store);
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, originalRef, allowLookup, flakeCache);
state.registerAccessor(accessor);
return readFlake(state, originalRef, resolvedRef, lockedRef, state.rootPath(state.store->toRealPath(storePath)), lockRootPath);
}
return readFlake(state, originalRef, resolvedRef, lockedRef, SourcePath {accessor, CanonPath::root}, lockRootPath);
Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup, FlakeCache & flakeCache)
{
return getFlake(state, originalRef, allowLookup, flakeCache, {});
}
Flake getFlake(EvalState & state, const FlakeRef & originalRef, bool allowLookup)
{
return getFlake(state, originalRef, allowLookup, {});
FlakeCache flakeCache;
return getFlake(state, originalRef, allowLookup, flakeCache);
}
static LockFile readLockFile(
@@ -305,9 +355,11 @@ LockedFlake lockFlake(
{
experimentalFeatureSettings.require(Xp::Flakes);
FlakeCache flakeCache;
auto useRegistries = lockFlags.useRegistries.value_or(settings.useRegistries);
auto flake = getFlake(state, topRef, useRegistries);
auto flake = getFlake(state, topRef, useRegistries, flakeCache);
if (lockFlags.applyNixConfig) {
flake.config.apply(settings);
@@ -501,7 +553,7 @@ LockedFlake lockFlake(
}
if (mustRefetch) {
auto inputFlake = getFlake(state, oldLock->lockedRef, false, inputPath);
auto inputFlake = getFlake(state, oldLock->lockedRef, false, flakeCache, inputPath);
nodePaths.emplace(childNode, inputFlake.path.parent());
computeLocks(inputFlake.inputs, childNode, inputPath, oldLock, lockRootPath, parentPath, false);
} else {
@@ -534,7 +586,7 @@ LockedFlake lockFlake(
if (localRef.input.getType() == "path")
localPath = absPath(*input.ref->input.getSourcePath(), parentPath);
auto inputFlake = getFlake(state, localRef, useRegistries, inputPath);
auto inputFlake = getFlake(state, localRef, useRegistries, flakeCache, inputPath);
auto childNode = make_ref<LockedNode>(inputFlake.lockedRef, ref);
@@ -563,17 +615,12 @@ LockedFlake lockFlake(
}
else {
auto [path, lockedRef] = [&]() -> std::tuple<SourcePath, FlakeRef>
{
auto resolvedRef = maybeResolve(state, *input.ref, useRegistries);
auto [accessor, lockedRef] = resolvedRef.lazyFetch(state.store);
state.registerAccessor(accessor);
return {SourcePath(accessor), lockedRef};
}();
auto [storePath, resolvedRef, lockedRef] = fetchOrSubstituteTree(
state, *input.ref, useRegistries, flakeCache);
auto childNode = make_ref<LockedNode>(lockedRef, ref, false);
nodePaths.emplace(childNode, path);
nodePaths.emplace(childNode, state.rootPath(state.store->toRealPath(storePath)));
node->inputs.insert_or_assign(id, childNode);
}
@@ -676,7 +723,8 @@ LockedFlake lockFlake(
repo, so we should re-read it. FIXME: we could
also just clear the 'rev' field... */
auto prevLockedRef = flake.lockedRef;
flake = getFlake(state, topRef, useRegistries);
FlakeCache dummyCache;
flake = getFlake(state, topRef, useRegistries, dummyCache);
if (lockFlags.commitLockFile &&
flake.lockedRef.input.getRev() &&
@@ -703,6 +751,21 @@ LockedFlake lockFlake(
}
}
std::pair<StorePath, Path> sourcePathToStorePath(
ref<Store> store,
const SourcePath & _path)
{
auto path = _path.path.abs();
if (auto store2 = store.dynamic_pointer_cast<LocalFSStore>()) {
auto realStoreDir = store2->getRealStoreDir();
if (isInDir(path, realStoreDir))
path = store2->storeDir + path.substr(realStoreDir.size());
}
return store->toStorePath(path);
}
void callFlake(EvalState & state,
const LockedFlake & lockedFlake,
Value & vRes)
@@ -720,23 +783,11 @@ void callFlake(EvalState & state,
auto lockedNode = node.dynamic_pointer_cast<const LockedNode>();
/*
// FIXME: This is a hack to support chroot stores. Remove this
// once we can pass a sourcePath rather than a storePath to
// call-flake.nix.
// auto path = sourcePath.path.abs();
// if (auto store = state.store.dynamic_pointer_cast<LocalFSStore>()) {
// auto realStoreDir = store->getRealStoreDir();
// if (isInDir(path, realStoreDir))
// path = store->storeDir + path.substr(realStoreDir.size());
// }
//
// auto [storePath, subdir] = state.store->toStorePath(path);
*/
auto [storePath, subdir] = sourcePathToStorePath(state.store, sourcePath);
emitTreeAttrs(
state,
SourcePath(sourcePath.accessor),
storePath,
lockedNode ? lockedNode->lockedRef.input : lockedFlake.flake.lockedRef.input,
vSourceInfo,
false,
@@ -747,7 +798,7 @@ void callFlake(EvalState & state,
override
.alloc(state.symbols.create("dir"))
.mkString(sourcePath.path.rel());
.mkString(CanonPath(subdir).rel());
overrides.alloc(state.symbols.create(key->second)).mkAttrs(override);
}
+11 -1
View File
@@ -214,11 +214,21 @@ void callFlake(
const LockedFlake & lockedFlake,
Value & v);
/**
* Map a `SourcePath` to the corresponding store path. This is a
* temporary hack to support chroot stores while we don't have full
* lazy trees. FIXME: Remove this once we can pass a sourcePath rather
* than a storePath to call-flake.nix.
*/
std::pair<StorePath, Path> sourcePathToStorePath(
ref<Store> store,
const SourcePath & path);
}
void emitTreeAttrs(
EvalState & state,
const SourcePath & storePath,
const StorePath & storePath,
const fetchers::Input & input,
Value & v,
bool emptyRevFallback = false,
+3 -3
View File
@@ -287,10 +287,10 @@ FlakeRef FlakeRef::fromAttrs(
fetchers::maybeGetStrAttr(attrs, "dir").value_or(""));
}
std::pair<ref<SourceAccessor>, FlakeRef> FlakeRef::lazyFetch(ref<Store> store) const
std::pair<StorePath, FlakeRef> FlakeRef::fetchTree(ref<Store> store) const
{
auto [accessor, lockedInput] = input.getAccessor(store);
return {accessor, FlakeRef(std::move(lockedInput), subdir)};
auto [storePath, lockedInput] = input.fetchToStore(store);
return {std::move(storePath), FlakeRef(std::move(lockedInput), subdir)};
}
std::tuple<FlakeRef, std::string, ExtendedOutputsSpec> parseFlakeRefWithFragmentAndExtendedOutputsSpec(
+1 -1
View File
@@ -63,7 +63,7 @@ struct FlakeRef
const fetchers::Settings & fetchSettings,
const fetchers::Attrs & attrs);
std::pair<ref<SourceAccessor>, FlakeRef> lazyFetch(ref<Store> store) const;
std::pair<StorePath, FlakeRef> fetchTree(ref<Store> store) const;
};
std::ostream & operator << (std::ostream & str, const FlakeRef & flakeRef);
+3 -2
View File
@@ -54,12 +54,13 @@ StorePath LockedNode::computeStorePath(Store & store) const
}
static std::shared_ptr<Node> doFind(const ref<Node>& root, const InputPath & path, std::vector<InputPath>& visited) {
static std::shared_ptr<Node> doFind(const ref<Node> & root, const InputPath & path, std::vector<InputPath> & visited)
{
auto pos = root;
auto found = std::find(visited.cbegin(), visited.cend(), path);
if(found != visited.end()) {
if (found != visited.end()) {
std::vector<std::string> cycle;
std::transform(found, visited.cend(), std::back_inserter(cycle), printInputPath);
cycle.push_back(printInputPath(path));
+20 -1
View File
@@ -77,6 +77,9 @@ private:
bool haveUpdate = true;
};
/** Helps avoid unnecessary redraws, see `redraw()` */
Sync<std::string> lastOutput_;
Sync<State> state_;
std::thread updateThread;
@@ -359,6 +362,22 @@ public:
updateCV.notify_one();
}
/**
* Redraw, if the output has changed.
*
* Excessive redrawing is noticable on slow terminals, and it interferes
* with text selection in some terminals, including libvte-based terminal
* emulators.
*/
void redraw(std::string newOutput)
{
auto lastOutput(lastOutput_.lock());
if (newOutput != *lastOutput) {
writeToStderr(newOutput);
*lastOutput = std::move(newOutput);
}
}
std::chrono::milliseconds draw(State & state)
{
auto nextWakeup = std::chrono::milliseconds::max();
@@ -412,7 +431,7 @@ public:
auto width = getWindowSize().second;
if (width <= 0) width = std::numeric_limits<decltype(width)>::max();
writeToStderr("\r" + filterANSIEscapes(line, false, width) + ANSI_NORMAL + "\e[K");
redraw("\r" + filterANSIEscapes(line, false, width) + ANSI_NORMAL + "\e[K");
return nextWakeup;
}
+12
View File
@@ -144,3 +144,15 @@ StorePath * nix_store_path_clone(const StorePath * p)
{
return new StorePath{p->path};
}
nix_err nix_store_copy_closure(nix_c_context * context, Store * srcStore, Store * dstStore, StorePath * path)
{
if (context)
context->last_err_code = NIX_OK;
try {
nix::RealisedPath::Set paths;
paths.insert(path->path);
nix::copyClosure(*srcStore->ptr, *dstStore->ptr, paths);
}
NIXC_CATCH_ERRS
}
+10
View File
@@ -161,6 +161,16 @@ nix_err nix_store_realise(
nix_err
nix_store_get_version(nix_c_context * context, Store * store, nix_get_string_callback callback, void * user_data);
/**
* @brief Copy the closure of `path` from `srcStore` to `dstStore`.
*
* @param[out] context Optional, stores error information
* @param[in] srcStore nix source store reference
* @param[in] srcStore nix destination store reference
* @param[in] path Path to copy
*/
nix_err nix_store_copy_closure(nix_c_context * context, Store * srcStore, Store * dstStore, StorePath * path);
// cffi end
#ifdef __cplusplus
}
+1 -1
View File
@@ -183,7 +183,7 @@ Goal::Co PathSubstitutionGoal::tryToRun(StorePath subPath, nix::ref<Store> sub,
/* Make sure that we are allowed to start a substitution. Note that even
if maxSubstitutionJobs == 0, we still allow a substituter to run. This
prevents infinite waiting. */
if (worker.getNrSubstitutions() >= std::max(1U, (unsigned int) settings.maxSubstitutionJobs)) {
while (worker.getNrSubstitutions() >= std::max(1U, (unsigned int) settings.maxSubstitutionJobs)) {
worker.waitForBuildSlot(shared_from_this());
co_await Suspend{};
}
+2 -2
View File
@@ -184,13 +184,13 @@ void Worker::wakeUp(GoalPtr goal)
}
unsigned Worker::getNrLocalBuilds()
size_t Worker::getNrLocalBuilds()
{
return nrLocalBuilds;
}
unsigned Worker::getNrSubstitutions()
size_t Worker::getNrSubstitutions()
{
return nrSubstitutions;
}
+4 -4
View File
@@ -92,12 +92,12 @@ private:
* Number of build slots occupied. This includes local builds but does not
* include substitutions or remote builds via the build hook.
*/
unsigned int nrLocalBuilds;
size_t nrLocalBuilds;
/**
* Number of substitution slots occupied.
*/
unsigned int nrSubstitutions;
size_t nrSubstitutions;
/**
* Maps used to prevent multiple instantiations of a goal for the
@@ -235,12 +235,12 @@ public:
* Return the number of local build processes currently running (but not
* remote builds via the build hook).
*/
unsigned int getNrLocalBuilds();
size_t getNrLocalBuilds();
/**
* Return the number of substitution processes currently running.
*/
unsigned int getNrSubstitutions();
size_t getNrSubstitutions();
/**
* Registers a running child process. `inBuildSlot` means that
+31 -9
View File
@@ -3,31 +3,53 @@
namespace nix {
namespace fs { using namespace std::filesystem; }
void builtinUnpackChannel(
const BasicDerivation & drv,
const std::map<std::string, Path> & outputs)
{
auto getAttr = [&](const std::string & name) {
auto getAttr = [&](const std::string & name) -> const std::string & {
auto i = drv.env.find(name);
if (i == drv.env.end()) throw Error("attribute '%s' missing", name);
return i->second;
};
auto out = outputs.at("out");
auto channelName = getAttr("channelName");
auto src = getAttr("src");
fs::path out{outputs.at("out")};
auto & channelName = getAttr("channelName");
auto & src = getAttr("src");
createDirs(out);
if (fs::path{channelName}.filename().string() != channelName) {
throw Error("channelName is not allowed to contain filesystem seperators, got %1%", channelName);
}
try {
fs::create_directories(out);
} catch (fs::filesystem_error &) {
throw SysError("creating directory '%1%'", out.string());
}
unpackTarfile(src, out);
auto entries = std::filesystem::directory_iterator{out};
auto fileName = entries->path().string();
auto fileCount = std::distance(std::filesystem::begin(entries), std::filesystem::end(entries));
size_t fileCount;
std::string fileName;
try {
auto entries = fs::directory_iterator{out};
fileName = entries->path().string();
fileCount = std::distance(fs::begin(entries), fs::end(entries));
} catch (fs::filesystem_error &) {
throw SysError("failed to read directory %1%", out.string());
}
if (fileCount != 1)
throw Error("channel tarball '%s' contains more than one file", src);
std::filesystem::rename(fileName, (out + "/" + channelName));
auto target = out / channelName;
try {
fs::rename(fileName, target);
} catch (fs::filesystem_error &) {
throw SysError("failed to rename %1% to %2%", fileName, target.string());
}
}
}
+3
View File
@@ -402,6 +402,9 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
logger->startWork();
auto pathInfo = [&]() {
// NB: FramedSource must be out of scope before logger->stopWork();
// FIXME: this means that if there is an error
// half-way through, the client will keep sending
// data, since we haven't sent it the error yet.
auto [contentAddressMethod, hashAlgo] = ContentAddressMethod::parseWithAlgo(camStr);
FramedSource source(conn.from);
FileSerialisationMethod dumpMethod;
+1 -1
View File
@@ -333,7 +333,7 @@ static std::string quoteRegexChars(const std::string & raw)
}
#if __linux__
static void readFileRoots(const char * path, UncheckedRoots & roots)
static void readFileRoots(const std::filesystem::path & path, UncheckedRoots & roots)
{
try {
roots[readFile(path)].emplace(path);
+14 -2
View File
@@ -12,6 +12,7 @@
#include <mutex>
#include <thread>
#include <curl/curl.h>
#include <nlohmann/json.hpp>
#ifndef _WIN32
@@ -135,7 +136,7 @@ std::vector<Path> getUserConfigFiles()
std::vector<Path> files;
auto dirs = getConfigDirs();
for (auto & dir : dirs) {
files.insert(files.end(), dir + "/nix/nix.conf");
files.insert(files.end(), dir + "/nix.conf");
}
return files;
}
@@ -363,10 +364,21 @@ void initLibStore(bool loadConfig) {
preloadNSS();
/* Because of an objc quirk[1], calling curl_global_init for the first time
after fork() will always result in a crash.
Up until now the solution has been to set OBJC_DISABLE_INITIALIZE_FORK_SAFETY
for every nix process to ignore that error.
Instead of working around that error we address it at the core -
by calling curl_global_init here, which should mean curl will already
have been initialized by the time we try to do so in a forked process.
[1] https://github.com/apple-oss-distributions/objc4/blob/01edf1705fbc3ff78a423cd21e03dfc21eb4d780/runtime/objc-initialize.mm#L614-L636
*/
curl_global_init(CURL_GLOBAL_ALL);
#if __APPLE__
/* On macOS, don't use the per-session TMPDIR (as set e.g. by
sshd). This breaks build users because they don't have access
to the TMPDIR, in particular in ‘nix-store --serve’. */
#if __APPLE__
if (hasPrefix(defaultTempDir(), "/var/folders/"))
unsetenv("TMPDIR");
#endif
+9 -1
View File
@@ -399,10 +399,18 @@ public:
default is `true`.
)"};
Setting<bool> fsyncStorePaths{this, false, "fsync-store-paths",
R"(
Whether to call `fsync()` on store paths before registering them, to
flush them to disk. This improves robustness in case of system crashes,
but reduces performance. The default is `false`.
)"};
Setting<bool> useSQLiteWAL{this, !isWSL1(), "use-sqlite-wal",
"Whether SQLite should use WAL mode."};
#ifndef _WIN32
// FIXME: remove this option, `fsync-store-paths` is faster.
Setting<bool> syncBeforeRegistering{this, false, "sync-before-registering",
"Whether to call `sync()` before registering a path as valid."};
#endif
@@ -1196,7 +1204,7 @@ public:
If the user is trusted (see `trusted-users` option), when building
a fixed-output derivation, environment variables set in this option
will be passed to the builder if they are listed in [`impureEnvVars`](@docroot@/language/advanced-attributes.md##adv-attr-impureEnvVars).
will be passed to the builder if they are listed in [`impureEnvVars`](@docroot@/language/advanced-attributes.md#adv-attr-impureEnvVars).
This option is useful for, e.g., setting `https_proxy` for
fixed-output derivations and in a multi-user Nix installation, or
+1 -1
View File
@@ -31,7 +31,7 @@ LocalOverlayStore::LocalOverlayStore(std::string_view scheme, PathView path, con
if (checkMount.get()) {
std::smatch match;
std::string mountInfo;
auto mounts = readFile("/proc/self/mounts");
auto mounts = readFile(std::filesystem::path{"/proc/self/mounts"});
auto regex = std::regex(R"((^|\n)overlay )" + realStoreDir.get() + R"( .*(\n|$))");
// Mount points can be stacked, so there might be multiple matching entries.
+2 -2
View File
@@ -77,13 +77,13 @@ The parts of a local overlay store are as follows:
The lower store directory and upper layer directory are combined via OverlayFS to create this directory.
Nix doesn't do this itself, because it typically wouldn't have the permissions to do so, so it is the responsibility of the user to set this up first.
Nix can, however, optionally check that that the OverlayFS mount settings appear as expected, matching Nix's own settings.
Nix can, however, optionally check that the OverlayFS mount settings appear as expected, matching Nix's own settings.
- **Upper SQLite database**:
> Not directly specified.
> The location of the database instead depends on the [`state`](#store-experimental-local-overlay-store-state) setting.
> It is is always `${state}/db`.
> It is always `${state}/db`.
This contains the metadata of all of the upper layer [store objects][store object] (everything beyond their file system objects), and also duplicate copies of some lower layer store object's metadta.
The duplication is so the metadata for the [closure](@docroot@/glossary.md#gloss-closure) of upper layer [store objects][store object] can be found entirely within the upper layer.
+13 -62
View File
@@ -42,7 +42,6 @@
# include <sched.h>
# include <sys/statvfs.h>
# include <sys/mount.h>
# include <sys/ioctl.h>
#endif
#ifdef __CYGWIN__
@@ -366,8 +365,6 @@ LocalStore::LocalStore(
have performed the upgrade already. */
curSchema = getSchema();
if (curSchema < 7) { upgradeStore7(); }
openDB(*state, false);
if (curSchema < 8) {
@@ -1137,7 +1134,7 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
TeeSource wrapperSource { source, hashSink };
narRead = true;
restorePath(realPath, wrapperSource);
restorePath(realPath, wrapperSource, settings.fsyncStorePaths);
auto hashResult = hashSink.finish();
@@ -1191,6 +1188,11 @@ void LocalStore::addToStore(const ValidPathInfo & info, Source & source,
optimisePath(realPath, repair); // FIXME: combine with hashPath()
if (settings.fsyncStorePaths) {
recursiveSync(realPath);
syncParent(realPath);
}
registerValidPath(info);
}
@@ -1271,7 +1273,7 @@ StorePath LocalStore::addToStoreFromDump(
delTempDir = std::make_unique<AutoDelete>(tempDir);
tempPath = tempDir / "x";
restorePath(tempPath.string(), bothSource, dumpMethod);
restorePath(tempPath.string(), bothSource, dumpMethod, settings.fsyncStorePaths);
dumpBuffer.reset();
dump = {};
@@ -1318,7 +1320,7 @@ StorePath LocalStore::addToStoreFromDump(
switch (fim) {
case FileIngestionMethod::Flat:
case FileIngestionMethod::NixArchive:
restorePath(realPath, dumpSource, (FileSerialisationMethod) fim);
restorePath(realPath, dumpSource, (FileSerialisationMethod) fim, settings.fsyncStorePaths);
break;
case FileIngestionMethod::Git:
// doesn't correspond to serialization method, so
@@ -1343,6 +1345,11 @@ StorePath LocalStore::addToStoreFromDump(
optimisePath(realPath, repair);
if (settings.fsyncStorePaths) {
recursiveSync(realPath);
syncParent(realPath);
}
ValidPathInfo info {
*this,
name,
@@ -1596,62 +1603,6 @@ std::optional<TrustedFlag> LocalStore::isTrustedClient()
}
#if defined(FS_IOC_SETFLAGS) && defined(FS_IOC_GETFLAGS) && defined(FS_IMMUTABLE_FL)
static void makeMutable(const Path & path)
{
checkInterrupt();
auto st = lstat(path);
if (!S_ISDIR(st.st_mode) && !S_ISREG(st.st_mode)) return;
if (S_ISDIR(st.st_mode)) {
for (auto & i : readDirectory(path))
makeMutable(path + "/" + i.name);
}
/* The O_NOFOLLOW is important to prevent us from changing the
mutable bit on the target of a symlink (which would be a
security hole). */
AutoCloseFD fd = open(path.c_str(), O_RDONLY | O_NOFOLLOW
#ifndef _WIN32
| O_CLOEXEC
#endif
);
if (fd == INVALID_DESCRIPTOR) {
if (errno == ELOOP) return; // it's a symlink
throw SysError("opening file '%1%'", path);
}
unsigned int flags = 0, old;
/* Silently ignore errors getting/setting the immutable flag so
that we work correctly on filesystems that don't support it. */
if (ioctl(fd, FS_IOC_GETFLAGS, &flags)) return;
old = flags;
flags &= ~FS_IMMUTABLE_FL;
if (old == flags) return;
if (ioctl(fd, FS_IOC_SETFLAGS, &flags)) return;
}
/* Upgrade from schema 6 (Nix 0.15) to schema 7 (Nix >= 1.3). */
void LocalStore::upgradeStore7()
{
if (!isRootUser()) return;
printInfo("removing immutable bits from the Nix store (this may take a while)...");
makeMutable(realStoreDir);
}
#else
void LocalStore::upgradeStore7()
{
}
#endif
void LocalStore::vacuumDB()
{
auto state(_state.lock());
-2
View File
@@ -373,8 +373,6 @@ private:
void updatePathInfo(State & state, const ValidPathInfo & info);
void upgradeStore6();
void upgradeStore7();
PathSet queryValidPathsOld();
ValidPathInfo queryPathInfoOld(const Path & path);
+17 -5
View File
@@ -7,6 +7,7 @@ project('nix-store', 'cpp',
'debug=true',
'optimization=2',
'errorlogs=true', # Please print logs for tests that fail
'localstatedir=/nix/var',
],
meson_version : '>= 1.1',
license : 'LGPL-2.1-or-later',
@@ -324,7 +325,7 @@ fs = import('fs')
prefix = get_option('prefix')
# For each of these paths, assume that it is relative to the prefix unless
# it is already an absolute path (which is the default for store-dir, state-dir, and log-dir).
# it is already an absolute path (which is the default for store-dir, localstatedir, and log-dir).
path_opts = [
# Meson built-ins.
'datadir',
@@ -334,13 +335,13 @@ path_opts = [
'libexecdir',
# Homecooked Nix directories.
'store-dir',
'state-dir',
'localstatedir',
'log-dir',
]
# For your grepping pleasure, this loop sets the following variables that aren't mentioned
# literally above:
# store_dir
# state_dir
# localstatedir
# log_dir
# profile_dir
foreach optname : path_opts
@@ -364,12 +365,12 @@ lsof = find_program('lsof', required : false)
# Aside from prefix itself, each of these was made into an absolute path
# by joining it with prefix, unless it was already an absolute path
# (which is the default for store-dir, state-dir, and log-dir).
# (which is the default for store-dir, localstatedir, and log-dir).
cpp_str_defines = {
'NIX_PREFIX': prefix,
'NIX_STORE_DIR': store_dir,
'NIX_DATA_DIR': datadir,
'NIX_STATE_DIR': state_dir / 'nix',
'NIX_STATE_DIR': localstatedir / 'nix',
'NIX_LOG_DIR': log_dir,
'NIX_CONF_DIR': sysconfdir / 'nix',
'NIX_MAN_DIR': mandir,
@@ -421,4 +422,15 @@ install_headers(headers, subdir : 'nix', preserve_path : true)
libraries_private = []
extra_pkg_config_variables = {
'storedir' : get_option('store-dir'),
}
# Working around https://github.com/mesonbuild/meson/issues/13584
if host_machine.system() != 'darwin'
extra_pkg_config_variables += {
'localstatedir' : get_option('localstatedir'),
}
endif
subdir('build-utils-meson/export')
-4
View File
@@ -16,10 +16,6 @@ option('store-dir', type : 'string', value : '/nix/store',
description : 'path of the Nix store',
)
option('state-dir', type : 'string', value : '/nix/var',
description : 'path to store state in for Nix',
)
option('log-dir', type : 'string', value : '/nix/var/log/nix',
description : 'path to store logs in for Nix',
)
+1 -1
View File
@@ -87,7 +87,7 @@ public:
Sync<State> _state;
NarInfoDiskCacheImpl(Path dbPath = getCacheDir() + "/nix/binary-cache-v6.sqlite")
NarInfoDiskCacheImpl(Path dbPath = getCacheDir() + "/binary-cache-v6.sqlite")
{
auto state(_state.lock());
+1 -1
View File
@@ -49,7 +49,7 @@ struct RemoteStore::ConnectionHandle
RemoteStore::Connection & operator * () { return *handle; }
RemoteStore::Connection * operator -> () { return &*handle; }
void processStderr(Sink * sink = 0, Source * source = 0, bool flush = true);
void processStderr(Sink * sink = 0, Source * source = 0, bool flush = true, bool block = true);
void withFramedSink(std::function<void(Sink & sink)> fun);
};
+8 -34
View File
@@ -153,9 +153,9 @@ RemoteStore::ConnectionHandle::~ConnectionHandle()
}
}
void RemoteStore::ConnectionHandle::processStderr(Sink * sink, Source * source, bool flush)
void RemoteStore::ConnectionHandle::processStderr(Sink * sink, Source * source, bool flush, bool block)
{
handle->processStderr(&daemonException, sink, source, flush);
handle->processStderr(&daemonException, sink, source, flush, block);
}
@@ -926,43 +926,17 @@ void RemoteStore::ConnectionHandle::withFramedSink(std::function<void(Sink & sin
{
(*this)->to.flush();
std::exception_ptr ex;
/* Handle log messages / exceptions from the remote on a separate
thread. */
std::thread stderrThread([&]()
{
try {
ReceiveInterrupts receiveInterrupts;
processStderr(nullptr, nullptr, false);
} catch (...) {
ex = std::current_exception();
}
});
Finally joinStderrThread([&]()
{
if (stderrThread.joinable()) {
stderrThread.join();
if (ex) {
try {
std::rethrow_exception(ex);
} catch (...) {
ignoreException();
}
}
}
});
{
FramedSink sink((*this)->to, ex);
FramedSink sink((*this)->to, [&]() {
/* Periodically process stderr messages and exceptions
from the daemon. */
processStderr(nullptr, nullptr, false, false);
});
fun(sink);
sink.flush();
}
stderrThread.join();
if (ex)
std::rethrow_exception(ex);
processStderr(nullptr, nullptr, false);
}
}
+30 -3
View File
@@ -3,6 +3,7 @@
#include "current-process.hh"
#include "environment-variables.hh"
#include "util.hh"
#include "exec.hh"
namespace nix {
@@ -44,6 +45,10 @@ void SSHMaster::addCommonSSHOpts(Strings & args)
if (compress)
args.push_back("-C");
// We use this to make ssh signal back to us that the connection is established.
// It really does run locally; see createSSHEnv which sets up SHELL to make
// it launch more reliably. The local command runs synchronously, so presumably
// the remote session won't be garbled if the local command is slow.
args.push_back("-oPermitLocalCommand=yes");
args.push_back("-oLocalCommand=echo started");
}
@@ -56,6 +61,27 @@ bool SSHMaster::isMasterRunning() {
return res.first == 0;
}
Strings createSSHEnv()
{
// Copy the environment and set SHELL=/bin/sh
std::map<std::string, std::string> env = getEnv();
// SSH will invoke the "user" shell for -oLocalCommand, but that means
// $SHELL. To keep things simple and avoid potential issues with other
// shells, we set it to /bin/sh.
// Technically, we don't need that, and we could reinvoke ourselves to print
// "started". Self-reinvocation is tricky with library consumers, but mostly
// solved; refer to the development history of nixExePath in libstore/globals.cc.
env.insert_or_assign("SHELL", "/bin/sh");
Strings r;
for (auto & [k, v] : env) {
r.push_back(k + "=" + v);
}
return r;
}
std::unique_ptr<SSHMaster::Connection> SSHMaster::startCommand(
Strings && command, Strings && extraSshArgs)
{
@@ -104,8 +130,8 @@ std::unique_ptr<SSHMaster::Connection> SSHMaster::startCommand(
}
args.splice(args.end(), std::move(command));
execvp(args.begin()->c_str(), stringsToCharPtrs(args).data());
auto env = createSSHEnv();
nix::execvpe(args.begin()->c_str(), stringsToCharPtrs(args).data(), stringsToCharPtrs(env).data());
// could not exec ssh/bash
throw SysError("unable to execute '%s'", args.front());
@@ -172,7 +198,8 @@ Path SSHMaster::startMaster()
if (verbosity >= lvlChatty)
args.push_back("-v");
addCommonSSHOpts(args);
execvp(args.begin()->c_str(), stringsToCharPtrs(args).data());
auto env = createSSHEnv();
nix::execvpe(args.begin()->c_str(), stringsToCharPtrs(args).data(), stringsToCharPtrs(env).data());
throw SysError("unable to execute '%s'", args.front());
}, options);
+15 -4
View File
@@ -822,14 +822,25 @@ StorePathSet Store::queryValidPaths(const StorePathSet & paths, SubstituteFlag m
auto doQuery = [&](const StorePath & path) {
checkInterrupt();
queryPathInfo(path, {[path, &state_, &wakeup](std::future<ref<const ValidPathInfo>> fut) {
auto state(state_.lock());
bool exists = false;
std::exception_ptr newExc{};
try {
auto info = fut.get();
state->valid.insert(path);
exists = true;
} catch (InvalidPath &) {
} catch (...) {
state->exc = std::current_exception();
newExc = std::current_exception();
}
auto state(state_.lock());
if (exists)
state->valid.insert(path);
if (newExc)
state->exc = newExc;
assert(state->left);
if (!--state->left)
wakeup.notify_one();
@@ -1304,7 +1315,7 @@ ref<Store> openStore(StoreReference && storeURI)
/* If /nix doesn't exist, there is no daemon socket, and
we're not root, then automatically set up a chroot
store in ~/.local/share/nix/root. */
auto chrootStore = getDataDir() + "/nix/root";
auto chrootStore = getDataDir() + "/root";
if (!pathExists(chrootStore)) {
try {
createDirs(chrootStore);
@@ -102,14 +102,7 @@ void handleDiffHook(
}
}
// We want $HOME to be un-creatable in the sandbox. On Linux,
// you can't create anything inside /proc since it's a virtual filesystem.
// On Darwin it seems that `/homeless-shelter` is good enough.
#if __linux__
const Path LocalDerivationGoal::homeDir = "/proc/homeless-shelter";
#else
const Path LocalDerivationGoal::homeDir = "/homeless-shelter";
#endif
LocalDerivationGoal::~LocalDerivationGoal()
@@ -451,25 +444,22 @@ void LocalDerivationGoal::startBuilder()
#if __linux__
experimentalFeatureSettings.require(Xp::Cgroups);
/* If we're running from the daemon, then this will return the
root cgroup of the service. Otherwise, it will return the
current cgroup. */
auto rootCgroup = getRootCgroup();
auto cgroupFS = getCgroupFS();
if (!cgroupFS)
throw Error("cannot determine the cgroups file system");
auto ourCgroups = getCgroups("/proc/self/cgroup");
auto ourCgroup = ourCgroups[""];
if (ourCgroup == "")
throw Error("cannot determine cgroup name from /proc/self/cgroup");
auto ourCgroupPath = canonPath(*cgroupFS + "/" + ourCgroup);
if (!pathExists(ourCgroupPath))
throw Error("expected cgroup directory '%s'", ourCgroupPath);
auto rootCgroupPath = canonPath(*cgroupFS + "/" + rootCgroup);
if (!pathExists(rootCgroupPath))
throw Error("expected cgroup directory '%s'", rootCgroupPath);
static std::atomic<unsigned int> counter{0};
cgroup = buildUser
? fmt("%s/nix-build-uid-%d", ourCgroupPath, buildUser->getUID())
: fmt("%s/nix-build-pid-%d-%d", ourCgroupPath, getpid(), counter++);
? fmt("%s/nix-build-uid-%d", rootCgroupPath, buildUser->getUID())
: fmt("%s/nix-build-pid-%d-%d", rootCgroupPath, getpid(), counter++);
debug("using cgroup '%s'", *cgroup);
@@ -1993,7 +1983,7 @@ void LocalDerivationGoal::runChild()
throw SysError("changing into '%1%'", tmpDir);
/* Close all other file descriptors. */
unix::closeMostFDs({STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO});
unix::closeExtraFDs();
#if __linux__
linux::setPersonality(drv->platform);
@@ -3007,6 +2997,7 @@ void LocalDerivationGoal::deleteTmpDir(bool force)
might have privileged stuff (like a copy of netrc). */
if (settings.keepFailed && !force && !drv->isBuiltin()) {
printError("note: keeping build directory '%s'", tmpDir);
chmod(topTmpDir.c_str(), 0755);
chmod(tmpDir.c_str(), 0755);
}
else
+11 -4
View File
@@ -32,7 +32,8 @@ static Logger::Fields readFields(Source & from)
return fields;
}
std::exception_ptr WorkerProto::BasicClientConnection::processStderrReturn(Sink * sink, Source * source, bool flush)
std::exception_ptr
WorkerProto::BasicClientConnection::processStderrReturn(Sink * sink, Source * source, bool flush, bool block)
{
if (flush)
to.flush();
@@ -41,6 +42,9 @@ std::exception_ptr WorkerProto::BasicClientConnection::processStderrReturn(Sink
while (true) {
if (!block && !from.hasData())
break;
auto msg = readNum<uint64_t>(from);
if (msg == STDERR_WRITE) {
@@ -95,8 +99,10 @@ std::exception_ptr WorkerProto::BasicClientConnection::processStderrReturn(Sink
logger->result(act, type, fields);
}
else if (msg == STDERR_LAST)
else if (msg == STDERR_LAST) {
assert(block);
break;
}
else
throw Error("got unknown message type %x from Nix daemon", msg);
@@ -130,9 +136,10 @@ std::exception_ptr WorkerProto::BasicClientConnection::processStderrReturn(Sink
}
}
void WorkerProto::BasicClientConnection::processStderr(bool * daemonException, Sink * sink, Source * source, bool flush)
void WorkerProto::BasicClientConnection::processStderr(
bool * daemonException, Sink * sink, Source * source, bool flush, bool block)
{
auto ex = processStderrReturn(sink, source, flush);
auto ex = processStderrReturn(sink, source, flush, block);
if (ex) {
*daemonException = true;
std::rethrow_exception(ex);
Loaded 100 of 208 files, more files were not shown because too many files have changed in this diff. Show more