Compare commits

...
Author SHA1 Message Date
Eelco Dolstra 4816b640a2 Fix HUP detection on macOS
As of macOS 15.4, passing 0 or POLLHUP doesn't seem to work at all for
sockets any more (though it does work for `notifyPipe`). As a
workaround, also pass POLLIN. That does cause us to receive a bunch of
POLLIN events we don't care about, so we sleep for a bit when receiving
POLLIN.
2025-08-27 21:11:05 +02:00
John Ericson 79211b6110 Merge pull request #13846 from obsidiansystems/derivation-builder-params-aggregate-initialize
No more `DerivationBuilderParams:` constructor!
2025-08-27 12:30:49 -04:00
John Ericson f5f9e32f54 No more DerivationBuilderParams: constructor!
I am not sure how/why this started working. C++23?
2025-08-27 11:40:02 -04:00
Jörg Thalheim 564593bcb9 Merge pull request #13837 from xokdvium/bump-nixpkgs
flake: Update nixpkgs
2025-08-27 09:33:04 +02:00
Sergei Zimmerman 8ee74792fe Merge pull request #13819 from obsidiansystems/relative-url
Implement `parseURLRelative`, use in `HttpBinaryCacheStore`
2025-08-27 03:34:57 +03:00
John Ericson e82210b3b2 Implement parseURLRelative, use in HttpBinaryCacheStore
This allows us to replace some very hacky and not correct string
concatentation in `HttpBinaryCacheStore`. It will especially be useful
with #13752, when today's hacks started to cause problems in practice,
not just theory.

Also make `fixGitURL` returned a `ParsedURL`.
2025-08-26 19:45:10 -04:00
Sergei Zimmerman 625477a7df flake: Update nixpkgs
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
  → 'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26)

This update contains d1266642a8722f2a05e311fa151c1413d2b9653c, which
is necessary for the TOML timestamps to get tested via nixpkgsLibTests job.
2025-08-27 02:23:05 +03:00
Sergei Zimmerman 231f3af535 Merge pull request #13835 from obsidiansystems/better-string-split
Better `stringSplit`
2025-08-27 01:20:46 +03:00
John EricsonandSergei Zimmerman cc4aa70e6e Better stringSplit
I need this for some `ParseURL` improvements, but I figure this is
better to send as its own PR.

I changed the tests willy-nilly to sometimes use
`std::list<std::string_view>` instead of `Strings` (which is
`std::list<std::string>`).

Co-Authored-By: Sergei Zimmerman <sergei@zimmerman.foo>
2025-08-26 18:03:23 -04:00
John Ericson 0bd9d6a28e Merge pull request #13832 from kip93/fix/empty-ports
Handle empty ports with new URL parsing
2025-08-26 13:55:59 -04:00
Leandro Reina 7989e3192d Handle empty ports 2025-08-26 17:41:27 +02:00
Robert Hensing 1e16a54ee5 Merge pull request #13828 from NixOS/readme-meeting-times
Update work meeting time in README
2025-08-26 01:37:07 +02:00
Robert Hensing afade27123 Update work meeting time in README 2025-08-26 00:50:12 +02:00
Robert Hensing c1e2396d58 Merge pull request #13826 from xokdvium/sqlite-zfs-hack
SQLite: fsync db.sqlite-shm before opening the database
2025-08-26 00:05:40 +02:00
John Ericson ca94905593 Merge pull request #13825 from obsidiansystems/slight-optimize-s3ToHttpsURL
`ParsedS3URL::toHttpsUrl` Slight optimize
2025-08-25 17:51:09 -04:00
Eelco Dolstra e492c64c8e SQLite: fsync db.sqlite-shm before opening the database
This is a workaround for https://github.com/NixOS/nix/issues/13515
(opening the SQLite DB randomly taking a couple of seconds on ZFS).

(cherry picked from commit a7fceb5eec)
2025-08-26 00:42:18 +03:00
John Ericson e4e8a615fa ParsedS3URL::toHttpsUrl Slight optimize
I didn't want to block that PR on further code review while I figured
out these new (to us) C++23 goodies.
2025-08-25 16:53:39 -04:00
John Ericson fac34ad20f Merge pull request #13824 from xokdvium/fix-formatting
libexpr: Fix weird formatting after treewide reformat
2025-08-25 15:26:17 -04:00
John Ericson 024d3954af Merge pull request #13823 from lovesegfault/extract-s3ToHttpsURL
feat(libstore/s3): add toHttpsUrl
2025-08-25 15:11:38 -04:00
Sergei Zimmerman f0e4af4365 libexpr: Fix weird formatting after treewide reformat 2025-08-25 22:09:18 +03:00
Bernardo Meurer Costa 5985d67906 feat(libstore/s3): add toHttpsUrl
This is extracted from the work in #13752
2025-08-25 18:48:19 +00:00
Eelco Dolstra 9bee0fa6ac Merge pull request #13822 from NixOS/bump-2.32.0
Bump version to 2.32
2025-08-25 17:22:57 +02:00
Eelco Dolstra adec28bf85 Update release-process.md 2025-08-25 10:30:21 +02:00
Eelco Dolstra f5e09d9b58 Update mergify.yml 2025-08-25 10:28:47 +02:00
Eelco Dolstra f67daa4a87 Bump version 2025-08-25 10:27:46 +02:00
John Ericson c9211b0b2d Merge pull request #13803 from obsidiansystems/more-parsed-urls
Make more URLs parsed, most notably `FileTransferRequest::url`
2025-08-23 10:54:39 -04:00
John Ericson 2fa2c0b09f Merge pull request #13812 from obsidiansystems/url-parse-leniency
Limit to lenient parsing of non-standard URLs only where needed
2025-08-23 10:53:59 -04:00
Jörg Thalheim ebf1cf5227 Merge pull request #13807 from roberth/release-notes-todo
maintainers: Add script for release notes todo list
2025-08-23 08:52:22 +02:00
John Ericson 3e86d75c9d Make more URLs parsed, most notably FileTransferRequest::url
Trying to gradually replace the use of strings with better types in ways
that makes sense.
2025-08-22 12:42:48 -04:00
John Ericson 72a548ed6a Limit to lenient parsing of non-standard URLs only where needed
This allows us to put `parseURL` in more spots without furthering
technical debt.
2025-08-22 12:37:37 -04:00
John Ericson 4083eff0c0 decodeQuery Take std::string_view not string ref 2025-08-22 12:26:48 -04:00
Eelco Dolstra 9c6019dd1a Merge pull request #13804 from NixOS/release-notes
Release notes for Nix 2.31
2025-08-22 17:18:15 +02:00
Eelco Dolstra 82d3662f09 Tweak 2025-08-22 11:03:05 +02:00
Sergei Zimmerman 01691e1180 Merge pull request #13810 from hgl/patch-1 2025-08-22 11:49:41 +03:00
Glen Huang c0246460a0 doc: Fix typo 2025-08-22 15:39:18 +08:00
John Ericson 0c46c2c37f Mention experimental SHA-256 git hashing in release notes 2025-08-21 23:17:12 -04:00
John Ericson 58278974b6 Merge pull request #13809 from xokdvium/enumerate
libutil: Replace hand-rolled enumerate with std::views::{zip,iota}
2025-08-21 22:58:19 -04:00
Sergei Zimmerman b853994e7a libutil: Replace hand-rolled enumerate with std::views::{zip,iota}
It would have been nice to use std::views::enumerate here, but
it uses a signed difference type for the value_type:

> value_type = std::tuple<difference_type, ranges::range_value_t<Base>>

zip + iota has the same semantics as the code used to have, so there's
no behavior change here.
2025-08-22 03:10:33 +03:00
Robert Hensing a1b3934a78 maintainers: Add script for release notes todo list 2025-08-21 14:19:22 +02:00
Eelco Dolstra ae09483498 Add more release notes 2025-08-21 13:37:38 +02:00
Eelco Dolstra 4dcfb36c1e Cleanup 2025-08-21 13:37:38 +02:00
Eelco Dolstra 2e7bb61a83 Document that you need to set GITHUB_TOKEN 2025-08-21 13:37:38 +02:00
Eelco Dolstra 0eaed891f4 Update release credits 2025-08-21 13:37:38 +02:00
Eelco Dolstra 6c391b7446 release notes: 2.31.0 2025-08-21 13:37:38 +02:00
Eelco Dolstra d7612f350f Merge pull request #13806 from NixOS/fix-meson-format
Fix meson formatting error
2025-08-21 13:31:50 +02:00
Eelco Dolstra 4fe700b7cb Fix meson-format 2025-08-21 13:17:06 +02:00
Eelco Dolstra 3826d51a65 Merge pull request #13565 from NixOS/prefetch-inputs
Add `nix flake prefetch-inputs` command
2025-08-21 12:30:17 +02:00
Eelco Dolstra 615b10cb44 Merge pull request #13800 from NixOS/concurrent-eval-cache
Use WAL mode for SQLite cache databases
2025-08-21 11:42:15 +02:00
John Ericson e2b984704a Merge pull request #13801 from obsidiansystems/move-sandbox-path-closure-code
Make sure `settings.sandboxedPaths` is closed outside `DerivationBuilder`
2025-08-20 19:15:16 -04:00
John Ericson 4c76db8e7c Make sure settings.sandboxedPaths is closed outside DerivationBuilder
This is a nicer separation of concerns --- `DerivationBuilder` just
mounts the extra paths you tell it too, and the outside world is
responsible for making sure those extra paths make sense.

Since the closure only depends on global settings, and not
per-derivation information, we also have the option of moving this up
further and caching it across all local builds. (I only just realized
this after having done this refactor. I am not doing that change at this
time, however.)
2025-08-20 18:49:11 -04:00
John Ericson 08e42e20fa Merge pull request #13769 from obsidiansystems/simplify-derivation-building-goal
Handle structured attrs, "export references graph" outside of `DerivationBuilder`
2025-08-20 18:05:04 -04:00
John Ericson 1d3ddb21fa Further consolidate environment variable processing outside DerivationBuilder
Now, `DerivationBuilder` only concerns itself with `finalEnv` and
`extraFiles`, in straightforward unconditional code. All the fancy
desugaring logic is consolidated in `DerivationBuildingGoal`.

We should better share the pulled-out logic with `nix-shell`/`nix
develop`, which would fill in some missing features, arguably fixing
bugs.
2025-08-20 16:54:17 -04:00
John Ericson e3c74f5a13 Desugar structured attrs, "export reference graph" outside DerivationBuilder
I think this is a better separation of concerns. `DerivationBuilder`
doesn't need to to the final, query-heavy details about how these things
are constructed. It just operates on the level of "simple, stupid" files
and environment variables.
2025-08-20 16:54:17 -04:00
John Ericson 92b10cf3f5 DerivationBuilderImpl::writeStructuredAttrs remove a rewrite
As much as I prefer rewriting the parsed rather than unparsed JSON for
elegance, this gets in the way of the separation of concerns that I am
trying to do.

As a practical matter, any rewriting that this did will also be done by
the second round of rewriting that remains below, so removing this code
should have no effect.
2025-08-20 16:54:17 -04:00
John Ericson 2767ae35d9 Deduplicate "export reference graph" logic a bit
The first part on `drvOptions.exportReferencesGraph` is the same in both
cases. It is just how the information is finally rendered that is
different.
2025-08-20 16:54:17 -04:00
John Ericson ca86d34077 Merge pull request #13799 from obsidiansystems/typed-sandbox-paths
Make `sandbox-settings` better typed, get `globals.hh` out of other headers
2025-08-20 16:50:34 -04:00
John Ericson a712445a7a Make Settings::sandboxPaths well-typed
Parsing logic is moved from `DerivationBuilder`, where is doesn't
belong, to `Settings` itself, where it does.
2025-08-20 16:24:37 -04:00
John Ericson 52212635db No more globals.hh in headers
This is needed to rearrange include order, but I also think it is a good
thing anyways, as we seek to reduce the use of global settings variables
over time.
2025-08-20 16:24:37 -04:00
John Ericson bce29ab2cf Move header outside Unix-only portion
We'll neeed some definitions elsewhere
2025-08-20 16:24:37 -04:00
John Ericson 8463fef161 Expose ChrootPath, PathsInChroot
Will want these for settings in a moment.
2025-08-20 16:24:37 -04:00
John Ericson d53c7b816b Push #include down to .cc file
That is where it should be.
2025-08-20 16:24:37 -04:00
John Ericson 4ab579b469 Remove constructor from ChrootPath
I rather use designated initializers.
2025-08-20 16:24:36 -04:00
Eelco Dolstra 0df147b145 Bump the version of the SQLite caches
This avoids problems with older versions of Nix that don't put the
caches in WAL mode. That's generally not a problem, until you do something like

  nix build --print-out-paths ... | cachix

which deadlocks because cachix tries to switch the caches to truncate
mode, which requires exclusive access. But the first process cannot
make progress because the cachix process isn't reading from the pipe.
2025-08-20 20:44:58 +02:00
Eelco Dolstra 4ab8ff5b4c SQLite: Use std::filesystem::path 2025-08-20 20:44:21 +02:00
Eelco Dolstra 349d2c58e5 Use WAL mode for SQLite cache databases
With "truncate" mode, if we try to write to the database while another
process has an active write transaction, we'll block until the other
transaction finishes. This is a problem for the evaluation cache in
particular, since it uses long-running transactions.

WAL mode does not have this issue: it just returns "busy" right away,
so Nix will print

  error (ignored): SQLite database '/home/eelco/.cache/nix/eval-cache-v5/...' is busy

and stop trying to write to the evaluation cache. (This was the
intended/original behaviour, see AttrDb::doSQLite().)
2025-08-20 20:34:43 +02:00
Jörg Thalheim bb600e1048 Merge pull request #13796 from NixOS/onboarding-room-list
onboarding.md: List the private rooms
2025-08-19 23:47:17 +02:00
John Ericson 088cc176f1 Merge pull request #13795 from xokdvium/factor-out-s3url
libstore: Introduce ParsedS3URL type
2025-08-19 17:28:52 -04:00
Robert Hensing 4134258c03 onboarding.md: List the private rooms
We forgot one in the latest onboarding
2025-08-19 23:00:32 +02:00
Sergei ZimmermanandBernardo Meurer Costa 69fcc2cfc1 libstore: Introduce ParsedS3URL type
This systematizes the way our s3:// URLs are parsed in filetransfer.cc.
Yoinked out and refactored out of [1].

[1]: https://github.com/NixOS/nix/pull/13752

Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
2025-08-19 23:39:18 +03:00
John Ericson a92a996dba Merge pull request #13793 from xokdvium/c++23
treewide: Bump C++ standard version to C++23
2025-08-19 12:37:17 -04:00
Jörg Thalheim 5c0eff24d5 Merge pull request #13784 from NixOS/queryPathInfo-dont-disconnect
Fix client disconnect when queryPathInfo() returns a negative result
2025-08-19 17:16:47 +02:00
Sergei Zimmerman 03101cc839 treewide: Bump C++ standard version to C++23
Compilers in nixpkgs have caught up and major distros
should also have recent enough compilers. It would be
nice to have newer features like more full featured
ranges and deducing this.
2025-08-19 15:33:27 +03:00
tomberek f3bd18f2b4 Merge pull request #13790 from xokdvium/fix-ci
libstore: Fix formatting
2025-08-18 16:49:00 -04:00
Sergei Zimmerman 62018b3a7f libstore: Fix formatting
Wasn't caught by CI because the PR wasn't fresh enough
and didn't have formatting checks enabled.
2025-08-18 23:24:10 +03:00
tomberek dc2478fe79 Merge pull request #13340 from mkenigs/better-error
Improve error when can't acquire db lock
2025-08-18 15:27:58 -04:00
Sergei Zimmerman 8283a20c6a Merge pull request #13789 from xokdvium/nix-2.3-insecure-fix
hydra: Fix otherNixes.nix_2_3
2025-08-18 21:58:35 +03:00
Sergei Zimmerman 051290b155 hydra: Fix otherNixes.nix_2_3
25.05 has it marked as insecure, but we don't care about it
for testing purposes.
2025-08-18 21:56:20 +03:00
Sergei Zimmerman b263bfcecd Merge pull request #13785 from xokdvium/25.05-nixpkgs
flake: nixpkgs: nixos-unstable -> nixos-25.05-small
2025-08-18 20:57:46 +03:00
Sergei Zimmerman 73f6729147 git-blame-ignore-revs: Add nixfmt 1.0.0 reformat 2025-08-18 20:30:54 +03:00
Sergei Zimmerman 1d943f5819 flake: Apply nixfmt 1.0.0 2025-08-18 20:29:45 +03:00
Robert Hensing aa0dc0d7e7 Merge pull request #13757 from fzakaria/issue-13215
libfetchers/git: add support for '.' in gitmodules
2025-08-18 19:16:54 +02:00
Sergei Zimmerman fc33681583 flake: nixpkgs: nixos-unstable -> nixos-25.05-small
About time we upgraded our nixpkgs flake input. Ideally
we'd have automation to do this.

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/adaa24fbf46737f3f1b5497bf64bae750f82942e?narHash=sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY%3D' (2025-05-13)
  → 'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
2025-08-18 20:04:57 +03:00
Eelco Dolstra f51779ee25 RemoteStore::addToStoreFromDump(): Invalidate cache entry for added path 2025-08-18 18:12:42 +02:00
Eelco Dolstra c82b67fa05 BasicClientConnection::queryPathInfo(): Don't throw exception for invalid paths
This caused RemoteStore::queryPathInfoUncached() to mark the
connection as invalid (see
RemoteStore::ConnectionHandle::~ConnectionHandle()), causing it to
disconnect and reconnect after every lookup of an invalid path. This
caused huge slowdowns in conjunction with
19f89eb684 and lazy-trees.
2025-08-18 18:12:42 +02:00
Eelco Dolstra 48b600d995 Merge pull request #13783 from xokdvium/lowdown-cross
packaging: Fix lowdown cross
2025-08-18 13:17:13 +02:00
Sergei Zimmerman 0216775ab7 packaging: Fix lowdown cross
Stupid mistake on my part. `binaryTarballCross.x86_64-linux.x86_64-unknown-freebsd` now
builds as expected.
2025-08-18 12:57:04 +03:00
Jörg Thalheim d48e04b05a Merge pull request #13780 from xokdvium/lowdown-2
libcmd: Fix rendering of absolute URLs in markdown, update lowdown to 2.0.2
2025-08-18 08:54:05 +02:00
Farid Zakaria b21304fe4c libfetchers/git: Add support for '.' in gitsubmodules
Period '.' is a special branch name in the gitsubmodule file which
represents the branch of the parent repository [1].

We add support for this by registering the ref of the InputAccessor to
be that of the parent input if '.' is encountered.

Fixes #13215

[1]: man gitmodules
2025-08-18 00:28:20 +03:00
Sergei Zimmerman 3ffeed497a packaging: Update lowdown to 2.0.2
Nixpkgs has been updated with this version and seems
like next stable nixos release will ship with lowdown 2.0.
2025-08-17 20:49:50 +03:00
Sergei Zimmerman da8759bb41 libcmd: Fix rendering of absolute URLs in markdown
lowdown >= 1.4.0 supports LOWDOWN_TERM_NORELLINK to render
absolute urls. This is useful, since we want to keep links to
web resources and such intact.
2025-08-17 20:31:34 +03:00
Jörg Thalheim f6db80a553 Merge pull request #13779 from vog/patch-1
Fix NIX_CONF_DIR description
2025-08-17 18:20:37 +02:00
Volker Diels-Grabsch 1d7ac6603c Fix NIX_CONF_DIR description
Fix description of `NIX_CONF_DIR`. It currently say that it defaults to `prefix/etc/nix`, which would mean `/nix/etc/nix` on default installations, and contradicts the description in `conf-file-prefix.md`.

This fix makes the description of `NIX_CONF_DIR` consistent with `conf-file-prefix.md`, assuming that the latter is correct.
2025-08-17 17:28:46 +02:00
Sergei Zimmerman 4284497d98 Merge pull request #13778 from xokdvium/fake-references-in-benchmark-drvs
libstore-tests: Obfuscate derivation paths in drv files for benchmarks
2025-08-17 17:25:12 +03:00
Sergei Zimmerman e244e32308 libstore-tests: Obfuscate derivation paths in drv files for benchmarks
Otherwise this might form a valid reference and nix repo will not be fetched
as a FOD.
2025-08-17 16:57:04 +03:00
Jörg Thalheim 72c8aa8bb1 Merge pull request #13774 from xokdvium/fix-authorization
tests/nixos: Fix daemon store reference in authorization test
2025-08-17 14:48:15 +02:00
Jörg Thalheim 723656daab Merge pull request #13775 from xokdvium/more-pch
treewide: Futher improve compile times for clangStdenv builds
2025-08-17 14:47:41 +02:00
Jörg Thalheim 1cf0fe3186 Merge pull request #13773 from xokdvium/quotes-fragment
libutil: Fix handling of unescaped spaces, quotes and shevrons in queries and fragments
2025-08-17 14:45:42 +02:00
Sergei Zimmerman bb9148fd5d Merge pull request #13777 from PedroManse/master 2025-08-17 12:20:25 +03:00
Manse ccc0a31f39 fix doc code example
Fixed nix shell's doc's shebang python example
2025-08-17 00:39:41 -03:00
Sergei Zimmerman 8be1cc6e96 libstore-tests: Improve compile times with PCH
(Before)

**** Time summary:
Compilation (25 times):
  Parsing (frontend):          145.5 s
  Codegen & opts (backend):     65.9 s

(After)

**** Time summary:
Compilation (26 times):
  Parsing (frontend):           39.9 s
  Codegen & opts (backend):     55.1 s
2025-08-17 01:08:31 +03:00
Sergei Zimmerman 347ed8d9ba libfetchers: Improve compile times with PCH
(Before)

**** Time summary:
Compilation (17 times):
  Parsing (frontend):           72.3 s
  Codegen & opts (backend):     47.8 s

(After)

Compilation (19 times):
  Parsing (frontend):           27.5 s
  Codegen & opts (backend):     43.8 s
2025-08-17 00:57:14 +03:00
Sergei Zimmerman f13aeaf2f1 libutil-tests: Improve compile times with PCH
(Before)

**** Time summary:
Compilation (30 times):
  Parsing (frontend):           98.2 s
  Codegen & opts (backend):     27.7 s

(After)

**** Time summary:
Compilation (31 times):
  Parsing (frontend):           34.9 s
  Codegen & opts (backend):     28.1 s
2025-08-17 00:57:06 +03:00
Sergei Zimmerman d6973e105c libexpr-tests: Improve compile times with PCH
(Before)

**** Time summary:
Compilation (14 times):
  Parsing (frontend):           60.1 s
  Codegen & opts (backend):     18.0 s

(After)

**** Time summary:
Compilation (15 times):
  Parsing (frontend):           16.6 s
  Codegen & opts (backend):     17.4 s
2025-08-17 00:56:54 +03:00
Sergei Zimmerman 3c0a5e0a51 nix-cli: Improve compile times
(Before)

**** Time summary:
Compilation (61 times):
  Parsing (frontend):          143.2 s
  Codegen & opts (backend):    105.8 s

(After)

**** Time summary:
Compilation (61 times):
  Parsing (frontend):          113.8 s
  Codegen & opts (backend):    111.7 s
2025-08-17 00:56:41 +03:00
Sergei Zimmerman 695f3bc7e3 tests/nixos: Fix daemon store reference in authorization test
Recent store reference cleanups have started canonicalizing
daemon -> unix://.
2025-08-16 23:31:57 +03:00
Sergei Zimmerman dc1b2012af libutil: Fix handling of unescaped spaces, quotes and shevrons in queries and fragments
Turns out we didn't have tests for some of the important behavior introduced
for flake reference fragments and url queries [1]. This is rather important
and is relied upon by existing tooling. This fixes up these exact cases before
handing off the URL to the Boost.URL parser.

To the best of my knowledge this implements the same behavior as prior regex-based
parser did [2]:

> fragmentRegex = "(?:" + pcharRegex + "|[/? \"^])*";
> queryRegex = "(?:" + pcharRegex + "|[/? \"])*";

[1]: 9c0a09f09f
[2]: https://github.com/NixOS/nix/blob/2.30.2/src/libutil/include/nix/util/url-parts.hh
2025-08-16 23:00:31 +03:00
John Ericson 0fd9ef0cf3 Merge pull request #13770 from obsidiansystems/delete-wrong-decl
Delete dangling declaration
2025-08-16 11:50:20 -04:00
John Ericson d5102ab30c Merge pull request #13771 from xokdvium/gdb-dev-shell
dev-shell: Add gdb
2025-08-16 11:45:24 -04:00
Sergei Zimmerman 0348030ae2 dev-shell: Add gdb
It's about time we added debuggers to the dev-shell. Having it in build inputs
does some magic so pretty printers for standard library types work better.
2025-08-16 18:27:39 +03:00
John Ericson 048cfb9cd6 Delete dangling declaration
There is no definition associated with this.
2025-08-16 11:27:10 -04:00
Jörg Thalheim 9fb62805b0 Merge pull request #13749 from Mic92/merge-queues
prepare merge queues for nix
2025-08-16 09:47:57 +02:00
John Ericson 3b03872ebf Merge pull request #13766 from obsidiansystems/more-store-dir
Make a few more things use `StoreDirConfig` instead of `Store`
2025-08-15 16:20:39 -04:00
John Ericson 97c966cc69 Merge pull request #13765 from obsidiansystems/simplify-derivation-building-goal
Simplify `DerivationBuildingGoal`
2025-08-15 16:19:44 -04:00
John Ericson 9ccbe23056 DerivationBuilder Change missing initialOutput if..throw to assert
Since this goal has no (goal-wide) notion of "wanted outputs" (we're
building the derivation, and thus making all outputs), we should have
`initialOutputs` for all outputs, and if we're missing one that's an
internal error caused by a bug in Nix.

Concretely, `DerivationBuildingGoal::gaveUpOnSubstitution` now clearly
does create `initialOutputs` for all outputs, whereas a few commits ago
that was not obvious, so I feel confident in saying that this invariant
that should be upheld, in fact is upheld.

`scatchOutputs` is initialized for every initial output, so the same
change to it follows for the same reasons.
2025-08-15 15:50:46 -04:00
John Ericson 870bb68d38 DerivationBuilder::registerOutputs: Inline checkSuffix
It is a simple constant that is only used once.
2025-08-15 15:50:41 -04:00
John Ericson 4bc9ae67c7 Give DerivationBuilder a LocalStore not Store
This is just more honest, since we downcasted it to `LocalStore` in many
places. We had the downcast before because it wasn't needed in the hook
case, just the local building case, but now that `DerivationBuilder` is
separated and just does the building case, we have formalized the
boundary where the single downcast should occur.
2025-08-15 15:50:36 -04:00
John Ericson 14e355d87d Remove InitialOutput::wanted
No derivation goal type has a notion of variable wanted outputs any
more. They either want them all, or they just care about a single
output, in which case we would just store this information for the one
output in question.
2025-08-15 15:50:35 -04:00
John Ericson 79fb9b0d3c Make a few more things use StoreDirConfig instead of Store 2025-08-15 15:35:51 -04:00
John Ericson 316fef35dc Merge pull request #13764 from obsidiansystems/simplify-store-dir
Simplify "Store dir" superclass
2025-08-15 15:24:47 -04:00
Sergei Zimmerman 1ed3ae8a9d Merge pull request #13763 from obsidiansystems/more-flexible-get
More flexible typing for `get` in `util.hh`
2025-08-15 22:09:57 +03:00
John EricsonandSergei Zimmerman 0ef818dd92 More flexible typing for get in util.hh
This is good for e.g. `std::string_view` and `StringMap`.

Needed by #11139

Co-authored-by: Sergei Zimmerman <145775305+xokdvium@users.noreply.github.com>
2025-08-15 14:23:37 -04:00
John Ericson 64c2ee3f45 Simplify "Store dir" superclass
We can cut out some gratuitous inhertence as follows:

- `MixStoreDirMethods` -> `StoreDirConfig`

- `StoreDirConfig` deleted because no longer needed. It is just folded
  into `StoreConfig`.

- `StoreDirConfigBase` -> `StoreConfigBase` same trick still needed, but
  now is for `StoreConfig` not `StoreDirConfig`

Here's how we got here:

1. I once factored out `StoreDirConfig` in #6236.

2. I factored out `MixStoreDirMethods` in #13154.

But, I didn't realize at point (2) that we didn't need `StoreDirConfig`
anymore, all uses of `StoreDirConfig` could instead be uses of
`MixStoreDirMethods`. Now I am doing that, and renaming
`MixStoreDirMethods` to just `StoreDirConfig` to reduce churn.
2025-08-15 14:12:37 -04:00
Sergei Zimmerman 22378ea093 Merge pull request #13762 from xokdvium/delete-footguns
libutil: Delete footgun overloads of get and getOr
2025-08-15 17:58:32 +03:00
Sergei Zimmerman 408c09a120 libutil: Delete footgun overloads of get and getOr
To avoid mistakes like the one in cea85e79ee.
These overloads are just asking for trouble.
2025-08-15 17:29:17 +03:00
Sergei Zimmerman c736db5320 Merge pull request #13761 from xokdvium/fix-dangling-pointers
libstore: Fix dangling pointers in DerivationGoal constructors
2025-08-15 17:03:49 +03:00
Sergei Zimmerman cea85e79ee libstore: Fix dangling pointers in DerivationGoal constructors
This leads to a use-after free, because staticOutputHashes returns a temporary
object that dies before we can do a `return *mOutputHash`.

This is most likely the cause for random failures in Hydra [1].

[1]: https://hydra.nixos.org/build/305091330/nixlog/2
2025-08-15 16:39:28 +03:00
Jörg Thalheim 4e776a5be8 Merge pull request #13753 from obsidiansystems/simplify-derivation-goal
Simplify `DerivationGoal` in many ways
2025-08-15 08:25:47 +02:00
Jörg Thalheim 677b1c0f8f prepare merge queues for nix 2025-08-15 08:05:01 +02:00
Sergei Zimmerman f64000e3f4 Merge pull request #13756 from xokdvium/fix-copy-path-message
libstore: Fix makeCopyPathMessage
2025-08-15 02:10:41 +03:00
Sergei Zimmerman e74ef417db libstore: Fix makeCopyPathMessage
Old code completely ignored query parameters and it seems ok to keep
that behavior. There's a lot of code out there that parses nix code
like nix-output-monitor and it can't parse messages like:

> copying path '/nix/store/wha2hi4yhkjmccqhivxavbfspsg1wrsj-source' from 'https://cache.nixos.org' to 'local://'...

Let's not break these tools without a good reason. This goes in line
with what other code does by ignoring parameters in logs.

The issue is just in detecting the shorthand notations for the store
reference - not in printing the url in logs.

By default the daemon opens a local store with ?path-info-cache-size=0,
so that leads to the erronenous 'local://'.
2025-08-15 00:55:03 +03:00
John Ericson 4b4895e750 Merge pull request #13755 from xokdvium/concise-uri-logs
treewide: Remove getUri and replace with getHumanReadableURI where appropriate
2025-08-14 15:39:09 -04:00
Sergei Zimmerman 1b7ffa53af treewide: Remove getUri and replace with getHumanReadableURI where appropriate
The problem with old code was that it used getUri for both the `diskCache`
as well as logging. This is really bad because it mixes the textual human
readable representation with the caching.

Also using getUri for the cache key is really problematic for the S3 store,
since it doesn't include the `endpoint` in the cache key, so it's totally broken.

This starts separating the logging / cache concerns by introducing a
`getHumanReadableURI` that should only be used for logging. The caching
logic now instead uses `getReference().render(/*withParams=*/false)` exclusively.
This would need to be fixed in follow-ups, because that's really fragile and
broken for some store types (but it was already broken before).
2025-08-14 16:47:05 +03:00
Jörg Thalheim 58eabe7479 Merge pull request #13754 from xokdvium/cleanup-docker-check-secrets
ci: Cleanup docker check secrets
2025-08-14 15:27:46 +02:00
Sergei Zimmerman e6f3a193d8 libstore: Fix makeCopyPathMessage after config getUri refactor 2025-08-14 15:52:24 +03:00
Sergei Zimmerman 26dbda6302 ci/docker_push_image: Remove dead step
This step is now part of the check_secrets job and
the output is completely unused.
2025-08-14 14:54:45 +03:00
Sergei Zimmerman 48d15ed1fb ci/check_secrets: Remove deprecated set-output 2025-08-14 14:54:20 +03:00
John Ericson 4a2de1dbab DerivationGoal Make some fields immutable
We can set both during construction, yay!
2025-08-14 00:39:00 -04:00
John Ericson f155dffe59 DerivationGoal::done Clean up parameter types
We don't need to ask all these callers to build these single-entry maps
for us.
2025-08-14 00:35:52 -04:00
John Ericson c940283750 DerivationBuilder
Move output result filtering logic and assert just into the branch where
it is not obviously a no op / meeting the assertion.

Add a comment too, while we are at it.
2025-08-14 00:35:34 -04:00
John Ericson 14441f9382 DerivationGoal inline gaveUpOnSubstitution lambda
We can shuffle around control flow so it's only called once. You'll
definitely want to review this diff ignoring whitespace.
2025-08-14 00:35:24 -04:00
John Ericson 88275e5723 DerivationGoal slight cleanup of some impure drv logic 2025-08-14 00:16:26 -04:00
John Ericson 7707d0acad Get rid of filterDrvOutputs
We don't need it any more, because we only used it in the
single-wanted-output `DerivationGoal`.
2025-08-14 00:09:08 -04:00
John Ericson 766a52ce87 DerivationOutput: Remove outputKnown state
Now that `DerivationGoal::checkPathValidity` is legible, we can see that
it only sets `outputKnown`, and doesn't read it. Likewise, with
co-routines, we don't have tiny scopes that make local variables
difficult. Between these two things, we can simply have
`checkPathValidity` return what it finds, rather than mutate some state,
and update everyting to use local variables.

The same transformation could probably be done to the other derivation
goal types (which currently, unfortunately, contain their own
`checkPathValidity`s, though they are diverging, and we hope and believe
that they continue to diverge).
2025-08-13 23:59:06 -04:00
John Ericson 2324fe3515 DerivationBuilder::checkPathValidity: Big simplify
`Store::queryPartialDerivationOutputMap` is nothing but checking
statically-known output paths, and then `Store::queryRealisation`, and
we were doing both of those things already. Inline that and simplify,
again taking advantage of the fact that we only care about one output.
2025-08-13 23:23:11 -04:00
John Ericson b6ca60cb82 DerivationBuilder::checkPathValidity: Simplify allValid calc
Now that the loops is gone, we can just inline this mutation to a single
simple expression.
2025-08-13 23:01:58 -04:00
John Ericson 2600391147 Simplify DerivationGoal loop -> if
More taking advantage of single wanted output. Also `auto *` not `auto`
for easy reading.
2025-08-13 22:44:10 -04:00
John Ericson 1a6f92837a Don't use InitialOutput in DerivationGoal
We don't need the `wanted` field. Just inline the other two fields.
2025-08-13 22:43:33 -04:00
John Ericson 14173d761c Simplify DerivationGoal by just storing a singular initialOutput
We know we want exactly want output in `DerivationGoal` now (since
recent refactors), so we can start simplifying things to take advantage
of this.
2025-08-13 22:07:59 -04:00
John Ericson 4b6edfcfc7 DerivationBuildingGoal: Check outputs beforehand
See the comment in the code for details. Some of the code is duplicated
for now, but we'll be cleaning that up soon.
2025-08-13 22:06:14 -04:00
Sergei Zimmerman cf7084a67c Merge pull request #13739 from obsidiansystems/getUri-not-string
Rewrite `StoreConfig::getUri` in terms of new `StoreConfig::getReference`
2025-08-14 02:46:48 +03:00
John Ericson 3e7879e6df Rewrite StoreConfig::getUri in terms of new StoreConfig::getReference
Rather than having store implementations return a free-form URI string,
have them return a `StoreReference`. This reflects that fact that this
method is supposed to invert `resolveStoreConfig`, which goes from a
`StoreReference` to some `StoreConfig` concrete derived class (based on
the registry).

`StoreConfig::getUri` is kept only as a convenience for the common case
that we want to immediately render the `StoreReference`.

A few tests were changed to use `local://` not `local`, since
`StoreReference` does not encode the `local` and `daemon` shorthands
(and instead desugars them to `local://` and `unix://` right away). I
think that is fine. `local` and `daemon` still work as input.
2025-08-13 19:06:59 -04:00
Sergei Zimmerman dfcbe70564 Merge pull request #13750 from obsidiansystems/simplify-derivation-goal-0
Simplify `DerivationGoal` in two ways
2025-08-14 02:02:03 +03:00
John Ericson ed55937002 Make many members of DerivationGoal private 2025-08-13 17:50:39 -04:00
John Ericson c37df9c87c Inline DerivationGoal::query{,Partial}DerivationOutputMap
The functions are used just once.
2025-08-13 17:50:35 -04:00
Jörg Thalheim be3a508b74 Merge pull request #13748 from Mic92/toml-revert
Revert "Merge pull request #13741 from xokdvium/toml-timestamps"
2025-08-13 22:51:55 +02:00
Jörg Thalheim 75740fbd75 Revert "Merge pull request #13741 from xokdvium/toml-timestamps"
This reverts commit 53ac49f72c, reversing
changes made to 8e5ca787f4.

This broke nixpkgs eval test that was depending overflowing integers...
2025-08-13 22:46:09 +02:00
Jörg Thalheim 5d3197bb6c Merge pull request #13497 from Mic92/repl-use-after-free-fix
Fix use-after-free warning in repl.cc
2025-08-13 21:55:01 +02:00
Jörg Thalheim 53ac49f72c Merge pull request #13741 from xokdvium/toml-timestamps
libexpr: Canonicalize TOML timestamps for toml11 > 4.0
2025-08-13 21:22:15 +02:00
Sergei Zimmerman 8e5ca787f4 Merge pull request #13743 from kip93/fix/lfs-ssh
Fix Git LFS SSH issues
2025-08-13 21:50:41 +03:00
Leandro Reina ccf658ed5c Fix Git LFS SSH issues
* Adds support for NIX_SSHOPTS
* Properly uses the parsed port from URL (fixes #13337)
* Don't guess the HTTP endpoint, use the response of git-lfs-authenticate
* Add an SSH Git LFS test
* Removed some unused test code
2025-08-13 21:04:29 +03:00
Jörg Thalheim 68839b9545 Merge pull request #13742 from xokdvium/fix-i686
libutil: Fix i686-linux build on clangStdenv
2025-08-13 09:03:25 +02:00
Sergei Zimmerman b5289fa130 libutil: Fix i686-linux build on clangStdenv
Clang refused to do a narrowing conversion in an initializer list:

```
 local-keys.cc:56:90: note: insert an explicit cast to silence this issue
return name + ":" + base64::encode(std::as_bytes(std::span<const unsigned char>{sig, sigLen}));
                                                                                    ^~~~~~
                                                                                    static_cast<size_type>( )
```
2025-08-12 22:48:15 +03:00
Sergei Zimmerman 320b2c74ef packaging: Build with toml11 4.4.0 2025-08-12 22:22:12 +03:00
Sergei Zimmerman dc769d72cb libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339
2025-08-12 21:49:30 +03:00
Jörg Thalheim ffe5c1ecb4 Merge pull request #13737 from NixOS/dependabot/github_actions/actions/download-artifact-5
build(deps): bump actions/download-artifact from 4 to 5
2025-08-12 15:43:43 +02:00
Sergei Zimmerman d8fc55a46e libexpr: Use table.size() instead of unnecessary loop 2025-08-12 14:58:53 +03:00
Sergei Zimmerman a80a5c4dba libexpr: Use recursive lambda instead of std::function
There's no reason to use a std::function for recursive lambdas
since there are polymorphic lambdas.
2025-08-12 14:55:29 +03:00
Sergei Zimmerman df4e55ffc1 libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.
2025-08-12 14:31:38 +03:00
Sergei Zimmerman 7ed0229d1a tests/functional/lang: Add more tests for TOML timestamps
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.
2025-08-12 14:28:39 +03:00
dependabot[bot] 4fb89eb2ea build(deps): bump actions/download-artifact from 4 to 5
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 5.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-12 09:05:31 +00:00
Sergei Zimmerman 6d481efd37 Merge pull request #13738 from NixOS/dependabot/github_actions/actions/checkout-5 2025-08-12 12:04:27 +03:00
dependabot[bot] e8d780642d build(deps): bump actions/checkout from 4 to 5
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-08-12 08:21:23 +00:00
Sergei Zimmerman b21f811ff6 Merge pull request #13736 from obsidiansystems/getUri-on-config
`getUri` should be `const` and on `Store::Config` not `Store`
2025-08-12 01:13:08 +03:00
John Ericson 0ef6f72c9c getUri should be const and on Store::Config not Store
It is a side-effect property of the configuration alone, not the rest of
the store.
2025-08-11 17:44:50 -04:00
Jörg Thalheim f93d25c0e7 Merge pull request #13735 from Mic92/override-dogfooding
ci: add workflow dispatch to bypass nix installer dogfooding
2025-08-11 20:24:08 +02:00
Jörg Thalheim dfb78af184 Merge pull request #13734 from xokdvium/get-uri-correct
libstore: Correct getUri methods for all stores
2025-08-11 20:16:08 +02:00
Jörg Thalheim f62b022872 ci: add workflow dispatch to bypass nix installer dogfooding
This helps to fix CI if our dogfooding Nix installer is broken
2025-08-11 19:51:46 +02:00
Sergei Zimmerman 41af531392 libstore: Correct getUri methods for all stores
Previously `getUri` didn't include store query parameters,
`ssh-ng` didn't include any information at all and the local
store didn't have the path:

```
$ nix store info --store "local?root=/tmp/aaa&require-sigs=false"
Store URL: local
Version: 2.31.0
Trusted: 1
$ nix store info --store "ssh-ng://localhost?remote-program=nix-daemon"
Store URL: ssh-ng://
Version: 2.31.0
Trusted: 1
$ nix store info --store "ssh://localhost?remote-program=nix-store"
Store URL: ssh://localhost
```

This commit changes this to:

```
$ nix store info --store "local?root=/tmp/aaa&require-sigs=false"
Store URL: local?require-sigs=false&root=/tmp/aaa
Version: 2.31.0
Trusted: 1
$ nix store info --store "ssh-ng://localhost?remote-program=nix-daemon"
Store URL: ssh-ng://localhost?remote-program=nix-daemon
Version: 2.31.0
Trusted: 1
$ nix store info --store "ssh://localhost?remote-program=nix-store"
Store URL: ssh://localhost?remote-program=nix-store
```
2025-08-11 20:44:31 +03:00
Sergei Zimmerman 73ebdf2497 libstore-tests: Restore commented out tests 2025-08-11 20:44:29 +03:00
Sergei Zimmerman 2c38ad2cfa libutil: Include necessary headers in config-impl.hh
These headers need to be included before everything else
(at least in GCC).
2025-08-11 20:44:27 +03:00
Sergei Zimmerman 28b74c3143 libstore: Add forward declarations for SandboxMode serializers
This is necessary to make libstore-tests for store configs work again.
2025-08-11 20:44:25 +03:00
Sergei Zimmerman a7fb257ec2 libutil: Make AbstractConfig::getSettings const 2025-08-11 20:44:23 +03:00
Sergei Zimmerman 920a878eaa Merge pull request #13733 from xokdvium/fix-ci-2
ci: Bump pinned stable nix version
2025-08-11 20:36:38 +03:00
Sergei Zimmerman 9d033733f2 ci: Bump pinned stable nix version
2.30.1 can't build our daemon tests. The fix is in 2.30.2.
2025-08-11 20:35:12 +03:00
Sergei Zimmerman 1030ed1e7d Merge pull request #13732 from xokdvium/disable-dogfood
ci: Disable dogfooding
2025-08-11 20:31:14 +03:00
Sergei Zimmerman 2b2c832f0a ci: Disable dogfooding
To unbreak the CI.
2025-08-11 20:30:03 +03:00
Jörg Thalheim 83c43d9311 Merge pull request #13731 from Mic92/fix-ci
Revert "Merge pull request #13709 from NixOS/boehm-coroutines-sp"
2025-08-11 19:23:21 +02:00
Jörg Thalheim 46b6907346 Revert "Merge pull request #13709 from NixOS/boehm-coroutines-sp"
This reverts commit 4b3ca9bd80, reversing
changes made to 867b69f533.

Since this commit we get reproducible segfaults building Nix ci in macos github runners:
https://github.com/NixOS/nix/actions/runs/16885882321/job/47837390248
2025-08-11 19:18:04 +02:00
Jörg Thalheim b2fb421386 Merge pull request #13490 from Mic92/clang-tidy-simple-warnings
Fix simple clang-tidy warnings
2025-08-11 18:21:46 +02:00
Jörg Thalheim 4b3ca9bd80 Merge pull request #13709 from NixOS/boehm-coroutines-sp
Restore fixupBoehmStackPointer
2025-08-11 16:17:14 +02:00
Jörg Thalheim 0675094861 Silence false positive clang-analyzer warning in repl.cc
The clang-analyzer incorrectly flags a use-after-free for GC-managed objects
when used with std::unique_ptr. Since NixRepl inherits from gc, its memory
is properly managed by Boehm GC and this is a false positive.

Added NOLINTNEXTLINE directive to suppress the warning.
2025-08-11 09:27:18 +02:00
Jörg Thalheim 867b69f533 Merge pull request #13729 from xokdvium/issue-13325
libfetchers: Remove badGitRefRegex and use libgit2 for reference validation
2025-08-11 09:15:35 +02:00
Sergei Zimmerman e8e9376a7b libfetchers: Remove badGitRefRegex and use libgit2 for reference validation
Fixes usage of `#` symbol in the reference name.
This also seems to identify several deficiencies in the libgit2 refname
validation code wrt to DEL symbol and a singular `@` symbol [1].

[1]: https://git-scm.com/docs/git-check-ref-format#_description
2025-08-11 02:38:45 +03:00
Jörg Thalheim b5f843a565 Merge pull request #13726 from a-kenji/ke-document-self-attrs
docs: add self-attrs documentation
2025-08-10 17:18:35 +02:00
a-kenji 937780ea08 docs: Add self-attribute documentation 2025-08-10 15:05:32 +02:00
Sergei Zimmerman 0b7f7e4b03 Merge pull request #13722 from xokdvium/faster-ci 2025-08-09 10:42:42 +03:00
Sergei Zimmerman 6ec88b93ba ci: Remove max-jobs = 1
This change was necessary when we were using `nix flake check` for CI
(see 6db6b269ed). Now this is not really
necessary, because we are running the checks in a much saner way, that
doesn't use up too much memory for evaluation.
2025-08-09 01:13:59 +03:00
Sergei Zimmerman 5451ad4a4c Merge pull request #13683 from fzakaria/fzakaria/meson-improvement
Make functional tests depend on nix binary so they auto recompile
2025-08-09 00:14:08 +03:00
John Ericson bd9ce27df0 Merge pull request #13713 from Mic92/freebsd-installer
add freebsd multi-user installer
2025-08-08 11:53:02 -04:00
John Ericson 8e35f3ca1e Merge pull request #13717 from Mic92/rc.d-script
add rc.d script for the nix-daemon
2025-08-08 11:49:59 -04:00
d2022189a1 Inline printHash16or32 and remove redundant helper (#13716)
Fixes ##13692

---------

Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-08-08 11:47:47 -04:00
Jörg Thalheim d4c6f24e9f Merge pull request #13718 from xokdvium/coverage-hydra-job-restore
hydra: Restore coverage job
2025-08-08 11:39:29 +02:00
Sergei Zimmerman 51c0e6bc63 hydra: Restore coverage job
Sometime ago we lost the coverage job in the midst of
meson migration. Until we have something like codecov
it'd be very useful to restore this job with the html
reports and historical metrics.

As a bonus we get more coverage metrics by switching to
LLVM tooling from LCOV.
2025-08-08 12:05:06 +03:00
Sergei Zimmerman cac8ec3b05 Merge pull request #13714 from xokdvium/move-ref-scan-sink
libutil: Move references.{hh,cc} to libstore, remove unused scanForReferences
2025-08-08 11:09:39 +03:00
Sergei Zimmerman 2e3ebfb829 libutil: Move references.{hh,cc} to libstore
The implicit dependency on refLength (which is the StorePath::HashLen)
is not good. Also the companion tests and benchmarks are already in libstore-tests.
2025-08-08 10:30:09 +03:00
Sergei Zimmerman 1b4aa5c1ef libstore: Remove unused overload of scanForReferences
This doesn't seem to be used anywhere at the moment.
It might be used out-of-tree, but this is a small convenience
function that is not worth keeping without in-tree usage.
2025-08-08 10:30:07 +03:00
Jörg Thalheim e74cfc52fe docs/uninstall: add instructions for freebsd 2025-08-08 09:10:35 +02:00
Jörg Thalheim 11d03893f8 add freebsd multi-user installer 2025-08-08 09:10:35 +02:00
Jörg Thalheim 73d09e67a7 add rc.d script for the nix-daemon 2025-08-08 09:09:58 +02:00
Farid ZakariaandSergei Zimmerman bf320465ae Make functional tests depend on nix binary so they auto recompile
With this I'm able to do a fresh config + meson test with all dependencies
correctly propagated.

Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
2025-08-08 03:18:38 +03:00
John Ericson 90aa2b83b5 Merge pull request #13715 from xokdvium/hash-result-struct
libutil: Make HashResult a proper struct
2025-08-07 20:13:39 -04:00
Sergei Zimmerman 143bd60136 libutil: Make HashResult a proper struct
This resolves an existing TODO and makes the
code slightly more readable.
2025-08-08 02:06:14 +03:00
John Ericson 241420a788 Merge pull request #13704 from obsidiansystems/slight-hash-cleanup
Slight clean up the constructors for `Hash`
2025-08-07 18:10:11 -04:00
John Ericson 628d423a12 Merge pull request #13712 from Mic92/freebsd-installer-basic
single-user-install: fix cp for freebsd
2025-08-07 18:04:02 -04:00
Jörg Thalheim ddd3034909 single-user-install: fix cp flags on freebsd 2025-08-07 23:51:37 +02:00
Jörg Thalheim 1cce358c40 single-user-install: just call uname once 2025-08-07 23:51:37 +02:00
Eelco Dolstra 3ba103865d Restore fixupBoehmStackPointer
This was removed in https://github.com/NixOS/nix/pull/11152. However,
we need it for the multi-threaded evaluator, because otherwise Boehm
GC will crash while scanning the thread stack:

  #0  GC_push_all_eager (bottom=<optimized out>, top=<optimized out>) at extra/../mark.c:1488
  #1  0x00007ffff74691d5 in GC_push_all_stack_sections (lo=<optimized out>, hi=<optimized out>, traced_stack_sect=0x0) at extra/../mark_rts.c:704
  #2  GC_push_all_stacks () at extra/../pthread_stop_world.c:876
  #3  GC_default_push_other_roots () at extra/../os_dep.c:2893
  #4  0x00007ffff746235c in GC_mark_some (cold_gc_frame=0x7ffee8ecaa50 "`\304G\367\377\177") at extra/../mark.c:374
  #5  0x00007ffff7465a8d in GC_stopped_mark (stop_func=stop_func@entry=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:875
  #6  0x00007ffff7466724 in GC_try_to_collect_inner (stop_func=0x7ffff7453c80 <GC_never_stop_func>) at extra/../alloc.c:624
  #7  0x00007ffff7466a22 in GC_collect_or_expand (needed_blocks=needed_blocks@entry=1, ignore_off_page=ignore_off_page@entry=0, retry=retry@entry=0) at extra/../alloc.c:1688
  #8  0x00007ffff746878f in GC_allocobj (gran=<optimized out>, kind=<optimized out>) at extra/../alloc.c:1798
  #9  GC_generic_malloc_inner (lb=<optimized out>, k=k@entry=1) at extra/../malloc.c:193
  #10 0x00007ffff746cd40 in GC_generic_malloc_many (lb=<optimized out>, k=<optimized out>, result=<optimized out>) at extra/../mallocx.c:477
  #11 0x00007ffff746cf35 in GC_malloc_kind (bytes=120, kind=1) at extra/../thread_local_alloc.c:187
  #12 0x00007ffff796ede5 in nix::allocBytes (n=<optimized out>, n=<optimized out>) at ../src/libexpr/include/nix/expr/eval-inline.hh:19

This is because it will use the stack pointer of the coroutine, so it
will scan a region of memory that doesn't exist, e.g.

  Stack for thread 0x7ffea4ff96c0 is [0x7ffe80197af0w,0x7ffea4ffa000)

(where 0x7ffe80197af0w is the sp of the coroutine and 0x7ffea4ffa000
is the base of the thread stack).

We don't scan coroutine stacks, because currently they don't have GC
roots (there is no evaluation happening in coroutines). So there is
currently no need to restore the other parts of the original patch,
such as BoehmGCStackAllocator.
2025-08-07 12:09:34 +02:00
John Ericson 9d8d8df729 Slight clean up the constructors for Hash
- No more private constructor that is kinda weird

- Two new static functions, `baseFromSize` and `baseFromSize`, that do
  one thing, and one thing only (simple).

- Two `Hash::parse*` that previously used the private constructor now
  can use these two functions directly.

- The remaining `Hash::parseAny*` methods, which are inherently more
  complex, are written in terms of a `parseAnyHelper` static function
  which is also complex, but keeps the complexity in one spot.
2025-08-06 20:04:59 -04:00
Eelco Dolstra 5fe6c53703 nix flake prefetch-inputs: Add
This command fetches all inputs of a flake in parallel.

Example runtime for

  $ chmod -R u+w /tmp/nix2; rm -rf /tmp/nix2; rm ~/.cache/nix/fetcher-cache-v3.sqlite*; rm -rf ~/.cache/nix/tarball-cache/ ~/.cache/nix/gitv3/; time nix flake prefetch-inputs --store /tmp/nix2 https://api.flakehub.com/f/pinned/informalsystems/cosmos.nix/0.3.0/018ce9ed-d0be-7ce5-81b6-a3c6e3ae1187/source.tar.gz

with http-connections = 1:

real    4m11.859s
user    2m6.931s
sys     0m25.619s

and http-connections = 25 (the default):

real    0m57.146s
user    2m49.506s
sys     0m36.008s
2025-07-28 18:53:49 +02:00
Jörg Thalheim 3c0cd73418 Fix uninitialized field in Attr constructor
The default constructor for Attr was not initializing the value pointer,
which could lead to undefined behavior when the uninitialized pointer is
accessed. This was caught by clang-tidy's UninitializedObject check.

This fixes the warning:
  1 uninitialized field at the end of the constructor call
  [clang-analyzer-optin.cplusplus.UninitializedObject]
2025-07-17 17:12:20 +02:00
Jörg Thalheim 6bf940d636 Fix clang-tidy uninitialized value warning in derivation-options.cc
Make lambda capture explicit to avoid clang-analyzer-core.CallAndMessage warning
2025-07-17 17:09:46 +02:00
Jörg Thalheim 4ba3b15a10 Fix s3.hh to include public config header
The s3.hh public header was incorrectly including store-config-private.hh
instead of the public config.hh. Since NIX_WITH_S3_SUPPORT is defined in
the public config, this caused clang-tidy to report it as undefined.
2025-07-17 17:09:38 +02:00
Jörg Thalheim 6e733b0544 Fix SIZE_MAX undefined warning in fchmodat2-compat.hh
Include <cstdint> to ensure SIZE_MAX is defined
2025-07-17 17:08:49 +02:00
Matthew Kenigsberg d60a8ee8b0 Improve database lock permission error context
Add helpful context when opening the Nix database lock fails due to
permission errors. Instead of just showing "Permission denied", now
provides guidance about possible causes:
- Running as non-root in a single-user Nix installation
- Nix daemon may have crashed
2025-06-11 08:13:52 +02:00
263 changed files with 3885 additions and 1727 deletions

No files matched your search

+2
View File
@@ -2,3 +2,5 @@
e4f62e46088919428a68bd8014201dc8e379fed7 # !autorebase ./maintainers/format.sh --until-stable
# meson re-formatting
385e2c3542c707d95e3784f7f6d623f67e77ab61 # !autorebase ./maintainers/format.sh --until-stable
# nixfmt 1.0.0
1d943f581908f35075a84a3d89c2eba3ff35067f # !autorebase ./maintainers/format.sh --until-stable
@@ -9,7 +9,7 @@ inputs:
install_url:
description: "URL of the Nix installer"
required: false
default: "https://releases.nixos.org/nix/nix-2.30.1/install"
default: "https://releases.nixos.org/nix/nix-2.30.2/install"
github_token:
description: "Github token"
required: true
+27 -27
View File
@@ -2,7 +2,15 @@ name: "CI"
on:
pull_request:
merge_group:
push:
workflow_dispatch:
inputs:
dogfood:
description: 'Use dogfood Nix build'
required: false
default: true
type: boolean
permissions: read-all
@@ -10,12 +18,12 @@ jobs:
eval:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: ./.github/actions/install-nix-action
with:
dogfood: true
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config:
experimental-features = nix-command flakes
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -48,17 +56,15 @@ jobs:
runs-on: ${{ matrix.runs-on }}
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: ./.github/actions/install-nix-action
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
dogfood: true
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
# The sandbox would otherwise be disabled by default on Darwin
extra_nix_config: |
sandbox = true
max-jobs = 1
extra_nix_config: "sandbox = true"
- uses: DeterminateSystems/magic-nix-cache-action@main
# Since ubuntu 22.30, unprivileged usernamespaces are no longer allowed to map to the root user:
# https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
@@ -110,9 +116,9 @@ jobs:
name: installer test ${{ matrix.scenario }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- name: Download installer tarball
uses: actions/download-artifact@v4
uses: actions/download-artifact@v5
with:
name: installer-${{matrix.os}}
path: out
@@ -141,17 +147,17 @@ jobs:
check_secrets:
permissions:
contents: none
name: Check Docker secrets present for installer tests
name: Check presence of secrets
runs-on: ubuntu-24.04
outputs:
docker: ${{ steps.secret.outputs.docker }}
steps:
- name: Check for secrets
- name: Check for DockerHub secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
echo "docker=${{ env._DOCKER_SECRETS != '' }}" >> $GITHUB_OUTPUT
docker_push_image:
needs: [tests, vm_tests, check_secrets]
@@ -164,13 +170,7 @@ jobs:
github.ref_name == 'master'
runs-on: ubuntu-24.04
steps:
- name: Check for secrets
id: secret
env:
_DOCKER_SECRETS: ${{ secrets.DOCKERHUB_USERNAME }}${{ secrets.DOCKERHUB_TOKEN }}
run: |
echo "::set-output name=docker::${{ env._DOCKER_SECRETS != '' }}"
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: cachix/install-nix-action@v31
@@ -216,10 +216,10 @@ jobs:
vm_tests:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
- uses: ./.github/actions/install-nix-action
with:
dogfood: true
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config:
experimental-features = nix-command flakes
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -237,20 +237,20 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout nix
uses: actions/checkout@v4
uses: actions/checkout@v5
- name: Checkout flake-regressions
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
repository: NixOS/flake-regressions
path: flake-regressions
- name: Checkout flake-regressions-data
uses: actions/checkout@v4
uses: actions/checkout@v5
with:
repository: NixOS/flake-regressions-data
path: flake-regressions/tests
- uses: ./.github/actions/install-nix-action
with:
dogfood: true
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config:
experimental-features = nix-command flakes
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -265,13 +265,13 @@ jobs:
github.event_name == 'push' &&
github.ref_name == 'master'
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v5
with:
fetch-depth: 0
- uses: ./.github/actions/install-nix-action
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
dogfood: true
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config: |
experimental-features = flakes nix-command ca-derivations impure-derivations
max-jobs = 1
+11
View File
@@ -161,3 +161,14 @@ pull_request_rules:
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.31
conditions:
- label=backport 2.31-maintenance
actions:
backport:
branches:
- "2.31-maintenance"
labels:
- automatic backport
- merge-queue
+1 -1
View File
@@ -1 +1 @@
2.31.0
2.32.0
+41 -23
View File
@@ -2,6 +2,12 @@
nixFlake ? builtins.getFlake ("git+file://" + toString ../../..),
system ? builtins.currentSystem,
pkgs ? nixFlake.inputs.nixpkgs.legacyPackages.${system},
nixComponents ? (
nixFlake.lib.makeComponents {
inherit pkgs;
inherit getStdenv;
}
),
getStdenv ? p: p.stdenv,
componentTestsPrefix ? "",
withSanitizers ? false,
@@ -49,13 +55,11 @@ let
};
# Done in a pre-configure hook, because $NIX_BUILD_TOP needs to be substituted.
preConfigure =
prevAttrs.preConfigure or ""
+ ''
mappingFlag=" -fcoverage-prefix-map=$NIX_BUILD_TOP/${finalAttrs.src.name}=${finalAttrs.src}"
CFLAGS+="$mappingFlag"
CXXFLAGS+="$mappingFlag"
'';
preConfigure = prevAttrs.preConfigure or "" + ''
mappingFlag=" -fcoverage-prefix-map=$NIX_BUILD_TOP/${finalAttrs.src.name}=${finalAttrs.src}"
CFLAGS+="$mappingFlag"
CXXFLAGS+="$mappingFlag"
'';
};
componentOverrides =
@@ -64,18 +68,13 @@ let
in
rec {
nixComponents =
(nixFlake.lib.makeComponents {
inherit pkgs;
inherit getStdenv;
}).overrideScope
(
final: prev: {
nix-store-tests = prev.nix-store-tests.override { withBenchmarks = true; };
nixComponentsInstrumented = nixComponents.overrideScope (
final: prev: {
nix-store-tests = prev.nix-store-tests.override { withBenchmarks = true; };
mesonComponentOverrides = lib.composeManyExtensions componentOverrides;
}
);
mesonComponentOverrides = lib.composeManyExtensions componentOverrides;
}
);
/**
Top-level tests for the flake outputs, as they would be built by hydra.
@@ -120,15 +119,15 @@ rec {
lib.concatMapAttrs (testName: test: {
"${componentTestsPrefix}${pkgName}-${testName}" = test;
}) (pkg.tests or { })
) nixComponents)
) nixComponentsInstrumented)
// lib.optionalAttrs (pkgs.stdenv.hostPlatform == pkgs.stdenv.buildPlatform) {
"${componentTestsPrefix}nix-functional-tests" = nixComponents.nix-functional-tests;
"${componentTestsPrefix}nix-functional-tests" = nixComponentsInstrumented.nix-functional-tests;
};
codeCoverage =
let
componentsTestsToProfile =
(builtins.mapAttrs (n: v: nixComponents.${n}.tests.run) {
(builtins.mapAttrs (n: v: nixComponentsInstrumented.${n}.tests.run) {
"nix-util-tests" = { };
"nix-store-tests" = { };
"nix-fetchers-tests" = { };
@@ -136,7 +135,7 @@ rec {
"nix-flake-tests" = { };
})
// {
inherit (nixComponents) nix-functional-tests;
inherit (nixComponentsInstrumented) nix-functional-tests;
};
coverageProfileDrvs = lib.mapAttrs (
@@ -170,12 +169,13 @@ rec {
coverageReports =
let
nixComponentDrvs = lib.filter (lib.isDerivation) (lib.attrValues nixComponents);
nixComponentDrvs = lib.filter (lib.isDerivation) (lib.attrValues nixComponentsInstrumented);
in
pkgs.runCommand "code-coverage-report"
{
nativeBuildInputs = [
pkgs.llvmPackages.libllvm
pkgs.jq
];
__structuredAttrs = true;
nixComponents = nixComponentDrvs;
@@ -201,6 +201,24 @@ rec {
echo
} >> $out/index.txt
llvm-cov export $arguments -instr-profile ${mergedProfdata} -format=text > $out/coverage.json
mkdir -p $out/nix-support
coverageTotals=$(jq ".data[0].totals" $out/coverage.json)
# Mostly inline from pkgs/build-support/setup-hooks/make-coverage-analysis-report.sh [1],
# which we can't use here, because we rely on LLVM's infra for source code coverage collection.
# [1]: https://github.com/NixOS/nixpkgs/blob/67bb48c4c8e327417d6d5aa7e538244b209e852b/pkgs/build-support/setup-hooks/make-coverage-analysis-report.sh#L16
declare -A metricsArray=(["lineCoverage"]="lines" ["functionCoverage"]="functions" ["branchCoverage"]="branches")
for metricName in "''\${!metricsArray[@]}"; do
key="''\${metricsArray[$metricName]}"
metric=$(echo "$coverageTotals" | jq ".$key.percent * 10 | round / 10")
echo "$metricName $metric %" >> $out/nix-support/hydra-metrics
done
echo "report coverage $out" >> $out/nix-support/hydra-build-products
'';
in
assert withCoverage;
+17 -18
View File
@@ -46,24 +46,23 @@ mkMesonDerivation (finalAttrs: {
];
# Hack for sake of the dev shell
passthru.externalNativeBuildInputs =
[
meson
ninja
(lib.getBin lowdown-unsandboxed)
mdbook
mdbook-linkcheck
jq
python3
rsync
changelog-d
]
++ lib.optionals (!officialRelease) [
# When not an official release, we likely have changelog entries that have
# yet to be rendered.
# When released, these are rendered into a committed file to save a dependency.
changelog-d
];
passthru.externalNativeBuildInputs = [
meson
ninja
(lib.getBin lowdown-unsandboxed)
mdbook
mdbook-linkcheck
jq
python3
rsync
changelog-d
]
++ lib.optionals (!officialRelease) [
# When not an official release, we likely have changelog entries that have
# yet to be rendered.
# When released, these are rendered into a committed file to save a dependency.
changelog-d
];
nativeBuildInputs = finalAttrs.passthru.externalNativeBuildInputs ++ [
nix-cli
@@ -1,6 +0,0 @@
---
synopsis: "`build-cores = 0` now auto-detects CPU cores"
prs: [13402]
---
When `build-cores` is set to `0`, nix now automatically detects the number of available CPU cores and passes this value via `NIX_BUILD_CORES`, instead of passing `0` directly. This matches the behavior when `build-cores` is unset. This prevents the builder from having to detect the number of cores.
-13
View File
@@ -1,13 +0,0 @@
---
synopsis: "Add support for user@address:port syntax in store URIs"
prs: [3425]
issues: [7044]
---
It's now possible to specify the port used for the SSH stores directly in the store URL in accordance with [RFC3986](https://datatracker.ietf.org/doc/html/rfc3986). Previously the only way to specify custom ports was via `ssh_config` or `NIX_SSHOPTS` environment variable, because Nix incorrectly passed the port number together with the host name to the SSH executable. This has now been fixed.
This change affects [store references](@docroot@/store/types/index.md#store-url-format) passed via the `--store` and similar flags in CLI as well as in the configuration for [remote builders](@docroot@/command-ref/conf-file.md#conf-builders). For example, the following store URIs now work:
- `ssh://127.0.0.1:2222`
- `ssh://[b573:6a48:e224:840b:6007:6275:f8f7:ebf3]:22`
- `ssh-ng://[b573:6a48:e224:840b:6007:6275:f8f7:ebf3]:22`
@@ -1,6 +0,0 @@
---
synopsis: "Represent IPv6 RFC4007 ZoneId literals in conformance with RFC6874"
prs: [13445]
---
Prior versions of Nix since [#4646](https://github.com/NixOS/nix/pull/4646) accepted [IPv6 scoped addresses](https://datatracker.ietf.org/doc/html/rfc4007) in URIs like [store references](@docroot@/store/types/index.md#store-url-format) in the textual representation with a literal percent character: `[fe80::1%18]`. This was ambiguous, because the the percent literal `%` is reserved by [RFC3986](https://datatracker.ietf.org/doc/html/rfc3986), since it's used to indicate percent encoding. Nix now requires that the percent `%` symbol is percent-encoded as `%25`. This implements [RFC6874](https://datatracker.ietf.org/doc/html/rfc6874), which defines the representation of zone identifiers in URIs. The example from above now has to be specified as `[fe80::1%2518]`.
+1
View File
@@ -138,6 +138,7 @@
- [Contributing](development/contributing.md)
- [Releases](release-notes/index.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.31 (2025-08-21)](release-notes/rl-2.31.md)
- [Release 2.30 (2025-07-07)](release-notes/rl-2.30.md)
- [Release 2.29 (2025-05-14)](release-notes/rl-2.29.md)
- [Release 2.28 (2025-04-02)](release-notes/rl-2.28.md)
+1 -1
View File
@@ -75,7 +75,7 @@ Most Nix commands interpret the following environment variables:
- <span id="env-NIX_CONF_DIR">[`NIX_CONF_DIR`](#env-NIX_CONF_DIR)</span>
Overrides the location of the system Nix configuration directory
(default `prefix/etc/nix`).
(default `sysconfdir/nix`, i.e. `/etc/nix` on most systems).
- <span id="env-NIX_CONFIG">[`NIX_CONFIG`](#env-NIX_CONFIG)</span>
@@ -10,7 +10,7 @@
- Bash Shell. The `./configure` script relies on bashisms, so Bash is
required.
- A version of GCC or Clang that supports C++20.
- A version of GCC or Clang that supports C++23.
- `pkg-config` to locate dependencies. If your distribution does not
provide it, you can get it from
@@ -41,6 +41,38 @@ There may also be references to Nix in
which you may remove.
### FreeBSD
1. Stop and remove the Nix daemon service:
```console
sudo service nix-daemon stop
sudo rm -f /usr/local/etc/rc.d/nix-daemon
sudo sysrc -x nix_daemon_enable
```
2. Remove files created by Nix:
```console
sudo rm -rf /etc/nix /usr/local/etc/profile.d/nix.sh /nix ~root/.nix-channels ~root/.nix-defexpr ~root/.nix-profile ~root/.cache/nix
```
3. Remove build users and their group:
```console
for i in $(seq 1 32); do
sudo pw userdel nixbld$i
done
sudo pw groupdel nixbld
```
4. There may also be references to Nix in:
- `/usr/local/etc/bashrc`
- `/usr/local/etc/zshrc`
- Shell configuration files in users' home directories
which you may remove.
### macOS
> **Updating to macOS 15 Sequoia**
@@ -0,0 +1,96 @@
# Release 2.31.0 (2025-08-21)
- `build-cores = 0` now auto-detects CPU cores [#13402](https://github.com/NixOS/nix/pull/13402)
When `build-cores` is set to `0`, Nix now automatically detects the number of available CPU cores and passes this value via `NIX_BUILD_CORES`, instead of passing `0` directly. This matches the behavior when `build-cores` is unset. This prevents the builder from having to detect the number of cores.
- Fix Git LFS SSH issues [#13337](https://github.com/NixOS/nix/issues/13337) [#13743](https://github.com/NixOS/nix/pull/13743)
Fixed some outstanding issues with Git LFS and SSH.
* Added support for `NIX_SSHOPTS`.
* Properly use the parsed port from URL.
* Better use of the response of `git-lfs-authenticate` to determine API endpoint when the API is not exposed on port 443.
- Add support for `user@address:port` syntax in store URIs [#7044](https://github.com/NixOS/nix/issues/7044) [#3425](https://github.com/NixOS/nix/pull/3425)
It's now possible to specify the port used for SSH stores directly in the store URL in accordance with [RFC3986](https://datatracker.ietf.org/doc/html/rfc3986). Previously the only way to specify custom ports was via `ssh_config` or the `NIX_SSHOPTS` environment variable, because Nix incorrectly passed the port number together with the host name to the SSH executable.
This change affects [store references](@docroot@/store/types/index.md#store-url-format) passed via the `--store` and similar flags in CLI as well as in the configuration for [remote builders](@docroot@/command-ref/conf-file.md#conf-builders). For example, the following store URIs now work:
- `ssh://127.0.0.1:2222`
- `ssh://[b573:6a48:e224:840b:6007:6275:f8f7:ebf3]:22`
- `ssh-ng://[b573:6a48:e224:840b:6007:6275:f8f7:ebf3]:22`
- Represent IPv6 RFC4007 ZoneId literals in conformance with RFC6874 [#13445](https://github.com/NixOS/nix/pull/13445)
Prior versions of Nix since [#4646](https://github.com/NixOS/nix/pull/4646) accepted [IPv6 scoped addresses](https://datatracker.ietf.org/doc/html/rfc4007) in URIs like [store references](@docroot@/store/types/index.md#store-url-format) in the textual representation with a literal percent character: `[fe80::1%18]`. This was ambiguous, because the the percent literal `%` is reserved by [RFC3986](https://datatracker.ietf.org/doc/html/rfc3986), since it's used to indicate percent encoding. Nix now requires that the percent `%` symbol is percent-encoded as `%25`. This implements [RFC6874](https://datatracker.ietf.org/doc/html/rfc6874), which defines the representation of zone identifiers in URIs. The example from above now has to be specified as `[fe80::1%2518]`.
- Use WAL mode for SQLite cache databases [#13800](https://github.com/NixOS/nix/pull/13800)
Previously, Nix used SQLite's "truncate" mode for caches. However, this could cause a Nix process to block if another process was updating the cache. This was a problem for the flake evaluation cache in particular, since it uses long-running transactions. Thus, concurrent Nix commands operating on the same flake could be blocked for an unbounded amount of time. WAL mode avoids this problem.
This change required updating the versions of the SQLite caches. For instance, `eval-cache-v5.sqlite` is now `eval-cache-v6.sqlite`.
- Enable parallel marking in bdwgc [#13708](https://github.com/NixOS/nix/pull/13708)
Previously marking was done by only one thread, which takes a long time if the heap gets big. Enabling parallel marking speeds up evaluation a lot, for example (on a Ryzen 9 5900X 12-Core):
* `nix search nixpkgs` from 24.3s to 18.9s.
* Evaluating the `NixOS/nix/2.21.2` flake regression test from 86.1s to 71.2s.
- New command `nix flake prefetch-inputs` [#13565](https://github.com/NixOS/nix/pull/13565)
This command fetches all inputs of a flake in parallel. This can be a lot faster than the serialized on-demand fetching during regular flake evaluation. The downside is that it may fetch inputs that aren't normally used.
- Add `warn-short-path-literals` setting [#13489](https://github.com/NixOS/nix/pull/13489)
This setting, when enabled, causes Nix to emit warnings when encountering relative path literals that don't start with `.` or `/`, for instance suggesting that `foo/bar` should be rewritten to `./foo/bar`.
- When updating a lock, respect the input's lock file [#13437](https://github.com/NixOS/nix/pull/13437)
For example, if a flake has a lock for `a` and `a/b`, and we change the flakeref for `a`, previously Nix would fetch the latest version of `b` rather than using the lock for `b` from `a`.
- Implement support for Git hashing with SHA-256 [#13543](https://github.com/NixOS/nix/pull/13543)
The experimental support for [Git-hashing](@docroot@/development/experimental-features.md#xp-feature-git-hashing) store objects now also includes support for SHA-256, not just SHA-1, in line with upstream Git.
## Contributors
This release was made possible by the following 34 contributors:
- John Soo [**(@jsoo1)**](https://github.com/jsoo1)
- Alan Urmancheev [**(@alurm)**](https://github.com/alurm)
- Manse [**(@PedroManse)**](https://github.com/PedroManse)
- Pol Dellaiera [**(@drupol)**](https://github.com/drupol)
- DavHau [**(@DavHau)**](https://github.com/DavHau)
- Leandro Emmanuel Reina Kiperman [**(@kip93)**](https://github.com/kip93)
- h0nIg [**(@h0nIg)**](https://github.com/h0nIg)
- Philip Taron [**(@philiptaron)**](https://github.com/philiptaron)
- Eelco Dolstra [**(@edolstra)**](https://github.com/edolstra)
- Connor Baker [**(@ConnorBaker)**](https://github.com/ConnorBaker)
- kenji [**(@a-kenji)**](https://github.com/a-kenji)
- Oleksandr Knyshuk [**(@k1gen)**](https://github.com/k1gen)
- Maciej Krüger [**(@mkg20001)**](https://github.com/mkg20001)
- Justin Bailey [**(@jgbailey-well)**](https://github.com/jgbailey-well)
- Emily [**(@emilazy)**](https://github.com/emilazy)
- Volker Diels-Grabsch [**(@vog)**](https://github.com/vog)
- gustavderdrache [**(@gustavderdrache)**](https://github.com/gustavderdrache)
- Elliot Cameron [**(@de11n)**](https://github.com/de11n)
- Alexander V. Nikolaev [**(@avnik)**](https://github.com/avnik)
- tomberek [**(@tomberek)**](https://github.com/tomberek)
- Matthew Kenigsberg [**(@mkenigs)**](https://github.com/mkenigs)
- Sergei Zimmerman [**(@xokdvium)**](https://github.com/xokdvium)
- Cosima Neidahl [**(@OPNA2608)**](https://github.com/OPNA2608)
- John Ericson [**(@Ericson2314)**](https://github.com/Ericson2314)
- m4dc4p [**(@m4dc4p)**](https://github.com/m4dc4p)
- Graham Christensen [**(@grahamc)**](https://github.com/grahamc)
- Jason Yundt [**(@Jayman2000)**](https://github.com/Jayman2000)
- Jens Petersen [**(@juhp)**](https://github.com/juhp)
- the-sun-will-rise-tomorrow [**(@the-sun-will-rise-tomorrow)**](https://github.com/the-sun-will-rise-tomorrow)
- Farid Zakaria [**(@fzakaria)**](https://github.com/fzakaria)
- AGawas [**(@aln730)**](https://github.com/aln730)
- Robert Hensing [**(@roberth)**](https://github.com/roberth)
- Dmitry Bogatov [**(@KAction)**](https://github.com/KAction)
- Jörg Thalheim [**(@Mic92)**](https://github.com/Mic92)
- Philipp Otterbein
+1 -1
View File
@@ -9,7 +9,7 @@ This is where Nix distinguishes itself.
## Store Derivation {#store-derivation}
A derivation is a specification for running an executable on precisely defined input to produce on more [store objects][store object].
A derivation is a specification for running an executable on precisely defined input to produce one or more [store objects][store object].
These store objects are known as the derivation's *outputs*.
Derivations are *built*, in which case the process is spawned according to the spec, and when it exits, required to leave behind files which will (after post-processing) become the outputs of the derivation.
+51 -52
View File
@@ -65,62 +65,61 @@ let
iana-etc
gitMinimal
openssh
] ++ extraPkgs;
]
++ extraPkgs;
users =
{
users = {
root = {
uid = 0;
shell = lib.getExe bashInteractive;
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = lib.getExe' shadow "nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.optionalAttrs (uid != 0) {
"${uname}" = {
uid = uid;
shell = lib.getExe bashInteractive;
home = "/home/${uname}";
gid = gid;
groups = [ "${gname}" ];
description = "Nix user";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups =
{
root.gid = 0;
nixbld.gid = 30000;
nobody.gid = 65534;
}
// lib.optionalAttrs (gid != 0) {
"${gname}".gid = gid;
root = {
uid = 0;
shell = lib.getExe bashInteractive;
home = "/root";
gid = 0;
groups = [ "root" ];
description = "System administrator";
};
nobody = {
uid = 65534;
shell = lib.getExe' shadow "nologin";
home = "/var/empty";
gid = 65534;
groups = [ "nobody" ];
description = "Unprivileged account (don't use!)";
};
}
// lib.optionalAttrs (uid != 0) {
"${uname}" = {
uid = uid;
shell = lib.getExe bashInteractive;
home = "/home/${uname}";
gid = gid;
groups = [ "${gname}" ];
description = "Nix user";
};
}
// lib.listToAttrs (
map (n: {
name = "nixbld${toString n}";
value = {
uid = 30000 + n;
gid = 30000;
groups = [ "nixbld" ];
description = "Nix build user ${toString n}";
};
}) (lib.lists.range 1 32)
);
groups = {
root.gid = 0;
nixbld.gid = 30000;
nobody.gid = 65534;
}
// lib.optionalAttrs (gid != 0) {
"${gname}".gid = gid;
};
userToPasswd = (
k:
{
Generated
+4 -4
View File
@@ -63,16 +63,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1747179050,
"narHash": "sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY=",
"lastModified": 1756178832,
"narHash": "sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "adaa24fbf46737f3f1b5497bf64bae750f82942e",
"rev": "d98ce345cdab58477ca61855540999c86577d19d",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"ref": "nixos-25.05-small",
"repo": "nixpkgs",
"type": "github"
}
+1 -1
View File
@@ -1,7 +1,7 @@
{
description = "The purely functional package manager";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05-small";
inputs.nixpkgs-regression.url = "github:NixOS/nixpkgs/215d4d0fd80ca5163643b03a33fde804a29cc1e2";
inputs.nixpkgs-23-11.url = "github:NixOS/nixpkgs/a62e6edd6d5e1fa0329b8653c801147986f8d446";
+1 -1
View File
@@ -46,7 +46,7 @@ The team meets twice a week (times are denoted in the [Europe/Amsterdam](https:/
- mark it as draft if it is blocked on the contributor
- escalate it back to the team by moving it to To discuss, and leaving a comment as to why the issue needs to be discussed again.
- Work meeting: Mondays 14:00-16:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
- Work meeting: Mondays 18:00-20:00 Europe/Amsterdam; see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
1. Code review on pull requests from [In review](#in-review).
2. Other chores and tasks.
@@ -185,5 +185,23 @@
"gwenn.lebihan7@gmail.com": "gwennlbh",
"hey@ewen.works": "gwennlbh",
"matt@sturgeon.me.uk": "MattSturgeon",
"pbsds@hotmail.com": "pbsds"
"pbsds@hotmail.com": "pbsds",
"sergei@zimmerman.foo": "xokdvium",
"v@njh.eu": "vog",
"pedro.manse@dmk3.com.br": "PedroManse",
"arnavgawas707@gmail.com": "aln730",
"mkg20001@gmail.com": "mkg20001",
"avn@avnik.info": "avnik",
"olk@disr.it": "k1gen",
"108410815+alurm@users.noreply.github.com": "alurm",
"kaction.cc@gmail.com": "KAction",
"juhpetersen@gmail.com": "juhp",
"opna2608@protonmail.com": "OPNA2608",
"jgbailey@gmail.com": "m4dc4p",
"justin.bailey@well.co": "jgbailey-well",
"130508846+de11n@users.noreply.github.com": "de11n",
"ConnorBaker01@Gmail.com": "ConnorBaker",
"jsoo1@asu.edu": "jsoo1",
"hsngrmpf+github@gmail.com": "DavHau",
"matthew@floxdev.com": "mkenigs"
}
@@ -162,5 +162,20 @@
"pbsds": "Peder Bergebakken Sundt",
"egorkonovalov": "Egor Konovalov",
"jayeshv": "jayeshv",
"vcunat": "Vladim\u00edr \u010cun\u00e1t"
"vcunat": "Vladim\u00edr \u010cun\u00e1t",
"mkenigs": "Matthew Kenigsberg",
"alurm": "Alan Urmancheev",
"jgbailey-well": "Justin Bailey",
"k1gen": "Oleksandr Knyshuk",
"juhp": "Jens Petersen",
"de11n": "Elliot Cameron",
"jsoo1": "John Soo",
"m4dc4p": null,
"PedroManse": "Manse",
"OPNA2608": "Cosima Neidahl",
"mkg20001": "Maciej Kr\u00fcger",
"avnik": "Alexander V. Nikolaev",
"DavHau": null,
"aln730": "AGawas",
"vog": "Volker Diels-Grabsch"
}
+5 -1
View File
@@ -3,5 +3,9 @@
- https://github.com/NixOS/nixos-homepage/
- https://github.com/orgs/NixOS/teams/nix-team
- Matrix room
- Matrix rooms
- [private] Nix maintainer team
- Nix ∪ Lix devs (also private)
- any open security issues if present and needed
- Team member should subscribe to notifications for the [Nix development category on Discourse](https://discourse.nixos.org/c/dev/nix/50)
+58
View File
@@ -0,0 +1,58 @@
#!/usr/bin/env bash
set -euo pipefail
# debug:
# set -x
START_REF="${1}"
END_REF="${2:-upstream/master}"
# Get the merge base
MERGE_BASE=$(git merge-base "$START_REF" "$END_REF")
unset START_REF
# Get date range
START_DATE=$(git show -s --format=%cI "$MERGE_BASE")
END_DATE=$(git show -s --format=%cI "$END_REF")
echo "Checking PRs merged between $START_DATE and $END_DATE" >&2
# Get all commits between merge base and HEAD
COMMITS=$(git rev-list "$MERGE_BASE..$END_REF")
# Convert to set for fast lookup
declare -A commit_set
for commit in $COMMITS; do
commit_set["$commit"]=1
done
# Get the current changelog
LOG_DONE="$(changelog-d doc/manual/rl-next)"
is_done(){
local nr="$1"
echo "$LOG_DONE" | grep -E "^- .*/pull/$nr)"
}
# Query merged PRs in date range
gh pr list \
--repo NixOS/nix \
--state merged \
--limit 1000 \
--json number,title,author,mergeCommit \
--search "merged:$START_DATE..$END_DATE" | \
jq -r '.[] | [.number, .mergeCommit.oid, .title, .author.login] | @tsv' | \
while IFS=$'\t' read -r pr_num merge_commit _title author; do
# Check if this PR's merge commit is in our branch
if [[ -n "${commit_set[$merge_commit]:-}" ]]; then
# Full detail, not suitable for comment due to mass ping and duplicate title
# echo "- #$pr_num $_title (@$author)"
echo "- #$pr_num ($author)"
if is_done "$pr_num"
then
echo " - [x] has note"
else
echo " - [ ] has note"
fi
echo " - [ ] skip"
fi
done
+10 -4
View File
@@ -24,11 +24,18 @@ release:
* In a checkout of the Nix repo, make sure you're on `master` and run
`git pull`.
* Compile a release notes to-do list by running
```console
$ ./maintainers/release-notes-todo PREV_RELEASE HEAD
```
* Compile the release notes by running
```console
$ export VERSION=X.YY
$ git checkout -b release-notes
$ export GITHUB_TOKEN=...
$ ./maintainers/release-notes
```
@@ -126,6 +133,8 @@ release:
Commit and push this to the maintenance branch.
* Create a backport label.
* Bump the version of `master`:
```console
@@ -133,6 +142,7 @@ release:
$ git pull
$ NEW_VERSION=2.13.0
$ echo $NEW_VERSION > .version
$ ... edit .mergify.yml to add the previous version ...
$ git checkout -b bump-$NEW_VERSION
$ git commit -a -m 'Bump version'
$ git push --set-upstream origin bump-$NEW_VERSION
@@ -140,10 +150,6 @@ release:
Make a pull request and auto-merge it.
* Create a backport label.
* Add the new backport label to `.mergify.yml`.
* Post an [announcement on Discourse](https://discourse.nixos.org/c/announcements/8), including the contents of
`rl-$VERSION.md`.
+10
View File
@@ -0,0 +1,10 @@
configure_file(
input : 'nix-daemon.in',
output : 'nix-daemon',
install : true,
install_dir : get_option('prefix') / 'etc/rc.d',
install_mode : 'rwxr-xr-x',
configuration : {
'bindir' : bindir,
},
)
+49
View File
@@ -0,0 +1,49 @@
#!/bin/sh
#
# PROVIDE: nix_daemon
# REQUIRE: DAEMON
# KEYWORD: shutdown
#
# Add the following lines to /etc/rc.conf to enable nix-daemon:
#
# nix_daemon_enable="YES"
#
# shellcheck source=/dev/null
. /etc/rc.subr
name="nix_daemon"
# shellcheck disable=SC2034
rcvar="nix_daemon_enable"
load_rc_config $name
: "${nix_daemon_enable:=NO}"
command="@bindir@/nix-daemon"
command_args=""
pidfile="/var/run/nix-daemon.pid"
# shellcheck disable=SC2034
start_cmd="${name}_start"
# shellcheck disable=SC2034
stop_cmd="${name}_stop"
nix_daemon_start() {
echo "Starting ${name}."
# command_args is intentionally unquoted to allow multiple arguments
# shellcheck disable=SC2086
/usr/sbin/daemon -c -f -p "${pidfile}" "${command}" ${command_args}
}
nix_daemon_stop() {
if [ -f "${pidfile}" ]; then
echo "Stopping ${name}."
kill -TERM "$(cat "${pidfile}")"
rm -f "${pidfile}"
else
echo "${name} is not running."
fi
}
run_rc_command "$1"
+4
View File
@@ -9,3 +9,7 @@ endif
if host_machine.system() == 'darwin'
subdir('launchd')
endif
if host_machine.system() == 'freebsd'
subdir('freebsd')
endif
+2 -2
View File
@@ -14,7 +14,7 @@ extra_pkg_config_variables = get_variable('extra_pkg_config_variables', {})
extra_cflags = []
if not meson.project_name().endswith('-c')
extra_cflags += [ '-std=c++2a' ]
extra_cflags += [ '-std=c++23' ]
endif
import('pkgconfig').generate(
@@ -34,7 +34,7 @@ meson.override_dependency(
declare_dependency(
include_directories : include_dirs,
link_with : this_library,
compile_args : [ '-std=c++2a' ],
compile_args : [ '-std=c++23' ],
dependencies : deps_public_subproject + deps_public,
variables : extra_pkg_config_variables,
),
+6
View File
@@ -37,6 +37,9 @@ runCommand "nix-binary-tarball-${version}" env ''
substitute ${../scripts/install-systemd-multi-user.sh} $TMPDIR/install-systemd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${../scripts/install-freebsd-multi-user.sh} $TMPDIR/install-freebsd-multi-user.sh \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
substitute ${../scripts/install-multi-user.sh} $TMPDIR/install-multi-user \
--subst-var-by nix ${nix} \
--subst-var-by cacert ${cacert}
@@ -48,6 +51,7 @@ runCommand "nix-binary-tarball-${version}" env ''
shellcheck $TMPDIR/create-darwin-volume.sh
shellcheck $TMPDIR/install-darwin-multi-user.sh
shellcheck $TMPDIR/install-systemd-multi-user.sh
shellcheck $TMPDIR/install-freebsd-multi-user.sh
# SC1091: Don't panic about not being able to source
# /etc/profile
@@ -64,6 +68,7 @@ runCommand "nix-binary-tarball-${version}" env ''
chmod +x $TMPDIR/create-darwin-volume.sh
chmod +x $TMPDIR/install-darwin-multi-user.sh
chmod +x $TMPDIR/install-systemd-multi-user.sh
chmod +x $TMPDIR/install-freebsd-multi-user.sh
chmod +x $TMPDIR/install-multi-user
dir=nix-${version}-${system}
fn=$out/$dir.tar.xz
@@ -82,6 +87,7 @@ runCommand "nix-binary-tarball-${version}" env ''
$TMPDIR/create-darwin-volume.sh \
$TMPDIR/install-darwin-multi-user.sh \
$TMPDIR/install-systemd-multi-user.sh \
$TMPDIR/install-freebsd-multi-user.sh \
$TMPDIR/install-multi-user \
$TMPDIR/reginfo \
$(cat ${installerClosureInfo}/store-paths)
+8 -7
View File
@@ -54,12 +54,12 @@ let
preConfigure =
prevAttrs.preConfigure or ""
+
# Update the repo-global .version file.
# Symlink ./.version points there, but by default only workDir is writable.
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
# Update the repo-global .version file.
# Symlink ./.version points there, but by default only workDir is writable.
''
chmod u+w ./.version
echo ${finalAttrs.version} > ./.version
'';
};
localSourceLayer =
@@ -148,7 +148,8 @@ let
nativeBuildInputs = [
meson
ninja
] ++ prevAttrs.nativeBuildInputs or [ ];
]
++ prevAttrs.nativeBuildInputs or [ ];
mesonCheckFlags = prevAttrs.mesonCheckFlags or [ ] ++ [
"--print-errorlogs"
];
+12
View File
@@ -64,6 +64,18 @@ scope: {
NIX_CFLAGS_COMPILE = "-DINITIAL_MARK_STACK_SIZE=1048576";
});
lowdown = pkgs.lowdown.overrideAttrs (prevAttrs: rec {
version = "2.0.2";
src = pkgs.fetchurl {
url = "https://kristaps.bsd.lv/lowdown/snapshots/lowdown-${version}.tar.gz";
hash = "sha512-cfzhuF4EnGmLJf5EGSIbWqJItY3npbRSALm+GarZ7SMU7Hr1xw0gtBFMpOdi5PBar4TgtvbnG4oRPh+COINGlA==";
};
nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ pkgs.buildPackages.bmake ];
postInstall =
lib.replaceStrings [ "lowdown.so.1" "lowdown.1.dylib" ] [ "lowdown.so.2" "lowdown.2.dylib" ]
prevAttrs.postInstall;
});
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost =
(pkgs.boost.override {
+24 -23
View File
@@ -71,17 +71,16 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
# We use this shell with the local checkout, not unpackPhase.
src = null;
env =
{
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
modular.pre-commit.settings.rawConfig
}";
}
// lib.optionalAttrs stdenv.hostPlatform.isLinux {
CC_LD = "mold";
CXX_LD = "mold";
};
env = {
# For `make format`, to work without installing pre-commit
_NIX_PRE_COMMIT_HOOKS_CONFIG = "${(pkgs.formats.yaml { }).generate "pre-commit-config.yaml"
modular.pre-commit.settings.rawConfig
}";
}
// lib.optionalAttrs stdenv.hostPlatform.isLinux {
CC_LD = "mold";
CXX_LD = "mold";
};
mesonFlags =
map (transformFlag "libutil") (ignoreCrossFile pkgs.nixComponents2.nix-util.mesonFlags)
@@ -119,23 +118,25 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
pkgs.buildPackages.gdb
]
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
lib.hiPrio pkgs.buildPackages.clang-tools
)
++ lib.optional stdenv.hostPlatform.isLinux pkgs.buildPackages.mold-wrapped;
buildInputs =
[ pkgs.gbenchmark ]
++ attrs.buildInputs or [ ]
++ pkgs.nixComponents2.nix-util.buildInputs
++ pkgs.nixComponents2.nix-store.buildInputs
++ pkgs.nixComponents2.nix-store-tests.externalBuildInputs
++ pkgs.nixComponents2.nix-fetchers.buildInputs
++ pkgs.nixComponents2.nix-expr.buildInputs
++ pkgs.nixComponents2.nix-expr.externalPropagatedBuildInputs
++ pkgs.nixComponents2.nix-cmd.buildInputs
++ lib.optionals havePerl pkgs.nixComponents2.nix-perl-bindings.externalBuildInputs
++ lib.optional havePerl pkgs.perl;
buildInputs = [
pkgs.gbenchmark
]
++ attrs.buildInputs or [ ]
++ pkgs.nixComponents2.nix-util.buildInputs
++ pkgs.nixComponents2.nix-store.buildInputs
++ pkgs.nixComponents2.nix-store-tests.externalBuildInputs
++ pkgs.nixComponents2.nix-fetchers.buildInputs
++ pkgs.nixComponents2.nix-expr.buildInputs
++ pkgs.nixComponents2.nix-expr.externalPropagatedBuildInputs
++ pkgs.nixComponents2.nix-cmd.buildInputs
++ lib.optionals havePerl pkgs.nixComponents2.nix-perl-bindings.externalBuildInputs
++ lib.optional havePerl pkgs.perl;
}
)
+31 -32
View File
@@ -47,25 +47,25 @@
}:
let
libs =
{
inherit
nix-util
nix-util-c
nix-store
nix-store-c
nix-fetchers
nix-fetchers-c
nix-expr
nix-expr-c
nix-flake
nix-flake-c
nix-main
nix-main-c
nix-cmd
;
}
// lib.optionalAttrs
libs = {
inherit
nix-util
nix-util-c
nix-store
nix-store-c
nix-fetchers
nix-fetchers-c
nix-expr
nix-expr-c
nix-flake
nix-flake-c
nix-main
nix-main-c
nix-cmd
;
}
//
lib.optionalAttrs
(!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform)
{
# Currently fails in static build
@@ -127,20 +127,19 @@ stdenv.mkDerivation (finalAttrs: {
*/
dontFixup = true;
checkInputs =
[
# Make sure the unit tests have passed
nix-util-tests.tests.run
nix-store-tests.tests.run
nix-expr-tests.tests.run
nix-fetchers-tests.tests.run
nix-flake-tests.tests.run
checkInputs = [
# Make sure the unit tests have passed
nix-util-tests.tests.run
nix-store-tests.tests.run
nix-expr-tests.tests.run
nix-fetchers-tests.tests.run
nix-flake-tests.tests.run
# Make sure the functional tests have passed
nix-functional-tests
]
++ lib.optionals
(!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform)
# Make sure the functional tests have passed
nix-functional-tests
]
++
lib.optionals (!stdenv.hostPlatform.isStatic && stdenv.buildPlatform.canExecute stdenv.hostPlatform)
[
# Perl currently fails in static build
# TODO: Split out tests into a separate derivation?
+11 -4
View File
@@ -223,10 +223,17 @@ in
dockerImage = lib.genAttrs linux64BitSystems (system: self.packages.${system}.dockerImage);
# # Line coverage analysis.
# coverage = nixpkgsFor.x86_64-linux.native.nix.override {
# pname = "nix-coverage";
# withCoverageChecks = true;
# };
coverage =
(import ./../ci/gha/tests rec {
withCoverage = true;
pkgs = nixpkgsFor.x86_64-linux.nativeForStdenv.clangStdenv;
nixComponents = pkgs.nixComponents2;
nixFlake = null;
getStdenv = p: p.clangStdenv;
}).codeCoverage.coverageReports.overrideAttrs
{
name = "nix-coverage"; # For historical consistency
};
# Nix's manual
manual = nixpkgsFor.x86_64-linux.native.nixComponents2.nix-manual;
+173
View File
@@ -0,0 +1,173 @@
#!/usr/bin/env bash
set -eu
set -o pipefail
# System specific settings
# FreeBSD typically uses UIDs from 1001+ for regular users,
# so we'll use a range that's unlikely to conflict
export NIX_FIRST_BUILD_UID="${NIX_FIRST_BUILD_UID:-30001}"
export NIX_BUILD_GROUP_ID="${NIX_BUILD_GROUP_ID:-30000}"
export NIX_BUILD_USER_NAME_TEMPLATE="nixbld%d"
# FreeBSD service paths
readonly SERVICE_SRC=/etc/rc.d/nix-daemon
readonly SERVICE_DEST=/usr/local/etc/rc.d/nix-daemon
poly_cure_artifacts() {
:
}
poly_service_installed_check() {
if [ -f "$SERVICE_DEST" ]; then
return 0
else
return 1
fi
}
poly_service_uninstall_directions() {
cat <<EOF
$1. Delete the rc.d service
sudo service nix-daemon stop
sudo rm -f $SERVICE_DEST
sudo sysrc -x nix_daemon_enable
EOF
}
poly_service_setup_note() {
cat <<EOF
- link the nix-daemon rc.d service to $SERVICE_DEST
EOF
}
poly_extra_try_me_commands() {
cat <<EOF
$ sudo service nix-daemon start
EOF
}
poly_configure_nix_daemon_service() {
task "Setting up the nix-daemon rc.d service"
# Ensure the rc.d directory exists
_sudo "to create the rc.d directory" \
mkdir -p /usr/local/etc/rc.d
# Link the pre-installed rc.d script
_sudo "to set up the nix-daemon service" \
ln -sfn "/nix/var/nix/profiles/default$SERVICE_SRC" "$SERVICE_DEST"
_sudo "to enable the nix-daemon service" \
sysrc nix_daemon_enable=YES
_sudo "to start the nix-daemon" \
service nix-daemon start
}
poly_group_exists() {
pw group show "$1" > /dev/null 2>&1
}
poly_group_id_get() {
pw group show "$1" | cut -d: -f3
}
poly_create_build_group() {
_sudo "Create the Nix build group, $NIX_BUILD_GROUP_NAME" \
pw groupadd -n "$NIX_BUILD_GROUP_NAME" -g "$NIX_BUILD_GROUP_ID" >&2
}
poly_user_exists() {
pw user show "$1" > /dev/null 2>&1
}
poly_user_id_get() {
pw user show "$1" | cut -d: -f3
}
poly_user_hidden_get() {
# FreeBSD doesn't have a concept of hidden users like macOS
echo "0"
}
poly_user_hidden_set() {
# No-op on FreeBSD
true
}
poly_user_home_get() {
pw user show "$1" | cut -d: -f9
}
poly_user_home_set() {
_sudo "in order to give $1 a safe home directory" \
pw usermod -n "$1" -d "$2"
}
poly_user_note_get() {
pw user show "$1" | cut -d: -f8
}
poly_user_note_set() {
_sudo "in order to give $1 a useful comment" \
pw usermod -n "$1" -c "$2"
}
poly_user_shell_get() {
pw user show "$1" | cut -d: -f10
}
poly_user_shell_set() {
_sudo "in order to prevent $1 from logging in" \
pw usermod -n "$1" -s "$2"
}
poly_user_in_group_check() {
groups "$1" 2>/dev/null | grep -q "\<$2\>"
}
poly_user_in_group_set() {
_sudo "Add $1 to the $2 group" \
pw groupmod -n "$2" -m "$1"
}
poly_user_primary_group_get() {
pw user show "$1" | cut -d: -f4
}
poly_user_primary_group_set() {
_sudo "to let the nix daemon use this user for builds" \
pw usermod -n "$1" -g "$2"
}
poly_create_build_user() {
username=$1
uid=$2
builder_num=$3
_sudo "Creating the Nix build user, $username" \
pw useradd \
-n "$username" \
-u "$uid" \
-g "$NIX_BUILD_GROUP_NAME" \
-G "$NIX_BUILD_GROUP_NAME" \
-d /var/empty \
-s /sbin/nologin \
-c "Nix build user $builder_num"
}
poly_prepare_to_install() {
# FreeBSD-specific preparation steps
:
}
poly_configure_default_profile_targets() {
# FreeBSD-specific profile locations
# FreeBSD uses /usr/local/etc for third-party shell configurations
# Include both profile (for login shells) and bashrc (for interactive shells)
echo "/usr/local/etc/profile /usr/local/etc/bashrc /usr/local/etc/profile.d/nix.sh /usr/local/etc/zshrc"
}
+26 -2
View File
@@ -33,7 +33,8 @@ readonly NIX_BUILD_GROUP_NAME="nixbld"
readonly NIX_ROOT="/nix"
readonly NIX_EXTRA_CONF=${NIX_EXTRA_CONF:-}
readonly PROFILE_TARGETS=("/etc/bashrc" "/etc/profile.d/nix.sh" "/etc/zshrc" "/etc/bash.bashrc" "/etc/zsh/zshrc")
# PROFILE_TARGETS will be set later after OS-specific scripts are loaded
PROFILE_TARGETS=()
readonly PROFILE_BACKUP_SUFFIX=".backup-before-nix"
readonly PROFILE_NIX_FILE="$NIX_ROOT/var/nix/profiles/default/etc/profile.d/nix-daemon.sh"
@@ -99,6 +100,14 @@ is_os_darwin() {
fi
}
is_os_freebsd() {
if [ "$(uname -s)" = "FreeBSD" ]; then
return 0
else
return 1
fi
}
contact_us() {
echo "You can open an issue at"
echo "https://github.com/NixOS/nix/issues/new?labels=installer&template=installer.md"
@@ -498,6 +507,10 @@ You have aborted the installation.
EOF
fi
fi
if is_os_freebsd; then
ok "Detected FreeBSD, will set up rc.d service for nix-daemon"
fi
}
setup_report() {
@@ -834,7 +847,7 @@ install_from_extracted_nix() {
(
cd "$EXTRACTED_NIX_PATH"
if is_os_darwin; then
if is_os_darwin || is_os_freebsd; then
_sudo "to copy the basic Nix files to the new store at $NIX_ROOT/store" \
cp -RPp ./store/* "$NIX_ROOT/store/"
else
@@ -989,11 +1002,22 @@ main() {
# shellcheck source=./install-systemd-multi-user.sh
. "$EXTRACTED_NIX_PATH/install-systemd-multi-user.sh" # most of this works on non-systemd distros also
check_required_system_specific_settings "install-systemd-multi-user.sh"
elif is_os_freebsd; then
# shellcheck source=./install-freebsd-multi-user.sh
. "$EXTRACTED_NIX_PATH/install-freebsd-multi-user.sh"
check_required_system_specific_settings "install-freebsd-multi-user.sh"
else
failure "Sorry, I don't know what to do on $(uname)"
fi
# Set profile targets after OS-specific scripts are loaded
if command -v poly_configure_default_profile_targets > /dev/null 2>&1; then
PROFILE_TARGETS=($(poly_configure_default_profile_targets))
else
PROFILE_TARGETS=("/etc/bashrc" "/etc/profile.d/nix.sh" "/etc/zshrc" "/etc/bash.bashrc" "/etc/zsh/zshrc")
fi
welcome_to_nix
if ! is_root; then
+14 -5
View File
@@ -26,8 +26,10 @@ if [ -z "$HOME" ]; then
exit 1
fi
OS="$(uname -s)"
# macOS support for 10.12.6 or higher
if [ "$(uname -s)" = "Darwin" ]; then
if [ "$OS" = "Darwin" ]; then
IFS='.' read -r macos_major macos_minor macos_patch << EOF
$(sw_vers -productVersion)
EOF
@@ -39,11 +41,11 @@ EOF
fi
# Determine if we could use the multi-user installer or not
if [ "$(uname -s)" = "Linux" ]; then
if [ "$OS" = "Linux" ] || [ "$OS" = "FreeBSD" ]; then
echo "Note: a multi-user installation is possible. See https://nix.dev/manual/nix/stable/installation/installing-binary.html#multi-user-installation" >&2
fi
case "$(uname -s)" in
case "$OS" in
"Darwin")
INSTALL_MODE=daemon;;
*)
@@ -60,7 +62,7 @@ while [ $# -gt 0 ]; do
ACTION=install
;;
--no-daemon)
if [ "$(uname -s)" = "Darwin" ]; then
if [ "$OS" = "Darwin" ]; then
printf '\e[1;31mError: --no-daemon installs are no-longer supported on Darwin/macOS!\e[0m\n' >&2
exit 1
fi
@@ -123,6 +125,13 @@ while [ $# -gt 0 ]; do
done
if [ "$INSTALL_MODE" = "daemon" ]; then
# Check for bash on systems that don't have it by default
if [ "$OS" = "FreeBSD" ] && ! command -v bash >/dev/null 2>&1; then
printf '\e[1;31mError: bash is required for multi-user installation but was not found.\e[0m\n' >&2
printf 'Please install bash first:\n' >&2
printf ' pkg install bash\n' >&2
exit 1
fi
printf '\e[1;31mSwitching to the Multi-user Installer\e[0m\n'
exec "$self/install-multi-user" $ACTION
exit 0
@@ -167,7 +176,7 @@ for i in $(cd "$self/store" >/dev/null && echo ./*); do
rm -rf "$i_tmp"
fi
if ! [ -e "$dest/store/$i" ]; then
if [ "$(uname -s)" = "Darwin" ]; then
if [ "$OS" = "Darwin" ] || [ "$OS" = "FreeBSD" ]; then
cp -RPp "$self/store/$i" "$i_tmp"
else
cp -RP --preserve=ownership,timestamps "$self/store/$i" "$i_tmp"
+1
View File
@@ -15,6 +15,7 @@
#include "nix/fetchers/fetch-to-store.hh"
#include "nix/cmd/compatibility-settings.hh"
#include "nix/expr/eval-settings.hh"
#include "nix/store/globals.hh"
namespace nix {
+10 -2
View File
@@ -37,9 +37,17 @@ static std::string doRenderMarkdownToTerminal(std::string_view markdown)
.vmargin = 0,
# endif
.feat = LOWDOWN_COMMONMARK | LOWDOWN_FENCED | LOWDOWN_DEFLIST | LOWDOWN_TABLES,
.oflags = LOWDOWN_TERM_NOLINK,
.oflags =
# if HAVE_LOWDOWN_1_4
LOWDOWN_TERM_NORELLINK // To render full links while skipping relative ones
# else
LOWDOWN_TERM_NOLINK
# endif
};
if (!isTTY())
opts.oflags |= LOWDOWN_TERM_NOANSI;
auto doc = lowdown_doc_new(&opts);
if (!doc)
throw Error("cannot allocate Markdown document");
@@ -65,7 +73,7 @@ static std::string doRenderMarkdownToTerminal(std::string_view markdown)
if (!rndr_res)
throw Error("allocation error while rendering Markdown");
return filterANSIEscapes(std::string(buf->data, buf->size), !isTTY());
return std::string(buf->data, buf->size);
}
std::string renderMarkdownToTerminal(std::string_view markdown)
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+2 -1
View File
@@ -53,7 +53,8 @@ mkMesonLibrary (finalAttrs: {
buildInputs = [
({ inherit editline readline; }.${readlineFlavor})
] ++ lib.optional enableMarkdown lowdown;
]
++ lib.optional enableMarkdown lowdown;
propagatedBuildInputs = [
nix-util
+4 -2
View File
@@ -574,14 +574,15 @@ ProcessLineResult NixRepl::processLine(std::string line)
for (auto & sub : subs) {
auto * logSubP = dynamic_cast<LogStore *>(&*sub);
if (!logSubP) {
printInfo("Skipped '%s' which does not support retrieving build logs", sub->getUri());
printInfo(
"Skipped '%s' which does not support retrieving build logs", sub->config.getHumanReadableURI());
continue;
}
auto & logSub = *logSubP;
auto log = logSub.getBuildLog(drvPath);
if (log) {
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.getUri());
printInfo("got build log for '%s' from '%s'", drvPathRaw, logSub.config.getHumanReadableURI());
logger->writeToStdout(*log);
foundLog = true;
break;
@@ -915,6 +916,7 @@ ReplExitStatus AbstractNixRepl::runSimple(ref<EvalState> evalState, const ValMap
return values;
};
LookupPath lookupPath = {};
// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDelete)
auto repl = std::make_unique<NixRepl>(
lookupPath,
openStore(),
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
@@ -1,3 +1,4 @@
#include <exception> // Needed by rapidcheck on Darwin
#include <rapidcheck.h>
#include "nix/store/tests/path.hh"
+1
View File
@@ -1,5 +1,6 @@
#include <nlohmann/json.hpp>
#include <gtest/gtest.h>
#include <exception> // Needed by rapidcheck on Darwin
#include <rapidcheck/gtest.h>
#include "nix/store/tests/derived-path.hh"
+2 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
@@ -75,6 +75,7 @@ this_exe = executable(
# TODO: -lrapidcheck, see ../libutil-support/build.meson
link_args : linker_export_flags + [ '-lrapidcheck' ],
install : true,
cpp_pch : do_pch ? [ 'pch/precompiled-headers.hh' ] : [],
)
test(
@@ -0,0 +1,4 @@
#include "nix/expr/tests/libexpr.hh"
#include <gtest/gtest.h>
#include <gmock/gmock.h>
+3 -2
View File
@@ -4,6 +4,7 @@
#include "nix/expr/eval.hh"
#include "nix/expr/eval-inline.hh"
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
// Need specialization involving `SymbolStr` just in this one module.
#include "nix/util/strings-inline.hh"
@@ -69,10 +70,10 @@ struct AttrDb
{
auto state(_state->lock());
Path cacheDir = getCacheDir() + "/eval-cache-v5";
auto cacheDir = std::filesystem::path(getCacheDir()) / "eval-cache-v6";
createDirs(cacheDir);
Path dbPath = cacheDir + "/" + fingerprint.to_string(HashFormat::Base16, false) + ".sqlite";
auto dbPath = cacheDir / (fingerprint.to_string(HashFormat::Base16, false) + ".sqlite");
state->db = SQLite(dbPath);
state->db.isCache();
-1
View File
@@ -1,6 +1,5 @@
#include "nix/expr/eval-profiler-settings.hh"
#include "nix/util/configuration.hh"
#include "nix/util/logging.hh" /* Needs to be included before config-impl.hh */
#include "nix/util/config-impl.hh"
#include "nix/util/abstract-setting-to-json.hh"
+1
View File
@@ -6,6 +6,7 @@
#include "nix/util/exit.hh"
#include "nix/util/types.hh"
#include "nix/util/util.hh"
#include "nix/util/environment-variables.hh"
#include "nix/store/store-api.hh"
#include "nix/store/derivations.hh"
#include "nix/store/downstream-placeholder.hh"
+1 -1
View File
@@ -22,7 +22,7 @@ struct Attr
way we keep Attr size at two words with no wasted space. */
Symbol name;
PosIdx pos;
Value * value;
Value * value = nullptr;
Attr(Symbol name, Value * value, PosIdx pos = noPos)
: name(name)
, pos(pos)
+9 -4
View File
@@ -595,12 +595,17 @@ struct ExprOpNot : Expr
{ \
return pos; \
} \
};
}
MakeBinOp(ExprOpEq, "==") MakeBinOp(ExprOpNEq, "!=") MakeBinOp(ExprOpAnd, "&&") MakeBinOp(ExprOpOr, "||")
MakeBinOp(ExprOpImpl, "->") MakeBinOp(ExprOpUpdate, "//") MakeBinOp(ExprOpConcatLists, "++")
MakeBinOp(ExprOpEq, "==");
MakeBinOp(ExprOpNEq, "!=");
MakeBinOp(ExprOpAnd, "&&");
MakeBinOp(ExprOpOr, "||");
MakeBinOp(ExprOpImpl, "->");
MakeBinOp(ExprOpUpdate, "//");
MakeBinOp(ExprOpConcatLists, "++");
struct ExprConcatStrings : Expr
struct ExprConcatStrings : Expr
{
PosIdx pos;
bool forceString;
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+4 -2
View File
@@ -70,13 +70,15 @@ mkMesonLibrary (finalAttrs: {
nix-util
nix-store
nix-fetchers
] ++ finalAttrs.passthru.externalPropagatedBuildInputs;
]
++ finalAttrs.passthru.externalPropagatedBuildInputs;
# Hack for sake of the dev shell
passthru.externalPropagatedBuildInputs = [
boost
nlohmann_json
] ++ lib.optional enableGC boehmgc;
]
++ lib.optional enableGC boehmgc;
mesonFlags = [
(lib.mesonEnable "gc" enableGC)
+1
View File
@@ -5,6 +5,7 @@
#include "nix/expr/eval-settings.hh"
#include "nix/expr/gc-small-vector.hh"
#include "nix/expr/json-to-value.hh"
#include "nix/store/globals.hh"
#include "nix/store/names.hh"
#include "nix/store/path-references.hh"
#include "nix/store/store-api.hh"
+1
View File
@@ -2,6 +2,7 @@
#include "nix/expr/eval-inline.hh"
#include "nix/store/derivations.hh"
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
namespace nix {
+2 -1
View File
@@ -3,6 +3,7 @@
#include "nix/store/realisation.hh"
#include "nix/store/make-content-addressed.hh"
#include "nix/util/url.hh"
#include "nix/util/environment-variables.hh"
namespace nix {
@@ -184,7 +185,7 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value ** args
{.msg = HintFmt("attribute '%s' is missing in call to 'fetchClosure'", "fromStore"),
.pos = state.positions[pos]});
auto parsedURL = parseURL(*fromStoreUrl);
auto parsedURL = parseURL(*fromStoreUrl, /*lenient=*/true);
if (parsedURL.scheme != "http" && parsedURL.scheme != "https"
&& !(getEnv("_NIX_IN_TEST").has_value() && parsedURL.scheme == "file"))
+2 -2
View File
@@ -113,7 +113,7 @@ static void fetchTree(
auto s = state.coerceToString(attr.pos, *attr.value, context, "", false, false).toOwned();
attrs.emplace(
state.symbols[attr.name],
params.isFetchGit && state.symbols[attr.name] == "url" ? fixGitURL(s) : s);
params.isFetchGit && state.symbols[attr.name] == "url" ? fixGitURL(s).to_string() : s);
} else if (attr.value->type() == nBool)
attrs.emplace(state.symbols[attr.name], Explicit<bool>{attr.value->boolean()});
else if (attr.value->type() == nInt) {
@@ -175,7 +175,7 @@ static void fetchTree(
if (params.isFetchGit) {
fetchers::Attrs attrs;
attrs.emplace("type", "git");
attrs.emplace("url", fixGitURL(url));
attrs.emplace("url", fixGitURL(url).to_string());
if (!attrs.contains("exportIgnore")
&& (!attrs.contains("submodules") || !*fetchers::maybeGetBoolAttr(attrs, "submodules"))) {
attrs.emplace("exportIgnore", Explicit<bool>{true});
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+54
View File
@@ -173,4 +173,58 @@ TEST_F(GitUtilsTest, peel_reference)
git_repository_free(rawRepo);
}
TEST(GitUtils, isLegalRefName)
{
ASSERT_TRUE(isLegalRefName("foox"));
ASSERT_TRUE(isLegalRefName("1337"));
ASSERT_TRUE(isLegalRefName("foo.baz"));
ASSERT_TRUE(isLegalRefName("foo/bar/baz"));
ASSERT_TRUE(isLegalRefName("foo./bar"));
ASSERT_TRUE(isLegalRefName("heads/foo@bar"));
ASSERT_TRUE(isLegalRefName("heads/fu\303\237"));
ASSERT_TRUE(isLegalRefName("foo-bar-baz"));
ASSERT_TRUE(isLegalRefName("branch#"));
ASSERT_TRUE(isLegalRefName("$1"));
ASSERT_TRUE(isLegalRefName("foo.locke"));
ASSERT_FALSE(isLegalRefName("refs///heads/foo"));
ASSERT_FALSE(isLegalRefName("heads/foo/"));
ASSERT_FALSE(isLegalRefName("///heads/foo"));
ASSERT_FALSE(isLegalRefName(".foo"));
ASSERT_FALSE(isLegalRefName("./foo"));
ASSERT_FALSE(isLegalRefName("./foo/bar"));
ASSERT_FALSE(isLegalRefName("foo/./bar"));
ASSERT_FALSE(isLegalRefName("foo/bar/."));
ASSERT_FALSE(isLegalRefName("foo bar"));
ASSERT_FALSE(isLegalRefName("foo?bar"));
ASSERT_FALSE(isLegalRefName("foo^bar"));
ASSERT_FALSE(isLegalRefName("foo~bar"));
ASSERT_FALSE(isLegalRefName("foo:bar"));
ASSERT_FALSE(isLegalRefName("foo[bar"));
ASSERT_FALSE(isLegalRefName("foo/bar/."));
ASSERT_FALSE(isLegalRefName(".refs/foo"));
ASSERT_FALSE(isLegalRefName("refs/heads/foo."));
ASSERT_FALSE(isLegalRefName("heads/foo..bar"));
ASSERT_FALSE(isLegalRefName("heads/foo?bar"));
ASSERT_FALSE(isLegalRefName("heads/foo.lock"));
ASSERT_FALSE(isLegalRefName("heads///foo.lock"));
ASSERT_FALSE(isLegalRefName("foo.lock/bar"));
ASSERT_FALSE(isLegalRefName("foo.lock///bar"));
ASSERT_FALSE(isLegalRefName("heads/v@{ation"));
ASSERT_FALSE(isLegalRefName("heads/foo\bar"));
ASSERT_FALSE(isLegalRefName("@"));
ASSERT_FALSE(isLegalRefName("\37"));
ASSERT_FALSE(isLegalRefName("\177"));
ASSERT_FALSE(isLegalRefName("foo/*"));
ASSERT_FALSE(isLegalRefName("*/foo"));
ASSERT_FALSE(isLegalRefName("foo/*/bar"));
ASSERT_FALSE(isLegalRefName("*"));
ASSERT_FALSE(isLegalRefName("foo/*/*"));
ASSERT_FALSE(isLegalRefName("*/foo/*"));
ASSERT_FALSE(isLegalRefName("/foo"));
ASSERT_FALSE(isLegalRefName(""));
}
} // namespace nix
+200
View File
@@ -0,0 +1,200 @@
#include "nix/store/store-open.hh"
#include "nix/store/globals.hh"
#include "nix/fetchers/fetch-settings.hh"
#include "nix/fetchers/fetchers.hh"
#include "nix/fetchers/git-utils.hh"
#include <git2.h>
#include <gtest/gtest.h>
#include <filesystem>
namespace {
template<class T, void (*F)(T *)>
struct Deleter
{
void operator()(T * p) const
{
if (p)
F(p);
}
};
template<class T, void (*F)(T *)>
using Handle = std::unique_ptr<T, Deleter<T, F>>;
using RepoHandle = Handle<git_repository, git_repository_free>;
using IndexHandle = Handle<git_index, git_index_free>;
using TreeHandle = Handle<git_tree, git_tree_free>;
using SigHandle = Handle<git_signature, git_signature_free>;
using RefHandle = Handle<git_reference, git_reference_free>;
using CommitHandle = Handle<git_commit, git_commit_free>;
using SubmoduleHandle = Handle<git_submodule, git_submodule_free>;
#define CHECK_LIBGIT(expr) ASSERT_TRUE((expr) >= 0) << git_error_last()
static void commitAll(git_repository * repo, const char * msg)
{
IndexHandle idx;
{
git_index * raw = nullptr;
CHECK_LIBGIT(git_repository_index(&raw, repo));
idx.reset(raw);
}
CHECK_LIBGIT(git_index_add_all(idx.get(), nullptr, 0, nullptr, nullptr));
CHECK_LIBGIT(git_index_write(idx.get()));
git_oid treeId{};
CHECK_LIBGIT(git_index_write_tree(&treeId, idx.get()));
TreeHandle tree;
{
git_tree * raw = nullptr;
CHECK_LIBGIT(git_tree_lookup(&raw, repo, &treeId));
tree.reset(raw);
}
SigHandle sig;
{
git_signature * raw = nullptr;
CHECK_LIBGIT(git_signature_now(&raw, "you", "you@example.com"));
sig.reset(raw);
}
git_oid commitId{};
if (git_repository_is_empty(repo) == 1) {
CHECK_LIBGIT(git_commit_create_v(&commitId, repo, "HEAD", sig.get(), sig.get(), nullptr, msg, tree.get(), 0));
CHECK_LIBGIT(git_reference_create(nullptr, repo, "refs/heads/main", &commitId, true, nullptr));
CHECK_LIBGIT(git_repository_set_head(repo, "refs/heads/main"));
} else {
RefHandle head;
{
git_reference * raw = nullptr;
CHECK_LIBGIT(git_repository_head(&raw, repo));
head.reset(raw);
}
CommitHandle parent;
{
git_commit * raw = nullptr;
CHECK_LIBGIT(git_commit_lookup(&raw, repo, git_reference_target(head.get())));
parent.reset(raw);
}
const git_commit * parents[] = {parent.get()};
CHECK_LIBGIT(git_commit_create(
&commitId,
repo,
"HEAD",
sig.get(),
sig.get(),
/*message_encoding=*/nullptr,
msg,
tree.get(),
/*parent_count=*/1,
&parents[0]));
}
}
} // namespace
using namespace nix;
class GitTest : public ::testing::Test
{
std::unique_ptr<AutoDelete> delTmpDir;
protected:
std::filesystem::path tmpDir;
void SetUp() override
{
tmpDir = createTempDir();
delTmpDir = std::make_unique<AutoDelete>(tmpDir, /*recursive=*/true);
nix::initLibStore(/*loadConfig=*/false);
git_libgit2_init();
}
void TearDown() override
{
delTmpDir.reset();
}
};
// Regression test for https://github.com/NixOS/nix/issues/13215
TEST_F(GitTest, submodulePeriodSupport)
{
auto storePath = tmpDir / "store";
auto repoPath = tmpDir / "repo";
auto submodulePath = tmpDir / "submodule";
// Set up our git directories: one top level and a submodule
// the submodule in the .gitmodules has the branch listed as '.'
// 1) Create sub repo
{
git_repository * raw = nullptr;
CHECK_LIBGIT(git_repository_init(&raw, submodulePath.string().c_str(), /*is_bare=*/0));
RepoHandle sub(raw);
writeFile(submodulePath / "lib.txt", "hello from submodule\n");
commitAll(sub.get(), "init sub");
}
// 2) Create super repo
RepoHandle super;
{
git_repository * raw = nullptr;
CHECK_LIBGIT(git_repository_init(&raw, repoPath.string().c_str(), /*is_bare=*/0));
super.reset(raw);
}
writeFile(repoPath / "README.md", "# super\n");
commitAll(super.get(), "init super");
// 3) Add submodule at deps/sub
{
git_repository * raw = nullptr;
git_clone_options cloneOpts = GIT_CLONE_OPTIONS_INIT;
// clone from local subPath into superPath/deps/sub
CHECK_LIBGIT(
git_clone(&raw, submodulePath.string().c_str(), (repoPath / "deps" / "sub").string().c_str(), &cloneOpts));
RepoHandle sub(raw);
}
// 4) Add submodule and set branch="."
SubmoduleHandle sm;
{
git_submodule * raw = nullptr;
CHECK_LIBGIT(git_submodule_add_setup(
&raw,
super.get(),
"../submodule",
"deps/sub",
/*use_gitlink=*/1));
sm.reset(raw);
}
CHECK_LIBGIT(git_submodule_set_branch(super.get(), git_submodule_name(sm.get()), /*branch=*/"."));
CHECK_LIBGIT(git_submodule_sync(sm.get()));
// 5) Finalize now that the worktree exists; libgit2 can read its HEAD OID
CHECK_LIBGIT(git_submodule_add_finalize(sm.get()));
// 6) Commit the addition in super
commitAll(super.get(), "Add submodule with branch='.'");
// TODO: Use dummy:// store with MemorySourceAccessor.
Path storeTmpDir = createTempDir();
auto storeTmpDirAutoDelete = AutoDelete(storeTmpDir, true);
ref<Store> store = openStore(storeTmpDir);
auto settings = fetchers::Settings{};
auto input = fetchers::Input::fromAttrs(
settings,
{
{"url", "file://" + repoPath.string()},
{"submodules", Explicit{true}},
{"type", "git"},
{"ref", "main"},
});
auto [accessor, i] = input.getAccessor(store);
ASSERT_EQ(accessor->readFile(CanonPath("deps/sub/lib.txt")), "hello from submodule\n");
}
+2 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
@@ -41,6 +41,7 @@ subdir('nix-meson-build-support/common')
sources = files(
'access-tokens.cc',
'git-utils.cc',
'git.cc',
'nix_api_fetchers.cc',
'public-key.cc',
)
+7 -11
View File
@@ -3,6 +3,7 @@
buildPackages,
stdenv,
mkMesonExecutable,
writableTmpDirAsHomeHook,
nix-fetchers,
nix-fetchers-c,
@@ -57,18 +58,13 @@ mkMesonExecutable (finalAttrs: {
runCommand "${finalAttrs.pname}-run"
{
meta.broken = !stdenv.hostPlatform.emulatorAvailable buildPackages;
buildInputs = [ writableTmpDirAsHomeHook ];
}
(
lib.optionalString stdenv.hostPlatform.isWindows ''
export HOME="$PWD/home-dir"
mkdir -p "$HOME"
''
+ ''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
''
);
''
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
'';
};
};
+2 -1
View File
@@ -4,6 +4,7 @@
#include "nix/store/sqlite.hh"
#include "nix/util/sync.hh"
#include "nix/store/store-api.hh"
#include "nix/store/globals.hh"
#include <nlohmann/json.hpp>
@@ -37,7 +38,7 @@ struct CacheImpl : Cache
{
auto state(_state.lock());
auto dbPath = getCacheDir() + "/fetcher-cache-v3.sqlite";
auto dbPath = getCacheDir() + "/fetcher-cache-v4.sqlite";
createDirs(dirOf(dbPath));
state->db = SQLite(dbPath);
+63 -34
View File
@@ -5,6 +5,7 @@
#include "nix/util/url.hh"
#include "nix/util/users.hh"
#include "nix/util/hash.hh"
#include "nix/store/ssh.hh"
#include <git2/attr.h>
#include <git2/config.h>
@@ -15,20 +16,19 @@
namespace nix::lfs {
// if authHeader is "", downloadToSink assumes no auth is expected
static void downloadToSink(
const std::string & url,
const std::string & authHeader,
const std::optional<std::string> & authHeader,
// FIXME: passing a StringSink is superfluous, we may as well
// return a string. Or use an abstract Sink for streaming.
StringSink & sink,
std::string sha256Expected,
size_t sizeExpected)
{
FileTransferRequest request(url);
FileTransferRequest request(parseURL(url));
Headers headers;
if (!authHeader.empty())
headers.push_back({"Authorization", authHeader});
if (authHeader.has_value())
headers.push_back({"Authorization", *authHeader});
request.headers = headers;
getFileTransfer()->download(std::move(request), sink);
@@ -42,30 +42,53 @@ static void downloadToSink(
"hash mismatch while fetching %s: expected sha256:%s but got sha256:%s", url, sha256Expected, sha256Actual);
}
static std::string getLfsApiToken(const ParsedURL & url)
namespace {
struct LfsApiInfo
{
std::string endpoint;
std::optional<std::string> authHeader;
};
} // namespace
static LfsApiInfo getLfsApi(const ParsedURL & url)
{
assert(url.authority.has_value());
if (url.scheme == "ssh") {
auto args = getNixSshOpts();
// FIXME: Not entirely correct.
auto [status, output] = runProgram(
RunOptions{
.program = "ssh",
.args = {url.authority->to_string(), "git-lfs-authenticate", url.path, "download"},
});
if (url.authority->port)
args.push_back(fmt("-p%d", *url.authority->port));
if (output.empty())
throw Error(
"git-lfs-authenticate: no output (cmd: ssh %s git-lfs-authenticate %s download)",
url.authority.value_or(ParsedURL::Authority{}).to_string(),
url.path);
std::ostringstream hostnameAndUser;
if (url.authority->user)
hostnameAndUser << *url.authority->user << "@";
hostnameAndUser << url.authority->host;
args.push_back(std::move(hostnameAndUser).str());
auto queryResp = nlohmann::json::parse(output);
if (!queryResp.contains("header"))
throw Error("no header in git-lfs-authenticate response");
if (!queryResp["header"].contains("Authorization"))
throw Error("no Authorization in git-lfs-authenticate response");
args.push_back("--");
args.push_back("git-lfs-authenticate");
args.push_back(url.path);
args.push_back("download");
return queryResp["header"]["Authorization"].get<std::string>();
auto [status, output] = runProgram({.program = "ssh", .args = args});
if (output.empty())
throw Error("git-lfs-authenticate: no output (cmd: 'ssh %s')", concatStringsSep(" ", args));
auto queryResp = nlohmann::json::parse(output);
auto headerIt = queryResp.find("header");
if (headerIt == queryResp.end())
throw Error("no header in git-lfs-authenticate response");
auto authIt = headerIt->find("Authorization");
if (authIt == headerIt->end())
throw Error("no Authorization in git-lfs-authenticate response");
return {queryResp.at("href").get<std::string>(), authIt->get<std::string>()};
}
return {url.to_string() + "/info/lfs", std::nullopt};
}
typedef std::unique_ptr<git_config, Deleter<git_config_free>> GitConfig;
@@ -156,7 +179,7 @@ Fetch::Fetch(git_repository * repo, git_oid rev)
const auto remoteUrl = lfs::getLfsEndpointUrl(repo);
this->url = nix::parseURL(nix::fixGitURL(remoteUrl)).canonicalise();
this->url = nix::fixGitURL(remoteUrl).canonicalise();
}
bool Fetch::shouldFetch(const CanonPath & path) const
@@ -181,13 +204,14 @@ static nlohmann::json pointerToPayload(const std::vector<Pointer> & items)
std::vector<nlohmann::json> Fetch::fetchUrls(const std::vector<Pointer> & pointers) const
{
ParsedURL httpUrl(url);
httpUrl.scheme = url.scheme == "ssh" ? "https" : url.scheme;
FileTransferRequest request(httpUrl.to_string() + "/info/lfs/objects/batch");
auto api = lfs::getLfsApi(this->url);
auto url = api.endpoint + "/objects/batch";
const auto & authHeader = api.authHeader;
FileTransferRequest request(parseURL(url));
request.post = true;
Headers headers;
if (this->url.scheme == "ssh")
headers.push_back({"Authorization", lfs::getLfsApiToken(this->url)});
if (authHeader.has_value())
headers.push_back({"Authorization", *authHeader});
headers.push_back({"Content-Type", "application/vnd.git-lfs+json"});
headers.push_back({"Accept", "application/vnd.git-lfs+json"});
request.headers = headers;
@@ -260,11 +284,16 @@ void Fetch::fetch(
try {
std::string sha256 = obj.at("oid"); // oid is also the sha256
std::string ourl = obj.at("actions").at("download").at("href");
std::string authHeader = "";
if (obj.at("actions").at("download").contains("header")
&& obj.at("actions").at("download").at("header").contains("Authorization")) {
authHeader = obj["actions"]["download"]["header"]["Authorization"];
}
auto authHeader = [&]() -> std::optional<std::string> {
const auto & download = obj.at("actions").at("download");
auto headerIt = download.find("header");
if (headerIt == download.end())
return std::nullopt;
auto authIt = headerIt->find("Authorization");
if (authIt == headerIt->end())
return std::nullopt;
return std::string(*authIt);
}();
const uint64_t size = obj.at("size");
sizeCallback(size);
downloadToSink(ourl, authHeader, sink, sha256, size);
+64 -2
View File
@@ -93,8 +93,11 @@ Hash toHash(const git_oid & oid)
static void initLibGit2()
{
if (git_libgit2_init() < 0)
throw Error("initialising libgit2: %s", git_error_last()->message);
static std::once_flag initialized;
std::call_once(initialized, []() {
if (git_libgit2_init() < 0)
throw Error("initialising libgit2: %s", git_error_last()->message);
});
}
git_oid hashToOID(const Hash & hash)
@@ -1308,4 +1311,63 @@ GitRepo::WorkdirInfo GitRepo::getCachedWorkdirInfo(const std::filesystem::path &
return workdirInfo;
}
/**
* Checks that the git reference is valid and normalizes slash '/' sequences.
*
* Accepts shorthand references (one-level refnames are allowed).
*/
bool isValidRefNameAllowNormalizations(const std::string & refName)
{
/* Unfortunately libgit2 doesn't expose the limit in headers, but its internal
limit is also 1024. */
std::array<char, 1024> normalizedRefBuffer;
/* It would be nice to have a better API like git_reference_name_is_valid, but
* with GIT_REFERENCE_FORMAT_REFSPEC_SHORTHAND flag. libgit2 uses it internally
* but doesn't expose it in public headers [1].
* [1]:
* https://github.com/libgit2/libgit2/blob/9d5f1bacc23594c2ba324c8f0d41b88bf0e9ef04/src/libgit2/refs.c#L1362-L1365
*/
auto res = git_reference_normalize_name(
normalizedRefBuffer.data(),
normalizedRefBuffer.size(),
refName.c_str(),
GIT_REFERENCE_FORMAT_ALLOW_ONELEVEL | GIT_REFERENCE_FORMAT_REFSPEC_SHORTHAND);
return res == 0;
}
bool isLegalRefName(const std::string & refName)
{
initLibGit2();
/* Since `git_reference_normalize_name` is the best API libgit2 has for verifying
* reference names with shorthands (see comment in normalizeRefName), we need to
* ensure that exceptions to the validity checks imposed by normalization [1] are checked
* explicitly.
* [1]: https://git-scm.com/docs/git-check-ref-format#Documentation/git-check-ref-format.txt---normalize
*/
/* Check for cases that don't get rejected by libgit2.
* FIXME: libgit2 should reject this. */
if (refName == "@")
return false;
/* Leading slashes and consecutive slashes are stripped during normalizatiton. */
if (refName.starts_with('/') || refName.find("//") != refName.npos)
return false;
/* Refer to libgit2. */
if (!isValidRefNameAllowNormalizations(refName))
return false;
/* libgit2 doesn't barf on DEL symbol.
* FIXME: libgit2 should reject this. */
if (refName.find('\177') != refName.npos)
return false;
return true;
}
} // namespace nix
+14 -10
View File
@@ -228,16 +228,12 @@ struct GitInputScheme : InputScheme
maybeGetBoolAttr(attrs, "verifyCommit");
if (auto ref = maybeGetStrAttr(attrs, "ref")) {
if (std::regex_search(*ref, badGitRefRegex))
throw BadURL("invalid Git branch/tag name '%s'", *ref);
}
if (auto ref = maybeGetStrAttr(attrs, "ref"); ref && !isLegalRefName(*ref))
throw BadURL("invalid Git branch/tag name '%s'", *ref);
Input input{settings};
input.attrs = attrs;
auto url = fixGitURL(getStrAttr(attrs, "url"));
parseURL(url);
input.attrs["url"] = url;
input.attrs["url"] = fixGitURL(getStrAttr(attrs, "url")).to_string();
getShallowAttr(input);
getSubmodulesAttr(input);
getAllRefsAttr(input);
@@ -741,8 +737,16 @@ struct GitInputScheme : InputScheme
fetchers::Attrs attrs;
attrs.insert_or_assign("type", "git");
attrs.insert_or_assign("url", resolved);
if (submodule.branch != "")
attrs.insert_or_assign("ref", submodule.branch);
if (submodule.branch != "") {
// A special value of . is used to indicate that the name of the branch in the submodule
// should be the same name as the current branch in the current repository.
// https://git-scm.com/docs/gitmodules
if (submodule.branch == ".") {
attrs.insert_or_assign("ref", ref);
} else {
attrs.insert_or_assign("ref", submodule.branch);
}
}
attrs.insert_or_assign("rev", submoduleRev.gitRev());
attrs.insert_or_assign("exportIgnore", Explicit<bool>{exportIgnore});
attrs.insert_or_assign("submodules", Explicit<bool>{true});
@@ -901,7 +905,7 @@ struct GitInputScheme : InputScheme
writeString(file.abs(), hashSink);
}
return makeFingerprint(*repoInfo.workdirInfo.headRev)
+ ";d=" + hashSink.finish().first.to_string(HashFormat::Base16, false);
+ ";d=" + hashSink.finish().hash.to_string(HashFormat::Base16, false);
}
return std::nullopt;
}
+7 -7
View File
@@ -19,7 +19,7 @@ namespace nix::fetchers {
struct DownloadUrl
{
std::string url;
ParsedURL url;
Headers headers;
};
@@ -48,7 +48,7 @@ struct GitArchiveInputScheme : InputScheme
if (size == 3) {
if (std::regex_match(path[2], revRegex))
rev = Hash::parseAny(path[2], HashAlgorithm::SHA1);
else if (std::regex_match(path[2], refRegex))
else if (isLegalRefName(path[2]))
ref = path[2];
else
throw BadURL("in URL '%s', '%s' is not a commit hash or branch/tag name", url, path[2]);
@@ -61,7 +61,7 @@ struct GitArchiveInputScheme : InputScheme
}
}
if (std::regex_match(rs, refRegex)) {
if (isLegalRefName(rs)) {
ref = rs;
} else {
throw BadURL("in URL '%s', '%s' is not a branch/tag name", url, rs);
@@ -75,7 +75,7 @@ struct GitArchiveInputScheme : InputScheme
throw BadURL("URL '%s' contains multiple commit hashes", url);
rev = Hash::parseAny(value, HashAlgorithm::SHA1);
} else if (name == "ref") {
if (!std::regex_match(value, refRegex))
if (!isLegalRefName(value))
throw BadURL("URL '%s' contains an invalid branch/tag name", url);
if (ref)
throw BadURL("URL '%s' contains multiple branch/tag names", url);
@@ -420,7 +420,7 @@ struct GitHubInputScheme : GitArchiveInputScheme
const auto url =
fmt(urlFmt, host, getOwner(input), getRepo(input), input.getRev()->to_string(HashFormat::Base16, false));
return DownloadUrl{url, headers};
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
@@ -500,7 +500,7 @@ struct GitLabInputScheme : GitArchiveInputScheme
input.getRev()->to_string(HashFormat::Base16, false));
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
return DownloadUrl{url, headers};
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
@@ -592,7 +592,7 @@ struct SourceHutInputScheme : GitArchiveInputScheme
input.getRev()->to_string(HashFormat::Base16, false));
Headers headers = makeHeadersWithAuthTokens(*input.settings, host, input);
return DownloadUrl{url, headers};
return DownloadUrl{parseURL(url), headers};
}
void clone(const Input & input, const Path & destDir) const override
@@ -157,4 +157,11 @@ struct Setter
}
};
/**
* Checks that the git reference is valid and normalized.
*
* Accepts shorthand references (one-level refnames are allowed).
*/
bool isLegalRefName(const std::string & refName);
} // namespace nix
+3 -2
View File
@@ -1,4 +1,5 @@
#include "nix/fetchers/fetchers.hh"
#include "nix/fetchers/git-utils.hh"
#include "nix/util/url-parts.hh"
#include "nix/store/path.hh"
@@ -22,12 +23,12 @@ struct IndirectInputScheme : InputScheme
} else if (path.size() == 2) {
if (std::regex_match(path[1], revRegex))
rev = Hash::parseAny(path[1], HashAlgorithm::SHA1);
else if (std::regex_match(path[1], refRegex))
else if (isLegalRefName(path[1]))
ref = path[1];
else
throw BadURL("in flake URL '%s', '%s' is not a commit hash or branch/tag name", url, path[1]);
} else if (path.size() == 3) {
if (!std::regex_match(path[1], refRegex))
if (!isLegalRefName(path[1]))
throw BadURL("in flake URL '%s', '%s' is not a branch/tag name", url, path[1]);
ref = path[1];
if (!std::regex_match(path[2], revRegex))
+2 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
@@ -66,6 +66,7 @@ this_library = library(
link_args : linker_export_flags,
prelink : true, # For C++ static initializers
install : true,
cpp_pch : do_pch ? [ 'pch/precompiled-headers.hh' ] : [],
)
install_headers(headers, subdir : 'nix/fetchers', preserve_path : true)
@@ -0,0 +1,3 @@
#include "nix/fetchers/fetchers.hh"
#include "nix/store/store-api.hh"
#include "nix/util/json-utils.hh"
+2 -2
View File
@@ -43,7 +43,7 @@ DownloadFileResult downloadFile(
if (cached && !cached->expired)
return useCached();
FileTransferRequest request(url);
FileTransferRequest request(parseURL(url));
request.headers = headers;
if (cached)
request.expectedETag = getStrAttr(cached->value, "etag");
@@ -153,7 +153,7 @@ static DownloadTarballResult downloadTarball_(
auto _res = std::make_shared<Sync<FileTransferResult>>();
auto source = sinkToSource([&](Sink & sink) {
FileTransferRequest req(url);
FileTransferRequest req(parseURL(url));
req.expectedETag = cached ? getStrAttr(cached->value, "etag") : "";
getFileTransfer()->download(std::move(req), sink, [_res](FileTransferResult r) { *_res->lock() = r; });
});
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+33
View File
@@ -57,6 +57,39 @@ TEST(parseFlakeRef, path)
}
}
TEST(parseFlakeRef, GitArchiveInput)
{
experimentalFeatureSettings.experimentalFeatures.get().insert(Xp::Flakes);
fetchers::Settings fetchSettings;
{
auto s = "github:foo/bar/branch%23"; // branch name with `#`
auto flakeref = parseFlakeRef(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "github:foo/bar/branch%23");
}
{
auto s = "github:foo/bar?ref=branch%23"; // branch name with `#`
auto flakeref = parseFlakeRef(fetchSettings, s);
ASSERT_EQ(flakeref.to_string(), "github:foo/bar/branch%23");
}
{
auto s = "github:foo/bar?ref=branch#\"name.with.dot\""; // unescaped quotes `"`
auto [flakeref, fragment] = parseFlakeRefWithFragment(fetchSettings, s);
ASSERT_EQ(fragment, "\"name.with.dot\"");
ASSERT_EQ(flakeref.to_string(), "github:foo/bar/branch");
}
{
auto s = "github:foo/bar#\"name.with.dot\""; // unescaped quotes `"`
auto [flakeref, fragment] = parseFlakeRefWithFragment(fetchSettings, s);
ASSERT_EQ(fragment, "\"name.with.dot\"");
ASSERT_EQ(flakeref.to_string(), "github:foo/bar");
}
}
TEST(to_string, doesntReencodeUrl)
{
fetchers::Settings fetchSettings;
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+4 -3
View File
@@ -13,8 +13,9 @@ TEST(getNameFromURL, getNameFromURL)
ASSERT_EQ(getNameFromURL(parseURL("path:~/repos/nixpkgs#packages.x86_64-linux.Hello")), "Hello");
ASSERT_EQ(getNameFromURL(parseURL("path:.#nonStandardAttr.mylaptop")), "mylaptop");
ASSERT_EQ(getNameFromURL(parseURL("path:./repos/myflake#nonStandardAttr.mylaptop")), "mylaptop");
ASSERT_EQ(getNameFromURL(parseURL("path:./nixpkgs#packages.x86_64-linux.complex^bin,man")), "complex");
ASSERT_EQ(getNameFromURL(parseURL("path:./myproj#packages.x86_64-linux.default^*")), "myproj");
ASSERT_EQ(
getNameFromURL(parseURL("path:./nixpkgs#packages.x86_64-linux.complex^bin,man", /*lenient=*/true)), "complex");
ASSERT_EQ(getNameFromURL(parseURL("path:./myproj#packages.x86_64-linux.default^*", /*lenient=*/true)), "myproj");
ASSERT_EQ(getNameFromURL(parseURL("path:./myproj#defaultPackage.x86_64-linux")), "myproj");
ASSERT_EQ(getNameFromURL(parseURL("github:NixOS/nixpkgs#packages.x86_64-linux.hello")), "hello");
@@ -80,6 +81,6 @@ TEST(getNameFromURL, getNameFromURL)
ASSERT_EQ(getNameFromURL(parseURL("path:.")), std::nullopt);
ASSERT_EQ(getNameFromURL(parseURL("file:.#")), std::nullopt);
ASSERT_EQ(getNameFromURL(parseURL("path:.#packages.x86_64-linux.default")), std::nullopt);
ASSERT_EQ(getNameFromURL(parseURL("path:.#packages.x86_64-linux.default^*")), std::nullopt);
ASSERT_EQ(getNameFromURL(parseURL("path:.#packages.x86_64-linux.default^*", /*lenient=*/true)), std::nullopt);
}
} // namespace nix
+3 -3
View File
@@ -82,7 +82,7 @@ std::pair<FlakeRef, std::string> parsePathFlakeRefWithFragment(
auto succeeds = std::regex_match(url, match, pathFlakeRegex);
assert(succeeds);
auto path = match[1].str();
auto query = decodeQuery(match[3]);
auto query = decodeQuery(match[3].str(), /*lenient=*/true);
auto fragment = percentDecode(match[5].str());
if (baseDir) {
@@ -210,7 +210,7 @@ std::optional<std::pair<FlakeRef, std::string>> parseURLFlakeRef(
bool isFlake)
{
try {
auto parsed = parseURL(url);
auto parsed = parseURL(url, /*lenient=*/true);
if (baseDir && (parsed.scheme == "path" || parsed.scheme == "git+file") && !isAbsolute(parsed.path))
parsed.path = absPath(parsed.path, *baseDir);
return fromParsedURL(fetchSettings, std::move(parsed), isFlake);
@@ -289,7 +289,7 @@ FlakeRef FlakeRef::canonicalize() const
filtering the `dir` query parameter from the URL. */
if (auto url = fetchers::maybeGetStrAttr(flakeRef.input.attrs, "url")) {
try {
auto parsed = parseURL(*url);
auto parsed = parseURL(*url, /*lenient=*/true);
if (auto dir2 = get(parsed.query, "dir")) {
if (flakeRef.subdir != "" && flakeRef.subdir == *dir2)
parsed.query.erase("dir");
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+1 -1
View File
@@ -62,7 +62,7 @@ nix_err nix_store_get_uri(nix_c_context * context, Store * store, nix_get_string
if (context)
context->last_err_code = NIX_OK;
try {
auto res = store->ptr->getUri();
auto res = store->ptr->config.getReference().render(/*withParams=*/true);
return call_nix_get_string_callback(res, callback, user_data);
}
NIXC_CATCH_ERRS
@@ -1,5 +1,6 @@
#include <regex>
#include <exception> // Needed by rapidcheck on Darwin
#include <rapidcheck.h>
#include "nix/store/tests/derived-path.hh"
@@ -6,6 +6,7 @@
#include "nix/store/store-api.hh"
#include "nix/store/store-open.hh"
#include "nix/store/globals.hh"
namespace nix {
@@ -1,6 +1,7 @@
#pragma once
///@file
#include <exception> // Needed by rapidcheck on Darwin
#include <rapidcheck/gen/Arbitrary.h>
#include "nix/store/outputs-spec.hh"
@@ -10,7 +10,7 @@
namespace nix {
template<class Proto, const char * protocolDir>
class ProtoTest : public CharacterizationTest, public LibStoreTest
class ProtoTest : public CharacterizationTest
{
std::filesystem::path unitTestData = getUnitTestData() / protocolDir;
@@ -18,6 +18,10 @@ class ProtoTest : public CharacterizationTest, public LibStoreTest
{
return unitTestData / (std::string{testStem + ".bin"});
}
public:
Path storeDir = "/nix/store";
StoreDirConfig store{storeDir};
};
template<class Proto, const char * protocolDir>
@@ -34,7 +38,7 @@ public:
T got = ({
StringSource from{encoded};
Proto::template Serialise<T>::read(
*LibStoreTest::store,
this->store,
typename Proto::ReadConn{
.from = from,
.version = version,
@@ -54,7 +58,7 @@ public:
CharacterizationTest::writeTest(testStem, [&]() {
StringSink to;
Proto::template Serialise<T>::write(
*LibStoreTest::store,
this->store,
typename Proto::WriteConn{
.to = to,
.version = version,
+1 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
+2 -1
View File
@@ -1,6 +1,7 @@
#include <rapidcheck/gen/Arbitrary.h>
#include <exception> // Needed by rapidcheck on Darwin
#include <regex>
#include <rapidcheck/gen/Arbitrary.h>
#include <rapidcheck.h>
#include "nix/store/path-regex.hh"
+2 -2
View File
@@ -25,7 +25,7 @@ public:
CharacterizationTest::readTest(testStem, [&](const auto & encoded) {
T got = ({
StringSource from{encoded};
CommonProto::Serialise<T>::read(*store, CommonProto::ReadConn{.from = from});
CommonProto::Serialise<T>::read(store, CommonProto::ReadConn{.from = from});
});
ASSERT_EQ(got, expected);
@@ -40,7 +40,7 @@ public:
{
CharacterizationTest::writeTest(testStem, [&]() -> std::string {
StringSink to;
CommonProto::Serialise<T>::write(*store, CommonProto::WriteConn{.to = to}, decoded);
CommonProto::Serialise<T>::write(store, CommonProto::WriteConn{.to = to}, decoded);
return to.s;
});
}
@@ -1 +1 @@
Derive([("out","/nix/store/jycqnr8rdfy035ckiwmar4yql406jjh6-firefox-140.0.4","","")],[("/nix/store/01qkwh6g0xqlrchziwa8zm3c8p6s5ylc-krb5-1.21.3.drv",["lib"]),("/nix/store/03ixv9yqpmi015vv0m4gza1d2ac8wql8-libglvnd-1.7.0.drv",["out"]),("/nix/store/2g73b2yd3qhzp774i9svckdnspyrfrn8-firefox-unwrapped-140.0.4.drv",["out"]),("/nix/store/2wf9wi34rvwyq7dxk0xlk41ccll282z0-make-shell-wrapper-hook.drv",["out"]),("/nix/store/3gr22851nw1nj8zmy34961yb3fbr2iz5-libva-2.22.0.drv",["out"]),("/nix/store/7gzk0w5125wqhgp8sh26l3k15lklm82c-policies.json.drv",["out"]),("/nix/store/b3jvr65plcm9nm4wn8ssw2ymnv2zshmq-extract-binary-wrapper-cmd.drv",["out"]),("/nix/store/ck7ckwkvqv49hz5rl8yp52c4v6p2f6vd-pipewire-1.4.6.drv",["out"]),("/nix/store/fm2b1q5m1wrvy237j6x9xcvhxw95wjfp-alsa-lib-1.2.14.drv",["out"]),("/nix/store/fvch7vyklf8gr8i53p9ybv2azyh6q30y-ffmpeg-7.1.1.drv",["lib"]),("/nix/store/ggzpx6yx9xd5h241ri730acdfpdirxjq-mesa-libgbm-25.1.0.drv",["out"]),("/nix/store/hmwbmap5c8zp3kl20bda0qk1dg4f408d-vulkan-loader-1.4.313.0.drv",["out"]),("/nix/store/hy8x576wmvz2fk9a778r05lqdb0dcbl4-speech-dispatcher-0.12.1.drv",["out"]),("/nix/store/knjlgd5nfd7dw45axa6z7w7d7rb11801-pciutils-3.14.0.drv",["out"]),("/nix/store/l7y4nybngpffyndkw8rwx2g3c8dw4yyj-xdg-utils-1.2.1.drv",["out"]),("/nix/store/lh0mbixp7jiz7z8cxfypm8b17lf8z9la-cups-2.4.12.drv",["lib"]),("/nix/store/lmfgw4dz59sdvplfh3zazqmin8d7gh9d-libnotify-0.8.6.drv",["out"]),("/nix/store/m4zp8p8y4wzxg2lhgq43kbgr2xl5l9px-firefox.desktop.drv",["out"]),("/nix/store/n8hy5gvnrw0kzz6cjdhky3rxr0q024sh-jq-1.8.1.drv",["dev"]),("/nix/store/nz0xp44xmbqcp2sfhsgicsswrrb2cpl7-libXScrnSaver-1.2.4.drv",["out"]),("/nix/store/p2nvbwxxiarbqaq26rqrjqkyw8dl0cf4-lndir-1.0.5.drv",["out"]),("/nix/store/p306lljvhw8906r1lxyyk7jn3bckdypn-sndio-1.10.0.drv",["out"]),("/nix/store/pbqbdcp76r373256ajlbp3fankpw5pdh-libpulseaudio-17.0.drv",["out"]),("/nix/store/q27q075ckyjkpqxx8cgvpghcwr0nzz5v-systemd-minimal-libs-257.6.drv",["out"]),("/nix/store/s4b8yadif84kiv8gyr9nxdi6zbg69b4g-bash-5.2p37.drv",["out"]),("/nix/store/sc2pgkzc1s6zp5dp8j7wsd4msilsnijn-stdenv-linux.drv",["out"]),("/nix/store/sy2kn7hwfw0nyvxq7rvzrmvm2ypxs9x6-gtk+3-3.24.49.drv",["dev"]),("/nix/store/v3sc4j11fhsah2ij0wkziw6nd3il9dyy-libjack2-1.9.22.drv",["out"]),("/nix/store/xy07yjv0x8bcdc3lzyq45lrbbz08hg5p-adwaita-icon-theme-48.0.drv",["out"]),("/nix/store/z8v9j1cf6faasgqrw3jw74xjyjiqwk82-libcanberra-0.30.drv",["out"])],["/nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh","/nix/store/vj1c3wf9c11a0qs6p3ymfvrnsdgsdcbq-source-stdenv.sh"],"x86_64-linux","/nix/store/p79bgyzmmmddi554ckwzbqlavbkw07zh-bash-5.2p37/bin/bash",["-e","/nix/store/vj1c3wf9c11a0qs6p3ymfvrnsdgsdcbq-source-stdenv.sh","/nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh"],[("__json","{\"NIX_MAIN_PROGRAM\":\"firefox\",\"buildCommand\":\"if [ ! -x \\\"/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4/bin/firefox\\\" ]\\nthen\\n echo \\\"cannot find executable file \\\\`/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4/bin/firefox'\\\"\\n exit 1\\nfi\\n\\n#########################\\n# #\\n# EXTRA PREF CHANGES #\\n# #\\n#########################\\n# Link the runtime. The executable itself has to be copied,\\n# because it will resolve paths relative to its true location.\\n# Any symbolic links have to be replicated as well.\\ncd \\\"/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4\\\"\\nfind . -type d -exec mkdir -p \\\"$out\\\"/{} \\\\;\\n\\nfind . -type f \\\\( -not -name \\\"firefox\\\" \\\\) -exec ln -sT \\\"/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4\\\"/{} \\\"$out\\\"/{} \\\\;\\n\\nfind . -type f \\\\( -name \\\"firefox\\\" -o -name \\\"firefox-bin\\\" \\\\) -print0 | while read -d $'\\\\0' f; do\\n cp -P --no-preserve=mode,ownership --remove-destination \\\"/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4/$f\\\" \\\"$out/$f\\\"\\n chmod a+rwx \\\"$out/$f\\\"\\ndone\\n\\n# fix links and absolute references\\n\\nfind . -type l -print0 | while read -d $'\\\\0' l; do\\n target=\\\"$(readlink \\\"$l\\\")\\\"\\n target=${target/#\\\"/nix/store/9ycwpmlavjsjgr0svaqdy0mmjg949nzq-firefox-unwrapped-140.0.4\\\"/\\\"$out\\\"}\\n ln -sfT \\\"$target\\\" \\\"$out/$l\\\"\\ndone\\n\\ncd \\\"$out\\\"\\n\\n\\n# create the wrapper\\n\\nexecutablePrefix=\\\"$out/bin\\\"\\nexecutablePath=\\\"$out/bin/firefox\\\"\\noldWrapperArgs=()\\n\\nif [[ -L $executablePath ]]; then\\n # Symbolic link: wrap the link's target.\\n oldExe=\\\"$(readlink -v --canonicalize-existing \\\"$executablePath\\\")\\\"\\n rm \\\"$executablePath\\\"\\nelif wrapperCmd=$(/nix/store/qczbm5rh1vfkql4jznp1p5lv9nyjz99r-extract-binary-wrapper-cmd \\\"$executablePath\\\"); [[ $wrapperCmd ]]; then\\n # If the executable is a binary wrapper, we need to update its target to\\n # point to $out, but we can't just edit the binary in-place because of length\\n # issues. SLine truncated
Derive([("out","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-140.0.4","","")],[("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-krb5-1.21.3.drv",["lib"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libglvnd-1.7.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-make-shell-wrapper-hook.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libva-2.22.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-policies.json.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-extract-binary-wrapper-cmd.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-pipewire-1.4.6.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-alsa-lib-1.2.14.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-ffmpeg-7.1.1.drv",["lib"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-mesa-libgbm-25.1.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-vulkan-loader-1.4.313.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-speech-dispatcher-0.12.1.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-pciutils-3.14.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-xdg-utils-1.2.1.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-cups-2.4.12.drv",["lib"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libnotify-0.8.6.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox.desktop.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-jq-1.8.1.drv",["dev"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libXScrnSaver-1.2.4.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-lndir-1.0.5.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-sndio-1.10.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libpulseaudio-17.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-systemd-minimal-libs-257.6.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bash-5.2p37.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-stdenv-linux.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-gtk+3-3.24.49.drv",["dev"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libjack2-1.9.22.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-adwaita-icon-theme-48.0.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-libcanberra-0.30.drv",["out"])],["/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-default-builder.sh","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-source-stdenv.sh"],"x86_64-linux","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bash-5.2p37/bin/bash",["-e","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-source-stdenv.sh","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-default-builder.sh"],[("__json","{\"NIX_MAIN_PROGRAM\":\"firefox\",\"buildCommand\":\"if [ ! -x \\\"/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4/bin/firefox\\\" ]\\nthen\\n echo \\\"cannot find executable file \\\\`/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4/bin/firefox'\\\"\\n exit 1\\nfi\\n\\n#########################\\n# #\\n# EXTRA PREF CHANGES #\\n# #\\n#########################\\n# Link the runtime. The executable itself has to be copied,\\n# because it will resolve paths relative to its true location.\\n# Any symbolic links have to be replicated as well.\\ncd \\\"/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4\\\"\\nfind . -type d -exec mkdir -p \\\"$out\\\"/{} \\\\;\\n\\nfind . -type f \\\\( -not -name \\\"firefox\\\" \\\\) -exec ln -sT \\\"/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4\\\"/{} \\\"$out\\\"/{} \\\\;\\n\\nfind . -type f \\\\( -name \\\"firefox\\\" -o -name \\\"firefox-bin\\\" \\\\) -print0 | while read -d $'\\\\0' f; do\\n cp -P --no-preserve=mode,ownership --remove-destination \\\"/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4/$f\\\" \\\"$out/$f\\\"\\n chmod a+rwx \\\"$out/$f\\\"\\ndone\\n\\n# fix links and absolute references\\n\\nfind . -type l -print0 | while read -d $'\\\\0' l; do\\n target=\\\"$(readlink \\\"$l\\\")\\\"\\n target=${target/#\\\"/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-firefox-unwrapped-140.0.4\\\"/\\\"$out\\\"}\\n ln -sfT \\\"$target\\\" \\\"$out/$l\\\"\\ndone\\n\\ncd \\\"$out\\\"\\n\\n\\n# create the wrapper\\n\\nexecutablePrefix=\\\"$out/bin\\\"\\nexecutablePath=\\\"$out/bin/firefox\\\"\\noldWrapperArgs=()\\n\\nif [[ -L $executablePath ]]; then\\n # Symbolic link: wrap the link's target.\\n oldExe=\\\"$(readlink -v --canonicalize-existing \\\"$executablePath\\\")\\\"\\n rm \\\"$executablePath\\\"\\nelif wrapperCmd=$(/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-extract-binary-wrapper-cmd \\\"$executablePath\\\"); [[ $wrapperCmd ]]; then\\n # If the executable is a binary wrapper, we need to update its target to\\n # point to $out, but we can't just edit the binary in-place because of length\\n # issues. SLine truncated
+1 -1
View File
@@ -1 +1 @@
Derive([("out","/nix/store/hhg83gh653wjw4ny49xn92f13v2j1za4-hello-2.12.2","","")],[("/nix/store/1xz4avqqrxqsxw7idz119vdzw837p1n1-version-check-hook.drv",["out"]),("/nix/store/bsv47sbqcar3205il55spxqacxp8j0fj-hello-2.12.2.tar.gz.drv",["out"]),("/nix/store/s4b8yadif84kiv8gyr9nxdi6zbg69b4g-bash-5.2p37.drv",["out"]),("/nix/store/sc2pgkzc1s6zp5dp8j7wsd4msilsnijn-stdenv-linux.drv",["out"])],["/nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh","/nix/store/vj1c3wf9c11a0qs6p3ymfvrnsdgsdcbq-source-stdenv.sh"],"x86_64-linux","/nix/store/p79bgyzmmmddi554ckwzbqlavbkw07zh-bash-5.2p37/bin/bash",["-e","/nix/store/vj1c3wf9c11a0qs6p3ymfvrnsdgsdcbq-source-stdenv.sh","/nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh"],[("NIX_MAIN_PROGRAM","hello"),("__structuredAttrs",""),("buildInputs",""),("builder","/nix/store/p79bgyzmmmddi554ckwzbqlavbkw07zh-bash-5.2p37/bin/bash"),("cmakeFlags",""),("configureFlags",""),("depsBuildBuild",""),("depsBuildBuildPropagated",""),("depsBuildTarget",""),("depsBuildTargetPropagated",""),("depsHostHost",""),("depsHostHostPropagated",""),("depsTargetTarget",""),("depsTargetTargetPropagated",""),("doCheck","1"),("doInstallCheck","1"),("mesonFlags",""),("name","hello-2.12.2"),("nativeBuildInputs","/nix/store/fxzn6kr5anxn5jgh511x56wrg8b3a99a-version-check-hook"),("out","/nix/store/hhg83gh653wjw4ny49xn92f13v2j1za4-hello-2.12.2"),("outputs","out"),("patches",""),("pname","hello"),("postInstallCheck","stat \"${!outputBin}/bin/hello\"\n"),("propagatedBuildInputs",""),("propagatedNativeBuildInputs",""),("src","/nix/store/dw402azxjrgrzrk6j0p66wkqrab5mwgw-hello-2.12.2.tar.gz"),("stdenv","/nix/store/a13rl87yjhzqrbkc4gb0mrwz2mfkivcf-stdenv-linux"),("strictDeps",""),("system","x86_64-linux"),("version","2.12.2")])
Derive([("out","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-hello-2.12.2","","")],[("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-version-check-hook.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-hello-2.12.2.tar.gz.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bash-5.2p37.drv",["out"]),("/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-stdenv-linux.drv",["out"])],["/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-default-builder.sh","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-source-stdenv.sh"],"x86_64-linux","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bash-5.2p37/bin/bash",["-e","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-source-stdenv.sh","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-default-builder.sh"],[("NIX_MAIN_PROGRAM","hello"),("__structuredAttrs",""),("buildInputs",""),("builder","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-bash-5.2p37/bin/bash"),("cmakeFlags",""),("configureFlags",""),("depsBuildBuild",""),("depsBuildBuildPropagated",""),("depsBuildTarget",""),("depsBuildTargetPropagated",""),("depsHostHost",""),("depsHostHostPropagated",""),("depsTargetTarget",""),("depsTargetTargetPropagated",""),("doCheck","1"),("doInstallCheck","1"),("mesonFlags",""),("name","hello-2.12.2"),("nativeBuildInputs","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-version-check-hook"),("out","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-hello-2.12.2"),("outputs","out"),("patches",""),("pname","hello"),("postInstallCheck","stat \"${!outputBin}/bin/hello\"\n"),("propagatedBuildInputs",""),("propagatedNativeBuildInputs",""),("src","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-hello-2.12.2.tar.gz"),("stdenv","/nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-stdenv-linux"),("strictDeps",""),("system","x86_64-linux"),("version","2.12.2")])
@@ -8,14 +8,14 @@ TEST(HttpBinaryCacheStore, constructConfig)
{
HttpBinaryCacheStoreConfig config{"http", "foo.bar.baz", {}};
EXPECT_EQ(config.cacheUri, "http://foo.bar.baz");
EXPECT_EQ(config.cacheUri.to_string(), "http://foo.bar.baz");
}
TEST(HttpBinaryCacheStore, constructConfigNoTrailingSlash)
{
HttpBinaryCacheStoreConfig config{"https", "foo.bar.baz/a/b/", {}};
EXPECT_EQ(config.cacheUri, "https://foo.bar.baz/a/b");
EXPECT_EQ(config.cacheUri.to_string(), "https://foo.bar.baz/a/b");
}
} // namespace nix
+7 -3
View File
@@ -6,21 +6,25 @@ namespace nix {
TEST(LegacySSHStore, constructConfig)
{
LegacySSHStoreConfig config{
LegacySSHStoreConfig config(
"ssh",
"localhost",
"me@localhost:2222",
StoreConfig::Params{
{
"remote-program",
// TODO #11106, no more split on space
"foo bar",
},
}};
});
EXPECT_EQ(
config.remoteProgram.get(),
(Strings{
"foo",
"bar",
}));
EXPECT_EQ(config.getReference().render(/*withParams=*/true), "ssh://me@localhost:2222?remote-program=foo%20bar");
EXPECT_EQ(config.getReference().render(/*withParams=*/false), "ssh://me@localhost:2222");
}
} // namespace nix
+2 -6
View File
@@ -1,9 +1,6 @@
// FIXME: Odd failures for templates that are causing the PR to break
// for now with discussion with @Ericson2314 to comment out.
#if 0
# include <gtest/gtest.h>
#include <gtest/gtest.h>
# include "nix/store/local-overlay-store.hh"
#include "nix/store/local-overlay-store.hh"
namespace nix {
@@ -31,4 +28,3 @@ TEST(LocalOverlayStore, constructConfig_rootPath)
}
} // namespace nix
#endif
+5 -9
View File
@@ -1,15 +1,12 @@
// FIXME: Odd failures for templates that are causing the PR to break
// for now with discussion with @Ericson2314 to comment out.
#if 0
# include <gtest/gtest.h>
#include <gtest/gtest.h>
# include "nix/store/local-store.hh"
#include "nix/store/local-store.hh"
// Needed for template specialisations. This is not good! When we
// overhaul how store configs work, this should be fixed.
# include "nix/util/args.hh"
# include "nix/util/config-impl.hh"
# include "nix/util/abstract-setting-to-json.hh"
#include "nix/util/args.hh"
#include "nix/util/config-impl.hh"
#include "nix/util/abstract-setting-to-json.hh"
namespace nix {
@@ -37,4 +34,3 @@ TEST(LocalStore, constructConfig_rootPath)
}
} // namespace nix
#endif
+4 -1
View File
@@ -3,7 +3,7 @@ project(
'cpp',
version : files('.version'),
default_options : [
'cpp_std=c++2a',
'cpp_std=c++23',
# TODO(Qyriad): increase the warning level
'warning_level=1',
'errorlogs=true', # Please print logs for tests that fail
@@ -74,6 +74,7 @@ sources = files(
'path.cc',
'references.cc',
's3-binary-cache-store.cc',
's3.cc',
'serve-protocol.cc',
'ssh-store.cc',
'store-reference.cc',
@@ -94,6 +95,7 @@ this_exe = executable(
link_args : linker_export_flags + [ '-lrapidcheck' ],
# get main from gtest
install : true,
cpp_pch : do_pch ? [ 'pch/precompiled-headers.hh' ] : [],
)
test(
@@ -127,6 +129,7 @@ if get_option('benchmarks')
include_directories : include_dirs,
link_args : linker_export_flags,
install : true,
cpp_pch : do_pch ? [ 'pch/precompiled-headers.hh' ] : [],
cpp_args : [
'-DNIX_UNIT_TEST_DATA="' + meson.current_source_dir() + '/data"',
],
+9 -2
View File
@@ -5,6 +5,7 @@
#include "nix/store/tests/nix_api_store.hh"
#include "nix/util/tests/string_callback.hh"
#include "nix/util/url.hh"
#include "store-tests-config.hh"
@@ -23,7 +24,13 @@ TEST_F(nix_api_store_test, nix_store_get_uri)
std::string str;
auto ret = nix_store_get_uri(ctx, store, OBSERVE_STRING(str));
ASSERT_EQ(NIX_OK, ret);
ASSERT_STREQ("local", str.c_str());
auto expectedStoreURI = "local://?"
+ nix::encodeQuery({
{"log", nixLogDir},
{"state", nixStateDir},
{"store", nixStoreDir},
});
ASSERT_EQ(expectedStoreURI, str);
}
TEST_F(nix_api_util_context, nix_store_get_storedir_default)
@@ -97,7 +104,7 @@ TEST_F(nix_api_util_context, nix_store_open_dummy)
nix_libstore_init(ctx);
Store * store = nix_store_open(ctx, "dummy://", nullptr);
ASSERT_EQ(NIX_OK, ctx->last_err_code);
ASSERT_STREQ("dummy", store->ptr->getUri().c_str());
ASSERT_STREQ("dummy://", store->ptr->config.getReference().render(/*withParams=*/true).c_str());
std::string str;
nix_store_get_version(ctx, store, OBSERVE_STRING(str));
Loaded 100 of 263 files, more files were not shown because too many files have changed in this diff. Show more