Compare commits

..
138 Commits
Author SHA1 Message Date
Sergei Zimmerman b5ec989b3f Merge pull request #15261 from obsidiansystems/backport-15252-to-2.31-maintenance
[Backport 2.31-maintenance] upload-release: disable containerd image store to preserve gzip layer compression
2026-02-17 02:16:00 +03:00
Amaan Qureshi 81f523af00 upload-release: disable containerd image store to preserve gzip layer compression
(cherry picked from commit 2ccb8a9a56)
2026-02-16 17:52:27 -05:00
internal-nix-ci[bot] b020bbba8d Merge pull request #15065 from NixOS/backport-15062-to-2.31-maintenance
[Backport 2.31-maintenance] ci: Drop magic-nix-cache
2026-01-23 00:58:40 +00:00
Sergei Zimmerman 1c23a7f62e ci: Drop magic-nix-cache
We are now seeing. I guess we are out with the cache. When the API responds with 418 (I'm a teapot)
it seems like the only reasonable solution is to oblige.

error: unable to download 'http://127.0.0.1:37515/7ms9f25xyxavf32pvdc3vb28nzzmkbn3.narinfo': HTTP error 418
       response body:
       GitHub API error: GitHub Actions Cache throttled Magic Nix Cache. Not trying to use it again on this run.
(cherry picked from commit dae41e06e8)
2026-01-23 00:35:17 +00:00
internal-nix-ci[bot] e26fe63e87 Merge pull request #15034 from NixOS/backport-15031-to-2.31-maintenance
[Backport 2.31-maintenance] ci: Bump magic-nix-cache to disable on 429
2026-01-21 05:13:21 +00:00
Sergei Zimmerman ecfdb18ea8 ci: Bump magic-nix-cache to disable on 429
(cherry picked from commit 1555677cd5)
2026-01-21 04:50:11 +00:00
Eelco Dolstra 69c48df066 Bump version 2026-01-07 21:36:13 +01:00
internal-nix-ci[bot] e4b8553d6e Merge pull request #14942 from NixOS/backport-14778-to-2.31-maintenance
[Backport 2.31-maintenance] test: add shebangs to shell.nix test scripts
2026-01-07 14:14:23 +00:00
Agustín Covarrubias 4fa81576cd test: add shebangs to shell.nix test scripts
Fix intermittent SIGSEGV (exit code 139) on macOS when running
  nix-shell and shebang tests inside the nix sandbox.

  The foo, bar, and ruby test scripts were created without shebangs,
  which causes intermittent crashes when executed via command
  substitution on macOS. Adding proper shebangs resolves the flakiness.

  Potentially closes: #13106

(cherry picked from commit 7b3d7eb634)
2026-01-07 13:50:15 +00:00
John Ericson 952c823960 Merge pull request #14909 from NixOS/backport-tarball-pct-decode-to-2.31-maintenance
libfetchers: Actually percentDecode path when using the path from fil…
2026-01-04 17:41:30 -05:00
internal-nix-ci[bot] 91236ddd27 Merge pull request #14915 from NixOS/backport-14729-to-2.31-maintenance
[Backport 2.31-maintenance] Fix Non-virtual interface pattern for `RestrictedStore::addDependency`
2026-01-02 15:51:49 +00:00
John Ericson eff5b4f8d7 Fix Non-virtual interface pattern for RestrictedStore::addDependency
I didn't do things quite right in 496e43ec72:

- Forgot to remove the now-redundant `isAllowed` check.

- Called the non-virtual, not the superclass's impl, in
  `addDependencyPrep`, causing bad recursion / UB.

Doing this fixes a crash I encountered with manual testing an Nix Ninja
--- hopefully we will get Nix Ninja or similar in a NixOS test longer
term to defend against this thing happening again.

(cherry picked from commit 4652345ac3)
2026-01-02 15:23:19 +00:00
Sergei Zimmerman c8e56b3664 libfetchers: Actually percentDecode path when using the path from file:// URI
Special characters like + e.t.c. are pct-encoded.
2026-01-02 00:50:10 +03:00
internal-nix-ci[bot] 2afdfc5836 Merge pull request #14906 from NixOS/backport-14903-to-2.31-maintenance
[Backport 2.31-maintenance] upload-release.pl: Fix up nix-channels bucket location, use awscli2
2026-01-01 21:43:48 +00:00
Sergei Zimmerman cc13939400 upload-release.pl: Fix up nix-channels bucket location, use awscli2
I messed up and accidentally configured the S3 client to use the same
host as the nix-releases bucket, but nix-channels is us-east-1 and
nix-releases is eu-west-1.

(cherry picked from commit 0900638f1d)
2026-01-01 21:17:44 +00:00
internal-nix-ci[bot] 4c1e03676b Merge pull request #14900 from NixOS/backport-14888-to-2.31-maintenance
[Backport 2.31-maintenance] ci: GitHub releng for release automation
2026-01-01 15:08:09 +00:00
Sergei Zimmerman e904e2cbc6 release-process: Document usage of upload-release.yml workflow
(cherry picked from commit 84ff2ef347)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman 06938e909a upload-release: Only upload the newly created tag
(cherry picked from commit 3933e45d52)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman effeeb9e0a upload-release: Also push to GHCR as part of the release process
(cherry picked from commit a1569458cc)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman 94311053e9 ci: Add upload-release.yml
This workflow is supposed to automate release uploads by using OIDC
for AWS setup. DockerHub still uses long-lived credentials, but that's
not fixable. In a follow-up we could set up release uploads to GHCR too.

(cherry picked from commit 4599daa10e)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman e5ec1ac162 maintainers: Document git tag signing
Previously it was only Eeclo doing releases that were signed with
B541D55301270E0BCF15CA5D8170B4726D7198DE. Other linux distributions
have the expectation (rightfully so) that our tags are signed. Let's
document this.

We could do cross-signing to make tracing the chain of trust easier
for all Nix team members [1].

[1]: https://nixos.org/community/teams/nix/

(cherry picked from commit 6cb8b58a47)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman a118c65b74 maintainers/upload-release.pl: Make more configurable
This allows for testing with a local minio deployment like:

./upload-release.pl --skip-docker --skip-git --s3-endpoint http://localhost:9000 --s3-host localhost:9000 1821360

(cherry picked from commit d19b8d5f99)
2026-01-01 14:45:27 +00:00
Sergei Zimmerman 126ee30fe3 Merge pull request #14881 from xokdvium/backport-14874-to-2.31-maintenance 2025-12-28 20:24:34 +03:00
Sergei Zimmerman c5f83ab995 ci: Make docker-push workflow more configurable
This should allow reusing this workflow (with more tweaks)
in the releng workflow.

(cherry picked from commit c867ed6726)
2025-12-28 19:11:46 +03:00
Sergei Zimmerman eb9b5535c9 ci: Pin actions in docker-push reusable workflow
(cherry picked from commit fb05f6de0d)
2025-12-28 19:11:45 +03:00
Sergei Zimmerman 1e1571be74 ci: Move docker_push_image into a separate workflow
Best reviewed with -w --color-moved. This just moves the code
into a separate workflow. This will allow us to reuse it in
the release job for github releng of releases.

(cherry picked from commit 745983dfc0)
2025-12-28 19:11:41 +03:00
Sergei Zimmerman dbabe7e13a ci: Bump magic-nix-cache with post-build-hook fix
No tagged release with the fix for [^].

[^]: https://github.com/DeterminateSystems/magic-nix-cache/commit/578f01e1473129cc289d579ba6dee9cdac40aeab

(cherry picked from commit 4c4eb5d07f)
2025-12-28 19:06:27 +03:00
Sergei Zimmerman a0be5be079 ci: Move magic-nix-cache-action into install-nix-action composite
This reduces duplication and pins the underlying version of magic-nix-cache,
as we already do with other actions.

(cherry picked from commit ad5c6a53b9)
2025-12-28 19:06:22 +03:00
Seth Flynn 16f93f9865 ci: allow for using the latest build of the experimental installer
Until these repos are potentially merged, this is good for dogfooding
alongside the experimental installer. It also uses the more official
`artifacts.nixos.org` endpoint to install stable releases now

More immediately though, we need a patch for the experimental installer
to really work in CI at all, and that hasn't landed in a tag yet. So,
this lets us use it right from `main`!

(cherry picked from commit 92d7381826)
2025-12-28 19:05:53 +03:00
Seth Flynn 4ed5eb89fe ci: enable use of the experimental installer
(cherry picked from commit 2cbbb63628)
2025-12-28 19:05:50 +03:00
internal-nix-ci[bot] 9fb9e6082c Merge pull request #14861 from NixOS/backport-14792-to-2.31-maintenance
[Backport 2.31-maintenance] Fix `curl` with `c-ares` failing to resolve DNS inside sandbox on macOS
2025-12-23 10:43:24 +00:00
Michael Hoang b141e29adc Fix curl with c-ares failing to resolve DNS inside sandbox on macOS
(cherry picked from commit 7541129f04)
2025-12-23 10:07:34 +00:00
internal-nix-ci[bot] 9b597aa64f Merge pull request #14848 from NixOS/backport-14785-to-2.31-maintenance
[Backport 2.31-maintenance] libstore: include path in the world-writable error
2025-12-21 19:05:54 +00:00
yawkar db39278d61 libstore: include path in the world-writable error
The previous error message was ambiguous about which specific directory failed the check.

This commit updates checkNotWorldWritable to return the failing path so it can be included in the error message, making debugging easier.

(cherry picked from commit a1e24fa6ce)
2025-12-21 18:37:20 +00:00
internal-nix-ci[bot] 4fecbf8a9a Merge pull request #14747 from NixOS/backport-14745-to-2.31-maintenance
[Backport 2.31-maintenance] Correct `build-dir` error in manual, link relevant settings
2025-12-09 02:01:40 +00:00
John Ericson 34c9d1a502 Correct build-dir error in manual, link relevant settings
This fixes out-of-date information that is no longer true, and makes the
up-to-date information more accessible.

(cherry picked from commit c5fa5e503a)
2025-12-09 01:32:34 +00:00
internal-nix-ci[bot] a4f5f36509 Merge pull request #14692 from NixOS/backport-14690-to-2.31-maintenance
[Backport 2.31-maintenance] Support mdbook 0.5
2025-12-02 14:30:51 +00:00
Robert Hensing 24096ecdd5 maint: Fix lowdown override compatibility with newer nixpkgs
Use `or ""` fallback for postInstall attribute which may not exist in
newer nixpkgs versions of lowdown.

(cherry picked from commit 0aef1ddb9e)
2025-12-02 14:06:21 +00:00
Robert Hensing f3cf58e469 maint: Remove mdbook-linkcheck and support mdbook 0.5.x
Fixes #14628

- Remove mdbook-linkcheck dependency and configuration (was blocking
  upgrades to mdbook 0.5.0+, configured with warning-policy = "ignore"
  due to false positives, and redundant with lychee-based link checking)
- Update substitute.py and anchors.jq to handle 'items' (mdbook 0.5.x)
  in addition to 'sections' (mdbook 0.4.x), as per mdbook 0.5.0
  changelog: "Book::sections was renamed to Book::items"
  https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md#05-migration-guide

(cherry picked from commit 2636f50dd4)
2025-12-02 14:06:21 +00:00
internal-nix-ci[bot] 315010c447 Merge pull request #14679 from NixOS/backport-14676-to-2.31-maintenance
[Backport 2.31-maintenance] libstore: Use makeTempPath in optimizePath_, assorted fs fixes
2025-12-01 03:23:00 +00:00
Sergei Zimmerman f67bdae6a6 libutil: Make AutoDelete non-copyable and non-movable
This is a good precaution, since we don't want to delete
directories twice accidentally.

(cherry picked from commit 40e3f5c0a4)
2025-12-01 02:48:04 +00:00
Sergei Zimmerman 3c01ff3148 libutil: Propagate error code in createSymlink
(cherry picked from commit bf7c53f2d3)
2025-12-01 02:48:03 +00:00
Sergei Zimmerman f5473c9624 libstore: Actually correctly call remove in case rename fails
(cherry picked from commit 1cc337bb5f)
2025-12-01 02:48:03 +00:00
Sergei Zimmerman 22c69f47b8 libstore: Use makeTempPath in optimizePath_
This was intended to be cherry-picked in 6aed9d877c,
but was left hanging. This is actually important for fixing [^]. emilazy let me know
of this bad cherry-pick and its significance.

[^]: https://github.com/NixOS/nix/issues/7273

Originally fixed by Lily Ballard <lily@ballards.net> in https://gerrit.lix.systems/c/lix/+/2100.

(cherry picked from commit d888846b68)
2025-12-01 02:48:03 +00:00
internal-nix-ci[bot] eee5c6ee72 Merge pull request #14533 from NixOS/backport-14531-to-2.31-maintenance
[Backport 2.31-maintenance] Restore isAllowed check in ChrootLinuxDerivationBuilder
2025-11-10 20:44:23 +00:00
Sergei Zimmerman b3e3be1e64 Restore isAllowed check in ChrootLinuxDerivationBuilder
This early return was lost in d4ef822add.

By doing some
https://en.wikipedia.org/wiki/Non-virtual_interface_pattern, we can
ensure that we don't make this mistake again --- implementations are no
longer responsible for implementing the caching/memoization mechanism.

(cherry picked from commit 496e43ec72)
2025-11-10 20:15:46 +00:00
John Ericson 5d14e91245 Merge pull request #14519 from NixOS/backport-14137-to-2.31-maintenance
[Backport 2.31-maintenance] fix(libstore/build/derivation-goal): don't assert on partially valid outputs
2025-11-09 14:29:14 -05:00
internal-nix-ci[bot] 1e92560665 Merge pull request #14523 from NixOS/backport-14515-to-2.31-maintenance
[Backport 2.31-maintenance] libexpr: Don't use nix::dirOf in prim_dirOf (fix 2.23 regression)
2025-11-09 18:07:05 +00:00
Sergei Zimmerman 3032fdbbe5 libexpr: Don't use nix::dirOf in prim_dirOf
This gets us back to pre-2.23 behavior of this primop.
Done by inlining the code of `nix::dirOf` from 2.2-maintenance.

(cherry picked from commit a33fccf55a)
2025-11-09 17:43:01 +00:00
Sergei Zimmerman f6702e94fa tests/functional: Add tests for builtins.dirOf
These will change in the next commit to fix the silent regression from 2.23
in the handling of multiple subsequent path separators.

(cherry picked from commit 86f090837b)
2025-11-09 17:43:01 +00:00
Bernardo Meurer Costa 795d7bb591 fix(libstore/build/derivation-goal): don't assert on partially valid outputs
Fixes: #14130
(cherry picked from commit 9eecee3d4e)
2025-11-09 12:24:50 -05:00
Eelco Dolstra 7358e50193 Revert "Merge pull request #14381 from NixOS/backport-14364-to-2.31-maintenance"
This reverts commit 264cc2e2b2, reversing
changes made to 3cca112d79.
2025-11-06 22:12:13 +01:00
internal-nix-ci[bot] d32b580243 Merge pull request #14497 from NixOS/backport-14491-to-2.31-maintenance
[Backport 2.31-maintenance] Don't crash on flakerefs containing newlines
2025-11-06 19:43:07 +00:00
Eelco Dolstra fe711b2f4c Don't crash on flakerefs containing newlines
Fixes #14311.

(cherry picked from commit c1317017e9)
2025-11-06 19:18:12 +00:00
internal-nix-ci[bot] bb375a0bcc Merge pull request #14460 from NixOS/backport-14450-to-2.31-maintenance
[Backport 2.31-maintenance] flake: Update, nixos-25.05-small -> nixos-25.05
2025-11-03 17:42:15 +00:00
Robert Hensing c3e5510622 Apply updated nixfmt
(cherry picked from commit 81a2809a52)
2025-11-03 17:08:27 +00:00
Robert Hensing 72a4832b66 flake: Update, nixos-25.05-small -> nixos-25.05
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26)
  → 'github:NixOS/nixpkgs/daf6dc47aa4b44791372d6139ab7b25269184d55?narHash=sha256-wxX7u6D2rpkJLWkZ2E932SIvDJW8%2BON/0Yy8%2Ba5vsDU%3D' (2025-10-27)

(cherry picked from commit 233bd250d1)
2025-11-03 17:08:27 +00:00
internal-nix-ci[bot] ce97fa9f05 Merge pull request #14437 from NixOS/backport-14434-to-2.31-maintenance
[Backport 2.31-maintenance] libstore: Improve store-reference back-compat with IPv6 ZoneId literals
2025-11-01 00:17:48 +00:00
Sergei Zimmerman 43f7704edc libstore: Improve store-reference back-compat with IPv6 ZoneId literals
This restores the pre-2.31 handling of ZoneID identifiers in store references.
It's the only place we reasonably care about this back-compat.

(cherry picked from commit 8dbc2475f7)
2025-10-31 23:50:19 +00:00
Sergei Zimmerman 60e906a535 Merge pull request #14433 from jonhermansen/backport-14134-to-2.31-maintenance 2025-11-01 00:57:54 +03:00
Jami Kettunen 9259a07e23 libstore: Include missing header to fix compile with libc++ 20
https://en.cppreference.com/w/cpp/thread.html

src/libstore/gc.cc:121:39: error: no member named 'sleep_for' in namespace 'std::this_thread'
  121 |                     std::this_thread::sleep_for(std::chrono::milliseconds(100));
      |                     ~~~~~~~~~~~~~~~~~~^

(cherry picked from commit 140b08ae3e)
2025-10-31 17:20:52 -04:00
internal-nix-ci[bot] 264cc2e2b2 Merge pull request #14381 from NixOS/backport-14364-to-2.31-maintenance
[Backport 2.31-maintenance] diff-closures: print sizes with dynamic unit
2025-10-27 20:37:15 +00:00
Marcel 6275de367f treewide: replace manual MiB calculations with renderSize
(cherry picked from commit 584a8e8a00)
2025-10-27 19:43:37 +00:00
Marcel 0c6585707a refactor(libutil): remove showBytes() in favor of renderSize()
The `showBytes()` function was redundant with `renderSize()` as the
latter automatically selects the appropriate unit (KiB, MiB, GiB, etc.)
based on the value, whereas `showBytes()` always formatted as MiB
regardless of size.

Co-authored-by: Bernardo Meurer Costa <beme@anthropic.com>
(cherry picked from commit f234633e27)
2025-10-27 19:43:37 +00:00
Marcel 1a22674806 diff-closures: print sizes with dynamic unit
(cherry picked from commit 9d4d10954a)
2025-10-27 19:43:37 +00:00
internal-nix-ci[bot] 3cca112d79 Merge pull request #14353 from NixOS/backport-14343-to-2.31-maintenance
[Backport 2.31-maintenance] Revert "libmain: Catch logger exceptions in `handleExceptions`"
2025-10-24 23:59:16 +00:00
Sergei Zimmerman 993a58e2f6 Revert "libmain: Catch logger exceptions in handleExceptions"
This reverts commit 90d1ff4805.

The initial issue with EPIPE was solved in 9f680874c5.
Now this patch does move bad than good by eating up boost::io::format_error that are
bugs.

(cherry picked from commit 4f5af471fb)
2025-10-24 23:28:48 +00:00
internal-nix-ci[bot] 61f09a36f8 Merge pull request #14258 from NixOS/backport-14253-to-2.31-maintenance
[Backport 2.31-maintenance] libfetchers/git-utils: Be more correct about validating refnames
2025-10-15 20:33:35 +00:00
Sergei Zimmerman 1e5a389a2f libfetchers/git-utils: Be more correct about validating refnames
Turns out there's a much better API for this that doesn't have the
footguns of the previous method.

isLegalRefName is somewhat of a misnomer, since it's mainly used to
validate user inputs that can be either references, branch names,
psedorefs or tags.

(cherry picked from commit 5d1178b817)
2025-10-15 20:08:39 +00:00
internal-nix-ci[bot] 71fe367e8c Merge pull request #14248 from NixOS/backport-13976-to-2.31-maintenance
[Backport 2.31-maintenance] packaging: Drop legacy apple sdk pattern
2025-10-14 21:35:37 +00:00
Sergei Zimmerman 421f650bb1 packaging: Drop legacy apple sdk pattern
This has been dropped on unstable an nix no longer
compiled with overridden nixpkgs input. On 25.05 these
overrides already do nothing.

Tested with:

nix build .#packages.x86_64-darwin.nix-cli -L --override-input nixpkgs https://releases.nixos.org/nixos/unstable/nixos-25.11pre859555.ab0f3607a6c7/nixexprs.tar.xz

Default deployment target on 25.05 is 11.3, so 10.13
sdk override doesn't have to be updated at all as evident
from the fact that we didn't observe any issues with it.

(cherry picked from commit 20b532eab0)
2025-10-14 21:15:28 +00:00
John Ericson dc87ed5eab Merge pull request #14240 from NixOS/2.31-maintenance-backport-13975
Fix hash error message
2025-10-13 18:07:16 -04:00
John Ericson 56751b1cd2 Fix hash error message
Wrong number of arguments was causing a format assertion.

(cherry picked from commit c6d06ce486)
2025-10-14 00:30:26 +03:00
internal-nix-ci[bot] a042b379fa Merge pull request #14211 from NixOS/backport-14210-to-2.31-maintenance
[Backport 2.31-maintenance] libstore: Fix double-quoting of paths in logs
2025-10-10 23:27:12 +00:00
Sergei Zimmerman ba38e59304 libstore: Fix double-quoting of paths in logs
std::filesystem::path is already quoted by boost::format with double quotes (").
(cherry picked from commit f30cb8667b)
2025-10-10 22:54:18 +00:00
internal-nix-ci[bot] f9f2f49252 Merge pull request #14195 from NixOS/backport-14189-to-2.31-maintenance
[Backport 2.31-maintenance] exportReferencesGraph: Handle heterogeneous arrays
2025-10-08 23:43:23 +00:00
Eelco Dolstra 2bfc564297 exportReferencesGraph: Handle heterogeneous arrays
This barfed with

   error: [json.exception.type_error.302] type must be string, but is array

on `nix build github:malt3/bazel-env#bazel-env` because it has a `exportReferencesGraph` with a value like `["string",...["string"]]`.

(cherry picked from commit 94f410b628)
2025-10-08 23:13:05 +00:00
internal-nix-ci[bot] 3019db2c87 Merge pull request #14192 from NixOS/backport-13861-to-2.31-maintenance
[Backport 2.31-maintenance] libutil: Try to call std::terminate for panic, use C++20 std::source_location
2025-10-08 20:46:42 +00:00
Sergei Zimmerman c750f0f630 libutil: Use std::source_location for unreachable
Make unreachable a function instead of a macro, since
C++20 provides a convenience class as a replacement for
older __FILE__, __LINE__ macros.

(cherry picked from commit d59b959c87)
2025-10-08 20:05:50 +00:00
Sergei Zimmerman a732f0354d libutil: Try to call std::terminate for panic
We now have a terminate handler that prints a
stack trace, which is useful to have when encountering
an unreachable.

(cherry picked from commit 1f607b5def)
2025-10-08 20:05:50 +00:00
internal-nix-ci[bot] 885499ace5 Merge pull request #14183 from NixOS/backport-14156-to-2.31-maintenance
[Backport 2.31-maintenance] treewide: Squash some user-after-free bugs
2025-10-08 00:28:23 +00:00
internal-nix-ci[bot] e4e41a17e6 Merge pull request #14185 from NixOS/backport-14086-to-2.31-maintenance
[Backport 2.31-maintenance] nix-cli: use pure/restricted eval for help pages
2025-10-08 00:20:41 +00:00
Seth Flynn 451f483ef0 nix-cli: use pure/restricted eval for help pages
This avoids any complications that can arise from the environment
affecting evaluation of the help pages (which don't need to be calling
out to anything external anyways)

A recent example of one of these problems is
https://github.com/NixOS/nix/issues/14085, which would break help pages
by causing them to make invalid calls to the dummy store they're
evaluated with

Fixes: https://github.com/NixOS/nix/issues/14062
Co-authored-by: Sergei Zimmerman <sergei@zimmerman.foo>
(cherry picked from commit ff82de86da)
2025-10-08 00:06:03 +00:00
Sergei Zimmerman fa8230167d libexpr: Use use-after-move in SampleStack::saveProfile()
(cherry picked from commit be1ade7373)
2025-10-08 00:03:50 +00:00
Sergei Zimmerman d1cec3d3ed libstore: Fix use-after-move in DerivationGoal::repairClosure
(cherry picked from commit 452ec09fe0)
2025-10-08 00:03:50 +00:00
internal-nix-ci[bot] 66f6ad1292 Merge pull request #14179 from NixOS/backport-14154-to-2.31-maintenance
[Backport 2.31-maintenance] treewide: Remove toView() because it leads to segfaults when compiled…
2025-10-07 22:15:04 +00:00
internal-nix-ci[bot] e8574843f1 Merge pull request #14178 from NixOS/backport-14150-to-2.31-maintenance
[Backport 2.31-maintenance] libstore: fixup fakeSSH check
2025-10-07 22:12:56 +00:00
internal-nix-ci[bot] 404bc1d93c Merge pull request #14177 from NixOS/backport-14168-to-2.31-maintenance
[Backport 2.31-maintenance] libutil: Throw if `str("contents")` not found
2025-10-07 22:09:54 +00:00
Sergei Zimmerman 64dbb6f812 treewide: Remove toView() because it leads to segfaults when compiled with newer nixpkgs
Firstly, this is now available on darwin where the default in llvm 19.
Secondly, this leads to very weird segfaults when building with newer nixpkgs for some reason.
(It's UB after all).

This appears when building with the following:

mesonComponentOverrides = finalAttrs: prevAttrs: {
mesonBuildType = "debugoptimized";
dontStrip = true;
doCheck = false;
separateDebugInfo = false;
preConfigure = (prevAttrs.preConfigure or "") + ''
  case "$mesonBuildType" in
  release|minsize|debugoptimized) appendToVar mesonFlags "-Db_lto=true"  ;;
  *)                              appendToVar mesonFlags "-Db_lto=false" ;;
  esac
'';
};

And with the following nixpkgs input:

nix build ".#nix-cli" -L --override-input nixpkgs "https://releases.nixos.org/nixos/unstable/nixos-25.11pre870157.7df7ff7d8e00/nixexprs.tar.xz"

Stacktrace:

 #0  0x00000000006afdc0 in ?? ()
 #1  0x00007ffff71cebb6 in _Unwind_ForcedUnwind_Phase2 () from /nix/store/41ym1jm1b7j3rhglk82gwg9jml26z1km-gcc-14.3.0-lib/lib/libgcc_s.so.1
 #2  0x00007ffff71cf5b5 in _Unwind_Resume () from /nix/store/41ym1jm1b7j3rhglk82gwg9jml26z1km-gcc-14.3.0-lib/lib/libgcc_s.so.1
 #3  0x00007ffff7eac7d8 in std::basic_ios<char, std::char_traits<char> >::~basic_ios (this=<optimized out>, this=<optimized out>)
     at /nix/store/82kmz7r96navanrc2fgckh2bamiqrgsw-gcc-14.3.0/include/c++/14.3.0/bits/basic_ios.h:286
 #4  std::__cxx11::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream (this=<optimized out>, this=<optimized out>)
     at /nix/store/82kmz7r96navanrc2fgckh2bamiqrgsw-gcc-14.3.0/include/c++/14.3.0/sstream:806
 #5  nix::SimpleLogger::logEI (this=<optimized out>, ei=...) at ../logging.cc:121
 #6  0x00007ffff7515794 in nix::Logger::logEI (this=0x675450, lvl=nix::lvlError, ei=...) at /nix/store/bkshji3nnxmrmgwa4n2kaxadajkwvn65-nix-util-2.32.0pre-dev/include/nix/util/logging.hh:144
 #7  nix::handleExceptions (programName=..., fun=...) at ../shared.cc:336
 #8  0x000000000047b76b in main (argc=<optimized out>, argv=<optimized out>) at /nix/store/82kmz7r96navanrc2fgckh2bamiqrgsw-gcc-14.3.0/include/c++/14.3.0/bits/new_allocator.h:88

(cherry picked from commit dce1a893d0)
2025-10-07 21:39:12 +00:00
Cole Helbling 823c630b2e libstore: fixup fakeSSH check
This broke invocations like:

    NIX_SSHOPTS='-p2222 -oUserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no' nix copy /nix/store/......-foo --to ssh-ng://root@localhost

In Nix 2.30.2, fakeSSH was enabled when the "thing I want to connect to"
was plain old "localhost". Previously, this check was written as:

         , fakeSSH(host == "localhost")

Given the above invocation, `host` would have been `root@localhost`, and
thus `fakeSSH` would be `false` because `root@localhost` != `localhost`.

However, since 49ba06175e, `authority.host`
returned _just_ the host (`localhost`, no user) and erroneously enabled
`fakeSSH` in this case, causing `NIX_SSHOPTS` to be ignored (since,
when `fakeSSH` is `true`, `SSHMaster::startCommand` doesn't call
`addCommonSSHOpts`).

`authority.to_string()` accurately returns the expected `root@localhost`
format (given the above invocation), fixing this.

(cherry picked from commit 7ec1427fc3)
2025-10-07 21:37:07 +00:00
Samuel Connelly 7a05ed9c12 libutil: Throw if str("contents") not found
This was broken in 7aa3e7e3a5 (since 2.25).

(cherry picked from commit 242f362567)
2025-10-07 21:33:19 +00:00
John Ericson 80f2ca4015 Merge pull request #14046 from NixOS/mergify/bp/2.31-maintenance/pr-13929
Remove unused function setChildSignalMask() (backport #13929)
2025-09-22 13:39:43 -04:00
Eelco Dolstra 9f34b6ca73 Remove unused function setChildSignalMask()
(cherry picked from commit a44dcbff13)
2025-09-22 16:32:00 +00:00
Eelco Dolstra 8e7d86d4cf Bump version 2025-09-18 13:38:38 +02:00
mergify[bot] fdea162417 Merge pull request #14010 from NixOS/mergify/bp/2.31-maintenance/pr-14009
Revert "tests/nixos: Fix daemon store reference in authorization test" (backport #14009)
2025-09-17 20:25:38 +00:00
Sergei Zimmerman 8989350d4e Revert "tests/nixos: Fix daemon store reference in authorization test"
This reverts commit 695f3bc7e3.

(cherry picked from commit 86ad8d49f9)
2025-09-17 19:59:49 +00:00
mergify[bot] a3df190232 Merge pull request #14004 from NixOS/mergify/bp/2.31-maintenance/pr-13839
don't include derivation name in temporary build directories (backport #13839)
2025-09-16 10:49:56 +00:00
Jörg Thalheim 7c3fd50617 don't include derivation name in temporary build directories
With the migration to /nix/var/nix/builds we now have failing builds
when the derivation name is too long.
This change removes the derivation name from the temporary build to have
a predictable prefix length:

Also see: https://github.com/NixOS/infra/pull/764
for context.

(cherry picked from commit 725a2f379f)
2025-09-16 10:23:44 +00:00
mergify[bot] 8fc22db0e1 Merge pull request #13989 from NixOS/mergify/bp/2.31-maintenance/pr-13985
libstore: Raise default connect-timeout to 15 secs (backport #13985)
2025-09-14 11:50:47 +00:00
dramforever a1ccb18abf libstore: Raise default connect-timeout to 15 secs
This allows the weird network or DNS server fallback mechanism inside
glibc to work, and prevents a "Resolving timed out after 5000
milliseconds" error. Read on for details.

The DNS request stuff (dns-hosts) in glibc uses this fallback procedure
to minimize network RTT in the ideal case while dealing with
ill-behaving networks and DNS servers gracefully (see resolv.conf(5)):

- Use sendmmsg() to send UDP DNS requests for IPv4 and IPv6 in parallel
- If that times out (meaning that none or only one of the responses have
  been received), send the requests one by one, waiting for the response
  before sending the next request ("single-request")
- If that still times out, try to use a different socket (hence
  different address) for each request ("single-request-reopen")

The default timeout inside glibc is 5 seconds. Therefore, setting
connect-timeout, and therefore CURLOPT_CONNECTTIMEOUT to 5 seconds
prevents the single-request fallback, and setting it to even 10 seconds
prevents the single-request-reopen fallback as well.

The fallback decision is saved by glibc, but only thread-locally, and
libcurl starts a new thread for getaddrinfo() for each connection.
Therefore for every connection the fallback starts from sendmmsg() all
over again. And since these are considered to have timed out by libcurl,
even though getaddrinfo() might return a successful result, it is not
cached in libcurl.

While a user could tweak these with resolv.conf(5) options (e.g. using
networking.resolvconf.extraOptions in NixOS), and indeed that is
probably needed to avoid annoying delays, it still means that the
default connect-timeout of 5 is too low. Raise it to give fallback a
chance.

(cherry picked from commit 7295034362)
2025-09-14 11:19:49 +00:00
Sergei Zimmerman f55f5dff34 Merge pull request #13974 from NixOS/mergify/bp/2.31-maintenance/pr-13970 2025-09-12 21:58:50 +00:00
Sergei Zimmerman 48eaf35828 Revert "meson: add soversion to libraries (#13960)"
This reverts commit bdbc739d6e.

Such a change needs more thought put into it. By versioning
shared libraries we'd make a false impression that libraries
themselves are actually versioned and have some sort of stable
ABI, which is not the case.

This will be useful when C bindings become stable, but as long
as they are experimental it does not make sense to set SONAME.

Also this change should not have been backported, since it's
severely breaking.

(cherry picked from commit 0db2b8c8fe)
2025-09-12 21:18:42 +00:00
mergify[bot] 6d862484d7 Merge pull request #13968 from NixOS/mergify/bp/2.31-maintenance/pr-13966
meson: add soversion to libraries (#13960) (backport #13966)
2025-09-12 07:00:40 +00:00
Jens Petersen c2c4ffc164 meson: add soversion to libraries (#13960)
(cherry picked from commit bdbc739d6e)
2025-09-12 06:26:41 +00:00
Sergei Zimmerman 489fad878b Merge pull request #13952 from NixOS/mergify/bp/2.31-maintenance/pr-13939
Pass `dir` in extraAttrs when overriding the registry (backport #13939)
2025-09-09 19:09:29 +00:00
Cole Helbling 26b862b6d2 Pass dir in extraAttrs when overriding the registry
This is handled similarly in the handler for `--override-flake` in
`MixEvalArgs`.

(cherry picked from commit 38663fb434)
2025-09-09 18:34:37 +00:00
Cole Helbling 3ba8b83f95 Test that using --inputs-from with a flakeref that has a dir works
Will not pass until the next commit.

(cherry picked from commit ed6ef7cdf4)
2025-09-09 18:34:36 +00:00
Eelco Dolstra c2ef01d26a Merge pull request #13946 from NixOS/mergify/bp/2.31-maintenance/pr-13934
Fix flake registry ignoring `dir` parameter (backport #13934)
2025-09-09 10:46:18 +02:00
Cole Helbling 7b59cafaed fixup: cached case
I couldn't come up with a test that failed before this, but my existing
test still passes so 🤷

(cherry picked from commit 9c832a08b0)
2025-09-09 07:39:17 +00:00
Cole Helbling ba46c7d0f2 Fix flake registry ignoring dir parameter
This broke in e3042f10af.

(cherry picked from commit bccdb95a86)
2025-09-09 07:39:17 +00:00
Cole Helbling 766a236014 Test that dir is propagated from registry entry
(cherry picked from commit 258d41bfb6)
2025-09-09 07:39:16 +00:00
mergify[bot] 1676a04197 Merge pull request #13943 from NixOS/mergify/bp/2.31-maintenance/pr-13940
libstore: Reallow unbracketed IPv6 addresses in store references (backport #13940)
2025-09-08 23:47:51 +00:00
Sergei Zimmerman 1ca1882e8c libstore: Reallow unbracketed IPv6 addresses in store references
This implements a special back-compat shim to specifically allow
unbracketed IPv6 addresses in store references. This is something
that is relied upon in the wild and the old parsing logic accepted
both ways (brackets were optional). This patch restores this behavior.
As always, we didn't have any tests for this.

Addresses #13937.

(cherry picked from commit 7cc654afa9)
2025-09-08 23:22:41 +00:00
mergify[bot] 72028d1fa1 Merge pull request #13931 from NixOS/mergify/bp/2.31-maintenance/pr-13911
libstore: Do not normalize daemon -> unix://, local -> local:// (backport #13911)
2025-09-07 20:59:27 +00:00
Sergei Zimmerman bbbb4ce330 libstore: Do not normalize daemon -> unix://, local -> local://
This is relied upon (specifically the `local` store) by existing
tooling [1] and we broke this in 3e7879e6df (which
was first released in 2.31).

To lessen the scope of the breakage we should not normalize "auto" references
and explicitly specified references like "local" or "daemon". It also makes
sense to canonicalize local://,daemon:// to be more compatible with prior
behavior.

[1]: https://github.com/maralorn/nix-output-monitor/blob/05e1b3cba2fa328a1781390a4e4515e9c432229e/lib/NOM/Builds.hs#L60-L64

(cherry picked from commit 3513ab13dc)
2025-09-07 23:38:14 +03:00
mergify[bot] 8e01f134a1 Merge pull request #13922 from NixOS/mergify/bp/2.31-maintenance/pr-13901
Fix macOS HUP detection using kqueue instead of poll (backport #13901)
2025-09-06 07:50:52 +00:00
Jörg Thalheim 2128753e46 Fix macOS HUP detection using kqueue instead of poll
On macOS, poll() is fundamentally broken for HUP detection. It loses event
subscriptions when EVFILT_READ fires without matching the requested events
in the pollfd. This causes daemon processes to linger after client disconnect.

This commit replaces poll() with kqueue on macOS, which is what poll()
uses internally but without the bugs. The kqueue implementation uses
EVFILT_READ which works for both sockets and pipes, avoiding EVFILT_SOCK
which only works for sockets.

On Linux and other platforms, we continue using poll() with the standard
POSIX behavior where POLLHUP is always reported regardless of requested events.

Based on work from the Lix project (https://git.lix.systems/lix-project/lix)
commit 69ba3c92db3ecca468bcd5ff7849fa8e8e0fc6c0

Fixes: https://github.com/NixOS/nix/issues/13847
Related: https://git.lix.systems/lix-project/lix/issues/729
Apple bugs: rdar://37537852 (poll), FB17447257 (poll)

Co-authored-by: Jade Lovelace <jadel@mercury.com>
(cherry picked from commit 1286d5db78)
2025-09-06 07:21:47 +00:00
John Ericson f3cb8050b2 Merge pull request #13915 from NixOS/mergify/bp/2.31-maintenance/pr-13900
Fix downstream MinGW build by not looking for Boost Regex (backport #13900)
2025-09-04 21:34:00 -04:00
John Ericson 702112a41c Fix downstream MinGW build by not looking for Boost Regex
(cherry picked from commit 6bdb5e8e09)
2025-09-05 01:07:02 +00:00
Eelco Dolstra e7540a269b Bump version 2025-09-02 11:22:41 +02:00
Sergei Zimmerman 4006d0fe11 Merge pull request #13882 from NixOS/mergify/bp/2.31-maintenance/pr-13741 2025-09-01 07:52:24 +00:00
Sergei Zimmerman 13d1be04b3 libexpr: Canonicalize TOML timestamps for toml11 > 4.0
This addresses several changes from toml11 4.0 bump in
nixpkgs [1].

1. Added more regression tests for timestamp formats.
   Special attention needs to be paid to the precision
   of the subsecond range for local-time. Prior versions select the closest
   (upwards) multiple of 3 with a hard cap of 9 digits.

2. Normalize local datetime and offset datetime to always
   use the uppercase separator `T`. This is actually the issue
   surfaced in [2]. This canonicalization is basically a requirement
   by (a certain reading) of rfc3339 section 5.6 [3].

3. If using toml11 >= 4.0 also keep the old behavior wrt
   to the number of digits used for subsecond part of the local-time.
   Newer versions cap it at 6 digits unconditionally.

[1]: https://www.github.com/NixOS/nixpkgs/pull/331649
[2]: https://www.github.com/NixOS/nix/issues/11441
[3]: https://datatracker.ietf.org/doc/html/rfc3339

(cherry picked from commit dc769d72cb)
2025-08-31 22:52:24 +00:00
Sergei Zimmerman e8a54769a1 libexpr: Use table.size() instead of unnecessary loop
(cherry picked from commit d8fc55a46e)
2025-08-31 22:52:24 +00:00
Sergei Zimmerman 1fc4d526a3 libexpr: Use recursive lambda instead of std::function
There's no reason to use a std::function for recursive lambdas
since there are polymorphic lambdas.

(cherry picked from commit a80a5c4dba)
2025-08-31 22:52:23 +00:00
Sergei Zimmerman c7e35e1ff8 libexpr: Remove extra trailing semicolons (NFC)
This looks really weird after the reformat.

(cherry picked from commit df4e55ffc1)
2025-08-31 22:52:23 +00:00
Sergei Zimmerman 92066f468e tests/functional/lang: Add more tests for TOML timestamps
Current test suite doesn't cover the subsecond formatting at
all and toml11 is quite finicky with that. We should at the very
least test its behavior to avoid silent breakages on updates.

(cherry picked from commit 7ed0229d1a)
2025-08-31 22:52:23 +00:00
mergify[bot] 1285e9485c Merge pull request #13870 from NixOS/mergify/bp/2.31-maintenance/pr-13867
nix/develop: Fix misleading ignored error when run with --arg/--argstr (backport #13867)
2025-08-29 21:44:55 +00:00
Sergei Zimmerman 05884fc103 nix/develop: Fix misleading ignored error when run with --arg/--argstr
This would print erroneous and misleading diagnostics like:

> error (ignored): error: '--arg' and '--argstr' are incompatible with flakes

When run with --expr/--file. Since this installable is used to get the
bash package it doesn't make sense to check this.

(cherry picked from commit b6f98b52a4)
2025-08-29 21:18:45 +00:00
Sergei Zimmerman 258e41004e Merge pull request #13843 from NixOS/mergify/bp/2.31-maintenance/pr-13837 2025-08-27 14:29:42 +03:00
Sergei Zimmerman f8245ffcee flake: Update nixpkgs
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/cd32a774ac52caaa03bcfc9e7591ac8c18617ced?narHash=sha256-VtMQg02B3kt1oejwwrGn50U9Xbjgzfbb5TV5Wtx8dKI%3D' (2025-08-17)
  → 'github:NixOS/nixpkgs/d98ce345cdab58477ca61855540999c86577d19d?narHash=sha256-O2CIn7HjZwEGqBrwu9EU76zlmA5dbmna7jL1XUmAId8%3D' (2025-08-26)

This update contains d1266642a8722f2a05e311fa151c1413d2b9653c, which
is necessary for the TOML timestamps to get tested via nixpkgsLibTests job.

(cherry picked from commit 625477a7df)
2025-08-27 07:53:44 +00:00
mergify[bot] 7aa0aca968 Merge pull request #13834 from NixOS/mergify/bp/2.31-maintenance/pr-13832
Handle empty ports with new URL parsing (backport #13832)
2025-08-26 18:28:14 +00:00
Leandro Reina 0cea128243 Handle empty ports
(cherry picked from commit 7989e3192d)
2025-08-26 17:57:06 +00:00
mergify[bot] 30682ec93b Merge pull request #13827 from NixOS/mergify/bp/2.31-maintenance/pr-13826
SQLite: fsync db.sqlite-shm before opening the database (backport #13826)
2025-08-25 22:35:18 +00:00
Eelco Dolstra 8e46456dfe SQLite: fsync db.sqlite-shm before opening the database
This is a workaround for https://github.com/NixOS/nix/issues/13515
(opening the SQLite DB randomly taking a couple of seconds on ZFS).

(cherry picked from commit a7fceb5eec)
(cherry picked from commit e492c64c8e)
2025-08-25 22:06:21 +00:00
Eelco Dolstra 9adbc08576 Bump version 2025-08-25 10:27:00 +02:00
John Ericson ec6ba866d1 Limit to lenient parsing of non-standard URLs only where needed
This allows us to put `parseURL` in more spots without furthering
technical debt.

(cherry picked from commit 72a548ed6a)
2025-08-23 12:03:01 -04:00
John Ericson 752d0ef1c0 decodeQuery Take std::string_view not string ref
(cherry picked from commit 4083eff0c0)
2025-08-23 12:02:56 -04:00
Eelco Dolstra f777aa70d3 Mark official release 2025-08-22 17:36:49 +02:00
551 changed files with 5631 additions and 12588 deletions

No files matched your search

-4
View File
@@ -15,10 +15,6 @@ so you understand the process and the expectations.
- volunteering contributions effectively
- how to get help and our review process.
PR stuck in review? We have two Nix team meetings per week online that are open for everyone in a jitsi conference:
- https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com
-->
## Motivation
-25
View File
@@ -30,31 +30,7 @@ jobs:
use_cache: false
- run: nix flake show --all-systems --json
pre-commit-checks:
name: pre-commit checks
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
- uses: ./.github/actions/install-nix-action
with:
dogfood: ${{ github.event_name == 'workflow_dispatch' && inputs.dogfood || github.event_name != 'workflow_dispatch' }}
extra_nix_config: experimental-features = nix-command flakes
github_token: ${{ secrets.GITHUB_TOKEN }}
- run: ./ci/gha/tests/pre-commit-checks
basic-checks:
name: aggregate basic checks
if: ${{ always() }}
runs-on: ubuntu-24.04
needs: [pre-commit-checks, eval]
steps:
- name: Exit with any errors
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: |
exit 1
tests:
needs: basic-checks
strategy:
fail-fast: false
matrix:
@@ -180,7 +156,6 @@ jobs:
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
vm_tests:
needs: basic-checks
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v5
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
runs-on: ubuntu-24.04
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/labeler@v6
- uses: actions/labeler@v5
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
sync-labels: false
+11
View File
@@ -39,6 +39,17 @@ jobs:
role-to-assume: "arn:aws:iam::080433136561:role/nix-release"
role-session-name: nix-release-oidc-${{ github.run_id }}
aws-region: eu-west-1
- name: Disable containerd image store
run: |
# Docker 28+ defaults to the containerd image store, which
# pushes layers uncompressed instead of gzip. OCI clients
# that only support gzip (e.g. go-containerregistry) fail
# with "gzip: invalid header". Disabling the containerd
# snapshotter restores the classic storage driver, which
# preserves gzip-compressed layers through the
# `docker load` / `docker push` pipeline.
echo '{"features":{"containerd-snapshotter":false}}' | sudo tee /etc/docker/daemon.json > /dev/null
sudo systemctl restart docker
- name: Login to Docker Hub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
-11
View File
@@ -161,14 +161,3 @@ pull_request_rules:
labels:
- automatic backport
- merge-queue
- name: backport patches to 2.31
conditions:
- label=backport 2.31-maintenance
actions:
backport:
branches:
- "2.31-maintenance"
labels:
- automatic backport
- merge-queue
+1 -1
View File
@@ -1 +1 @@
2.32.6
2.31.4
+14 -11
View File
@@ -1,8 +1,8 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
<https://fsf.org/>
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
@@ -10,7 +10,7 @@
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
@@ -112,7 +112,7 @@ modification follow. Pay close attention to the difference between a
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
@@ -146,7 +146,7 @@ such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
@@ -432,7 +432,7 @@ decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
@@ -455,7 +455,7 @@ FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
@@ -484,7 +484,8 @@ convey the exclusion of warranty; and each file should have at least the
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, see <https://www.gnu.org/licenses/>.
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
@@ -495,7 +496,9 @@ necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
<signature of Moe Ghoul>, 1 April 1990
Moe Ghoul, President of Vice
<signature of Ty Coon>, 1 April 1990
Ty Coon, President of Vice
That's all there is to it!
+10 -5
View File
@@ -24,7 +24,16 @@ let
enableSanitizersLayer = finalAttrs: prevAttrs: {
mesonFlags =
(prevAttrs.mesonFlags or [ ])
++ [ (lib.mesonOption "b_sanitize" "address,undefined") ]
++ [
# Run all tests with UBSAN enabled. Running both with ubsan and
# without doesn't seem to have much immediate benefit for doubling
# the GHA CI workaround.
#
# TODO: Work toward enabling "address,undefined" if it seems feasible.
# This would maybe require dropping Boost coroutines and ignoring intentional
# memory leaks with detect_leaks=0.
(lib.mesonOption "b_sanitize" "undefined")
]
++ (lib.optionals stdenv.cc.isClang [
# https://www.github.com/mesonbuild/meson/issues/764
(lib.mesonBool "b_lundef" false)
@@ -62,12 +71,8 @@ rec {
nixComponentsInstrumented = nixComponents.overrideScope (
final: prev: {
nix-store-tests = prev.nix-store-tests.override { withBenchmarks = true; };
# Boehm is incompatible with ASAN.
nix-expr = prev.nix-expr.override { enableGC = !withSanitizers; };
mesonComponentOverrides = lib.composeManyExtensions componentOverrides;
# Unclear how to make Perl bindings work with a dynamically linked ASAN.
nix-perl-bindings = if withSanitizers then null else prev.nix-perl-bindings;
}
);
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
system=$(nix eval --raw --impure --expr builtins.currentSystem)
echo "::group::Running pre-commit checks"
if nix build ".#checks.$system.pre-commit" -L; then
echo "::endgroup::"
exit 0
fi
echo "::error ::Changes do not pass pre-commit checks"
cat <<EOF
The code isn't formatted or doesn't pass lints. You can run pre-commit locally with:
nix develop -c ./maintainers/format.sh
EOF
echo "::endgroup::"
exit 1
-1
View File
@@ -15,7 +15,6 @@ pymod = import('python')
python = pymod.find_installation('python3')
nix_env_for_docs = {
'ASAN_OPTIONS' : 'abort_on_error=1:print_summary=1:detect_leaks=0',
'HOME' : '/dummy',
'NIX_CONF_DIR' : '/dummy',
'NIX_SSL_CERT_FILE' : '/dummy/no-ca-bundle.crt',
@@ -0,0 +1,6 @@
---
synopsis: "Temporary build directories no longer include derivation names"
prs: [13839]
---
Temporary build directories created during derivation builds no longer include the derivation name in their path to avoid build failures when the derivation name is too long. This change ensures predictable prefix lengths for build directories under `/nix/var/nix/builds`.
-1
View File
@@ -138,7 +138,6 @@
- [Contributing](development/contributing.md)
- [Releases](release-notes/index.md)
{{#include ./SUMMARY-rl-next.md}}
- [Release 2.32 (2025-10-06)](release-notes/rl-2.32.md)
- [Release 2.31 (2025-08-21)](release-notes/rl-2.31.md)
- [Release 2.30 (2025-07-07)](release-notes/rl-2.30.md)
- [Release 2.29 (2025-05-14)](release-notes/rl-2.29.md)
@@ -2,7 +2,6 @@ xp_features_json = custom_target(
command : [ nix, '__dump-xp-features' ],
capture : true,
output : 'xp-features.json',
env : nix_env_for_docs,
)
experimental_features_shortlist_md = custom_target(
+2 -2
View File
@@ -23,7 +23,7 @@ $ nix-shell
To get a shell with one of the other [supported compilation environments](#compilation-environments):
```console
$ nix-shell --attr devShells.x86_64-linux.native-clangStdenv
$ nix-shell --attr devShells.x86_64-linux.native-clangStdenvPackages
```
> **Note**
@@ -34,7 +34,7 @@ $ nix-shell --attr devShells.x86_64-linux.native-clangStdenv
To build Nix itself in this shell:
```console
[nix-shell]$ out="$(pwd)/outputs/out" dev=$out debug=$out mesonFlags+=" --prefix=${out}"
[nix-shell]$ mesonFlags+=" --prefix=$(pwd)/outputs/out"
[nix-shell]$ dontAddPrefix=1 configurePhase
[nix-shell]$ buildPhase
```
@@ -24,19 +24,6 @@ It is also possible to build without debugging for faster build:
(The first line is needed because `fortify` hardening requires at least some optimization.)
## Building Nix with sanitizers
Nix can be built with [Address](https://clang.llvm.org/docs/AddressSanitizer.html) and
[UB](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html) sanitizers using LLVM
or GCC. This is useful when debugging memory corruption issues.
```console
[nix-shell]$ export mesonBuildType=debugoptimized
[nix-shell]$ appendToVar mesonFlags "-Dlibexpr:gc=disabled" # Disable Boehm
[nix-shell]$ appendToVar mesonFlags "-Dbindings=false" # Disable nix-perl
[nix-shell]$ appendToVar mesonFlags "-Db_sanitize=address,undefined"
```
## Debugging the Nix Binary
Obtain your preferred debugger within the development shell:
@@ -7,6 +7,5 @@ experimental_feature_descriptions_md = custom_target(
xp_features_json,
],
capture : true,
env : nix_env_for_docs,
output : 'experimental-feature-descriptions.md',
)
+3 -19
View File
@@ -5,28 +5,12 @@ All built-ins are available through the global [`builtins`](#builtins-builtins)
Some built-ins are also exposed directly in the global scope:
<!-- TODO(@rhendric, #10970): this list is incomplete -->
- [`derivation`](#builtins-derivation)
- `derivationStrict`
- [`abort`](#builtins-abort)
- [`baseNameOf`](#builtins-baseNameOf)
- [`break`](#builtins-break)
- [`dirOf`](#builtins-dirOf)
- [`false`](#builtins-false)
- [`fetchGit`](#builtins-fetchGit)
- `fetchMercurial`
- [`fetchTarball`](#builtins-fetchTarball)
- [`fetchTree`](#builtins-fetchTree)
- [`fromTOML`](#builtins-fromTOML)
- [`import`](#builtins-import)
- [`isNull`](#builtins-isNull)
- [`map`](#builtins-map)
- [`null`](#builtins-null)
- [`placeholder`](#builtins-placeholder)
- [`removeAttrs`](#builtins-removeAttrs)
- `scopedImport`
- [`abort`](#builtins-abort)
- [`throw`](#builtins-throw)
- [`toString`](#builtins-toString)
- [`true`](#builtins-true)
<dl>
<dt id="builtins-derivation"><a href="#builtins-derivation"><code>derivation <var>attrs</var></code></a></dt>
+3 -26
View File
@@ -14,21 +14,6 @@ is a JSON object with the following fields:
The name of the derivation.
This is used when calculating the store paths of the derivation's outputs.
* `version`:
Must be `3`.
This is a guard that allows us to continue evolving this format.
The choice of `3` is fairly arbitrary, but corresponds to this informal version:
- Version 0: A-Term format
- Version 1: Original JSON format, with ugly `"r:sha256"` inherited from A-Term format.
- Version 2: Separate `method` and `hashAlgo` fields in output specs
- Verison 3: Drop store dir from store paths, just include base name.
Note that while this format is experimental, the maintenance of versions is best-effort, and not promised to identify every change.
* `outputs`:
Information about the output paths of the derivation.
This is a JSON object with one member per output, where the key is the output name and the value is a JSON object with these fields:
@@ -67,6 +52,7 @@ is a JSON object with the following fields:
> ```json
> "outputs": {
> "out": {
> "path": "/nix/store/2543j7c6jn75blc3drf4g5vhb1rhdq29-source",
> "method": "nar",
> "hashAlgo": "sha256",
> "hash": "6fc80dcc62179dbc12fc0b5881275898f93444833d21b89dfe5f7fbcbb1d0d62"
@@ -77,15 +63,6 @@ is a JSON object with the following fields:
* `inputSrcs`:
A list of store paths on which this derivation depends.
> **Example**
>
> ```json
> "inputSrcs": [
> "47y241wqdhac3jm5l7nv0x4975mb1975-separate-debug-info.sh",
> "56d0w71pjj9bdr363ym3wj1zkwyqq97j-fix-pop-var-context-error.patch"
> ]
> ```
* `inputDrvs`:
A JSON object specifying the derivations on which this derivation depends, and what outputs of those derivations.
@@ -93,8 +70,8 @@ is a JSON object with the following fields:
>
> ```json
> "inputDrvs": {
> "6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
> "fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
> "/nix/store/6lkh5yi7nlb7l6dr8fljlli5zfd9hq58-curl-7.73.0.drv": ["dev"],
> "/nix/store/fn3kgnfzl5dzym26j8g907gq3kbm8bfh-unzip-6.0.drv": ["out"]
> }
> ```
-165
View File
@@ -1,165 +0,0 @@
# Release 2.32.0 (2025-10-06)
## Incompatible changes
- Removed support for daemons and clients older than Nix 2.0 [#13951](https://github.com/NixOS/nix/pull/13951)
We have dropped support in the daemon worker protocol for daemons and clients that don't speak at least version 18 of the protocol. This first Nix release that supports this version is Nix 2.0, released in February 2018.
- Derivation JSON format now uses store path basenames only [#13570](https://github.com/NixOS/nix/issues/13570) [#13980](https://github.com/NixOS/nix/pull/13980)
Experience with many JSON frameworks (e.g. nlohmann/json in C++, Serde in Rust, and Aeson in Haskell) has shown that the use of the store directory in JSON formats is an impediment to systematic JSON formats, because it requires the serializer/deserializer to take an extra parameter (the store directory).
We ultimately want to rectify this issue with all JSON formats to the extent allowed by our stability promises. To start with, we are changing the JSON format for derivations because the `nix derivation` commands are — in addition to being formally unstable — less widely used than other unstable commands.
See the documentation on the [JSON format for derivations](@docroot@/protocols/json/derivation.md) for further details.
- C API: `nix_get_attr_name_byidx`, `nix_get_attr_byidx` take a `nix_value *` instead of `const nix_value *` [#13987](https://github.com/NixOS/nix/pull/13987)
In order to accommodate a more optimized internal representation of attribute set merges these functions require
a mutable `nix_value *` that might be modified on access. This does *not* break the ABI of these functions.
## New features
- C API: Add lazy attribute and list item accessors [#14030](https://github.com/NixOS/nix/pull/14030)
The C API now includes lazy accessor functions for retrieving values from lists and attribute sets without forcing evaluation:
- `nix_get_list_byidx_lazy()` - Get a list element without forcing its evaluation
- `nix_get_attr_byname_lazy()` - Get an attribute value by name without forcing evaluation
- `nix_get_attr_byidx_lazy()` - Get an attribute by index without forcing evaluation
These functions are useful when forwarding unevaluated sub-values to other lists, attribute sets, or function calls. They allow more efficient handling of Nix values by deferring evaluation until actually needed.
Additionally, bounds checking has been improved for all `_byidx` functions to properly validate indices before access, preventing potential out-of-bounds errors.
The documentation for `NIX_ERR_KEY` error handling has also been clarified to specify when this error code is returned.
- HTTP binary caches now support transparent compression for metadata
HTTP binary cache stores can now compress `.narinfo`, `.ls`, and build log files before uploading them,
reducing bandwidth usage and storage requirements. The compression is applied transparently using the
`Content-Encoding` header, allowing compatible clients to automatically decompress the files.
Three new configuration options control this behavior:
- `narinfo-compression`: Compression method for `.narinfo` files
- `ls-compression`: Compression method for `.ls` files
- `log-compression`: Compression method for build logs in `log/` directory
Example usage:
```
nix copy --to 'http://cache.example.com?narinfo-compression=gzip&ls-compression=gzip' /nix/store/...
nix store copy-log --to 'http://cache.example.com?log-compression=br' /nix/store/...
```
- Temporary build directories no longer include derivation names [#13839](https://github.com/NixOS/nix/pull/13839)
Temporary build directories created during derivation builds no longer include the derivation name in their path to avoid build failures when the derivation name is too long. This change ensures predictable prefix lengths for build directories under `/nix/var/nix/builds`.
- External derivation builders [#14145](https://github.com/NixOS/nix/pull/14145)
These are helper programs that Nix calls to perform derivations for specified system types, e.g. by using QEMU to emulate a different type of platform. For more information, see the [`external-builders` setting](../command-ref/conf-file.md#conf-external-builders).
This is currently an experimental feature.
## Performance improvements
- Optimize memory usage of attribute set merges [#13987](https://github.com/NixOS/nix/pull/13987)
[Attribute set update operations](@docroot@/language/operators.md#update) have been optimized to
reduce reallocations in cases when the second operand is small.
For typical evaluations of nixpkgs this optimization leads to ~20% less memory allocated in total
without significantly affecting evaluation performance.
See [eval-attrset-update-layer-rhs-threshold](@docroot@/command-ref/conf-file.md#conf-eval-attrset-update-layer-rhs-threshold)
- Substituted flake inputs are no longer re-copied to the store [#14041](https://github.com/NixOS/nix/pull/14041)
Since 2.25, Nix would fail to store a cache entry for substituted flake inputs, which in turn would cause them to be re-copied to the store on initial evaluation. Caching these inputs results in a near doubling of performance in some cases — especially on I/O-bound machines and when using commands that fetch many inputs, like `nix flake [archive|prefetch-inputs]`.
- `nix flake check` now skips derivations that can be substituted [#13574](https://github.com/NixOS/nix/pull/13574)
Previously, `nix flake check` would evaluate and build/substitute all
derivations. Now, it will skip downloading derivations that can be substituted.
This can drastically decrease the time invocations take in environments where
checks may already be cached (like in CI).
- `fetchTarball` and `fetchurl` now correctly substitute (#14138)
At some point we stopped substituting calls to `fetchTarball` and `fetchurl` with a set `narHash` to avoid incorrectly substituting things in `fetchTree`, even though it would be safe to substitute when calling the legacy `fetch{Tarball,url}`. This fixes that regression where it is safe.
- Started moving AST allocations into a bump allocator [#14088](https://github.com/NixOS/nix/issues/14088)
This leaves smaller, immutable structures in the AST. So far this saves about 2% memory on a NixOS config evaluation.
## Contributors
This release was made possible by the following 32 contributors:
- Farid Zakaria [**(@fzakaria)**](https://github.com/fzakaria)
- dram [**(@dramforever)**](https://github.com/dramforever)
- Ephraim Siegfried [**(@EphraimSiegfried)**](https://github.com/EphraimSiegfried)
- Robert Hensing [**(@roberth)**](https://github.com/roberth)
- Taeer Bar-Yam [**(@Radvendii)**](https://github.com/Radvendii)
- Emily [**(@emilazy)**](https://github.com/emilazy)
- Jens Petersen [**(@juhp)**](https://github.com/juhp)
- Bernardo Meurer [**(@lovesegfault)**](https://github.com/lovesegfault)
- Jörg Thalheim [**(@Mic92)**](https://github.com/Mic92)
- Leandro Emmanuel Reina Kiperman [**(@kip93)**](https://github.com/kip93)
- Marie [**(@NyCodeGHG)**](https://github.com/NyCodeGHG)
- Ethan Evans [**(@ethanavatar)**](https://github.com/ethanavatar)
- Yaroslav Bolyukin [**(@CertainLach)**](https://github.com/CertainLach)
- Matej Urbas [**(@urbas)**](https://github.com/urbas)
- Jami Kettunen [**(@JamiKettunen)**](https://github.com/JamiKettunen)
- Clayton [**(@netadr)**](https://github.com/netadr)
- Grégory Marti [**(@gmarti)**](https://github.com/gmarti)
- Eelco Dolstra [**(@edolstra)**](https://github.com/edolstra)
- rszyma [**(@rszyma)**](https://github.com/rszyma)
- Philip Wilk [**(@philipwilk)**](https://github.com/philipwilk)
- John Ericson [**(@Ericson2314)**](https://github.com/Ericson2314)
- Tom Westerhout [**(@twesterhout)**](https://github.com/twesterhout)
- Tristan Ross [**(@RossComputerGuy)**](https://github.com/RossComputerGuy)
- Sergei Zimmerman [**(@xokdvium)**](https://github.com/xokdvium)
- Jean-François Roche [**(@jfroche)**](https://github.com/jfroche)
- Seth Flynn [**(@getchoo)**](https://github.com/getchoo)
- éclairevoyant [**(@eclairevoyant)**](https://github.com/eclairevoyant)
- Glen Huang [**(@hgl)**](https://github.com/hgl)
- osman - オスマン [**(@osbm)**](https://github.com/osbm)
- David McFarland [**(@corngood)**](https://github.com/corngood)
- Cole Helbling [**(@cole-h)**](https://github.com/cole-h)
- Sinan Mohd [**(@sinanmohd)**](https://github.com/sinanmohd)
- Philipp Otterbein
# Release 2.32.5 (2026-01-02)
## Bug fixes
- Fix `builtins.fetchGit` with `ref = "HEAD"` [#13948](https://github.com/NixOS/nix/issues/13948) [#14673](https://github.com/NixOS/nix/pull/14673)
- Fix dynamic attributes that are simple string expressions [#14642](https://github.com/NixOS/nix/issues/14642) [#14646](https://github.com/NixOS/nix/pull/14646)
Dynamic attributes are typically not allowed in [`let` expressions](https://nix.dev/manual/nix/2.32/language/syntax.html#let-expressions), but simple [string literals](https://nix.dev/manual/nix/2.32/language/string-literals.html) are allowed. This special-case was broken in prior releases of 2.32.
- Fix null pointer dereference on reading non-existent derivations [#14571](https://github.com/NixOS/nix/issues/14571) [#14891](https://github.com/NixOS/nix/issues/14891) [#14572](https://github.com/NixOS/nix/pull/14572)
- Fix use-after-free in derivation build scheduler [#14782](https://github.com/NixOS/nix/pull/14782)
- Avoid querying remaining substituters after first success [#14836](https://github.com/NixOS/nix/issues/14836) [#14839](https://github.com/NixOS/nix/pull/14839)
- Improve temporary path creation in hard link store optimisation [#7273](https://github.com/NixOS/nix/issues/7273) [#14680](https://github.com/NixOS/nix/pull/14680)
`auto-optimise-store` has been known to be flaky on Darwin due the usage of `rand()` when constructing the path of a temporary hardlink.
- Move singletons out of headers [#14558](https://github.com/NixOS/nix/issues/14558)
Such variables have ["vague linkage"](https://gcc.gnu.org/onlinedocs/gcc-14.2.0/gcc/Vague-Linkage.html) and require special support from the dynamic linker to deduplicate across different shared libraries. Platforms such as Cygwin do not support this and duplicate the symbols instead.
- Fix `curl` with `c-ares` failing to resolve DNS inside sandbox on macOS [#14859](https://github.com/NixOS/nix/pull/14859)
- Fix `recursive-nix` [#14730](https://github.com/NixOS/nix/pull/14730)
## Improvements
- Support building manual with mdbook 0.5 [#14628](https://github.com/NixOS/nix/issues/14628) [#14693](https://github.com/NixOS/nix/pull/14693)
- Improve `parent directory is world-writable or a symlink` error message to include the offending path component [#13701](https://github.com/NixOS/nix/issues/13701) [#14849](https://github.com/NixOS/nix/pull/14849)
- Correct `build-dir` documentation in manual [#14748](https://github.com/NixOS/nix/pull/14748)
+1 -2
View File
@@ -20,8 +20,7 @@ The graph of references excluding self-references thus forms a [directed acyclic
[directed acyclic graph]: @docroot@/glossary.md#gloss-directed-acyclic-graph
We can take the [transitive closure] of the references graph, in which any pair of store objects have an edge if a *path* of one or more references exists from the first to the second object.
(A single reference always forms a path which is one reference long, but longer paths may connect objects which have no direct reference between them.)
We can take the [transitive closure] of the references graph, which any pair of store objects have an edge not if there is a single reference from the first to the second, but a path of one or more references from the first to the second.
The *requisites* of a store object are all store objects reachable by paths of references which start with given store object's references.
[transitive closure]: https://en.wikipedia.org/wiki/Transitive_closure
-3
View File
@@ -281,10 +281,7 @@ let
# may get replaced by pkgs.dockerTools.caCertificates
mkdir -p $out/etc/ssl/certs
# Old NixOS compatibility.
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs
# NixOS canonical location
ln -s /nix/var/nix/profiles/default/etc/ssl/certs/ca-bundle.crt $out/etc/ssl/certs/ca-certificates.crt
cat $passwdContentsPath > $out/etc/passwd
echo "" >> $out/etc/passwd
+1 -1
View File
@@ -46,7 +46,7 @@ The team meets twice a week (times are denoted in the [Europe/Amsterdam](https:/
- mark it as draft if it is blocked on the contributor
- escalate it back to the team by moving it to To discuss, and leaving a comment as to why the issue needs to be discussed again.
- Work meeting: Mondays 18:00-20:00 Europe/Amsterdam; see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
- Work meeting: Mondays 14:00-16:00 Europe/Amsterdam see [calendar](https://calendar.google.com/calendar/u/0/embed?src=b9o52fobqjak8oq8lfkhg3t0qg@group.calendar.google.com).
1. Code review on pull requests from [In review](#in-review).
2. Other chores and tasks.
@@ -203,26 +203,5 @@
"ConnorBaker01@Gmail.com": "ConnorBaker",
"jsoo1@asu.edu": "jsoo1",
"hsngrmpf+github@gmail.com": "DavHau",
"matthew@floxdev.com": "mkenigs",
"taeer@bar-yam.me": "Radvendii",
"beme@anthropic.com": "lovesegfault",
"osbm@osbm.dev": "osbm",
"jami.kettunen@protonmail.com": "JamiKettunen",
"ephraim.siegfried@hotmail.com": "EphraimSiegfried",
"rszyma.dev@gmail.com": "rszyma",
"tristan.ross@determinate.systems": "RossComputerGuy",
"corngood@gmail.com": "corngood",
"jfroche@pyxel.be": "jfroche",
"848000+eclairevoyant@users.noreply.github.com": "eclairevoyant",
"petersen@redhat.com": "juhp",
"dramforever@live.com": "dramforever",
"me@glenhuang.com": "hgl",
"philip.wilk@fivium.co.uk": "philipwilk",
"me@nycode.dev": "NyCodeGHG",
"14264576+twesterhout@users.noreply.github.com": "twesterhout",
"sinan@sinanmohd.com": "sinanmohd",
"42688647+netadr@users.noreply.github.com": "netadr",
"matej.urbas@gmail.com": "urbas",
"ethanalexevans@gmail.com": "ethanavatar",
"greg.marti@gmail.com": "gmarti"
"matthew@floxdev.com": "mkenigs"
}
@@ -177,24 +177,5 @@
"avnik": "Alexander V. Nikolaev",
"DavHau": null,
"aln730": "AGawas",
"vog": "Volker Diels-Grabsch",
"corngood": "David McFarland",
"twesterhout": "Tom Westerhout",
"JamiKettunen": "Jami Kettunen",
"dramforever": "dram",
"philipwilk": "Philip Wilk",
"netadr": "Clayton",
"NyCodeGHG": "Marie",
"jfroche": "Jean-Fran\u00e7ois Roche",
"urbas": "Matej Urbas",
"osbm": "osman - \u30aa\u30b9\u30de\u30f3",
"rszyma": null,
"eclairevoyant": "\u00e9clairevoyant",
"Radvendii": "Taeer Bar-Yam",
"sinanmohd": "Sinan Mohd",
"ethanavatar": "Ethan Evans",
"gmarti": "Gr\u00e9gory Marti",
"lovesegfault": "Bernardo Meurer",
"EphraimSiegfried": "Ephraim Siegfried",
"hgl": "Glen Huang"
"vog": "Volker Diels-Grabsch"
}
+145 -2
View File
@@ -79,8 +79,6 @@
# Not supported by nixfmt
''^tests/functional/lang/eval-okay-deprecate-cursed-or\.nix$''
''^tests/functional/lang/eval-okay-attrs5\.nix$''
''^tests/functional/lang/eval-fail-dynamic-attrs-inherit\.nix$''
''^tests/functional/lang/eval-fail-dynamic-attrs-inherit-2\.nix$''
# More syntax tests
# These tests, or parts of them, should have been parse-* test cases.
@@ -106,6 +104,151 @@
};
shellcheck = {
enable = true;
excludes = [
# We haven't linted these files yet
''^config/install-sh$''
''^misc/bash/completion\.sh$''
''^misc/fish/completion\.fish$''
''^misc/zsh/completion\.zsh$''
''^scripts/create-darwin-volume\.sh$''
''^scripts/install-darwin-multi-user\.sh$''
''^scripts/install-multi-user\.sh$''
''^scripts/install-systemd-multi-user\.sh$''
''^src/nix/get-env\.sh$''
''^tests/functional/ca/build-dry\.sh$''
''^tests/functional/ca/build-with-garbage-path\.sh$''
''^tests/functional/ca/common\.sh$''
''^tests/functional/ca/concurrent-builds\.sh$''
''^tests/functional/ca/eval-store\.sh$''
''^tests/functional/ca/gc\.sh$''
''^tests/functional/ca/import-from-derivation\.sh$''
''^tests/functional/ca/new-build-cmd\.sh$''
''^tests/functional/ca/nix-shell\.sh$''
''^tests/functional/ca/post-hook\.sh$''
''^tests/functional/ca/recursive\.sh$''
''^tests/functional/ca/repl\.sh$''
''^tests/functional/ca/selfref-gc\.sh$''
''^tests/functional/ca/why-depends\.sh$''
''^tests/functional/characterisation-test-infra\.sh$''
''^tests/functional/common/vars-and-functions\.sh$''
''^tests/functional/completions\.sh$''
''^tests/functional/compute-levels\.sh$''
''^tests/functional/config\.sh$''
''^tests/functional/db-migration\.sh$''
''^tests/functional/debugger\.sh$''
''^tests/functional/dependencies\.builder0\.sh$''
''^tests/functional/dependencies\.sh$''
''^tests/functional/dump-db\.sh$''
''^tests/functional/dyn-drv/build-built-drv\.sh$''
''^tests/functional/dyn-drv/common\.sh$''
''^tests/functional/dyn-drv/dep-built-drv\.sh$''
''^tests/functional/dyn-drv/eval-outputOf\.sh$''
''^tests/functional/dyn-drv/old-daemon-error-hack\.sh$''
''^tests/functional/dyn-drv/recursive-mod-json\.sh$''
''^tests/functional/eval-store\.sh$''
''^tests/functional/export-graph\.sh$''
''^tests/functional/export\.sh$''
''^tests/functional/extra-sandbox-profile\.sh$''
''^tests/functional/fetchClosure\.sh$''
''^tests/functional/fetchGit\.sh$''
''^tests/functional/fetchGitRefs\.sh$''
''^tests/functional/fetchGitSubmodules\.sh$''
''^tests/functional/fetchGitVerification\.sh$''
''^tests/functional/fetchMercurial\.sh$''
''^tests/functional/fixed\.builder1\.sh$''
''^tests/functional/fixed\.builder2\.sh$''
''^tests/functional/fixed\.sh$''
''^tests/functional/flakes/absolute-paths\.sh$''
''^tests/functional/flakes/check\.sh$''
''^tests/functional/flakes/config\.sh$''
''^tests/functional/flakes/flakes\.sh$''
''^tests/functional/flakes/follow-paths\.sh$''
''^tests/functional/flakes/prefetch\.sh$''
''^tests/functional/flakes/run\.sh$''
''^tests/functional/flakes/show\.sh$''
''^tests/functional/formatter\.sh$''
''^tests/functional/formatter\.simple\.sh$''
''^tests/functional/gc-auto\.sh$''
''^tests/functional/gc-concurrent\.builder\.sh$''
''^tests/functional/gc-concurrent\.sh$''
''^tests/functional/gc-concurrent2\.builder\.sh$''
''^tests/functional/gc-non-blocking\.sh$''
''^tests/functional/hash-convert\.sh$''
''^tests/functional/impure-derivations\.sh$''
''^tests/functional/impure-eval\.sh$''
''^tests/functional/install-darwin\.sh$''
''^tests/functional/legacy-ssh-store\.sh$''
''^tests/functional/linux-sandbox\.sh$''
''^tests/functional/local-overlay-store/add-lower-inner\.sh$''
''^tests/functional/local-overlay-store/add-lower\.sh$''
''^tests/functional/local-overlay-store/bad-uris\.sh$''
''^tests/functional/local-overlay-store/build-inner\.sh$''
''^tests/functional/local-overlay-store/build\.sh$''
''^tests/functional/local-overlay-store/check-post-init-inner\.sh$''
''^tests/functional/local-overlay-store/check-post-init\.sh$''
''^tests/functional/local-overlay-store/common\.sh$''
''^tests/functional/local-overlay-store/delete-duplicate-inner\.sh$''
''^tests/functional/local-overlay-store/delete-duplicate\.sh$''
''^tests/functional/local-overlay-store/delete-refs-inner\.sh$''
''^tests/functional/local-overlay-store/delete-refs\.sh$''
''^tests/functional/local-overlay-store/gc-inner\.sh$''
''^tests/functional/local-overlay-store/gc\.sh$''
''^tests/functional/local-overlay-store/optimise-inner\.sh$''
''^tests/functional/local-overlay-store/optimise\.sh$''
''^tests/functional/local-overlay-store/redundant-add-inner\.sh$''
''^tests/functional/local-overlay-store/redundant-add\.sh$''
''^tests/functional/local-overlay-store/remount\.sh$''
''^tests/functional/local-overlay-store/stale-file-handle-inner\.sh$''
''^tests/functional/local-overlay-store/stale-file-handle\.sh$''
''^tests/functional/local-overlay-store/verify-inner\.sh$''
''^tests/functional/local-overlay-store/verify\.sh$''
''^tests/functional/logging\.sh$''
''^tests/functional/misc\.sh$''
''^tests/functional/multiple-outputs\.sh$''
''^tests/functional/nested-sandboxing\.sh$''
''^tests/functional/nested-sandboxing/command\.sh$''
''^tests/functional/nix-build\.sh$''
''^tests/functional/nix-channel\.sh$''
''^tests/functional/nix-collect-garbage-d\.sh$''
''^tests/functional/nix-copy-ssh-common\.sh$''
''^tests/functional/nix-copy-ssh-ng\.sh$''
''^tests/functional/nix-copy-ssh\.sh$''
''^tests/functional/nix-daemon-untrusting\.sh$''
''^tests/functional/nix-profile\.sh$''
''^tests/functional/nix-shell\.sh$''
''^tests/functional/nix_path\.sh$''
''^tests/functional/optimise-store\.sh$''
''^tests/functional/output-normalization\.sh$''
''^tests/functional/parallel\.builder\.sh$''
''^tests/functional/parallel\.sh$''
''^tests/functional/pass-as-file\.sh$''
''^tests/functional/path-from-hash-part\.sh$''
''^tests/functional/path-info\.sh$''
''^tests/functional/placeholders\.sh$''
''^tests/functional/post-hook\.sh$''
''^tests/functional/pure-eval\.sh$''
''^tests/functional/push-to-store-old\.sh$''
''^tests/functional/push-to-store\.sh$''
''^tests/functional/read-only-store\.sh$''
''^tests/functional/readfile-context\.sh$''
''^tests/functional/recursive\.sh$''
''^tests/functional/referrers\.sh$''
''^tests/functional/remote-store\.sh$''
''^tests/functional/repair\.sh$''
''^tests/functional/restricted\.sh$''
''^tests/functional/search\.sh$''
''^tests/functional/secure-drv-outputs\.sh$''
''^tests/functional/selfref-gc\.sh$''
''^tests/functional/shell\.shebang\.sh$''
''^tests/functional/simple\.builder\.sh$''
''^tests/functional/supplementary-groups\.sh$''
''^tests/functional/toString-path\.sh$''
''^tests/functional/user-envs-migration\.sh$''
''^tests/functional/user-envs-test-case\.sh$''
''^tests/functional/user-envs\.builder\.sh$''
''^tests/functional/user-envs\.sh$''
''^tests/functional/why-depends\.sh$''
];
};
};
};
-58
View File
@@ -1,58 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# debug:
# set -x
START_REF="${1}"
END_REF="${2:-upstream/master}"
# Get the merge base
MERGE_BASE=$(git merge-base "$START_REF" "$END_REF")
unset START_REF
# Get date range
START_DATE=$(git show -s --format=%cI "$MERGE_BASE")
END_DATE=$(git show -s --format=%cI "$END_REF")
echo "Checking PRs merged between $START_DATE and $END_DATE" >&2
# Get all commits between merge base and HEAD
COMMITS=$(git rev-list "$MERGE_BASE..$END_REF")
# Convert to set for fast lookup
declare -A commit_set
for commit in $COMMITS; do
commit_set["$commit"]=1
done
# Get the current changelog
LOG_DONE="$(changelog-d doc/manual/rl-next)"
is_done(){
local nr="$1"
echo "$LOG_DONE" | grep -E "^- .*/pull/$nr)"
}
# Query merged PRs in date range
gh pr list \
--repo NixOS/nix \
--state merged \
--limit 1000 \
--json number,title,author,mergeCommit \
--search "merged:$START_DATE..$END_DATE" | \
jq -r '.[] | [.number, .mergeCommit.oid, .title, .author.login] | @tsv' | \
while IFS=$'\t' read -r pr_num merge_commit _title author; do
# Check if this PR's merge commit is in our branch
if [[ -n "${commit_set[$merge_commit]:-}" ]]; then
# Full detail, not suitable for comment due to mass ping and duplicate title
# echo "- #$pr_num $_title (@$author)"
echo "- #$pr_num ($author)"
if is_done "$pr_num"
then
echo " - [x] has note"
else
echo " - [ ] has note"
fi
echo " - [ ] skip"
fi
done
+4 -9
View File
@@ -24,12 +24,6 @@ release:
* In a checkout of the Nix repo, make sure you're on `master` and run
`git pull`.
* Compile a release notes to-do list by running
```console
$ ./maintainers/release-notes-todo PREV_RELEASE HEAD
```
* Compile the release notes by running
```console
@@ -129,8 +123,6 @@ release:
Commit and push this to the maintenance branch.
* Create a backport label.
* Bump the version of `master`:
```console
@@ -138,7 +130,6 @@ release:
$ git pull
$ NEW_VERSION=2.13.0
$ echo $NEW_VERSION > .version
$ ... edit .mergify.yml to add the previous version ...
$ git checkout -b bump-$NEW_VERSION
$ git commit -a -m 'Bump version'
$ git push --set-upstream origin bump-$NEW_VERSION
@@ -146,6 +137,10 @@ release:
Make a pull request and auto-merge it.
* Create a backport label.
* Add the new backport label to `.mergify.yml`.
* Post an [announcement on Discourse](https://discourse.nixos.org/c/announcements/8), including the contents of
`rl-$VERSION.md`.
+1 -3
View File
@@ -41,10 +41,8 @@ subproject('libexpr-c')
subproject('libflake-c')
subproject('libmain-c')
asan_enabled = 'address' in get_option('b_sanitize')
# Language Bindings
if get_option('bindings') and not meson.is_cross_build() and not asan_enabled
if get_option('bindings') and not meson.is_cross_build()
subproject('perl')
endif
-1
View File
@@ -1,4 +1,3 @@
# shellcheck shell=bash
function _complete_nix {
local -a words
local cword cur
-1
View File
@@ -1,4 +1,3 @@
# shellcheck disable=all
function _nix_complete
# Get the current command up to a cursor.
# - Behaves correctly even with pipes and nested in commands like env.
-1
View File
@@ -1,5 +1,4 @@
#compdef nix
# shellcheck disable=all
function _nix() {
local ifs_bk="$IFS"
@@ -1,12 +0,0 @@
asan_test_options_env = {
'ASAN_OPTIONS' : 'abort_on_error=1:print_summary=1:detect_leaks=0',
}
# Clang gets grumpy about missing libasan symbols if -shared-libasan is not
# passed when building shared libs, at least on Linux
if cxx.get_id() == 'clang' and ('address' in get_option('b_sanitize') or 'undefined' in get_option(
'b_sanitize',
))
add_project_link_arguments('-shared-libasan', language : 'cpp')
endif
@@ -1,32 +0,0 @@
can_wrap_assert_fail_test_code = '''
#include <cstdlib>
#include <cassert>
int main()
{
assert(0);
}
extern "C" void * __real___assert_fail(const char *, const char *, unsigned int, const char *);
extern "C" void *
__wrap___assert_fail(const char *, const char *, unsigned int, const char *)
{
return __real___assert_fail(nullptr, nullptr, 0, nullptr);
}
'''
wrap_assert_fail_args = [ '-Wl,--wrap=__assert_fail' ]
can_wrap_assert_fail = cxx.links(
can_wrap_assert_fail_test_code,
args : wrap_assert_fail_args,
name : 'linker can wrap __assert_fail',
)
if can_wrap_assert_fail
deps_other += declare_dependency(
sources : 'wrap-assert-fail.cc',
link_args : wrap_assert_fail_args,
)
endif
@@ -1,17 +0,0 @@
#include "nix/util/error.hh"
#include <cstdio>
#include <cstdlib>
#include <cinttypes>
#include <string_view>
extern "C" [[noreturn]] void __attribute__((weak))
__wrap___assert_fail(const char * assertion, const char * file, unsigned int line, const char * function)
{
char buf[512];
int n =
snprintf(buf, sizeof(buf), "Assertion '%s' failed in %s at %s:%" PRIuLEAST32, assertion, function, file, line);
if (n < 0)
nix::panic("Assertion failed and could not format error message");
nix::panic(std::string_view(buf, std::min(static_cast<int>(sizeof(buf)), n)));
}
+6 -32
View File
@@ -5,15 +5,6 @@ if not (host_machine.system() == 'windows' and cxx.get_id() == 'gcc')
deps_private += dependency('threads')
endif
if host_machine.system() == 'cygwin'
# -std=gnu on cygwin defines 'unix', which conflicts with the namespace
add_project_arguments(
'-D_POSIX_C_SOURCE=200809L',
'-D_GNU_SOURCE',
language : 'cpp',
)
endif
add_project_arguments(
'-Wdeprecated-copy',
'-Werror=suggest-override',
@@ -42,27 +33,10 @@ if cxx.get_id() == 'clang'
add_project_arguments('-fpch-instantiate-templates', language : 'cpp')
endif
# Detect if we're using libstdc++ (GCC's standard library)
# libstdc++ uses Intel TBB as backend for C++17 parallel algorithms when <execution> is included.
# boost::concurrent_flat_map includes <execution>, which would require linking against TBB.
# Since we don't actually use parallel algorithms, disable the TBB backend to avoid the dependency.
# TBB is a dependency of blake3 and leaking into our build environment.
is_using_libstdcxx = cxx.compiles(
'''
#include <ciso646>
#ifndef __GLIBCXX__
#error "not libstdc++"
#endif
int main() { return 0; }
''',
name : 'using libstdc++',
)
if is_using_libstdcxx
add_project_arguments('-D_GLIBCXX_USE_TBB_PAR_BACKEND=0', language : 'cpp')
# Clang gets grumpy about missing libasan symbols if -shared-libasan is not
# passed when building shared libs, at least on Linux
if cxx.get_id() == 'clang' and ('address' in get_option('b_sanitize') or 'undefined' in get_option(
'b_sanitize',
))
add_project_link_arguments('-shared-libasan', language : 'cpp')
endif
# Darwin ld doesn't like "X.Y.ZpreABCD+W"
nix_soversion = meson.project_version().split('+')[0].split('pre')[0]
subdir('assert-fail')
@@ -3,6 +3,6 @@
# This is needed for std::atomic on some platforms
# We did not manage to test this reliably on all platforms, so we hardcode
# it for now.
if host_machine.cpu_family() in [ 'arm', 'ppc' ]
if host_machine.cpu_family() == 'arm'
deps_other += cxx.find_library('atomic')
endif
-18
View File
@@ -164,24 +164,6 @@ let
};
mesonLibraryLayer = finalAttrs: prevAttrs: {
preConfigure =
let
interpositionFlags = [
"-fno-semantic-interposition"
"-Wl,-Bsymbolic-functions"
];
in
# NOTE: By default GCC disables interprocedular optimizations (in particular inlining) for
# position-independent code and thus shared libraries.
# Since LD_PRELOAD tricks aren't worth losing out on optimizations, we disable it for good.
# This is not the case for Clang, where inlining is done by default even without -fno-semantic-interposition.
# https://reviews.llvm.org/D102453
# https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
prevAttrs.preConfigure or ""
+ lib.optionalString stdenv.cc.isGNU ''
export CFLAGS="''${CFLAGS:-} ${toString interpositionFlags}"
export CXXFLAGS="''${CXXFLAGS:-} ${toString interpositionFlags}"
'';
outputs = prevAttrs.outputs or [ "out" ] ++ [ "dev" ];
};
-15
View File
@@ -57,21 +57,6 @@ scope: {
(prevAttrs.postInstall or "");
});
# TODO: Remove this when https://github.com/NixOS/nixpkgs/pull/442682 is included in a stable release
toml11 =
if lib.versionAtLeast pkgs.toml11.version "4.4.0" then
pkgs.toml11
else
pkgs.toml11.overrideAttrs rec {
version = "4.4.0";
src = pkgs.fetchFromGitHub {
owner = "ToruNiina";
repo = "toml11";
tag = "v${version}";
hash = "sha256-sgWKYxNT22nw376ttGsTdg0AMzOwp8QH3E8mx0BZJTQ=";
};
};
# TODO Hack until https://github.com/NixOS/nixpkgs/issues/45462 is fixed.
boost =
(pkgs.boost.override {
-1
View File
@@ -118,7 +118,6 @@ pkgs.nixComponents2.nix-util.overrideAttrs (
modular.pre-commit.settings.package
(pkgs.writeScriptBin "pre-commit-hooks-install" modular.pre-commit.settings.installationScript)
pkgs.buildPackages.nixfmt-rfc-style
pkgs.buildPackages.shellcheck
pkgs.buildPackages.gdb
]
++ lib.optional (stdenv.cc.isClang && stdenv.hostPlatform == stdenv.buildPlatform) (
+10 -21
View File
@@ -55,22 +55,18 @@ readonly NIX_INSTALLED_NIX="@nix@"
readonly NIX_INSTALLED_CACERT="@cacert@"
#readonly NIX_INSTALLED_NIX="/nix/store/j8dbv5w6jl34caywh2ygdy88knx1mdf7-nix-2.3.6"
#readonly NIX_INSTALLED_CACERT="/nix/store/7dxhzymvy330i28ii676fl1pqwcahv2f-nss-cacert-3.49.2"
EXTRACTED_NIX_PATH="$(dirname "$0")"
readonly EXTRACTED_NIX_PATH
readonly EXTRACTED_NIX_PATH="$(dirname "$0")"
# allow to override identity change command
NIX_BECOME=${NIX_BECOME:-sudo}
readonly NIX_BECOME
readonly NIX_BECOME=${NIX_BECOME:-sudo}
ROOT_HOME=~root
readonly ROOT_HOME
readonly ROOT_HOME=~root
if [ -t 0 ] && [ -z "${NIX_INSTALLER_YES:-}" ]; then
IS_HEADLESS='no'
readonly IS_HEADLESS='no'
else
IS_HEADLESS='yes'
readonly IS_HEADLESS='yes'
fi
readonly IS_HEADLESS
headless() {
if [ "$IS_HEADLESS" = "yes" ]; then
@@ -160,7 +156,6 @@ EOF
}
nix_user_for_core() {
# shellcheck disable=SC2059
printf "$NIX_BUILD_USER_NAME_TEMPLATE" "$1"
}
@@ -386,12 +381,10 @@ _sudo() {
# Ensure that $TMPDIR exists if defined.
if [[ -n "${TMPDIR:-}" ]] && [[ ! -d "${TMPDIR:-}" ]]; then
# shellcheck disable=SC2174
mkdir -m 0700 -p "${TMPDIR:-}"
fi
SCRATCH=$(mktemp -d)
readonly SCRATCH
readonly SCRATCH=$(mktemp -d)
finish_cleanup() {
rm -rf "$SCRATCH"
}
@@ -684,8 +677,7 @@ create_directories() {
# hiding behind || true, and the general state
# should be one the user can repair once they
# figure out where chown is...
local get_chr_own
get_chr_own="$(PATH="$(getconf PATH 2>/dev/null)" command -vp chown)"
local get_chr_own="$(PATH="$(getconf PATH 2>/dev/null)" command -vp chown)"
if [[ -z "$get_chr_own" ]]; then
get_chr_own="$(command -v chown)"
fi
@@ -923,11 +915,9 @@ configure_shell_profile() {
fi
if [ -e "$profile_target" ]; then
{
shell_source_lines
cat "$profile_target"
} | _sudo "extend your $profile_target with nix-daemon settings" \
tee "$profile_target"
shell_source_lines \
| _sudo "extend your $profile_target with nix-daemon settings" \
tee -a "$profile_target"
fi
done
@@ -1023,7 +1013,6 @@ main() {
# Set profile targets after OS-specific scripts are loaded
if command -v poly_configure_default_profile_targets > /dev/null 2>&1; then
# shellcheck disable=SC2207
PROFILE_TARGETS=($(poly_configure_default_profile_targets))
else
PROFILE_TARGETS=("/etc/bashrc" "/etc/profile.d/nix.sh" "/etc/zshrc" "/etc/bash.bashrc" "/etc/zsh/zshrc")
+1 -1
View File
@@ -39,7 +39,7 @@ create_systemd_proxy_env() {
vars="http_proxy https_proxy ftp_proxy all_proxy no_proxy HTTP_PROXY HTTPS_PROXY FTP_PROXY ALL_PROXY NO_PROXY"
for v in $vars; do
if [ "x${!v:-}" != "x" ]; then
echo "Environment=${v}=$(escape_systemd_env "${!v}")"
echo "Environment=${v}=$(escape_systemd_env ${!v})"
fi
done
}
+2 -12
View File
@@ -83,22 +83,12 @@ nlohmann::json SingleBuiltPath::Built::toJSON(const StoreDirConfig & store) cons
nlohmann::json SingleBuiltPath::toJSON(const StoreDirConfig & store) const
{
return std::visit(
overloaded{
[&](const SingleBuiltPath::Opaque & o) -> nlohmann::json { return store.printStorePath(o.path); },
[&](const SingleBuiltPath::Built & b) { return b.toJSON(store); },
},
raw());
return std::visit([&](const auto & buildable) { return buildable.toJSON(store); }, raw());
}
nlohmann::json BuiltPath::toJSON(const StoreDirConfig & store) const
{
return std::visit(
overloaded{
[&](const BuiltPath::Opaque & o) -> nlohmann::json { return store.printStorePath(o.path); },
[&](const BuiltPath::Built & b) { return b.toJSON(store); },
},
raw());
return std::visit([&](const auto & buildable) { return buildable.toJSON(store); }, raw());
}
RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
-13
View File
@@ -2,7 +2,6 @@
#include <nlohmann/json.hpp>
#include "nix/cmd/command.hh"
#include "nix/cmd/legacy.hh"
#include "nix/cmd/markdown.hh"
#include "nix/store/store-open.hh"
#include "nix/store/local-fs-store.hh"
@@ -15,18 +14,6 @@
namespace nix {
RegisterCommand::Commands & RegisterCommand::commands()
{
static RegisterCommand::Commands commands;
return commands;
}
RegisterLegacyCommand::Commands & RegisterLegacyCommand::commands()
{
static RegisterLegacyCommand::Commands commands;
return commands;
}
nix::Commands RegisterCommand::getCommandsFor(const std::vector<std::string> & prefix)
{
nix::Commands res;
+5 -1
View File
@@ -286,7 +286,11 @@ struct RegisterCommand
{
typedef std::map<std::vector<std::string>, std::function<ref<Command>()>> Commands;
static Commands & commands();
static Commands & commands()
{
static Commands commands;
return commands;
}
RegisterCommand(std::vector<std::string> && name, std::function<ref<Command>()> command)
{
+5 -1
View File
@@ -13,7 +13,11 @@ struct RegisterLegacyCommand
{
typedef std::map<std::string, MainFunction> Commands;
static Commands & commands();
static Commands & commands()
{
static Commands commands;
return commands;
}
RegisterLegacyCommand(const std::string & name, MainFunction fun)
{
+4 -4
View File
@@ -105,8 +105,8 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
std::optional<NixInt::Inner> priority;
if (attr->maybeGetAttr(state->s.outputSpecified)) {
} else if (auto aMeta = attr->maybeGetAttr(state->s.meta)) {
if (attr->maybeGetAttr(state->sOutputSpecified)) {
} else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aPriority = aMeta->maybeGetAttr("priority"))
priority = aPriority->getInt().value;
}
@@ -119,12 +119,12 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
overloaded{
[&](const ExtendedOutputsSpec::Default & d) -> OutputsSpec {
StringSet outputsToInstall;
if (auto aOutputSpecified = attr->maybeGetAttr(state->s.outputSpecified)) {
if (auto aOutputSpecified = attr->maybeGetAttr(state->sOutputSpecified)) {
if (aOutputSpecified->getBool()) {
if (auto aOutputName = attr->maybeGetAttr("outputName"))
outputsToInstall = {aOutputName->getString()};
}
} else if (auto aMeta = attr->maybeGetAttr(state->s.meta)) {
} else if (auto aMeta = attr->maybeGetAttr(state->sMeta)) {
if (auto aOutputsToInstall = aMeta->maybeGetAttr("outputsToInstall"))
for (auto & s : aOutputsToInstall->getListOfStrings())
outputsToInstall.insert(s);
+8 -10
View File
@@ -604,28 +604,28 @@ std::vector<BuiltPathWithResult> Installable::build(
static void throwBuildErrors(std::vector<KeyedBuildResult> & buildResults, const Store & store)
{
std::vector<std::pair<const KeyedBuildResult *, const KeyedBuildResult::Failure *>> failed;
std::vector<KeyedBuildResult> failed;
for (auto & buildResult : buildResults) {
if (auto * failure = buildResult.tryGetFailure()) {
failed.push_back({&buildResult, failure});
if (!buildResult.success()) {
failed.push_back(buildResult);
}
}
auto failedResult = failed.begin();
if (failedResult != failed.end()) {
if (failed.size() == 1) {
failedResult->second->rethrow();
failedResult->rethrow();
} else {
StringSet failedPaths;
for (; failedResult != failed.end(); failedResult++) {
if (!failedResult->second->errorMsg.empty()) {
if (!failedResult->errorMsg.empty()) {
logError(
ErrorInfo{
.level = lvlError,
.msg = failedResult->second->errorMsg,
.msg = failedResult->errorMsg,
});
}
failedPaths.insert(failedResult->first->path.to_string(store));
failedPaths.insert(failedResult->path.to_string(store));
}
throw Error("build of %s failed", concatStringsSep(", ", quoteStrings(failedPaths)));
}
@@ -695,14 +695,12 @@ std::vector<std::pair<ref<Installable>, BuiltPathWithResult>> Installable::build
auto buildResults = store->buildPathsWithResults(pathsToBuild, bMode, evalStore);
throwBuildErrors(buildResults, *store);
for (auto & buildResult : buildResults) {
// If we didn't throw, they must all be sucesses
auto & success = std::get<nix::BuildResult::Success>(buildResult.inner);
for (auto & aux : backmap[buildResult.path]) {
std::visit(
overloaded{
[&](const DerivedPath::Built & bfd) {
std::map<std::string, StorePath> outputs;
for (auto & [outputName, realisation] : success.builtOutputs)
for (auto & [outputName, realisation] : buildResult.builtOutputs)
outputs.emplace(outputName, realisation.outPath);
res.push_back(
{aux.installable,
-2
View File
@@ -67,7 +67,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'built-path.cc',
@@ -96,7 +95,6 @@ this_library = library(
'nixcmd',
sources,
config_priv_h,
soversion : nix_soversion,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args : linker_export_flags,
+8 -7
View File
@@ -760,7 +760,7 @@ void NixRepl::loadFlake(const std::string & flakeRefS)
void NixRepl::initEnv()
{
env = &state->mem.allocEnv(envSize);
env = &state->allocEnv(envSize);
env->up = &state->baseEnv;
displ = 0;
staticEnv->vars.clear();
@@ -869,8 +869,14 @@ void NixRepl::addVarToScope(const Symbol name, Value & v)
Expr * NixRepl::parseString(std::string s)
{
return state->parseExprFromString(std::move(s), state->rootPath("."), staticEnv);
}
void NixRepl::evalString(std::string s, Value & v)
{
Expr * e;
try {
return state->parseExprFromString(std::move(s), state->rootPath("."), staticEnv);
e = parseString(s);
} catch (ParseError & e) {
if (e.msg().find("unexpected end of file") != std::string::npos)
// For parse errors on incomplete input, we continue waiting for the next line of
@@ -879,11 +885,6 @@ Expr * NixRepl::parseString(std::string s)
else
throw;
}
}
void NixRepl::evalString(std::string s, Value & v)
{
Expr * e = parseString(s);
e->eval(*state, *env, v);
state->forceValue(v, v.determinePos(noPos));
}
-2
View File
@@ -28,7 +28,6 @@ deps_public_maybe_subproject = [
subdir('nix-meson-build-support/subprojects')
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'nix_api_expr.cc',
@@ -51,7 +50,6 @@ subdir('nix-meson-build-support/windows-version')
this_library = library(
'nixexprc',
sources,
soversion : nix_soversion,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args : linker_export_flags,
+20 -16
View File
@@ -16,7 +16,7 @@
#include "nix_api_util_internal.h"
#if NIX_USE_BOEHMGC
# include <boost/unordered/concurrent_flat_map.hpp>
# include <mutex>
#endif
/**
@@ -40,8 +40,6 @@ static T * unsafe_new_with_self(F && init)
return new (p) T(init(static_cast<T *>(p)));
}
extern "C" {
nix_err nix_libexpr_init(nix_c_context * context)
{
if (context)
@@ -137,7 +135,7 @@ nix_eval_state_builder * nix_eval_state_builder_new(nix_c_context * context, Sto
void nix_eval_state_builder_free(nix_eval_state_builder * builder)
{
operator delete(builder, static_cast<std::align_val_t>(alignof(nix_eval_state_builder)));
delete builder;
}
nix_err nix_eval_state_builder_load(nix_c_context * context, nix_eval_state_builder * builder)
@@ -203,24 +201,32 @@ EvalState * nix_state_create(nix_c_context * context, const char ** lookupPath_c
void nix_state_free(EvalState * state)
{
operator delete(state, static_cast<std::align_val_t>(alignof(EvalState)));
delete state;
}
#if NIX_USE_BOEHMGC
boost::concurrent_flat_map<
std::unordered_map<
const void *,
unsigned int,
std::hash<const void *>,
std::equal_to<const void *>,
traceable_allocator<std::pair<const void * const, unsigned int>>>
nix_refcounts{};
nix_refcounts;
std::mutex nix_refcount_lock;
nix_err nix_gc_incref(nix_c_context * context, const void * p)
{
if (context)
context->last_err_code = NIX_OK;
try {
nix_refcounts.insert_or_visit({p, 1}, [](auto & kv) { kv.second++; });
std::scoped_lock lock(nix_refcount_lock);
auto f = nix_refcounts.find(p);
if (f != nix_refcounts.end()) {
f->second++;
} else {
nix_refcounts[p] = 1;
}
}
NIXC_CATCH_ERRS
}
@@ -231,12 +237,12 @@ nix_err nix_gc_decref(nix_c_context * context, const void * p)
if (context)
context->last_err_code = NIX_OK;
try {
bool fail = true;
nix_refcounts.erase_if(p, [&](auto & kv) {
fail = false;
return !--kv.second;
});
if (fail)
std::scoped_lock lock(nix_refcount_lock);
auto f = nix_refcounts.find(p);
if (f != nix_refcounts.end()) {
if (--f->second == 0)
nix_refcounts.erase(f);
} else
throw std::runtime_error("nix_gc_decref: object was not referenced");
}
NIXC_CATCH_ERRS
@@ -281,5 +287,3 @@ void nix_gc_register_finalizer(void * obj, void * cd, void (*finalizer)(void * o
GC_REGISTER_FINALIZER(obj, finalizer, cd, 0, 0);
#endif
}
} // extern "C"
-4
View File
@@ -8,8 +8,6 @@
#include "nix_api_value.h"
#include "nix/expr/search-path.hh"
extern "C" {
struct nix_eval_state_builder
{
nix::ref<nix::Store> store;
@@ -63,6 +61,4 @@ struct nix_realised_string
std::vector<StorePath> storePaths;
};
} // extern "C"
#endif // NIX_API_EXPR_INTERNAL_H
-8
View File
@@ -14,8 +14,6 @@
#include <nlohmann/json.hpp>
extern "C" {
void nix_set_string_return(nix_string_return * str, const char * c)
{
str->str = c;
@@ -42,8 +40,6 @@ nix_err nix_external_add_string_context(nix_c_context * context, nix_string_cont
NIXC_CATCH_ERRS
}
} // extern "C"
class NixCExternalValue : public nix::ExternalValueBase
{
NixCExternalValueDesc & desc;
@@ -174,8 +170,6 @@ public:
virtual ~NixCExternalValue() override {};
};
extern "C" {
ExternalValue * nix_create_external_value(nix_c_context * context, NixCExternalValueDesc * desc, void * v)
{
if (context)
@@ -204,5 +198,3 @@ void * nix_get_external_value_content(nix_c_context * context, ExternalValue * b
}
NIXC_CATCH_ERRS_NULL
}
} // extern "C"
+5 -94
View File
@@ -111,8 +111,6 @@ static void nix_c_primop_wrapper(
v = vTmp;
}
extern "C" {
PrimOp * nix_alloc_primop(
nix_c_context * context,
PrimOpFun fun,
@@ -326,10 +324,6 @@ nix_value * nix_get_list_byidx(nix_c_context * context, const nix_value * value,
try {
auto & v = check_value_in(value);
assert(v.type() == nix::nList);
if (ix >= v.listSize()) {
nix_set_err_msg(context, NIX_ERR_KEY, "list index out of bounds");
return nullptr;
}
auto * p = v.listView()[ix];
nix_gc_incref(nullptr, p);
if (p != nullptr)
@@ -339,26 +333,6 @@ nix_value * nix_get_list_byidx(nix_c_context * context, const nix_value * value,
NIXC_CATCH_ERRS_NULL
}
nix_value *
nix_get_list_byidx_lazy(nix_c_context * context, const nix_value * value, EvalState * state, unsigned int ix)
{
if (context)
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
assert(v.type() == nix::nList);
if (ix >= v.listSize()) {
nix_set_err_msg(context, NIX_ERR_KEY, "list index out of bounds");
return nullptr;
}
auto * p = v.listView()[ix];
nix_gc_incref(nullptr, p);
// Note: intentionally NOT calling forceValue() to keep the element lazy
return as_nix_value_ptr(p);
}
NIXC_CATCH_ERRS_NULL
}
nix_value * nix_get_attr_byname(nix_c_context * context, const nix_value * value, EvalState * state, const char * name)
{
if (context)
@@ -379,27 +353,6 @@ nix_value * nix_get_attr_byname(nix_c_context * context, const nix_value * value
NIXC_CATCH_ERRS_NULL
}
nix_value *
nix_get_attr_byname_lazy(nix_c_context * context, const nix_value * value, EvalState * state, const char * name)
{
if (context)
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
assert(v.type() == nix::nAttrs);
nix::Symbol s = state->state.symbols.create(name);
auto attr = v.attrs()->get(s);
if (attr) {
nix_gc_incref(nullptr, attr->value);
// Note: intentionally NOT calling forceValue() to keep the attribute lazy
return as_nix_value_ptr(attr->value);
}
nix_set_err_msg(context, NIX_ERR_KEY, "missing attribute");
return nullptr;
}
NIXC_CATCH_ERRS_NULL
}
bool nix_has_attr_byname(nix_c_context * context, const nix_value * value, EvalState * state, const char * name)
{
if (context)
@@ -416,28 +369,13 @@ bool nix_has_attr_byname(nix_c_context * context, const nix_value * value, EvalS
NIXC_CATCH_ERRS_RES(false);
}
static void collapse_attrset_layer_chain_if_needed(nix::Value & v, EvalState * state)
{
auto & attrs = *v.attrs();
if (attrs.isLayered()) {
auto bindings = state->state.buildBindings(attrs.size());
std::ranges::copy(attrs, std::back_inserter(bindings));
v.mkAttrs(bindings);
}
}
nix_value *
nix_get_attr_byidx(nix_c_context * context, nix_value * value, EvalState * state, unsigned int i, const char ** name)
nix_value * nix_get_attr_byidx(
nix_c_context * context, const nix_value * value, EvalState * state, unsigned int i, const char ** name)
{
if (context)
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
collapse_attrset_layer_chain_if_needed(v, state);
if (i >= v.attrs()->size()) {
nix_set_err_msg(context, NIX_ERR_KEY, "attribute index out of bounds");
return nullptr;
}
const nix::Attr & a = (*v.attrs())[i];
*name = state->state.symbols[a.name].c_str();
nix_gc_incref(nullptr, a.value);
@@ -447,38 +385,13 @@ nix_get_attr_byidx(nix_c_context * context, nix_value * value, EvalState * state
NIXC_CATCH_ERRS_NULL
}
nix_value * nix_get_attr_byidx_lazy(
nix_c_context * context, nix_value * value, EvalState * state, unsigned int i, const char ** name)
const char *
nix_get_attr_name_byidx(nix_c_context * context, const nix_value * value, EvalState * state, unsigned int i)
{
if (context)
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
collapse_attrset_layer_chain_if_needed(v, state);
if (i >= v.attrs()->size()) {
nix_set_err_msg(context, NIX_ERR_KEY, "attribute index out of bounds (Nix C API contract violation)");
return nullptr;
}
const nix::Attr & a = (*v.attrs())[i];
*name = state->state.symbols[a.name].c_str();
nix_gc_incref(nullptr, a.value);
// Note: intentionally NOT calling forceValue() to keep the attribute lazy
return as_nix_value_ptr(a.value);
}
NIXC_CATCH_ERRS_NULL
}
const char * nix_get_attr_name_byidx(nix_c_context * context, nix_value * value, EvalState * state, unsigned int i)
{
if (context)
context->last_err_code = NIX_OK;
try {
auto & v = check_value_in(value);
collapse_attrset_layer_chain_if_needed(v, state);
if (i >= v.attrs()->size()) {
nix_set_err_msg(context, NIX_ERR_KEY, "attribute index out of bounds (Nix C API contract violation)");
return nullptr;
}
const nix::Attr & a = (*v.attrs())[i];
return state->state.symbols[a.name].c_str();
}
@@ -679,7 +592,7 @@ nix_err nix_bindings_builder_insert(nix_c_context * context, BindingsBuilder * b
context->last_err_code = NIX_OK;
try {
auto & v = check_value_not_null(value);
nix::Symbol s = bb->builder.symbols.get().create(name);
nix::Symbol s = bb->builder.state.symbols.create(name);
bb->builder.insert(s, &v);
}
NIXC_CATCH_ERRS
@@ -738,5 +651,3 @@ const StorePath * nix_realised_string_get_store_path(nix_realised_string * s, si
{
return &s->storePaths[i];
}
} // extern "C"
+9 -86
View File
@@ -265,24 +265,9 @@ ExternalValue * nix_get_external(nix_c_context * context, nix_value * value);
*/
nix_value * nix_get_list_byidx(nix_c_context * context, const nix_value * value, EvalState * state, unsigned int ix);
/** @brief Get the ix'th element of a list without forcing evaluation of the element
*
* Returns the list element without forcing its evaluation, allowing access to lazy values.
* The list value itself must already be evaluated.
*
* Owned by the GC. Use nix_gc_decref when you're done with the pointer
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect (must be an evaluated list)
* @param[in] state nix evaluator state
* @param[in] ix list element to get
* @return value, NULL in case of errors
*/
nix_value *
nix_get_list_byidx_lazy(nix_c_context * context, const nix_value * value, EvalState * state, unsigned int ix);
/** @brief Get an attr by name
*
* Use nix_gc_decref when you're done with the pointer
* Owned by the GC. Use nix_gc_decref when you're done with the pointer
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect
* @param[in] state nix evaluator state
@@ -291,21 +276,6 @@ nix_get_list_byidx_lazy(nix_c_context * context, const nix_value * value, EvalSt
*/
nix_value * nix_get_attr_byname(nix_c_context * context, const nix_value * value, EvalState * state, const char * name);
/** @brief Get an attribute value by attribute name, without forcing evaluation of the attribute's value
*
* Returns the attribute value without forcing its evaluation, allowing access to lazy values.
* The attribute set value itself must already be evaluated.
*
* Use nix_gc_decref when you're done with the pointer
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect (must be an evaluated attribute set)
* @param[in] state nix evaluator state
* @param[in] name attribute name
* @return value, NULL in case of errors
*/
nix_value *
nix_get_attr_byname_lazy(nix_c_context * context, const nix_value * value, EvalState * state, const char * name);
/** @brief Check if an attribute name exists on a value
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect
@@ -315,21 +285,11 @@ nix_get_attr_byname_lazy(nix_c_context * context, const nix_value * value, EvalS
*/
bool nix_has_attr_byname(nix_c_context * context, const nix_value * value, EvalState * state, const char * name);
/** @brief Get an attribute by index
/** @brief Get an attribute by index in the sorted bindings
*
* Also gives you the name.
*
* Attributes are returned in an unspecified order which is NOT suitable for
* reproducible operations. In Nix's domain, reproducibility is paramount. The caller
* is responsible for sorting the attributes or storing them in an ordered map to
* ensure deterministic behavior in your application.
*
* @note When Nix does sort attributes, which it does for virtually all intermediate
* operations and outputs, it uses byte-wise lexicographic order (equivalent to
* lexicographic order by Unicode scalar value for valid UTF-8). We recommend
* applying this same ordering for consistency.
*
* Use nix_gc_decref when you're done with the pointer
* Owned by the GC. Use nix_gc_decref when you're done with the pointer
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect
* @param[in] state nix evaluator state
@@ -337,50 +297,12 @@ bool nix_has_attr_byname(nix_c_context * context, const nix_value * value, EvalS
* @param[out] name will store a pointer to the attribute name
* @return value, NULL in case of errors
*/
nix_value *
nix_get_attr_byidx(nix_c_context * context, nix_value * value, EvalState * state, unsigned int i, const char ** name);
nix_value * nix_get_attr_byidx(
nix_c_context * context, const nix_value * value, EvalState * state, unsigned int i, const char ** name);
/** @brief Get an attribute by index, without forcing evaluation of the attribute's value
/** @brief Get an attribute name by index in the sorted bindings
*
* Also gives you the name.
*
* Returns the attribute value without forcing its evaluation, allowing access to lazy values.
* The attribute set value itself must already have been evaluated.
*
* Attributes are returned in an unspecified order which is NOT suitable for
* reproducible operations. In Nix's domain, reproducibility is paramount. The caller
* is responsible for sorting the attributes or storing them in an ordered map to
* ensure deterministic behavior in your application.
*
* @note When Nix does sort attributes, which it does for virtually all intermediate
* operations and outputs, it uses byte-wise lexicographic order (equivalent to
* lexicographic order by Unicode scalar value for valid UTF-8). We recommend
* applying this same ordering for consistency.
*
* Use nix_gc_decref when you're done with the pointer
* @param[out] context Optional, stores error information
* @param[in] value Nix value to inspect (must be an evaluated attribute set)
* @param[in] state nix evaluator state
* @param[in] i attribute index
* @param[out] name will store a pointer to the attribute name
* @return value, NULL in case of errors
*/
nix_value * nix_get_attr_byidx_lazy(
nix_c_context * context, nix_value * value, EvalState * state, unsigned int i, const char ** name);
/** @brief Get an attribute name by index
*
* Returns the attribute name without forcing evaluation of the attribute's value.
*
* Attributes are returned in an unspecified order which is NOT suitable for
* reproducible operations. In Nix's domain, reproducibility is paramount. The caller
* is responsible for sorting the attributes or storing them in an ordered map to
* ensure deterministic behavior in your application.
*
* @note When Nix does sort attributes, which it does for virtually all intermediate
* operations and outputs, it uses byte-wise lexicographic order (equivalent to
* lexicographic order by Unicode scalar value for valid UTF-8). We recommend
* applying this same ordering for consistency.
* Useful when you want the name but want to avoid evaluation.
*
* Owned by the nix EvalState
* @param[out] context Optional, stores error information
@@ -389,7 +311,8 @@ nix_value * nix_get_attr_byidx_lazy(
* @param[in] i attribute index
* @return name, NULL in case of errors
*/
const char * nix_get_attr_name_byidx(nix_c_context * context, nix_value * value, EvalState * state, unsigned int i);
const char *
nix_get_attr_name_byidx(nix_c_context * context, const nix_value * value, EvalState * state, unsigned int i);
/**@}*/
/** @name Initializers
@@ -26,20 +26,11 @@ public:
}
protected:
LibExprTest(ref<Store> store, auto && makeEvalSettings)
LibExprTest()
: LibStoreTest()
, evalSettings(makeEvalSettings(readOnlyMode))
, state({}, store, fetchSettings, evalSettings, nullptr)
{
}
LibExprTest()
: LibExprTest(openStore("dummy://"), [](bool & readOnlyMode) {
EvalSettings settings{readOnlyMode};
settings.nixPath = {};
return settings;
})
{
evalSettings.nixPath = {};
}
Value eval(std::string input, bool forceValue = true)
-2
View File
@@ -31,7 +31,6 @@ rapidcheck = dependency('rapidcheck')
deps_public += rapidcheck
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'tests/value/context.cc',
@@ -45,7 +44,6 @@ subdir('nix-meson-build-support/windows-version')
this_library = library(
'nix-expr-test-support',
sources,
soversion : nix_soversion,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
# TODO: Remove `-lrapidcheck` when https://github.com/emil-e/rapidcheck/pull/326
-38
View File
@@ -3,7 +3,6 @@
#include "nix/expr/eval.hh"
#include "nix/expr/tests/libexpr.hh"
#include "nix/util/memory-source-accessor.hh"
namespace nix {
@@ -175,41 +174,4 @@ TEST_F(EvalStateTest, getBuiltin_fail)
ASSERT_THROW(state.getBuiltin("nonexistent"), EvalError);
}
class PureEvalTest : public LibExprTest
{
public:
PureEvalTest()
: LibExprTest(openStore("dummy://", {{"read-only", "false"}}), [](bool & readOnlyMode) {
EvalSettings settings{readOnlyMode};
settings.pureEval = true;
settings.restrictEval = true;
return settings;
})
{
}
};
TEST_F(PureEvalTest, pathExists)
{
ASSERT_THAT(eval("builtins.pathExists /."), IsFalse());
ASSERT_THAT(eval("builtins.pathExists /nix"), IsFalse());
ASSERT_THAT(eval("builtins.pathExists /nix/store"), IsFalse());
{
std::string contents = "Lorem ipsum";
StringSource s{contents};
auto path = state.store->addToStoreFromDump(
s, "source", FileSerialisationMethod::Flat, ContentAddressMethod::Raw::Text, HashAlgorithm::SHA256);
auto printed = store->printStorePath(path);
ASSERT_THROW(eval(fmt("builtins.readFile %s", printed)), RestrictedPathError);
ASSERT_THAT(eval(fmt("builtins.pathExists %s", printed)), IsFalse());
ASSERT_THROW(eval("builtins.readDir /."), RestrictedPathError);
state.allowPath(path); // FIXME: This shouldn't behave this way.
ASSERT_THAT(eval("builtins.readDir /."), IsAttrsOfSize(0));
}
}
} // namespace nix
+2 -2
View File
@@ -54,7 +54,7 @@ TEST_F(JSONValueTest, IntNegative)
TEST_F(JSONValueTest, String)
{
Value v;
v.mkStringNoCopy("test");
v.mkString("test");
ASSERT_EQ(getJSONValue(v), "\"test\"");
}
@@ -62,7 +62,7 @@ TEST_F(JSONValueTest, StringQuotes)
{
Value v;
v.mkStringNoCopy("test\"");
v.mkString("test\"");
ASSERT_EQ(getJSONValue(v), "\"test\\\"\"");
}
+31 -6
View File
@@ -1,15 +1,40 @@
#include <gtest/gtest.h>
#include "nix/store/tests/test-main.hh"
#include "nix/util/config-global.hh"
#include <cstdlib>
#include "nix/store/globals.hh"
#include "nix/util/logging.hh"
using namespace nix;
int main(int argc, char ** argv)
{
auto res = testMainForBuidingPre(argc, argv);
if (res)
return res;
if (argc > 1 && std::string_view(argv[1]) == "__build-remote") {
printError("test-build-remote: not supported in libexpr unit tests");
return 1;
}
// Disable build hook. We won't be testing remote builds in these unit tests. If we do, fix the above build hook.
settings.buildHook = {};
#ifdef __linux__ // should match the conditional around sandboxBuildDir declaration.
// When building and testing nix within the host's Nix sandbox, our store dir will be located in the host's
// sandboxBuildDir, e.g.: Host
// storeDir = /nix/store
// sandboxBuildDir = /build
// This process
// storeDir = /build/foo/bar/store
// sandboxBuildDir = /build
// However, we have a rule that the store dir must not be inside the storeDir, so we need to pick a different
// sandboxBuildDir.
settings.sandboxBuildDir = "/test-build-dir-instead-of-usual-build-dir";
#endif
#ifdef __APPLE__
// Avoid this error, when already running in a sandbox:
// sandbox-exec: sandbox_apply: Operation not permitted
settings.sandboxMode = smDisabled;
setEnv("_NIX_TEST_NO_SANDBOX", "1");
#endif
// For pipe operator tests in trivial.cc
experimentalFeatureSettings.set("experimental-features", "pipe-operators");
+1 -3
View File
@@ -45,7 +45,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
sources = files(
'derived-path.cc',
@@ -56,7 +55,6 @@ sources = files(
'nix_api_expr.cc',
'nix_api_external.cc',
'nix_api_value.cc',
'nix_api_value_internal.cc',
'primops.cc',
'search-path.cc',
'trivial.cc',
@@ -83,7 +81,7 @@ this_exe = executable(
test(
meson.project_name(),
this_exe,
env : asan_test_options_env + {
env : {
'_NIX_TEST_UNIT_DATA' : meson.current_source_dir() / 'data',
},
protocol : 'gtest',
+16 -88
View File
@@ -1,5 +1,7 @@
#include "nix_api_store.h"
#include "nix_api_store_internal.h"
#include "nix_api_util.h"
#include "nix_api_util_internal.h"
#include "nix_api_expr.h"
#include "nix_api_value.h"
@@ -149,8 +151,8 @@ TEST_F(nix_api_expr_test, nix_expr_realise_context_bad_value)
assert_ctx_ok();
auto r = nix_string_realise(ctx, state, value, false);
ASSERT_EQ(nullptr, r);
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("cannot coerce"));
ASSERT_EQ(ctx->last_err_code, NIX_ERR_NIX_ERROR);
ASSERT_THAT(ctx->last_err, testing::Optional(testing::HasSubstr("cannot coerce")));
}
TEST_F(nix_api_expr_test, nix_expr_realise_context_bad_build)
@@ -166,8 +168,8 @@ TEST_F(nix_api_expr_test, nix_expr_realise_context_bad_build)
assert_ctx_ok();
auto r = nix_string_realise(ctx, state, value, false);
ASSERT_EQ(nullptr, r);
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("failed with exit code 1"));
ASSERT_EQ(ctx->last_err_code, NIX_ERR_NIX_ERROR);
ASSERT_THAT(ctx->last_err, testing::Optional(testing::HasSubstr("failed with exit code 1")));
}
TEST_F(nix_api_expr_test, nix_expr_realise_context)
@@ -379,11 +381,12 @@ TEST_F(nix_api_expr_test, nix_expr_primop_bad_no_return)
nix_value * result = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_value_call(ctx, state, primopValue, three, result);
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_EQ(ctx->last_err_code, NIX_ERR_NIX_ERROR);
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr),
testing::HasSubstr("Implementation error in custom function: return value was not initialized"));
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("badNoReturn"));
ctx->last_err,
testing::Optional(
testing::HasSubstr("Implementation error in custom function: return value was not initialized")));
ASSERT_THAT(ctx->last_err, testing::Optional(testing::HasSubstr("badNoReturn")));
}
static void primop_bad_return_thunk(
@@ -416,60 +419,12 @@ TEST_F(nix_api_expr_test, nix_expr_primop_bad_return_thunk)
assert_ctx_ok();
NIX_VALUE_CALL(ctx, state, result, primopValue, toString, four);
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_EQ(ctx->last_err_code, NIX_ERR_NIX_ERROR);
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr),
testing::HasSubstr("Implementation error in custom function: return value must not be a thunk"));
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("badReturnThunk"));
}
static void primop_with_nix_err_key(
void * user_data, nix_c_context * context, EvalState * state, nix_value ** args, nix_value * ret)
{
nix_set_err_msg(context, NIX_ERR_KEY, "Test error from primop");
}
TEST_F(nix_api_expr_test, nix_expr_primop_nix_err_key_conversion)
{
// Test that NIX_ERR_KEY from a custom primop gets converted to a generic EvalError
//
// RATIONALE: NIX_ERR_KEY must not be propagated from custom primops because it would
// create semantic confusion. NIX_ERR_KEY indicates missing keys/indices in C API functions
// (like nix_get_attr_byname, nix_get_list_byidx). If custom primops could return NIX_ERR_KEY,
// an evaluation error would be indistinguishable from an actual missing attribute.
//
// For example, if nix_get_attr_byname returned NIX_ERR_KEY when the attribute is present
// but the value evaluation fails, callers expecting NIX_ERR_KEY to mean "missing attribute"
// would incorrectly handle evaluation failures as missing attributes. In places where
// missing attributes are tolerated (like optional attributes), this would cause the
// program to continue after swallowing the error, leading to silent failures.
PrimOp * primop = nix_alloc_primop(
ctx, primop_with_nix_err_key, 1, "testErrorPrimop", nullptr, "a test primop that sets NIX_ERR_KEY", nullptr);
assert_ctx_ok();
nix_value * primopValue = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_init_primop(ctx, primopValue, primop);
assert_ctx_ok();
nix_value * arg = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_init_int(ctx, arg, 42);
assert_ctx_ok();
nix_value * result = nix_alloc_value(ctx, state);
assert_ctx_ok();
nix_value_call(ctx, state, primopValue, arg, result);
// Verify that NIX_ERR_KEY gets converted to NIX_ERR_NIX_ERROR (generic evaluation error)
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("Error from custom function"));
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("Test error from primop"));
ASSERT_THAT(nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("testErrorPrimop"));
// Clean up
nix_gc_decref(ctx, primopValue);
nix_gc_decref(ctx, arg);
nix_gc_decref(ctx, result);
ctx->last_err,
testing::Optional(
testing::HasSubstr("Implementation error in custom function: return value must not be a thunk")));
ASSERT_THAT(ctx->last_err, testing::Optional(testing::HasSubstr("badReturnThunk")));
}
TEST_F(nix_api_expr_test, nix_value_call_multi_no_args)
@@ -486,31 +441,4 @@ TEST_F(nix_api_expr_test, nix_value_call_multi_no_args)
assert_ctx_ok();
ASSERT_EQ(3, rInt);
}
TEST_F(nix_api_expr_test, nix_expr_attrset_update)
{
nix_expr_eval_from_string(ctx, state, "{ a = 0; b = 2; } // { a = 1; b = 3; } // { a = 2; }", ".", value);
assert_ctx_ok();
ASSERT_EQ(nix_get_attrs_size(ctx, value), 2);
assert_ctx_ok();
std::array<std::pair<std::string_view, nix_value *>, 2> values;
for (unsigned int i = 0; i < 2; ++i) {
const char * name;
values[i].second = nix_get_attr_byidx(ctx, value, state, i, &name);
assert_ctx_ok();
values[i].first = name;
}
std::sort(values.begin(), values.end(), [](const auto & lhs, const auto & rhs) { return lhs.first < rhs.first; });
nix_value * a = values[0].second;
ASSERT_EQ("a", values[0].first);
ASSERT_EQ(nix_get_int(ctx, a), 2);
assert_ctx_ok();
nix_value * b = values[1].second;
ASSERT_EQ("b", values[1].first);
ASSERT_EQ(nix_get_int(ctx, b), 3);
assert_ctx_ok();
}
} // namespace nixC
+4 -1
View File
@@ -1,6 +1,9 @@
#include "nix_api_store.h"
#include "nix_api_store_internal.h"
#include "nix_api_util.h"
#include "nix_api_util_internal.h"
#include "nix_api_expr.h"
#include "nix_api_expr_internal.h"
#include "nix_api_value.h"
#include "nix_api_external.h"
@@ -36,7 +39,7 @@ private:
std::string type_string = "nix-external<MyExternalValueDesc( ";
type_string += std::to_string(obj->_x);
type_string += " )>";
nix_set_string_return(res, &*type_string.begin());
res->str = &*type_string.begin();
}
};
+14 -334
View File
@@ -1,7 +1,10 @@
#include "nix_api_store.h"
#include "nix_api_store_internal.h"
#include "nix_api_util.h"
#include "nix_api_util_internal.h"
#include "nix_api_expr.h"
#include "nix_api_value.h"
#include "nix_api_expr_internal.h"
#include "nix/expr/tests/nix_api_expr.hh"
#include "nix/util/tests/string_callback.hh"
@@ -13,6 +16,14 @@
namespace nixC {
TEST_F(nix_api_expr_test, as_nix_value_ptr)
{
// nix_alloc_value casts nix::Value to nix_value
// It should be obvious from the decl that that works, but if it doesn't,
// the whole implementation would be utterly broken.
ASSERT_EQ(sizeof(nix::Value), sizeof(nix_value));
}
TEST_F(nix_api_expr_test, nix_value_get_int_invalid)
{
ASSERT_EQ(0, nix_get_int(ctx, nullptr));
@@ -162,114 +173,6 @@ TEST_F(nix_api_expr_test, nix_build_and_init_list)
nix_gc_decref(ctx, intValue);
}
TEST_F(nix_api_expr_test, nix_get_list_byidx_large_indices)
{
// Create a small list to test extremely large out-of-bounds access
ListBuilder * builder = nix_make_list_builder(ctx, state, 2);
nix_value * intValue = nix_alloc_value(ctx, state);
nix_init_int(ctx, intValue, 42);
nix_list_builder_insert(ctx, builder, 0, intValue);
nix_list_builder_insert(ctx, builder, 1, intValue);
nix_make_list(ctx, builder, value);
nix_list_builder_free(builder);
// Test extremely large indices that would definitely crash without bounds checking
ASSERT_EQ(nullptr, nix_get_list_byidx(ctx, value, state, 1000000));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_list_byidx(ctx, value, state, UINT_MAX / 2));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_list_byidx(ctx, value, state, UINT_MAX / 2 + 1000000));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
// Clean up
nix_gc_decref(ctx, intValue);
}
TEST_F(nix_api_expr_test, nix_get_list_byidx_lazy)
{
// Create a list with a throwing lazy element, an already-evaluated int, and a lazy function call
// 1. Throwing lazy element - create a function application thunk that will throw when forced
nix_value * throwingFn = nix_alloc_value(ctx, state);
nix_value * throwingValue = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(
ctx,
state,
R"(
_: throw "This should not be evaluated by the lazy accessor"
)",
"<test>",
throwingFn);
assert_ctx_ok();
nix_init_apply(ctx, throwingValue, throwingFn, throwingFn);
assert_ctx_ok();
// 2. Already evaluated int (not lazy)
nix_value * intValue = nix_alloc_value(ctx, state);
nix_init_int(ctx, intValue, 42);
assert_ctx_ok();
// 3. Lazy function application that would compute increment 5 = 6
nix_value * lazyApply = nix_alloc_value(ctx, state);
nix_value * incrementFn = nix_alloc_value(ctx, state);
nix_value * argFive = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(ctx, state, "x: x + 1", "<test>", incrementFn);
assert_ctx_ok();
nix_init_int(ctx, argFive, 5);
// Create a lazy application: (x: x + 1) 5
nix_init_apply(ctx, lazyApply, incrementFn, argFive);
assert_ctx_ok();
ListBuilder * builder = nix_make_list_builder(ctx, state, 3);
nix_list_builder_insert(ctx, builder, 0, throwingValue);
nix_list_builder_insert(ctx, builder, 1, intValue);
nix_list_builder_insert(ctx, builder, 2, lazyApply);
nix_make_list(ctx, builder, value);
nix_list_builder_free(builder);
// Test 1: Lazy accessor should return the throwing element without forcing evaluation
nix_value * lazyThrowingElement = nix_get_list_byidx_lazy(ctx, value, state, 0);
assert_ctx_ok();
ASSERT_NE(nullptr, lazyThrowingElement);
// Verify the element is still lazy by checking that forcing it throws
nix_value_force(ctx, state, lazyThrowingElement);
assert_ctx_err();
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("This should not be evaluated by the lazy accessor"));
// Test 2: Lazy accessor should return the already-evaluated int
nix_value * intElement = nix_get_list_byidx_lazy(ctx, value, state, 1);
assert_ctx_ok();
ASSERT_NE(nullptr, intElement);
ASSERT_EQ(42, nix_get_int(ctx, intElement));
// Test 3: Lazy accessor should return the lazy function application without forcing
nix_value * lazyFunctionElement = nix_get_list_byidx_lazy(ctx, value, state, 2);
assert_ctx_ok();
ASSERT_NE(nullptr, lazyFunctionElement);
// Force the lazy function application - should compute 5 + 1 = 6
nix_value_force(ctx, state, lazyFunctionElement);
assert_ctx_ok();
ASSERT_EQ(6, nix_get_int(ctx, lazyFunctionElement));
// Clean up
nix_gc_decref(ctx, throwingFn);
nix_gc_decref(ctx, throwingValue);
nix_gc_decref(ctx, intValue);
nix_gc_decref(ctx, lazyApply);
nix_gc_decref(ctx, incrementFn);
nix_gc_decref(ctx, argFive);
nix_gc_decref(ctx, lazyThrowingElement);
nix_gc_decref(ctx, intElement);
nix_gc_decref(ctx, lazyFunctionElement);
}
TEST_F(nix_api_expr_test, nix_build_and_init_attr_invalid)
{
ASSERT_EQ(nullptr, nix_get_attr_byname(ctx, nullptr, state, 0));
@@ -352,225 +255,6 @@ TEST_F(nix_api_expr_test, nix_build_and_init_attr)
free(out_name);
}
TEST_F(nix_api_expr_test, nix_get_attr_byidx_large_indices)
{
// Create a small attribute set to test extremely large out-of-bounds access
const char ** out_name = (const char **) malloc(sizeof(char *));
BindingsBuilder * builder = nix_make_bindings_builder(ctx, state, 2);
nix_value * intValue = nix_alloc_value(ctx, state);
nix_init_int(ctx, intValue, 42);
nix_bindings_builder_insert(ctx, builder, "test", intValue);
nix_make_attrs(ctx, value, builder);
nix_bindings_builder_free(builder);
// Test extremely large indices that would definitely crash without bounds checking
ASSERT_EQ(nullptr, nix_get_attr_byidx(ctx, value, state, 1000000, out_name));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_attr_byidx(ctx, value, state, UINT_MAX / 2, out_name));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_attr_byidx(ctx, value, state, UINT_MAX / 2 + 1000000, out_name));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
// Test nix_get_attr_name_byidx with large indices too
ASSERT_EQ(nullptr, nix_get_attr_name_byidx(ctx, value, state, 1000000));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_attr_name_byidx(ctx, value, state, UINT_MAX / 2));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
ASSERT_EQ(nullptr, nix_get_attr_name_byidx(ctx, value, state, UINT_MAX / 2 + 1000000));
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
// Clean up
nix_gc_decref(ctx, intValue);
free(out_name);
}
TEST_F(nix_api_expr_test, nix_get_attr_byname_lazy)
{
// Create an attribute set with a throwing lazy attribute, an already-evaluated int, and a lazy function call
// 1. Throwing lazy element - create a function application thunk that will throw when forced
nix_value * throwingFn = nix_alloc_value(ctx, state);
nix_value * throwingValue = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(
ctx,
state,
R"(
_: throw "This should not be evaluated by the lazy accessor"
)",
"<test>",
throwingFn);
assert_ctx_ok();
nix_init_apply(ctx, throwingValue, throwingFn, throwingFn);
assert_ctx_ok();
// 2. Already evaluated int (not lazy)
nix_value * intValue = nix_alloc_value(ctx, state);
nix_init_int(ctx, intValue, 42);
assert_ctx_ok();
// 3. Lazy function application that would compute increment 7 = 8
nix_value * lazyApply = nix_alloc_value(ctx, state);
nix_value * incrementFn = nix_alloc_value(ctx, state);
nix_value * argSeven = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(ctx, state, "x: x + 1", "<test>", incrementFn);
assert_ctx_ok();
nix_init_int(ctx, argSeven, 7);
// Create a lazy application: (x: x + 1) 7
nix_init_apply(ctx, lazyApply, incrementFn, argSeven);
assert_ctx_ok();
BindingsBuilder * builder = nix_make_bindings_builder(ctx, state, 3);
nix_bindings_builder_insert(ctx, builder, "throwing", throwingValue);
nix_bindings_builder_insert(ctx, builder, "normal", intValue);
nix_bindings_builder_insert(ctx, builder, "lazy", lazyApply);
nix_make_attrs(ctx, value, builder);
nix_bindings_builder_free(builder);
// Test 1: Lazy accessor should return the throwing attribute without forcing evaluation
nix_value * lazyThrowingAttr = nix_get_attr_byname_lazy(ctx, value, state, "throwing");
assert_ctx_ok();
ASSERT_NE(nullptr, lazyThrowingAttr);
// Verify the attribute is still lazy by checking that forcing it throws
nix_value_force(ctx, state, lazyThrowingAttr);
assert_ctx_err();
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("This should not be evaluated by the lazy accessor"));
// Test 2: Lazy accessor should return the already-evaluated int
nix_value * intAttr = nix_get_attr_byname_lazy(ctx, value, state, "normal");
assert_ctx_ok();
ASSERT_NE(nullptr, intAttr);
ASSERT_EQ(42, nix_get_int(ctx, intAttr));
// Test 3: Lazy accessor should return the lazy function application without forcing
nix_value * lazyFunctionAttr = nix_get_attr_byname_lazy(ctx, value, state, "lazy");
assert_ctx_ok();
ASSERT_NE(nullptr, lazyFunctionAttr);
// Force the lazy function application - should compute 7 + 1 = 8
nix_value_force(ctx, state, lazyFunctionAttr);
assert_ctx_ok();
ASSERT_EQ(8, nix_get_int(ctx, lazyFunctionAttr));
// Test 4: Missing attribute should return NULL with NIX_ERR_KEY
nix_value * missingAttr = nix_get_attr_byname_lazy(ctx, value, state, "nonexistent");
ASSERT_EQ(nullptr, missingAttr);
ASSERT_EQ(NIX_ERR_KEY, nix_err_code(ctx));
// Clean up
nix_gc_decref(ctx, throwingFn);
nix_gc_decref(ctx, throwingValue);
nix_gc_decref(ctx, intValue);
nix_gc_decref(ctx, lazyApply);
nix_gc_decref(ctx, incrementFn);
nix_gc_decref(ctx, argSeven);
nix_gc_decref(ctx, lazyThrowingAttr);
nix_gc_decref(ctx, intAttr);
nix_gc_decref(ctx, lazyFunctionAttr);
}
TEST_F(nix_api_expr_test, nix_get_attr_byidx_lazy)
{
// Create an attribute set with a throwing lazy attribute, an already-evaluated int, and a lazy function call
// 1. Throwing lazy element - create a function application thunk that will throw when forced
nix_value * throwingFn = nix_alloc_value(ctx, state);
nix_value * throwingValue = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(
ctx,
state,
R"(
_: throw "This should not be evaluated by the lazy accessor"
)",
"<test>",
throwingFn);
assert_ctx_ok();
nix_init_apply(ctx, throwingValue, throwingFn, throwingFn);
assert_ctx_ok();
// 2. Already evaluated int (not lazy)
nix_value * intValue = nix_alloc_value(ctx, state);
nix_init_int(ctx, intValue, 99);
assert_ctx_ok();
// 3. Lazy function application that would compute increment 10 = 11
nix_value * lazyApply = nix_alloc_value(ctx, state);
nix_value * incrementFn = nix_alloc_value(ctx, state);
nix_value * argTen = nix_alloc_value(ctx, state);
nix_expr_eval_from_string(ctx, state, "x: x + 1", "<test>", incrementFn);
assert_ctx_ok();
nix_init_int(ctx, argTen, 10);
// Create a lazy application: (x: x + 1) 10
nix_init_apply(ctx, lazyApply, incrementFn, argTen);
assert_ctx_ok();
BindingsBuilder * builder = nix_make_bindings_builder(ctx, state, 3);
nix_bindings_builder_insert(ctx, builder, "a_throwing", throwingValue);
nix_bindings_builder_insert(ctx, builder, "b_normal", intValue);
nix_bindings_builder_insert(ctx, builder, "c_lazy", lazyApply);
nix_make_attrs(ctx, value, builder);
nix_bindings_builder_free(builder);
// Proper usage: first get the size and gather all attributes into a map
unsigned int attrCount = nix_get_attrs_size(ctx, value);
assert_ctx_ok();
ASSERT_EQ(3u, attrCount);
// Gather all attributes into a map (proper contract usage)
std::map<std::string, nix_value *> attrMap;
const char * name;
for (unsigned int i = 0; i < attrCount; i++) {
nix_value * attr = nix_get_attr_byidx_lazy(ctx, value, state, i, &name);
assert_ctx_ok();
ASSERT_NE(nullptr, attr);
attrMap[std::string(name)] = attr;
}
// Now test the gathered attributes
ASSERT_EQ(3u, attrMap.size());
ASSERT_TRUE(attrMap.count("a_throwing"));
ASSERT_TRUE(attrMap.count("b_normal"));
ASSERT_TRUE(attrMap.count("c_lazy"));
// Test 1: Throwing attribute should be lazy
nix_value * throwingAttr = attrMap["a_throwing"];
nix_value_force(ctx, state, throwingAttr);
assert_ctx_err();
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr), testing::HasSubstr("This should not be evaluated by the lazy accessor"));
// Test 2: Normal attribute should be already evaluated
nix_value * normalAttr = attrMap["b_normal"];
ASSERT_EQ(99, nix_get_int(ctx, normalAttr));
// Test 3: Lazy function should compute when forced
nix_value * lazyAttr = attrMap["c_lazy"];
nix_value_force(ctx, state, lazyAttr);
assert_ctx_ok();
ASSERT_EQ(11, nix_get_int(ctx, lazyAttr));
// Clean up
nix_gc_decref(ctx, throwingFn);
nix_gc_decref(ctx, throwingValue);
nix_gc_decref(ctx, intValue);
nix_gc_decref(ctx, lazyApply);
nix_gc_decref(ctx, incrementFn);
nix_gc_decref(ctx, argTen);
for (auto & pair : attrMap) {
nix_gc_decref(ctx, pair.second);
}
}
TEST_F(nix_api_expr_test, nix_value_init)
{
// Setup
@@ -636,10 +320,8 @@ TEST_F(nix_api_expr_test, nix_value_init_apply_error)
// Evaluate it
nix_value_force(ctx, state, v);
ASSERT_EQ(nix_err_code(ctx), NIX_ERR_NIX_ERROR);
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr),
testing::HasSubstr("attempt to call something which is not a function but"));
ASSERT_EQ(ctx->last_err_code, NIX_ERR_NIX_ERROR);
ASSERT_THAT(ctx->last_err.value(), testing::HasSubstr("attempt to call something which is not a function but"));
// Clean up
nix_gc_decref(ctx, some_string);
@@ -698,9 +380,7 @@ TEST_F(nix_api_expr_test, nix_value_init_apply_lazy_arg)
// nix_get_attr_byname isn't lazy (it could have been) so it will throw the exception
nix_value * foo = nix_get_attr_byname(ctx, r, state, "foo");
ASSERT_EQ(nullptr, foo);
ASSERT_THAT(
nix_err_msg(nullptr, ctx, nullptr),
testing::HasSubstr("error message for test case nix_value_init_apply_lazy_arg"));
ASSERT_THAT(ctx->last_err.value(), testing::HasSubstr("error message for test case nix_value_init_apply_lazy_arg"));
// Clean up
nix_gc_decref(ctx, f);
@@ -1,25 +0,0 @@
#include "nix_api_store.h"
#include "nix_api_util.h"
#include "nix_api_expr.h"
#include "nix_api_value.h"
#include "nix_api_expr_internal.h"
#include "nix/expr/tests/nix_api_expr.hh"
#include "nix/util/tests/string_callback.hh"
#include <gmock/gmock.h>
#include <cstddef>
#include <cstdlib>
#include <gtest/gtest.h>
namespace nixC {
TEST_F(nix_api_expr_test, as_nix_value_ptr)
{
// nix_alloc_value casts nix::Value to nix_value
// It should be obvious from the decl that that works, but if it doesn't,
// the whole implementation would be utterly broken.
ASSERT_EQ(sizeof(nix::Value), sizeof(nix_value));
}
} // namespace nixC
-1
View File
@@ -62,7 +62,6 @@ mkMesonExecutable (finalAttrs: {
mkdir -p "$HOME"
''
+ ''
export ASAN_OPTIONS=abort_on_error=1:print_summary=1:detect_leaks=0
export _NIX_TEST_UNIT_DATA=${resolvePath ./data}
${stdenv.hostPlatform.emulator buildPackages} ${lib.getExe finalAttrs.finalPackage}
touch $out
+15 -15
View File
@@ -195,18 +195,18 @@ TEST_F(PrimOpTest, unsafeGetAttrPos)
auto v = eval(expr);
ASSERT_THAT(v, IsAttrsOfSize(3));
auto file = v.attrs()->get(createSymbol("file"));
auto file = v.attrs()->find(createSymbol("file"));
ASSERT_NE(file, nullptr);
ASSERT_THAT(*file->value, IsString());
auto s = baseNameOf(file->value->string_view());
ASSERT_EQ(s, "foo.nix");
auto line = v.attrs()->get(createSymbol("line"));
auto line = v.attrs()->find(createSymbol("line"));
ASSERT_NE(line, nullptr);
state.forceValue(*line->value, noPos);
ASSERT_THAT(*line->value, IsIntEq(4));
auto column = v.attrs()->get(createSymbol("column"));
auto column = v.attrs()->find(createSymbol("column"));
ASSERT_NE(column, nullptr);
state.forceValue(*column->value, noPos);
ASSERT_THAT(*column->value, IsIntEq(3));
@@ -246,7 +246,7 @@ TEST_F(PrimOpTest, removeAttrsRetains)
{
auto v = eval("builtins.removeAttrs { x = 1; y = 2; } [\"x\"]");
ASSERT_THAT(v, IsAttrsOfSize(1));
ASSERT_NE(v.attrs()->get(createSymbol("y")), nullptr);
ASSERT_NE(v.attrs()->find(createSymbol("y")), nullptr);
}
TEST_F(PrimOpTest, listToAttrsEmptyList)
@@ -266,7 +266,7 @@ TEST_F(PrimOpTest, listToAttrs)
{
auto v = eval("builtins.listToAttrs [ { name = \"key\"; value = 123; } ]");
ASSERT_THAT(v, IsAttrsOfSize(1));
auto key = v.attrs()->get(createSymbol("key"));
auto key = v.attrs()->find(createSymbol("key"));
ASSERT_NE(key, nullptr);
ASSERT_THAT(*key->value, IsIntEq(123));
}
@@ -275,7 +275,7 @@ TEST_F(PrimOpTest, intersectAttrs)
{
auto v = eval("builtins.intersectAttrs { a = 1; b = 2; } { b = 3; c = 4; }");
ASSERT_THAT(v, IsAttrsOfSize(1));
auto b = v.attrs()->get(createSymbol("b"));
auto b = v.attrs()->find(createSymbol("b"));
ASSERT_NE(b, nullptr);
ASSERT_THAT(*b->value, IsIntEq(3));
}
@@ -293,11 +293,11 @@ TEST_F(PrimOpTest, functionArgs)
auto v = eval("builtins.functionArgs ({ x, y ? 123}: 1)");
ASSERT_THAT(v, IsAttrsOfSize(2));
auto x = v.attrs()->get(createSymbol("x"));
auto x = v.attrs()->find(createSymbol("x"));
ASSERT_NE(x, nullptr);
ASSERT_THAT(*x->value, IsFalse());
auto y = v.attrs()->get(createSymbol("y"));
auto y = v.attrs()->find(createSymbol("y"));
ASSERT_NE(y, nullptr);
ASSERT_THAT(*y->value, IsTrue());
}
@@ -307,13 +307,13 @@ TEST_F(PrimOpTest, mapAttrs)
auto v = eval("builtins.mapAttrs (name: value: value * 10) { a = 1; b = 2; }");
ASSERT_THAT(v, IsAttrsOfSize(2));
auto a = v.attrs()->get(createSymbol("a"));
auto a = v.attrs()->find(createSymbol("a"));
ASSERT_NE(a, nullptr);
ASSERT_THAT(*a->value, IsThunk());
state.forceValue(*a->value, noPos);
ASSERT_THAT(*a->value, IsIntEq(10));
auto b = v.attrs()->get(createSymbol("b"));
auto b = v.attrs()->find(createSymbol("b"));
ASSERT_NE(b, nullptr);
ASSERT_THAT(*b->value, IsThunk());
state.forceValue(*b->value, noPos);
@@ -642,7 +642,7 @@ class ToStringPrimOpTest : public PrimOpTest,
TEST_P(ToStringPrimOpTest, toString)
{
const auto & [input, output] = GetParam();
const auto [input, output] = GetParam();
auto v = eval(input);
ASSERT_THAT(v, IsStringEq(output));
}
@@ -798,7 +798,7 @@ class CompareVersionsPrimOpTest : public PrimOpTest,
TEST_P(CompareVersionsPrimOpTest, compareVersions)
{
const auto & [expression, expectation] = GetParam();
auto [expression, expectation] = GetParam();
auto v = eval(expression);
ASSERT_THAT(v, IsIntEq(expectation));
}
@@ -834,16 +834,16 @@ class ParseDrvNamePrimOpTest
TEST_P(ParseDrvNamePrimOpTest, parseDrvName)
{
const auto & [input, expectedName, expectedVersion] = GetParam();
auto [input, expectedName, expectedVersion] = GetParam();
const auto expr = fmt("builtins.parseDrvName \"%1%\"", input);
auto v = eval(expr);
ASSERT_THAT(v, IsAttrsOfSize(2));
auto name = v.attrs()->get(createSymbol("name"));
auto name = v.attrs()->find(createSymbol("name"));
ASSERT_TRUE(name);
ASSERT_THAT(*name->value, IsStringEq(expectedName));
auto version = v.attrs()->get(createSymbol("version"));
auto version = v.attrs()->find(createSymbol("version"));
ASSERT_TRUE(version);
ASSERT_THAT(*version->value, IsStringEq(expectedVersion));
}
+6 -6
View File
@@ -75,11 +75,11 @@ TEST_F(TrivialExpressionTest, updateAttrs)
{
auto v = eval("{ a = 1; } // { b = 2; a = 3; }");
ASSERT_THAT(v, IsAttrsOfSize(2));
auto a = v.attrs()->get(createSymbol("a"));
auto a = v.attrs()->find(createSymbol("a"));
ASSERT_NE(a, nullptr);
ASSERT_THAT(*a->value, IsIntEq(3));
auto b = v.attrs()->get(createSymbol("b"));
auto b = v.attrs()->find(createSymbol("b"));
ASSERT_NE(b, nullptr);
ASSERT_THAT(*b->value, IsIntEq(2));
}
@@ -176,7 +176,7 @@ TEST_P(AttrSetMergeTrvialExpressionTest, attrsetMergeLazy)
auto v = eval(expr);
ASSERT_THAT(v, IsAttrsOfSize(1));
auto a = v.attrs()->get(createSymbol("a"));
auto a = v.attrs()->find(createSymbol("a"));
ASSERT_NE(a, nullptr);
ASSERT_THAT(*a->value, IsThunk());
@@ -184,11 +184,11 @@ TEST_P(AttrSetMergeTrvialExpressionTest, attrsetMergeLazy)
ASSERT_THAT(*a->value, IsAttrsOfSize(2));
auto b = a->value->attrs()->get(createSymbol("b"));
auto b = a->value->attrs()->find(createSymbol("b"));
ASSERT_NE(b, nullptr);
ASSERT_THAT(*b->value, IsIntEq(1));
auto c = a->value->attrs()->get(createSymbol("c"));
auto c = a->value->attrs()->find(createSymbol("c"));
ASSERT_NE(c, nullptr);
ASSERT_THAT(*c->value, IsIntEq(2));
}
@@ -330,7 +330,7 @@ TEST_F(TrivialExpressionTest, bindOr)
{
auto v = eval("{ or = 1; }");
ASSERT_THAT(v, IsAttrsOfSize(1));
auto b = v.attrs()->get(createSymbol("or"));
auto b = v.attrs()->find(createSymbol("or"));
ASSERT_NE(b, nullptr);
ASSERT_THAT(*b->value, IsIntEq(1));
}
+30 -28
View File
@@ -35,14 +35,14 @@ TEST_F(ValuePrintingTests, tBool)
TEST_F(ValuePrintingTests, tString)
{
Value vString;
vString.mkStringNoCopy("some-string");
vString.mkString("some-string");
test(vString, "\"some-string\"");
}
TEST_F(ValuePrintingTests, tPath)
{
Value vPath;
vPath.mkStringNoCopy("/foo");
vPath.mkString("/foo");
test(vPath, "\"/foo\"");
}
@@ -61,7 +61,7 @@ TEST_F(ValuePrintingTests, tAttrs)
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
@@ -196,11 +196,11 @@ TEST_F(ValuePrintingTests, depthAttrs)
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builderEmpty = state.buildBindings(0);
BindingsBuilder builderEmpty(state, state.allocBindings(0));
Value vAttrsEmpty;
vAttrsEmpty.mkAttrs(builderEmpty.finish());
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
builder.insert(state.symbols.create("nested"), &vAttrsEmpty);
@@ -208,7 +208,7 @@ TEST_F(ValuePrintingTests, depthAttrs)
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
BindingsBuilder builder2 = state.buildBindings(10);
BindingsBuilder builder2(state, state.allocBindings(10));
builder2.insert(state.symbols.create("one"), &vOne);
builder2.insert(state.symbols.create("two"), &vTwo);
builder2.insert(state.symbols.create("nested"), &vAttrs);
@@ -233,14 +233,14 @@ TEST_F(ValuePrintingTests, depthList)
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
BindingsBuilder builder2 = state.buildBindings(10);
BindingsBuilder builder2(state, state.allocBindings(10));
builder2.insert(state.symbols.create("one"), &vOne);
builder2.insert(state.symbols.create("two"), &vTwo);
builder2.insert(state.symbols.create("nested"), &vAttrs);
@@ -290,12 +290,12 @@ TEST_F(StringPrintingTests, maxLengthTruncation)
TEST_F(ValuePrintingTests, attrsTypeFirst)
{
Value vType;
vType.mkStringNoCopy("puppy");
vType.mkString("puppy");
Value vApple;
vApple.mkStringNoCopy("apple");
vApple.mkString("apple");
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("type"), &vType);
builder.insert(state.symbols.create("apple"), &vApple);
@@ -334,7 +334,7 @@ TEST_F(ValuePrintingTests, ansiColorsBool)
TEST_F(ValuePrintingTests, ansiColorsString)
{
Value v;
v.mkStringNoCopy("puppy");
v.mkString("puppy");
test(v, ANSI_MAGENTA "\"puppy\"" ANSI_NORMAL, PrintOptions{.ansiColors = true});
}
@@ -342,7 +342,7 @@ TEST_F(ValuePrintingTests, ansiColorsString)
TEST_F(ValuePrintingTests, ansiColorsStringElided)
{
Value v;
v.mkStringNoCopy("puppy");
v.mkString("puppy");
test(
v,
@@ -374,7 +374,7 @@ TEST_F(ValuePrintingTests, ansiColorsAttrs)
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
@@ -390,10 +390,10 @@ TEST_F(ValuePrintingTests, ansiColorsAttrs)
TEST_F(ValuePrintingTests, ansiColorsDerivation)
{
Value vDerivation;
vDerivation.mkStringNoCopy("derivation");
vDerivation.mkString("derivation");
BindingsBuilder builder = state.buildBindings(10);
builder.insert(state.s.type, &vDerivation);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.sType, &vDerivation);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
@@ -413,7 +413,7 @@ TEST_F(ValuePrintingTests, ansiColorsError)
{
Value throw_ = state.getBuiltin("throw");
Value message;
message.mkStringNoCopy("uh oh!");
message.mkString("uh oh!");
Value vError;
vError.mkApp(&throw_, &message);
@@ -430,16 +430,16 @@ TEST_F(ValuePrintingTests, ansiColorsDerivationError)
{
Value throw_ = state.getBuiltin("throw");
Value message;
message.mkStringNoCopy("uh oh!");
message.mkString("uh oh!");
Value vError;
vError.mkApp(&throw_, &message);
Value vDerivation;
vDerivation.mkStringNoCopy("derivation");
vDerivation.mkString("derivation");
BindingsBuilder builder = state.buildBindings(10);
builder.insert(state.s.type, &vDerivation);
builder.insert(state.s.drvPath, &vError);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.sType, &vDerivation);
builder.insert(state.sDrvPath, &vError);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
@@ -553,12 +553,12 @@ TEST_F(ValuePrintingTests, ansiColorsBlackhole)
TEST_F(ValuePrintingTests, ansiColorsAttrsRepeated)
{
BindingsBuilder emptyBuilder = state.buildBindings(1);
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("a"), &vEmpty);
builder.insert(state.symbols.create("b"), &vEmpty);
@@ -570,7 +570,7 @@ TEST_F(ValuePrintingTests, ansiColorsAttrsRepeated)
TEST_F(ValuePrintingTests, ansiColorsListRepeated)
{
BindingsBuilder emptyBuilder = state.buildBindings(1);
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
@@ -586,7 +586,7 @@ TEST_F(ValuePrintingTests, ansiColorsListRepeated)
TEST_F(ValuePrintingTests, listRepeated)
{
BindingsBuilder emptyBuilder = state.buildBindings(1);
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vEmpty;
vEmpty.mkAttrs(emptyBuilder.finish());
@@ -609,7 +609,7 @@ TEST_F(ValuePrintingTests, ansiColorsAttrsElided)
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder = state.buildBindings(10);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
@@ -635,6 +635,8 @@ TEST_F(ValuePrintingTests, ansiColorsAttrsElided)
TEST_F(ValuePrintingTests, ansiColorsListElided)
{
BindingsBuilder emptyBuilder(state, state.allocBindings(1));
Value vOne;
vOne.mkInt(1);
+2 -2
View File
@@ -110,8 +110,8 @@ std::pair<SourcePath, uint32_t> findPackageFilename(EvalState & state, Value & v
{
Value * v2;
try {
auto & dummyArgs = Bindings::emptyBindings;
v2 = findAlongAttrPath(state, "meta.position", dummyArgs, v).first;
auto dummyArgs = state.allocBindings(0);
v2 = findAlongAttrPath(state, "meta.position", *dummyArgs, v).first;
} catch (Error &) {
throw NoPositionInfo("package '%s' has no source location information", what);
}
+10 -11
View File
@@ -5,37 +5,36 @@
namespace nix {
Bindings Bindings::emptyBindings;
/* Allocate a new array of attributes for an attribute set with a specific
capacity. The space is implicitly reserved after the Bindings
structure. */
Bindings * EvalMemory::allocBindings(size_t capacity)
Bindings * EvalState::allocBindings(size_t capacity)
{
if (capacity == 0)
return &Bindings::emptyBindings;
if (capacity > std::numeric_limits<Bindings::size_type>::max())
return &emptyBindings;
if (capacity > std::numeric_limits<Bindings::size_t>::max())
throw Error("attribute set of size %d is too big", capacity);
stats.nrAttrsets++;
stats.nrAttrsInAttrsets += capacity;
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings();
nrAttrsets++;
nrAttrsInAttrsets += capacity;
return new (allocBytes(sizeof(Bindings) + sizeof(Attr) * capacity)) Bindings((Bindings::size_t) capacity);
}
Value & BindingsBuilder::alloc(Symbol name, PosIdx pos)
{
auto value = mem.get().allocValue();
auto value = state.allocValue();
bindings->push_back(Attr(name, value, pos));
return *value;
}
Value & BindingsBuilder::alloc(std::string_view name, PosIdx pos)
{
return alloc(symbols.get().create(name), pos);
return alloc(state.symbols.create(name), pos);
}
void Bindings::sort()
{
std::sort(attrs, attrs + numAttrs);
if (size_)
std::sort(begin(), end());
}
Value & Value::mkAttrs(BindingsBuilder & bindings)
+2 -2
View File
@@ -330,7 +330,7 @@ AttrCursor::AttrCursor(
AttrKey AttrCursor::getKey()
{
if (!parent)
return {0, root->state.s.epsilon};
return {0, root->state.sEpsilon};
if (!parent->first->cachedValue) {
parent->first->cachedValue = root->db->getAttr(parent->first->getKey());
assert(parent->first->cachedValue);
@@ -702,7 +702,7 @@ bool AttrCursor::isDerivation()
StorePath AttrCursor::forceDerivation()
{
auto aDrvPath = getAttr(root->state.s.drvPath);
auto aDrvPath = getAttr(root->state.sDrvPath);
auto drvPath = root->state.store->parseStorePath(aDrvPath->getString());
drvPath.requireDerivation();
if (!root->state.store->isValidPath(drvPath) && !settings.readOnlyMode) {
-13
View File
@@ -16,7 +16,6 @@
# endif
# include <gc/gc_allocator.h>
# include <gc/gc_tiny_fl.h> // For GC_GRANULE_BYTES
# include <boost/coroutine2/coroutine.hpp>
# include <boost/coroutine2/protected_fixedsize_stack.hpp>
@@ -27,18 +26,6 @@
namespace nix {
#if NIX_USE_BOEHMGC
/*
* Ensure that Boehm satisfies our alignment requirements. This is the default configuration [^]
* and this assertion should never break for any platform. Let's assert it just in case.
*
* This alignment is particularly useful to be able to use aligned
* load/store instructions for loading/writing Values.
*
* [^]: https://github.com/bdwgc/bdwgc/blob/54ac18ccbc5a833dd7edaff94a10ab9b65044d61/include/gc/gc_tiny_fl.h#L31-L33
*/
static_assert(sizeof(void *) * 2 == GC_GRANULE_BYTES, "Boehm GC must use GC_GRANULE_WORDS = 2");
/* Called when the Boehm GC runs out of memory. */
static void * oomHandler(size_t requested)
{
+2 -2
View File
@@ -185,7 +185,7 @@ FrameInfo SampleStack::getPrimOpFrameInfo(const PrimOp & primOp, std::span<Value
/* Error context strings don't actually matter, since we ignore all eval errors. */
state.forceAttrs(*args[0], pos, "");
auto attrs = args[0]->attrs();
auto nameAttr = state.getAttr(state.s.name, attrs, "");
auto nameAttr = state.getAttr(state.sName, attrs, "");
auto drvName = std::string(state.forceStringNoCtx(*nameAttr->value, pos, ""));
return DerivationStrictFrameInfo{.callPos = pos, .drvName = std::move(drvName)};
} catch (...) {
@@ -211,7 +211,7 @@ FrameInfo SampleStack::getFrameInfoFromValueAndPos(const Value & v, std::span<Va
/* Resolve primOp eagerly. Must not hold on to a reference to a Value. */
return PrimOpFrameInfo{.expr = v.primOpAppPrimOp(), .callPos = pos};
else if (state.isFunctor(v)) {
const auto functor = v.attrs()->get(state.s.functor);
const auto functor = v.attrs()->get(state.sFunctor);
if (auto pos_ = posCache.lookup(pos); std::holds_alternative<std::monostate>(pos_.origin))
/* HACK: In case callsite position is unresolved. */
return FunctorFrameInfo{.pos = functor->pos};
+275 -299
View File
@@ -17,13 +17,11 @@
#include "nix/expr/print.hh"
#include "nix/fetchers/filtering-source-accessor.hh"
#include "nix/util/memory-source-accessor.hh"
#include "nix/util/mounted-source-accessor.hh"
#include "nix/expr/gc-small-vector.hh"
#include "nix/util/url.hh"
#include "nix/fetchers/fetch-to-store.hh"
#include "nix/fetchers/tarball.hh"
#include "nix/fetchers/input-cache.hh"
#include "nix/util/current-process.hh"
#include "parser-tab.hh"
@@ -39,7 +37,10 @@
#include <nlohmann/json.hpp>
#include <boost/container/small_vector.hpp>
#include <boost/unordered/concurrent_flat_map.hpp>
#ifndef _WIN32 // TODO use portable implementation
# include <sys/resource.h>
#endif
#include "nix/util/strings-inline.hh"
@@ -194,15 +195,6 @@ static Symbol getName(const AttrName & name, EvalState & state, Env & env)
static constexpr size_t BASE_ENV_SIZE = 128;
EvalMemory::EvalMemory()
#if NIX_USE_BOEHMGC
: valueAllocCache(std::allocate_shared<void *>(traceable_allocator<void *>(), nullptr))
, env1AllocCache(std::allocate_shared<void *>(traceable_allocator<void *>(), nullptr))
#endif
{
assertGCInitialized();
}
EvalState::EvalState(
const LookupPath & lookupPathFromArguments,
ref<Store> store,
@@ -211,75 +203,138 @@ EvalState::EvalState(
std::shared_ptr<Store> buildStore)
: fetchSettings{fetchSettings}
, settings{settings}
, symbols(StaticEvalSymbols::staticSymbolTable())
, sWith(symbols.create("<with>"))
, sOutPath(symbols.create("outPath"))
, sDrvPath(symbols.create("drvPath"))
, sType(symbols.create("type"))
, sMeta(symbols.create("meta"))
, sName(symbols.create("name"))
, sValue(symbols.create("value"))
, sSystem(symbols.create("system"))
, sOverrides(symbols.create("__overrides"))
, sOutputs(symbols.create("outputs"))
, sOutputName(symbols.create("outputName"))
, sIgnoreNulls(symbols.create("__ignoreNulls"))
, sFile(symbols.create("file"))
, sLine(symbols.create("line"))
, sColumn(symbols.create("column"))
, sFunctor(symbols.create("__functor"))
, sToString(symbols.create("__toString"))
, sRight(symbols.create("right"))
, sWrong(symbols.create("wrong"))
, sStructuredAttrs(symbols.create("__structuredAttrs"))
, sJson(symbols.create("__json"))
, sAllowedReferences(symbols.create("allowedReferences"))
, sAllowedRequisites(symbols.create("allowedRequisites"))
, sDisallowedReferences(symbols.create("disallowedReferences"))
, sDisallowedRequisites(symbols.create("disallowedRequisites"))
, sMaxSize(symbols.create("maxSize"))
, sMaxClosureSize(symbols.create("maxClosureSize"))
, sBuilder(symbols.create("builder"))
, sArgs(symbols.create("args"))
, sContentAddressed(symbols.create("__contentAddressed"))
, sImpure(symbols.create("__impure"))
, sOutputHash(symbols.create("outputHash"))
, sOutputHashAlgo(symbols.create("outputHashAlgo"))
, sOutputHashMode(symbols.create("outputHashMode"))
, sRecurseForDerivations(symbols.create("recurseForDerivations"))
, sDescription(symbols.create("description"))
, sSelf(symbols.create("self"))
, sEpsilon(symbols.create(""))
, sStartSet(symbols.create("startSet"))
, sOperator(symbols.create("operator"))
, sKey(symbols.create("key"))
, sPath(symbols.create("path"))
, sPrefix(symbols.create("prefix"))
, sOutputSpecified(symbols.create("outputSpecified"))
, exprSymbols{
.sub = symbols.create("__sub"),
.lessThan = symbols.create("__lessThan"),
.mul = symbols.create("__mul"),
.div = symbols.create("__div"),
.or_ = symbols.create("or"),
.findFile = symbols.create("__findFile"),
.nixPath = symbols.create("__nixPath"),
.body = symbols.create("body"),
}
, repair(NoRepair)
, storeFS(makeMountedSourceAccessor({
{CanonPath::root, makeEmptySourceAccessor()},
/* In the pure eval case, we can simply require
valid paths. However, in the *impure* eval
case this gets in the way of the union
mechanism, because an invalid access in the
upper layer will *not* be caught by the union
source accessor, but instead abort the entire
lookup.
, emptyBindings(0)
, storeFS(
makeMountedSourceAccessor(
{
{CanonPath::root, makeEmptySourceAccessor()},
/* In the pure eval case, we can simply require
valid paths. However, in the *impure* eval
case this gets in the way of the union
mechanism, because an invalid access in the
upper layer will *not* be caught by the union
source accessor, but instead abort the entire
lookup.
This happens when the store dir in the
ambient file system has a path (e.g. because
another Nix store there), but the relocated
store does not.
This happens when the store dir in the
ambient file system has a path (e.g. because
another Nix store there), but the relocated
store does not.
TODO make the various source accessors doing
access control all throw the same type of
exception, and make union source accessor
catch it, so we don't need to do this hack.
*/
{CanonPath(store->storeDir), store->getFSAccessor(settings.pureEval)},
}))
, rootFS([&] {
/* In pure eval mode, we provide a filesystem that only
contains the Nix store.
TODO make the various source accessors doing
access control all throw the same type of
exception, and make union source accessor
catch it, so we don't need to do this hack.
*/
{CanonPath(store->storeDir), store->getFSAccessor(settings.pureEval)},
}))
, rootFS(
({
/* In pure eval mode, we provide a filesystem that only
contains the Nix store.
Otherwise, use a union accessor to make the augmented store
available at its logical location while still having the
underlying directory available. This is necessary for
instance if we're evaluating a file from the physical
/nix/store while using a chroot store, and also for lazy
mounted fetchTree. */
auto accessor = settings.pureEval ? storeFS.cast<SourceAccessor>()
: makeUnionSourceAccessor({getFSSourceAccessor(), storeFS});
If we have a chroot store and pure eval is not enabled,
use a union accessor to make the chroot store available
at its logical location while still having the
underlying directory available. This is necessary for
instance if we're evaluating a file from the physical
/nix/store while using a chroot store. */
auto accessor = getFSSourceAccessor();
/* Apply access control if needed. */
if (settings.restrictEval || settings.pureEval)
accessor = AllowListSourceAccessor::create(
accessor, {}, {}, [&settings](const CanonPath & path) -> RestrictedPathError {
auto modeInformation = settings.pureEval ? "in pure evaluation mode (use '--impure' to override)"
: "in restricted mode";
throw RestrictedPathError("access to absolute path '%1%' is forbidden %2%", path, modeInformation);
});
auto realStoreDir = dirOf(store->toRealPath(StorePath::dummy));
if (settings.pureEval || store->storeDir != realStoreDir) {
accessor = settings.pureEval
? storeFS
: makeUnionSourceAccessor({accessor, storeFS});
}
return accessor;
}())
/* Apply access control if needed. */
if (settings.restrictEval || settings.pureEval)
accessor = AllowListSourceAccessor::create(accessor, {}, {},
[&settings](const CanonPath & path) -> RestrictedPathError {
auto modeInformation = settings.pureEval
? "in pure evaluation mode (use '--impure' to override)"
: "in restricted mode";
throw RestrictedPathError("access to absolute path '%1%' is forbidden %2%", path, modeInformation);
});
accessor;
}))
, corepkgsFS(make_ref<MemorySourceAccessor>())
, internalFS(make_ref<MemorySourceAccessor>())
, derivationInternal{internalFS->addFile(
CanonPath("derivation-internal.nix"),
, derivationInternal{corepkgsFS->addFile(
CanonPath("derivation-internal.nix"),
#include "primops/derivation.nix.gen.hh"
)}
)}
, store(store)
, buildStore(buildStore ? buildStore : store)
, inputCache(fetchers::InputCache::create())
, debugRepl(nullptr)
, debugStop(false)
, trylevel(0)
, srcToStore(make_ref<decltype(srcToStore)::element_type>())
, importResolutionCache(make_ref<decltype(importResolutionCache)::element_type>())
, fileEvalCache(make_ref<decltype(fileEvalCache)::element_type>())
, regexCache(makeRegexCache())
#if NIX_USE_BOEHMGC
, baseEnvP(std::allocate_shared<Env *>(traceable_allocator<Env *>(), &mem.allocEnv(BASE_ENV_SIZE)))
, valueAllocCache(std::allocate_shared<void *>(traceable_allocator<void *>(), nullptr))
, env1AllocCache(std::allocate_shared<void *>(traceable_allocator<void *>(), nullptr))
, baseEnvP(std::allocate_shared<Env *>(traceable_allocator<Env *>(), &allocEnv(BASE_ENV_SIZE)))
, baseEnv(**baseEnvP)
#else
, baseEnv(mem.allocEnv(BASE_ENV_SIZE))
, baseEnv(allocEnv(BASE_ENV_SIZE))
#endif
, staticBaseEnv{std::make_shared<StaticEnv>(nullptr, nullptr)}
{
@@ -288,8 +343,18 @@ EvalState::EvalState(
countCalls = getEnv("NIX_COUNT_CALLS").value_or("0") != "0";
assertGCInitialized();
static_assert(sizeof(Env) <= 16, "environment must be <= 16 bytes");
static_assert(sizeof(Counter) == 64, "counters must be 64 bytes");
vEmptyList.mkList(buildList(0));
vNull.mkNull();
vTrue.mkBool(true);
vFalse.mkBool(false);
vStringRegular.mkString("regular");
vStringDirectory.mkString("directory");
vStringSymlink.mkString("symlink");
vStringUnknown.mkString("unknown");
/* Construct the Nix expression search path. */
assert(lookupPath.elements.empty());
@@ -336,7 +401,7 @@ EvalState::EvalState(
EvalState::~EvalState() {}
void EvalState::allowPathLegacy(const Path & path)
void EvalState::allowPath(const Path & path)
{
if (auto rootFS2 = rootFS.dynamic_pointer_cast<AllowListSourceAccessor>())
rootFS2->allowPrefix(CanonPath(path));
@@ -589,7 +654,7 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
}
if (isFunctor(v)) {
try {
Value & functor = *v.attrs()->get(s.functor)->value;
Value & functor = *v.attrs()->find(sFunctor)->value;
Value * vp[] = {&v};
Value partiallyApplied;
// The first parameter is not user-provided, and may be
@@ -818,7 +883,7 @@ DebugTraceStacker::DebugTraceStacker(EvalState & evalState, DebugTrace t)
void Value::mkString(std::string_view s)
{
mkStringNoCopy(makeImmutableString(s));
mkString(makeImmutableString(s));
}
static const char ** encodeContext(const NixStringContext & context)
@@ -837,12 +902,12 @@ static const char ** encodeContext(const NixStringContext & context)
void Value::mkString(std::string_view s, const NixStringContext & context)
{
mkStringNoCopy(makeImmutableString(s), encodeContext(context));
mkString(makeImmutableString(s), encodeContext(context));
}
void Value::mkStringMove(const char * s, const NixStringContext & context)
{
mkStringNoCopy(s, encodeContext(context));
mkString(s, encodeContext(context));
}
void Value::mkPath(const SourcePath & path)
@@ -883,18 +948,19 @@ inline Value * EvalState::lookupVar(Env * env, const ExprVar & var, bool noEval)
}
}
ListBuilder::ListBuilder(size_t size)
ListBuilder::ListBuilder(EvalState & state, size_t size)
: size(size)
, elems(size <= 2 ? inlineElems : (Value **) allocBytes(size * sizeof(Value *)))
{
state.nrListElems += size;
}
Value * EvalState::getBool(bool b)
{
return b ? &Value::vTrue : &Value::vFalse;
return b ? &vTrue : &vFalse;
}
static Counter nrThunks;
unsigned long nrThunks = 0;
static inline void mkThunk(Value & v, Env & env, Expr * expr)
{
@@ -912,8 +978,8 @@ void EvalState::mkPos(Value & v, PosIdx p)
auto origin = positions.originOf(p);
if (auto path = std::get_if<SourcePath>(&origin)) {
auto attrs = buildBindings(3);
attrs.alloc(s.file).mkString(path->path.abs());
makePositionThunks(*this, p, attrs.alloc(s.line), attrs.alloc(s.column));
attrs.alloc(sFile).mkString(path->path.abs());
makePositionThunks(*this, p, attrs.alloc(sLine), attrs.alloc(sColumn));
v.mkAttrs(attrs);
} else
v.mkNull();
@@ -985,6 +1051,10 @@ void EvalState::mkSingleDerivedPathString(const SingleDerivedPath & p, Value & v
});
}
/* Create a thunk for the delayed computation of the given expression
in the given environment. But if the expression is a variable,
then look it up right away. This significantly reduces the number
of thunks allocated. */
Value * Expr::maybeThunk(EvalState & state, Env & env)
{
Value * v = state.allocValue();
@@ -1028,90 +1098,61 @@ Value * ExprPath::maybeThunk(EvalState & state, Env & env)
return &v;
}
/**
* A helper `Expr` class to lets us parse and evaluate Nix expressions
* from a thunk, ensuring that every file is parsed/evaluated only
* once (via the thunk stored in `EvalState::fileEvalCache`).
*/
struct ExprParseFile : Expr, gc
{
// FIXME: make this a reference (see below).
SourcePath path;
bool mustBeTrivial;
ExprParseFile(SourcePath & path, bool mustBeTrivial)
: path(path)
, mustBeTrivial(mustBeTrivial)
{
}
void eval(EvalState & state, Env & env, Value & v) override
{
printTalkative("evaluating file '%s'", path);
auto e = state.parseExprFromFile(path);
try {
auto dts =
state.debugRepl
? makeDebugTraceStacker(
state, *e, state.baseEnv, e->getPos(), "while evaluating the file '%s':", path.to_string())
: nullptr;
// Enforce that 'flake.nix' is a direct attrset, not a
// computation.
if (mustBeTrivial && !(dynamic_cast<ExprAttrs *>(e)))
state.error<EvalError>("file '%s' must be an attribute set", path).debugThrow();
state.eval(e, v);
} catch (Error & e) {
state.addErrorTrace(e, "while evaluating the file '%s':", path.to_string());
throw;
}
}
};
void EvalState::evalFile(const SourcePath & path, Value & v, bool mustBeTrivial)
{
auto resolvedPath = getConcurrent(*importResolutionCache, path);
if (!resolvedPath) {
resolvedPath = resolveExprPath(path);
importResolutionCache->emplace(path, *resolvedPath);
}
if (auto v2 = getConcurrent(*fileEvalCache, *resolvedPath)) {
forceValue(**v2, noPos);
v = **v2;
FileEvalCache::iterator i;
if ((i = fileEvalCache.find(path)) != fileEvalCache.end()) {
v = i->second;
return;
}
Value * vExpr;
// FIXME: put ExprParseFile on the stack instead of the heap once
// https://github.com/NixOS/nix/pull/13930 is merged. That will ensure
// the post-condition that `expr` is unreachable after
// `forceValue()` returns.
auto expr = new ExprParseFile{*resolvedPath, mustBeTrivial};
auto resolvedPath = resolveExprPath(path);
if ((i = fileEvalCache.find(resolvedPath)) != fileEvalCache.end()) {
v = i->second;
return;
}
fileEvalCache->try_emplace_and_cvisit(
*resolvedPath,
nullptr,
[&](auto & i) {
vExpr = allocValue();
vExpr->mkThunk(&baseEnv, expr);
i.second = vExpr;
},
[&](auto & i) { vExpr = i.second; });
printTalkative("evaluating file '%1%'", resolvedPath);
Expr * e = nullptr;
forceValue(*vExpr, noPos);
auto j = fileParseCache.find(resolvedPath);
if (j != fileParseCache.end())
e = j->second;
v = *vExpr;
if (!e)
e = parseExprFromFile(resolvedPath);
fileParseCache.emplace(resolvedPath, e);
try {
auto dts = debugRepl ? makeDebugTraceStacker(
*this,
*e,
this->baseEnv,
e->getPos(),
"while evaluating the file '%1%':",
resolvedPath.to_string())
: nullptr;
// Enforce that 'flake.nix' is a direct attrset, not a
// computation.
if (mustBeTrivial && !(dynamic_cast<ExprAttrs *>(e)))
error<EvalError>("file '%s' must be an attribute set", path).debugThrow();
eval(e, v);
} catch (Error & e) {
addErrorTrace(e, "while evaluating the file '%1%':", resolvedPath.to_string());
throw;
}
fileEvalCache.emplace(resolvedPath, v);
if (path != resolvedPath)
fileEvalCache.emplace(path, v);
}
void EvalState::resetFileCache()
{
importResolutionCache->clear();
fileEvalCache->clear();
fileEvalCache.clear();
fileParseCache.clear();
inputCache->clear();
}
@@ -1180,7 +1221,7 @@ void ExprPath::eval(EvalState & state, Env & env, Value & v)
Env * ExprAttrs::buildInheritFromEnv(EvalState & state, Env & up)
{
Env & inheritEnv = state.mem.allocEnv(inheritFromExprs->size());
Env & inheritEnv = state.allocEnv(inheritFromExprs->size());
inheritEnv.up = &up;
Displacement displ = 0;
@@ -1199,12 +1240,12 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
if (recursive) {
/* Create a new environment that contains the attributes in
this `rec'. */
Env & env2(state.mem.allocEnv(attrs.size()));
Env & env2(state.allocEnv(attrs.size()));
env2.up = &env;
dynamicEnv = &env2;
Env * inheritEnv = inheritFromExprs ? buildInheritFromEnv(state, env2) : nullptr;
AttrDefs::iterator overrides = attrs.find(state.s.overrides);
AttrDefs::iterator overrides = attrs.find(state.sOverrides);
bool hasOverrides = overrides != attrs.end();
/* The recursive attributes are evaluated in the new
@@ -1236,7 +1277,7 @@ void ExprAttrs::eval(EvalState & state, Env & env, Value & v)
*vOverrides,
[&]() { return vOverrides->determinePos(noPos); },
"while evaluating the `__overrides` attribute");
bindings.grow(state.buildBindings(bindings.capacity() + vOverrides->attrs()->size()));
bindings.grow(state.allocBindings(bindings.capacity() + vOverrides->attrs()->size()));
for (auto & i : *vOverrides->attrs()) {
AttrDefs::iterator j = attrs.find(i.name);
if (j != attrs.end()) {
@@ -1291,7 +1332,7 @@ void ExprLet::eval(EvalState & state, Env & env, Value & v)
{
/* Create a new environment that contains the attributes in this
`let'. */
Env & env2(state.mem.allocEnv(attrs->attrs.size()));
Env & env2(state.allocEnv(attrs->attrs.size()));
env2.up = &env;
Env * inheritEnv = attrs->inheritFromExprs ? attrs->buildInheritFromEnv(state, env2) : nullptr;
@@ -1322,7 +1363,7 @@ void ExprList::eval(EvalState & state, Env & env, Value & v)
Value * ExprList::maybeThunk(EvalState & state, Env & env)
{
if (elems.empty()) {
return &Value::vEmptyList;
return &state.vEmptyList;
}
return Expr::maybeThunk(state, env);
}
@@ -1334,7 +1375,7 @@ void ExprVar::eval(EvalState & state, Env & env, Value & v)
v = *v2;
}
static std::string showAttrPath(EvalState & state, Env & env, std::span<const AttrName> attrPath)
static std::string showAttrPath(EvalState & state, Env & env, const AttrPath & attrPath)
{
std::ostringstream out;
bool first = true;
@@ -1370,10 +1411,10 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
env,
getPos(),
"while evaluating the attribute '%1%'",
showAttrPath(state, env, getAttrPath()))
showAttrPath(state, env, attrPath))
: nullptr;
for (auto & i : getAttrPath()) {
for (auto & i : attrPath) {
state.nrLookups++;
const Attr * j;
auto name = getName(i, state, env);
@@ -1411,7 +1452,7 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
auto origin = std::get_if<SourcePath>(&pos2r.origin);
if (!(origin && *origin == state.derivationInternal))
state.addErrorTrace(
e, pos2, "while evaluating the attribute '%1%'", showAttrPath(state, env, getAttrPath()));
e, pos2, "while evaluating the attribute '%1%'", showAttrPath(state, env, attrPath));
}
throw;
}
@@ -1422,13 +1463,13 @@ void ExprSelect::eval(EvalState & state, Env & env, Value & v)
Symbol ExprSelect::evalExceptFinalSelect(EvalState & state, Env & env, Value & attrs)
{
Value vTmp;
Symbol name = getName(attrPathStart[nAttrPath - 1], state, env);
Symbol name = getName(attrPath[attrPath.size() - 1], state, env);
if (nAttrPath == 1) {
if (attrPath.size() == 1) {
e->eval(state, env, vTmp);
} else {
ExprSelect init(*this);
init.nAttrPath--;
init.attrPath.pop_back();
init.eval(state, env, vTmp);
}
attrs = vTmp;
@@ -1497,7 +1538,7 @@ void EvalState::callFunction(Value & fun, std::span<Value *> args, Value & vRes,
ExprLambda & lambda(*vCur.lambda().fun);
auto size = (!lambda.arg ? 0 : 1) + (lambda.hasFormals() ? lambda.formals->formals.size() : 0);
Env & env2(mem.allocEnv(size));
Env & env2(allocEnv(size));
env2.up = vCur.lambda().env;
Displacement displ = 0;
@@ -1676,7 +1717,7 @@ void EvalState::callFunction(Value & fun, std::span<Value *> args, Value & vRes,
}
}
else if (vCur.type() == nAttrs && (functor = vCur.attrs()->get(s.functor))) {
else if (vCur.type() == nAttrs && (functor = vCur.attrs()->get(sFunctor))) {
/* 'vCur' may be allocated on the stack of the calling
function, but for functors we may keep a reference, so
heap-allocate a copy and use that instead. */
@@ -1738,8 +1779,8 @@ void EvalState::autoCallFunction(const Bindings & args, Value & fun, Value & res
forceValue(fun, pos);
if (fun.type() == nAttrs) {
auto found = fun.attrs()->get(s.functor);
if (found) {
auto found = fun.attrs()->find(sFunctor);
if (found != fun.attrs()->end()) {
Value * v = allocValue();
callFunction(*found->value, fun, *v, pos);
forceValue(*v, pos);
@@ -1786,7 +1827,7 @@ https://nix.dev/manual/nix/stable/language/syntax.html#functions.)",
void ExprWith::eval(EvalState & state, Env & env, Value & v)
{
Env & env2(state.mem.allocEnv(1));
Env & env2(state.allocEnv(1));
env2.up = &env;
env2.values[0] = attrs->maybeThunk(state, env);
@@ -1868,113 +1909,51 @@ void ExprOpImpl::eval(EvalState & state, Env & env, Value & v)
|| state.evalBool(env, e2, pos, "in the right operand of the IMPL (->) operator"));
}
void ExprOpUpdate::eval(EvalState & state, Value & v, Value & v1, Value & v2)
void ExprOpUpdate::eval(EvalState & state, Env & env, Value & v)
{
Value v1, v2;
state.evalAttrs(env, e1, v1, pos, "in the left operand of the update (//) operator");
state.evalAttrs(env, e2, v2, pos, "in the right operand of the update (//) operator");
state.nrOpUpdates++;
const Bindings & bindings1 = *v1.attrs();
if (bindings1.empty()) {
if (v1.attrs()->size() == 0) {
v = v2;
return;
}
const Bindings & bindings2 = *v2.attrs();
if (bindings2.empty()) {
if (v2.attrs()->size() == 0) {
v = v1;
return;
}
/* Simple heuristic for determining whether attrs2 should be "layered" on top of
attrs1 instead of copying to a new Bindings. */
bool shouldLayer = [&]() -> bool {
if (bindings1.isLayerListFull())
return false;
if (bindings2.size() > state.settings.bindingsUpdateLayerRhsSizeThreshold)
return false;
return true;
}();
if (shouldLayer) {
auto attrs = state.buildBindings(bindings2.size());
attrs.layerOnTopOf(bindings1);
std::ranges::copy(bindings2, std::back_inserter(attrs));
v.mkAttrs(attrs.alreadySorted());
state.nrOpUpdateValuesCopied += bindings2.size();
return;
}
auto attrs = state.buildBindings(bindings1.size() + bindings2.size());
auto attrs = state.buildBindings(v1.attrs()->size() + v2.attrs()->size());
/* Merge the sets, preferring values from the second set. Make
sure to keep the resulting vector in sorted order. */
auto i = bindings1.begin();
auto j = bindings2.begin();
auto i = v1.attrs()->begin();
auto j = v2.attrs()->begin();
while (i != bindings1.end() && j != bindings2.end()) {
while (i != v1.attrs()->end() && j != v2.attrs()->end()) {
if (i->name == j->name) {
attrs.insert(*j);
++i;
++j;
} else if (i->name < j->name) {
attrs.insert(*i);
++i;
} else {
attrs.insert(*j);
++j;
}
} else if (i->name < j->name)
attrs.insert(*i++);
else
attrs.insert(*j++);
}
while (i != bindings1.end()) {
attrs.insert(*i);
++i;
}
while (j != bindings2.end()) {
attrs.insert(*j);
++j;
}
while (i != v1.attrs()->end())
attrs.insert(*i++);
while (j != v2.attrs()->end())
attrs.insert(*j++);
v.mkAttrs(attrs.alreadySorted());
state.nrOpUpdateValuesCopied += v.attrs()->size();
}
void ExprOpUpdate::eval(EvalState & state, Env & env, Value & v)
{
UpdateQueue q;
evalForUpdate(state, env, q);
v.mkAttrs(&Bindings::emptyBindings);
for (auto & rhs : std::views::reverse(q)) {
/* Remember that queue is sorted rightmost attrset first. */
eval(state, /*v=*/v, /*v1=*/v, /*v2=*/rhs);
}
}
void Expr::evalForUpdate(EvalState & state, Env & env, UpdateQueue & q, std::string_view errorCtx)
{
Value v;
state.evalAttrs(env, this, v, getPos(), errorCtx);
q.push_back(v);
}
void ExprOpUpdate::evalForUpdate(EvalState & state, Env & env, UpdateQueue & q)
{
/* Output rightmost attrset first to the merge queue as the one
with the most priority. */
e2->evalForUpdate(state, env, q, "in the right operand of the update (//) operator");
e1->evalForUpdate(state, env, q, "in the left operand of the update (//) operator");
}
void ExprOpUpdate::evalForUpdate(EvalState & state, Env & env, UpdateQueue & q, std::string_view errorCtx)
{
evalForUpdate(state, env, q);
}
void ExprOpConcatLists::eval(EvalState & state, Env & env, Value & v)
{
Value v1;
@@ -2251,10 +2230,10 @@ bool EvalState::forceBool(Value & v, const PosIdx pos, std::string_view errorCtx
return v.boolean();
}
const Attr * EvalState::getAttr(Symbol attrSym, const Bindings * attrSet, std::string_view errorCtx)
Bindings::const_iterator EvalState::getAttr(Symbol attrSym, const Bindings * attrSet, std::string_view errorCtx)
{
auto value = attrSet->get(attrSym);
if (!value) {
auto value = attrSet->find(attrSym);
if (value == attrSet->end()) {
error<TypeError>("attribute '%s' missing", symbols[attrSym]).withTrace(noPos, errorCtx).debugThrow();
}
return value;
@@ -2262,7 +2241,7 @@ const Attr * EvalState::getAttr(Symbol attrSym, const Bindings * attrSet, std::s
bool EvalState::isFunctor(const Value & fun) const
{
return fun.type() == nAttrs && fun.attrs()->get(s.functor);
return fun.type() == nAttrs && fun.attrs()->find(sFunctor) != fun.attrs()->end();
}
void EvalState::forceFunction(Value & v, const PosIdx pos, std::string_view errorCtx)
@@ -2331,7 +2310,7 @@ bool EvalState::isDerivation(Value & v)
{
if (v.type() != nAttrs)
return false;
auto i = v.attrs()->get(s.type);
auto i = v.attrs()->get(sType);
if (!i)
return false;
forceValue(*i->value, i->pos);
@@ -2343,8 +2322,8 @@ bool EvalState::isDerivation(Value & v)
std::optional<std::string>
EvalState::tryAttrsToString(const PosIdx pos, Value & v, NixStringContext & context, bool coerceMore, bool copyToStore)
{
auto i = v.attrs()->get(s.toString);
if (i) {
auto i = v.attrs()->find(sToString);
if (i != v.attrs()->end()) {
Value v1;
callFunction(*i->value, v, v1, pos);
return coerceToString(
@@ -2389,8 +2368,8 @@ BackedStringView EvalState::coerceToString(
auto maybeString = tryAttrsToString(pos, v, context, coerceMore, copyToStore);
if (maybeString)
return std::move(*maybeString);
auto i = v.attrs()->get(s.outPath);
if (!i) {
auto i = v.attrs()->find(sOutPath);
if (i == v.attrs()->end()) {
error<TypeError>(
"cannot coerce %1% to a string: %2%", showType(v), ValuePrinter(*this, v, errorPrintOptions))
.withTrace(pos, errorCtx)
@@ -2458,7 +2437,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
if (nix::isDerivation(path.path.abs()))
error<EvalError>("file names are not allowed to end in '%1%'", drvExtension).debugThrow();
auto dstPathCached = getConcurrent(*srcToStore, path);
auto dstPathCached = get(*srcToStore.lock(), path);
auto dstPath = dstPathCached ? *dstPathCached : [&]() {
auto dstPath = fetchToStore(
@@ -2471,7 +2450,7 @@ StorePath EvalState::copyPathToStore(NixStringContext & context, const SourcePat
nullptr,
repair);
allowPath(dstPath);
srcToStore->try_emplace(path, dstPath);
srcToStore.lock()->try_emplace(path, dstPath);
printMsg(lvlChatty, "copied source '%1%' -> '%2%'", path, store->printStorePath(dstPath));
return dstPath;
}();
@@ -2496,8 +2475,8 @@ SourcePath EvalState::coerceToPath(const PosIdx pos, Value & v, NixStringContext
/* Similarly, handle __toString where the result may be a path
value. */
if (v.type() == nAttrs) {
auto i = v.attrs()->get(s.toString);
if (i) {
auto i = v.attrs()->find(sToString);
if (i != v.attrs()->end()) {
Value v1;
callFunction(*i->value, v, v1, pos);
return coerceToPath(pos, v1, context, errorCtx);
@@ -2686,8 +2665,8 @@ void EvalState::assertEqValues(Value & v1, Value & v2, const PosIdx pos, std::st
case nAttrs: {
if (isDerivation(v1) && isDerivation(v2)) {
auto i = v1.attrs()->get(s.outPath);
auto j = v2.attrs()->get(s.outPath);
auto i = v1.attrs()->get(sOutPath);
auto j = v2.attrs()->get(sOutPath);
if (i && j) {
try {
assertEqValues(*i->value, *j->value, pos, errorCtx);
@@ -2840,8 +2819,8 @@ bool EvalState::eqValues(Value & v1, Value & v2, const PosIdx pos, std::string_v
/* If both sets denote a derivation (type = "derivation"),
then compare their outPaths. */
if (isDerivation(v1) && isDerivation(v2)) {
auto i = v1.attrs()->get(s.outPath);
auto j = v2.attrs()->get(s.outPath);
auto i = v1.attrs()->get(sOutPath);
auto j = v2.attrs()->get(sOutPath);
if (i && j)
return eqValues(*i->value, *j->value, pos, errorCtx);
}
@@ -2893,11 +2872,11 @@ bool EvalState::fullGC()
#endif
}
bool Counter::enabled = getEnv("NIX_SHOW_STATS").value_or("0") != "0";
void EvalState::maybePrintStats()
{
if (Counter::enabled) {
bool showStats = getEnv("NIX_SHOW_STATS").value_or("0") != "0";
if (showStats) {
// Make the final heap size more deterministic.
#if NIX_USE_BOEHMGC
if (!fullGC()) {
@@ -2910,15 +2889,16 @@ void EvalState::maybePrintStats()
void EvalState::printStatistics()
{
std::chrono::microseconds cpuTimeDuration = getCpuUserTime();
float cpuTime = std::chrono::duration_cast<std::chrono::duration<float>>(cpuTimeDuration).count();
#ifndef _WIN32 // TODO use portable implementation
struct rusage buf;
getrusage(RUSAGE_SELF, &buf);
float cpuTime = buf.ru_utime.tv_sec + ((float) buf.ru_utime.tv_usec / 1000000);
#endif
auto & memstats = mem.getStats();
uint64_t bEnvs = memstats.nrEnvs * sizeof(Env) + memstats.nrValuesInEnvs * sizeof(Value *);
uint64_t bLists = memstats.nrListElems * sizeof(Value *);
uint64_t bValues = memstats.nrValues * sizeof(Value);
uint64_t bAttrsets = memstats.nrAttrsets * sizeof(Bindings) + memstats.nrAttrsInAttrsets * sizeof(Attr);
uint64_t bEnvs = nrEnvs * sizeof(Env) + nrValuesInEnvs * sizeof(Value *);
uint64_t bLists = nrListElems * sizeof(Value *);
uint64_t bValues = nrValues * sizeof(Value);
uint64_t bAttrsets = nrAttrsets * sizeof(Bindings) + nrAttrsInAttrsets * sizeof(Attr);
#if NIX_USE_BOEHMGC
GC_word heapSize, totalBytes;
@@ -2935,27 +2915,33 @@ void EvalState::printStatistics()
if (outPath != "-")
fs.open(outPath, std::fstream::out);
json topObj = json::object();
#ifndef _WIN32 // TODO implement
topObj["cpuTime"] = cpuTime;
#endif
topObj["time"] = {
#ifndef _WIN32 // TODO implement
{"cpu", cpuTime},
#endif
#if NIX_USE_BOEHMGC
{GC_is_incremental_mode() ? "gcNonIncremental" : "gc", gcFullOnlyTime},
# ifndef _WIN32 // TODO implement
{GC_is_incremental_mode() ? "gcNonIncrementalFraction" : "gcFraction", gcFullOnlyTime / cpuTime},
# endif
#endif
};
topObj["envs"] = {
{"number", memstats.nrEnvs.load()},
{"elements", memstats.nrValuesInEnvs.load()},
{"number", nrEnvs},
{"elements", nrValuesInEnvs},
{"bytes", bEnvs},
};
topObj["nrExprs"] = Expr::nrExprs.load();
topObj["nrExprs"] = Expr::nrExprs;
topObj["list"] = {
{"elements", memstats.nrListElems.load()},
{"elements", nrListElems},
{"bytes", bLists},
{"concats", nrListConcats.load()},
{"concats", nrListConcats},
};
topObj["values"] = {
{"number", memstats.nrValues.load()},
{"number", nrValues},
{"bytes", bValues},
};
topObj["symbols"] = {
@@ -2963,9 +2949,9 @@ void EvalState::printStatistics()
{"bytes", symbols.totalSize()},
};
topObj["sets"] = {
{"number", memstats.nrAttrsets.load()},
{"number", nrAttrsets},
{"bytes", bAttrsets},
{"elements", memstats.nrAttrsInAttrsets.load()},
{"elements", nrAttrsInAttrsets},
};
topObj["sizes"] = {
{"Env", sizeof(Env)},
@@ -2973,13 +2959,13 @@ void EvalState::printStatistics()
{"Bindings", sizeof(Bindings)},
{"Attr", sizeof(Attr)},
};
topObj["nrOpUpdates"] = nrOpUpdates.load();
topObj["nrOpUpdateValuesCopied"] = nrOpUpdateValuesCopied.load();
topObj["nrThunks"] = nrThunks.load();
topObj["nrAvoided"] = nrAvoided.load();
topObj["nrLookups"] = nrLookups.load();
topObj["nrPrimOpCalls"] = nrPrimOpCalls.load();
topObj["nrFunctionCalls"] = nrFunctionCalls.load();
topObj["nrOpUpdates"] = nrOpUpdates;
topObj["nrOpUpdateValuesCopied"] = nrOpUpdateValuesCopied;
topObj["nrThunks"] = nrThunks;
topObj["nrAvoided"] = nrAvoided;
topObj["nrLookups"] = nrLookups;
topObj["nrPrimOpCalls"] = nrPrimOpCalls;
topObj["nrFunctionCalls"] = nrFunctionCalls;
#if NIX_USE_BOEHMGC
topObj["gc"] = {
{"heapSize", heapSize},
@@ -3126,11 +3112,6 @@ SourcePath EvalState::findFile(const LookupPath & lookupPath, const std::string_
auto res = (r / CanonPath(suffix)).resolveSymlinks();
if (res.pathExists())
return res;
// Backward compatibility hack: throw an exception if access
// to this path is not allowed.
if (auto accessor = res.accessor.dynamic_pointer_cast<FilteringSourceAccessor>())
accessor->checkAccess(res.path);
}
if (hasPrefix(path, "nix/"))
@@ -3185,7 +3166,7 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
/* Allow access to paths in the search path. */
if (initAccessControl) {
allowPathLegacy(path.path.abs());
allowPath(path.path.abs());
if (store->isInStore(path.path.abs())) {
try {
allowClosure(store->toStorePath(path.path.abs()).first);
@@ -3197,11 +3178,6 @@ std::optional<SourcePath> EvalState::resolveLookupPathPath(const LookupPath::Pat
if (path.resolveSymlinks().pathExists())
return finish(std::move(path));
else {
// Backward compatibility hack: throw an exception if access
// to this path is not allowed.
if (auto accessor = path.accessor.dynamic_pointer_cast<FilteringSourceAccessor>())
accessor->checkAccess(path.path);
logWarning({.msg = HintFmt("Nix search path entry '%1%' does not exist, ignoring", value)});
}
}
@@ -3221,7 +3197,7 @@ Expr * EvalState::parse(
}
auto result = parseExprFromBuf(
text, length, origin, basePath, mem.exprs.alloc, symbols, settings, positions, *docComments, rootFS);
text, length, origin, basePath, symbols, settings, positions, *docComments, rootFS, exprSymbols);
result->bindVars(*this, staticEnv);
+16 -15
View File
@@ -45,8 +45,8 @@ PackageInfo::PackageInfo(EvalState & state, ref<Store> store, const std::string
std::string PackageInfo::queryName() const
{
if (name == "" && attrs) {
auto i = attrs->get(state->s.name);
if (!i)
auto i = attrs->find(state->sName);
if (i == attrs->end())
state->error<TypeError>("derivation name missing").debugThrow();
name = state->forceStringNoCtx(*i->value, noPos, "while evaluating the 'name' attribute of a derivation");
}
@@ -56,10 +56,11 @@ std::string PackageInfo::queryName() const
std::string PackageInfo::querySystem() const
{
if (system == "" && attrs) {
auto i = attrs->get(state->s.system);
auto i = attrs->find(state->sSystem);
system =
!i ? "unknown"
: state->forceStringNoCtx(*i->value, i->pos, "while evaluating the 'system' attribute of a derivation");
i == attrs->end()
? "unknown"
: state->forceStringNoCtx(*i->value, i->pos, "while evaluating the 'system' attribute of a derivation");
}
return system;
}
@@ -67,7 +68,7 @@ std::string PackageInfo::querySystem() const
std::optional<StorePath> PackageInfo::queryDrvPath() const
{
if (!drvPath && attrs) {
if (auto i = attrs->get(state->s.drvPath)) {
if (auto i = attrs->get(state->sDrvPath)) {
NixStringContext context;
auto found = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the 'drvPath' attribute of a derivation");
@@ -94,9 +95,9 @@ StorePath PackageInfo::requireDrvPath() const
StorePath PackageInfo::queryOutPath() const
{
if (!outPath && attrs) {
auto i = attrs->get(state->s.outPath);
auto i = attrs->find(state->sOutPath);
NixStringContext context;
if (i)
if (i != attrs->end())
outPath = state->coerceToStorePath(
i->pos, *i->value, context, "while evaluating the output path of a derivation");
}
@@ -110,7 +111,7 @@ PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsT
if (outputs.empty()) {
/* Get the ‘outputs’ list. */
const Attr * i;
if (attrs && (i = attrs->get(state->s.outputs))) {
if (attrs && (i = attrs->get(state->sOutputs))) {
state->forceList(*i->value, i->pos, "while evaluating the 'outputs' attribute of a derivation");
/* For each output... */
@@ -126,7 +127,7 @@ PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsT
state->forceAttrs(*out->value, i->pos, "while evaluating an output of a derivation");
/* And evaluate its ‘outPath’ attribute. */
auto outPath = out->value->attrs()->get(state->s.outPath);
auto outPath = out->value->attrs()->get(state->sOutPath);
if (!outPath)
continue; // FIXME: throw error?
NixStringContext context;
@@ -145,7 +146,7 @@ PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsT
return outputs;
const Attr * i;
if (attrs && (i = attrs->get(state->s.outputSpecified))
if (attrs && (i = attrs->get(state->sOutputSpecified))
&& state->forceBool(*i->value, i->pos, "while evaluating the 'outputSpecified' attribute of a derivation")) {
Outputs result;
auto out = outputs.find(queryOutputName());
@@ -180,7 +181,7 @@ PackageInfo::Outputs PackageInfo::queryOutputs(bool withPaths, bool onlyOutputsT
std::string PackageInfo::queryOutputName() const
{
if (outputName == "" && attrs) {
auto i = attrs->get(state->s.outputName);
auto i = attrs->get(state->sOutputName);
outputName =
i ? state->forceStringNoCtx(*i->value, noPos, "while evaluating the output name of a derivation") : "";
}
@@ -193,7 +194,7 @@ const Bindings * PackageInfo::getMeta()
return meta;
if (!attrs)
return 0;
auto a = attrs->get(state->s.meta);
auto a = attrs->get(state->sMeta);
if (!a)
return 0;
state->forceAttrs(*a->value, a->pos, "while evaluating the 'meta' attribute of a derivation");
@@ -220,7 +221,7 @@ bool PackageInfo::checkMeta(Value & v)
return false;
return true;
} else if (v.type() == nAttrs) {
if (v.attrs()->get(state->s.outPath))
if (v.attrs()->get(state->sOutPath))
return false;
for (auto & i : *v.attrs())
if (!checkMeta(*i.value))
@@ -410,7 +411,7 @@ static void getDerivations(
should we recurse into it? => Only if it has a
`recurseForDerivations = true' attribute. */
if (i->value->type() == nAttrs) {
auto j = i->value->attrs()->get(state.s.recurseForDerivations);
auto j = i->value->attrs()->get(state.sRecurseForDerivations);
if (j
&& state.forceBool(
*j->value, j->pos, "while evaluating the attribute `recurseForDerivations`"))
+58 -408
View File
@@ -4,17 +4,11 @@
#include "nix/expr/nixexpr.hh"
#include "nix/expr/symbol-table.hh"
#include <boost/container/static_vector.hpp>
#include <boost/iterator/function_output_iterator.hpp>
#include <algorithm>
#include <functional>
#include <ranges>
#include <optional>
namespace nix {
class EvalMemory;
class EvalState;
struct Value;
/**
@@ -52,466 +46,141 @@ static_assert(
* by its size and its capacity, the capacity being the number of Attr
* elements allocated after this structure, while the size corresponds to
* the number of elements already inserted in this structure.
*
* Bindings can be efficiently `//`-composed into an intrusive linked list of "layers"
* that saves on copies and allocations. Each lookup (@see Bindings::get) traverses
* this linked list until a matching attribute is found (thus overlays earlier in
* the list take precedence). For iteration over the whole Bindings, an on-the-fly
* k-way merge is performed by Bindings::iterator class.
*/
class Bindings
{
public:
using size_type = uint32_t;
typedef uint32_t size_t;
PosIdx pos;
/**
* An instance of bindings objects with 0 attributes.
* This object must never be modified.
*/
static Bindings emptyBindings;
private:
/**
* Number of attributes in the attrs FAM (Flexible Array Member).
*/
size_type numAttrs = 0;
/**
* Number of attributes with unique names in the layer chain.
*
* This is the *real* user-facing size of bindings, whereas @ref numAttrs is
* an implementation detail of the data structure.
*/
size_type numAttrsInChain = 0;
/**
* Length of the layers list.
*/
uint32_t numLayers = 1;
/**
* Bindings that this attrset is "layered" on top of.
*/
const Bindings * baseLayer = nullptr;
/**
* Flexible array member of attributes.
*/
size_t size_, capacity_;
Attr attrs[0];
Bindings() = default;
Bindings(const Bindings &) = delete;
Bindings(Bindings &&) = delete;
Bindings & operator=(const Bindings &) = delete;
Bindings & operator=(Bindings &&) = delete;
Bindings(size_t capacity)
: size_(0)
, capacity_(capacity)
{
}
friend class BindingsBuilder;
/**
* Maximum length of the Bindings layer chains.
*/
static constexpr unsigned maxLayers = 8;
Bindings(const Bindings & bindings) = delete;
public:
size_type size() const
size_t size() const
{
return numAttrsInChain;
return size_;
}
bool empty() const
{
return size() == 0;
return !size_;
}
class iterator
{
public:
using value_type = Attr;
using pointer = const value_type *;
using reference = const value_type &;
using difference_type = std::ptrdiff_t;
using iterator_category = std::forward_iterator_tag;
typedef Attr * iterator;
friend class Bindings;
private:
struct BindingsCursor
{
/**
* Attr that the cursor currently points to.
*/
pointer current;
/**
* One past the end pointer to the contiguous buffer of Attrs.
*/
pointer end;
/**
* Priority of the value. Lesser values have more priority (i.e. they override
* attributes that appear later in the linked list of Bindings).
*/
uint32_t priority;
pointer operator->() const noexcept
{
return current;
}
reference get() const noexcept
{
return *current;
}
bool empty() const noexcept
{
return current == end;
}
void increment() noexcept
{
++current;
}
void consume(Symbol name) noexcept
{
while (!empty() && current->name <= name)
++current;
}
GENERATE_CMP(BindingsCursor, me->current->name, me->priority)
};
using QueueStorageType = boost::container::static_vector<BindingsCursor, maxLayers>;
/**
* Comparator implementing the override priority / name ordering
* for BindingsCursor.
*/
static constexpr auto comp = std::greater<BindingsCursor>();
/**
* A priority queue used to implement an on-the-fly k-way merge.
*/
QueueStorageType cursorHeap;
/**
* The attribute the iterator currently points to.
*/
pointer current = nullptr;
/**
* Whether iterating over a single attribute and not a merge chain.
*/
bool doMerge = true;
void push(BindingsCursor cursor) noexcept
{
cursorHeap.push_back(cursor);
std::ranges::make_heap(cursorHeap, comp);
}
[[nodiscard]] BindingsCursor pop() noexcept
{
std::ranges::pop_heap(cursorHeap, comp);
auto cursor = cursorHeap.back();
cursorHeap.pop_back();
return cursor;
}
iterator & finished() noexcept
{
current = nullptr;
return *this;
}
void next(BindingsCursor cursor) noexcept
{
current = &cursor.get();
cursor.increment();
if (!cursor.empty())
push(cursor);
}
std::optional<BindingsCursor> consumeAllUntilCurrentName() noexcept
{
auto cursor = pop();
Symbol lastHandledName = current->name;
while (cursor->name <= lastHandledName) {
cursor.consume(lastHandledName);
if (!cursor.empty())
push(cursor);
if (cursorHeap.empty())
return std::nullopt;
cursor = pop();
}
return cursor;
}
explicit iterator(const Bindings & attrs) noexcept
: doMerge(attrs.baseLayer)
{
auto pushBindings = [this, priority = unsigned{0}](const Bindings & layer) mutable {
auto first = layer.attrs;
push(
BindingsCursor{
.current = first,
.end = first + layer.numAttrs,
.priority = priority++,
});
};
if (!doMerge) {
if (attrs.empty())
return;
current = attrs.attrs;
pushBindings(attrs);
return;
}
const Bindings * layer = &attrs;
while (layer) {
if (layer->numAttrs != 0)
pushBindings(*layer);
layer = layer->baseLayer;
}
if (cursorHeap.empty())
return;
next(pop());
}
public:
iterator() = default;
reference operator*() const noexcept
{
return *current;
}
pointer operator->() const noexcept
{
return current;
}
iterator & operator++() noexcept
{
if (!doMerge) {
++current;
if (current == cursorHeap.front().end)
return finished();
return *this;
}
if (cursorHeap.empty())
return finished();
auto cursor = consumeAllUntilCurrentName();
if (!cursor)
return finished();
next(*cursor);
return *this;
}
iterator operator++(int) noexcept
{
iterator tmp = *this;
++*this;
return tmp;
}
bool operator==(const iterator & rhs) const noexcept
{
return current == rhs.current;
}
};
using const_iterator = iterator;
typedef const Attr * const_iterator;
void push_back(const Attr & attr)
{
attrs[numAttrs++] = attr;
numAttrsInChain = numAttrs;
assert(size_ < capacity_);
attrs[size_++] = attr;
}
/**
* Get attribute by name or nullptr if no such attribute exists.
*/
const Attr * get(Symbol name) const noexcept
const_iterator find(Symbol name) const
{
auto getInChunk = [key = Attr{name, nullptr}](const Bindings & chunk) -> const Attr * {
auto first = chunk.attrs;
auto last = first + chunk.numAttrs;
const Attr * i = std::lower_bound(first, last, key);
if (i != last && i->name == key.name)
return i;
return nullptr;
};
const Bindings * currentChunk = this;
while (currentChunk) {
const Attr * maybeAttr = getInChunk(*currentChunk);
if (maybeAttr)
return maybeAttr;
currentChunk = currentChunk->baseLayer;
}
Attr key(name, 0);
const_iterator i = std::lower_bound(begin(), end(), key);
if (i != end() && i->name == name)
return i;
return end();
}
const Attr * get(Symbol name) const
{
Attr key(name, 0);
const_iterator i = std::lower_bound(begin(), end(), key);
if (i != end() && i->name == name)
return &*i;
return nullptr;
}
/**
* Check if the layer chain is full.
*/
bool isLayerListFull() const noexcept
iterator begin()
{
return numLayers == Bindings::maxLayers;
return &attrs[0];
}
/**
* Test if the length of the linked list of layers is greater than 1.
*/
bool isLayered() const noexcept
iterator end()
{
return numLayers > 1;
return &attrs[size_];
}
const_iterator begin() const
{
return const_iterator(*this);
return &attrs[0];
}
const_iterator end() const
{
return const_iterator();
return &attrs[size_];
}
Attr & operator[](size_type pos)
Attr & operator[](size_t pos)
{
if (isLayered()) [[unlikely]]
unreachable();
return attrs[pos];
}
const Attr & operator[](size_type pos) const
const Attr & operator[](size_t pos) const
{
if (isLayered()) [[unlikely]]
unreachable();
return attrs[pos];
}
void sort();
size_t capacity() const
{
return capacity_;
}
/**
* Returns the attributes in lexicographically sorted order.
*/
std::vector<const Attr *> lexicographicOrder(const SymbolTable & symbols) const
{
std::vector<const Attr *> res;
res.reserve(size());
std::ranges::transform(*this, std::back_inserter(res), [](const Attr & a) { return &a; });
std::ranges::sort(res, [&](const Attr * a, const Attr * b) {
res.reserve(size_);
for (size_t n = 0; n < size_; n++)
res.emplace_back(&attrs[n]);
std::sort(res.begin(), res.end(), [&](const Attr * a, const Attr * b) {
std::string_view sa = symbols[a->name], sb = symbols[b->name];
return sa < sb;
});
return res;
}
friend class EvalMemory;
friend class EvalState;
};
static_assert(std::forward_iterator<Bindings::iterator>);
static_assert(std::ranges::forward_range<Bindings>);
/**
* A wrapper around Bindings that ensures that its always in sorted
* order at the end. The only way to consume a BindingsBuilder is to
* call finish(), which sorts the bindings.
*/
class BindingsBuilder final
class BindingsBuilder
{
Bindings * bindings;
public:
// needed by std::back_inserter
using value_type = Attr;
using size_type = Bindings::size_type;
private:
Bindings * bindings;
Bindings::size_type capacity_;
EvalState & state;
friend class EvalMemory;
BindingsBuilder(EvalMemory & mem, SymbolTable & symbols, Bindings * bindings, size_type capacity)
BindingsBuilder(EvalState & state, Bindings * bindings)
: bindings(bindings)
, capacity_(capacity)
, mem(mem)
, symbols(symbols)
, state(state)
{
}
bool hasBaseLayer() const noexcept
{
return bindings->baseLayer;
}
/**
* If the bindings gets "layered" on top of another we need to recalculate
* the number of unique attributes in the chain.
*
* This is done by either iterating over the base "layer" and the newly added
* attributes and counting duplicates. If the base "layer" is big this approach
* is inefficient and we fall back to doing per-element binary search in the base
* "layer".
*/
void finishSizeIfNecessary()
{
if (!hasBaseLayer())
return;
auto & base = *bindings->baseLayer;
auto attrs = std::span(bindings->attrs, bindings->numAttrs);
Bindings::size_type duplicates = 0;
/* If the base bindings is smaller than the newly added attributes
iterate using std::set_intersection to run in O(|base| + |attrs|) =
O(|attrs|). Otherwise use an O(|attrs| * log(|base|)) per-attr binary
search to check for duplicates. Note that if we are in this code path then
|attrs| <= bindingsUpdateLayerRhsSizeThreshold, which 16 by default. We are
optimizing for the case when a small attribute set gets "layered" on top of
a much larger one. When attrsets are already small it's fine to do a linear
scan, but we should avoid expensive iterations over large "base" attrsets. */
if (attrs.size() > base.size()) {
std::set_intersection(
base.begin(),
base.end(),
attrs.begin(),
attrs.end(),
boost::make_function_output_iterator([&]([[maybe_unused]] auto && _) { ++duplicates; }));
} else {
for (const auto & attr : attrs) {
if (base.get(attr.name))
++duplicates;
}
}
bindings->numAttrsInChain = base.numAttrsInChain + attrs.size() - duplicates;
}
public:
std::reference_wrapper<EvalMemory> mem;
std::reference_wrapper<SymbolTable> symbols;
void insert(Symbol name, Value * value, PosIdx pos = noPos)
{
insert(Attr(name, value, pos));
@@ -524,26 +193,9 @@ public:
void push_back(const Attr & attr)
{
assert(bindings->numAttrs < capacity_);
bindings->push_back(attr);
}
/**
* "Layer" the newly constructured Bindings on top of another attribute set.
*
* This effectively performs an attribute set merge, while giving preference
* to attributes from the newly constructed Bindings in case of duplicate attribute
* names.
*
* This operation amortizes the need to copy over all attributes and allows
* for efficient implementation of attribute set merges (ExprOpUpdate::eval).
*/
void layerOnTopOf(const Bindings & base) noexcept
{
bindings->baseLayer = &base;
bindings->numLayers = base.numLayers + 1;
}
Value & alloc(Symbol name, PosIdx pos = noPos);
Value & alloc(std::string_view name, PosIdx pos = noPos);
@@ -551,26 +203,24 @@ public:
Bindings * finish()
{
bindings->sort();
finishSizeIfNecessary();
return bindings;
}
Bindings * alreadySorted()
{
finishSizeIfNecessary();
return bindings;
}
size_t capacity() const noexcept
size_t capacity()
{
return capacity_;
return bindings->capacity();
}
void grow(BindingsBuilder newBindings)
void grow(Bindings * newBindings)
{
for (auto & i : *bindings)
newBindings.push_back(i);
std::swap(*this, newBindings);
newBindings->push_back(i);
bindings = newBindings;
}
friend struct ExprAttrs;
-70
View File
@@ -1,70 +0,0 @@
#pragma once
#include <atomic>
#include <cstdint>
namespace nix {
/**
* An atomic counter aligned on a cache line to prevent false sharing.
* The counter is only enabled when the `NIX_SHOW_STATS` environment
* variable is set. This is to prevent contention on these counters
* when multi-threaded evaluation is enabled.
*/
struct alignas(64) Counter
{
using value_type = uint64_t;
std::atomic<value_type> inner{0};
static bool enabled;
Counter() {}
operator value_type() const noexcept
{
return inner;
}
void operator=(value_type n) noexcept
{
inner = n;
}
value_type load() const noexcept
{
return inner;
}
value_type operator++() noexcept
{
return enabled ? ++inner : 0;
}
value_type operator++(int) noexcept
{
return enabled ? inner++ : 0;
}
value_type operator--() noexcept
{
return enabled ? --inner : 0;
}
value_type operator--(int) noexcept
{
return enabled ? inner-- : 0;
}
value_type operator+=(value_type n) noexcept
{
return enabled ? inner += n : 0;
}
value_type operator-=(value_type n) noexcept
{
return enabled ? inner -= n : 0;
}
};
} // namespace nix
+5 -5
View File
@@ -26,7 +26,7 @@ inline void * allocBytes(size_t n)
}
[[gnu::always_inline]]
Value * EvalMemory::allocValue()
Value * EvalState::allocValue()
{
#if NIX_USE_BOEHMGC
/* We use the boehm batch allocator to speed up allocations of Values (of which there are many).
@@ -48,15 +48,15 @@ Value * EvalMemory::allocValue()
void * p = allocBytes(sizeof(Value));
#endif
stats.nrValues++;
nrValues++;
return (Value *) p;
}
[[gnu::always_inline]]
Env & EvalMemory::allocEnv(size_t size)
Env & EvalState::allocEnv(size_t size)
{
stats.nrEnvs++;
stats.nrValuesInEnvs += size;
nrEnvs++;
nrValuesInEnvs += size;
Env * env;
@@ -342,25 +342,6 @@ struct EvalSettings : Config
This is useful for improving code readability and making path literals
more explicit.
)"};
Setting<unsigned> bindingsUpdateLayerRhsSizeThreshold{
this,
sizeof(void *) == 4 ? 8192 : 16,
"eval-attrset-update-layer-rhs-threshold",
R"(
Tunes the maximum size of an attribute set that, when used
as a right operand in an [attribute set update expression](@docroot@/language/operators.md#update),
uses a more space-efficient linked-list representation of attribute sets.
Setting this to larger values generally leads to less memory allocations,
but may lead to worse evaluation performance.
A value of `0` disables this optimization completely.
This is an advanced performance tuning option and typically should not be changed.
The default value is chosen to balance performance and memory usage. On 32 bit systems
where memory is scarce, the default is a large value to reduce the amount of allocations.
)"};
};
/**
+112 -207
View File
@@ -16,14 +16,10 @@
#include "nix/expr/search-path.hh"
#include "nix/expr/repl-exit-status.hh"
#include "nix/util/ref.hh"
#include "nix/expr/counter.hh"
// For `NIX_USE_BOEHMGC`, and if that's set, `GC_THREADS`
#include "nix/expr/config.hh"
#include <boost/unordered/unordered_flat_map.hpp>
#include <boost/unordered/concurrent_flat_map_fwd.hpp>
#include <map>
#include <optional>
#include <functional>
@@ -42,7 +38,6 @@ class Store;
namespace fetchers {
struct Settings;
struct InputCache;
struct Input;
} // namespace fetchers
struct EvalSettings;
class EvalState;
@@ -50,7 +45,6 @@ class StorePath;
struct SingleDerivedPath;
enum RepairFlag : bool;
struct MemorySourceAccessor;
struct MountedSourceAccessor;
namespace eval_cache {
class EvalCache;
@@ -168,7 +162,7 @@ typedef std::
map<std::string, Value *, std::less<std::string>, traceable_allocator<std::pair<const std::string, Value *>>>
ValMap;
typedef boost::unordered_flat_map<PosIdx, DocComment, std::hash<PosIdx>> DocCommentMap;
typedef std::unordered_map<PosIdx, DocComment> DocCommentMap;
struct Env
{
@@ -219,163 +213,22 @@ struct DebugTrace
}
};
struct StaticEvalSymbols
{
Symbol with, outPath, drvPath, type, meta, name, value, system, overrides, outputs, outputName, ignoreNulls, file,
line, column, functor, toString, right, wrong, structuredAttrs, json, allowedReferences, allowedRequisites,
disallowedReferences, disallowedRequisites, maxSize, maxClosureSize, builder, args, contentAddressed, impure,
outputHash, outputHashAlgo, outputHashMode, recurseForDerivations, description, self, epsilon, startSet,
operator_, key, path, prefix, outputSpecified;
Expr::AstSymbols exprSymbols;
static constexpr auto preallocate()
{
StaticSymbolTable alloc;
StaticEvalSymbols staticSymbols = {
.with = alloc.create("<with>"),
.outPath = alloc.create("outPath"),
.drvPath = alloc.create("drvPath"),
.type = alloc.create("type"),
.meta = alloc.create("meta"),
.name = alloc.create("name"),
.value = alloc.create("value"),
.system = alloc.create("system"),
.overrides = alloc.create("__overrides"),
.outputs = alloc.create("outputs"),
.outputName = alloc.create("outputName"),
.ignoreNulls = alloc.create("__ignoreNulls"),
.file = alloc.create("file"),
.line = alloc.create("line"),
.column = alloc.create("column"),
.functor = alloc.create("__functor"),
.toString = alloc.create("__toString"),
.right = alloc.create("right"),
.wrong = alloc.create("wrong"),
.structuredAttrs = alloc.create("__structuredAttrs"),
.json = alloc.create("__json"),
.allowedReferences = alloc.create("allowedReferences"),
.allowedRequisites = alloc.create("allowedRequisites"),
.disallowedReferences = alloc.create("disallowedReferences"),
.disallowedRequisites = alloc.create("disallowedRequisites"),
.maxSize = alloc.create("maxSize"),
.maxClosureSize = alloc.create("maxClosureSize"),
.builder = alloc.create("builder"),
.args = alloc.create("args"),
.contentAddressed = alloc.create("__contentAddressed"),
.impure = alloc.create("__impure"),
.outputHash = alloc.create("outputHash"),
.outputHashAlgo = alloc.create("outputHashAlgo"),
.outputHashMode = alloc.create("outputHashMode"),
.recurseForDerivations = alloc.create("recurseForDerivations"),
.description = alloc.create("description"),
.self = alloc.create("self"),
.epsilon = alloc.create(""),
.startSet = alloc.create("startSet"),
.operator_ = alloc.create("operator"),
.key = alloc.create("key"),
.path = alloc.create("path"),
.prefix = alloc.create("prefix"),
.outputSpecified = alloc.create("outputSpecified"),
.exprSymbols = {
.sub = alloc.create("__sub"),
.lessThan = alloc.create("__lessThan"),
.mul = alloc.create("__mul"),
.div = alloc.create("__div"),
.or_ = alloc.create("or"),
.findFile = alloc.create("__findFile"),
.nixPath = alloc.create("__nixPath"),
.body = alloc.create("body"),
}};
return std::pair{staticSymbols, alloc};
}
static consteval StaticEvalSymbols create()
{
return preallocate().first;
}
static constexpr StaticSymbolTable staticSymbolTable()
{
return preallocate().second;
}
};
class EvalMemory
{
#if NIX_USE_BOEHMGC
/**
* Allocation cache for GC'd Value objects.
*/
std::shared_ptr<void *> valueAllocCache;
/**
* Allocation cache for size-1 Env objects.
*/
std::shared_ptr<void *> env1AllocCache;
#endif
public:
struct Statistics
{
Counter nrEnvs;
Counter nrValuesInEnvs;
Counter nrValues;
Counter nrAttrsets;
Counter nrAttrsInAttrsets;
Counter nrListElems;
};
EvalMemory();
EvalMemory(const EvalMemory &) = delete;
EvalMemory(EvalMemory &&) = delete;
EvalMemory & operator=(const EvalMemory &) = delete;
EvalMemory & operator=(EvalMemory &&) = delete;
inline Value * allocValue();
inline Env & allocEnv(size_t size);
Bindings * allocBindings(size_t capacity);
BindingsBuilder buildBindings(SymbolTable & symbols, size_t capacity)
{
return BindingsBuilder(*this, symbols, allocBindings(capacity), capacity);
}
ListBuilder buildList(size_t size)
{
stats.nrListElems += size;
return ListBuilder(size);
}
const Statistics & getStats() const &
{
return stats;
}
/**
* Storage for the AST nodes
*/
Exprs exprs;
private:
Statistics stats;
};
class EvalState : public std::enable_shared_from_this<EvalState>
{
public:
static constexpr StaticEvalSymbols s = StaticEvalSymbols::create();
const fetchers::Settings & fetchSettings;
const EvalSettings & settings;
SymbolTable symbols;
PosTable positions;
EvalMemory mem;
const Symbol sWith, sOutPath, sDrvPath, sType, sMeta, sName, sValue, sSystem, sOverrides, sOutputs, sOutputName,
sIgnoreNulls, sFile, sLine, sColumn, sFunctor, sToString, sRight, sWrong, sStructuredAttrs, sJson,
sAllowedReferences, sAllowedRequisites, sDisallowedReferences, sDisallowedRequisites, sMaxSize, sMaxClosureSize,
sBuilder, sArgs, sContentAddressed, sImpure, sOutputHash, sOutputHashAlgo, sOutputHashMode,
sRecurseForDerivations, sDescription, sSelf, sEpsilon, sStartSet, sOperator, sKey, sPath, sPrefix,
sOutputSpecified;
const Expr::AstSymbols exprSymbols;
/**
* If set, force copying files to the Nix store even if they
@@ -383,10 +236,47 @@ public:
*/
RepairFlag repair;
Bindings emptyBindings;
/**
* Empty list constant.
*/
Value vEmptyList;
/**
* `null` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vNull;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vTrue;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
Value vFalse;
/** `"regular"` */
Value vStringRegular;
/** `"directory"` */
Value vStringDirectory;
/** `"symlink"` */
Value vStringSymlink;
/** `"unknown"` */
Value vStringUnknown;
/**
* The accessor corresponding to `store`.
*/
const ref<MountedSourceAccessor> storeFS;
const ref<SourceAccessor> storeFS;
/**
* The accessor for the root filesystem.
@@ -428,7 +318,7 @@ public:
bool inDebugger = false;
int trylevel;
std::list<DebugTrace> debugTraces;
boost::unordered_flat_map<const Expr *, const std::shared_ptr<const StaticEnv>> exprEnvs;
std::map<const Expr *, const std::shared_ptr<const StaticEnv>> exprEnvs;
const std::shared_ptr<const StaticEnv> getStaticEnv(const Expr & expr) const
{
@@ -471,41 +361,59 @@ private:
/* Cache for calls to addToStore(); maps source paths to the store
paths. */
ref<boost::concurrent_flat_map<SourcePath, StorePath>> srcToStore;
Sync<std::unordered_map<SourcePath, StorePath>> srcToStore;
/**
* A cache that maps paths to "resolved" paths for importing Nix
* expressions, i.e. `/foo` to `/foo/default.nix`.
* A cache from path names to parse trees.
*/
ref<boost::concurrent_flat_map<SourcePath, SourcePath>> importResolutionCache;
/**
* A cache from resolved paths to values.
*/
ref<boost::concurrent_flat_map<
typedef std::unordered_map<
SourcePath,
Value *,
Expr *,
std::hash<SourcePath>,
std::equal_to<SourcePath>,
traceable_allocator<std::pair<const SourcePath, Value *>>>>
fileEvalCache;
traceable_allocator<std::pair<const SourcePath, Expr *>>>
FileParseCache;
FileParseCache fileParseCache;
/**
* A cache from path names to values.
*/
typedef std::unordered_map<
SourcePath,
Value,
std::hash<SourcePath>,
std::equal_to<SourcePath>,
traceable_allocator<std::pair<const SourcePath, Value>>>
FileEvalCache;
FileEvalCache fileEvalCache;
/**
* Associate source positions of certain AST nodes with their preceding doc comment, if they have one.
* Grouped by file.
*/
boost::unordered_flat_map<SourcePath, DocCommentMap> positionToDocComment;
std::unordered_map<SourcePath, DocCommentMap> positionToDocComment;
LookupPath lookupPath;
boost::unordered_flat_map<std::string, std::optional<SourcePath>, StringViewHash, std::equal_to<>>
lookupPathResolved;
std::map<std::string, std::optional<SourcePath>> lookupPathResolved;
/**
* Cache used by prim_match().
*/
std::shared_ptr<RegexCache> regexCache;
#if NIX_USE_BOEHMGC
/**
* Allocation cache for GC'd Value objects.
*/
std::shared_ptr<void *> valueAllocCache;
/**
* Allocation cache for size-1 Env objects.
*/
std::shared_ptr<void *> env1AllocCache;
#endif
public:
EvalState(
@@ -516,15 +424,6 @@ public:
std::shared_ptr<Store> buildStore = nullptr);
~EvalState();
/**
* A wrapper around EvalMemory::allocValue() to avoid code churn when it
* was introduced.
*/
inline Value * allocValue()
{
return mem.allocValue();
}
LookupPath getLookupPath()
{
return lookupPath;
@@ -552,11 +451,8 @@ public:
/**
* Allow access to a path.
*
* Only for restrict eval: pure eval just whitelist store paths,
* never arbitrary paths.
*/
void allowPathLegacy(const Path & path);
void allowPath(const Path & path);
/**
* Allow access to a store path. Note that this gets remapped to
@@ -576,11 +472,6 @@ public:
void checkURI(const std::string & uri);
/**
* Mount an input on the Nix store.
*/
StorePath mountInput(fetchers::Input & input, const fetchers::Input & originalInput, ref<SourceAccessor> accessor);
/**
* Parse a Nix expression from the specified file.
*/
@@ -679,7 +570,7 @@ public:
/**
* Get attribute from an attribute set and throw an error if it doesn't exist.
*/
const Attr * getAttr(Symbol attrSym, const Bindings * attrSet, std::string_view errorCtx);
Bindings::const_iterator getAttr(Symbol attrSym, const Bindings * attrSet, std::string_view errorCtx);
template<typename... Args>
[[gnu::noinline]]
@@ -778,11 +669,11 @@ public:
/**
* Internal primops not exposed to the user.
*/
boost::unordered_flat_map<
std::unordered_map<
std::string,
Value *,
StringViewHash,
std::equal_to<>,
std::hash<std::string>,
std::equal_to<std::string>,
traceable_allocator<std::pair<const std::string, Value *>>>
internalPrimOps;
@@ -901,14 +792,22 @@ public:
*/
void autoCallFunction(const Bindings & args, Value & fun, Value & res);
/**
* Allocation primitives.
*/
inline Value * allocValue();
inline Env & allocEnv(size_t size);
Bindings * allocBindings(size_t capacity);
BindingsBuilder buildBindings(size_t capacity)
{
return mem.buildBindings(symbols, capacity);
return BindingsBuilder(*this, allocBindings(capacity));
}
ListBuilder buildList(size_t size)
{
return mem.buildList(size);
return ListBuilder(*this, size);
}
/**
@@ -1025,20 +924,26 @@ private:
*/
std::string mkSingleDerivedPathStringRaw(const SingleDerivedPath & p);
Counter nrLookups;
Counter nrAvoided;
Counter nrOpUpdates;
Counter nrOpUpdateValuesCopied;
Counter nrListConcats;
Counter nrPrimOpCalls;
Counter nrFunctionCalls;
unsigned long nrEnvs = 0;
unsigned long nrValuesInEnvs = 0;
unsigned long nrValues = 0;
unsigned long nrListElems = 0;
unsigned long nrLookups = 0;
unsigned long nrAttrsets = 0;
unsigned long nrAttrsInAttrsets = 0;
unsigned long nrAvoided = 0;
unsigned long nrOpUpdates = 0;
unsigned long nrOpUpdateValuesCopied = 0;
unsigned long nrListConcats = 0;
unsigned long nrPrimOpCalls = 0;
unsigned long nrFunctionCalls = 0;
bool countCalls;
typedef boost::unordered_flat_map<std::string, size_t, StringViewHash, std::equal_to<>> PrimOpCalls;
typedef std::map<std::string, size_t> PrimOpCalls;
PrimOpCalls primOpCalls;
typedef boost::unordered_flat_map<ExprLambda *, size_t> FunctionCalls;
typedef std::map<ExprLambda *, size_t> FunctionCalls;
FunctionCalls functionCalls;
/** Evaluation/call profiler. */
@@ -1046,7 +951,7 @@ private:
void incrFunctionCall(ExprLambda * fun);
typedef boost::unordered_flat_map<PosIdx, size_t, std::hash<PosIdx>> AttrSelects;
typedef std::map<PosIdx, size_t> AttrSelects;
AttrSelects attrSelects;
friend struct ExprOpUpdate;
@@ -26,20 +26,4 @@ using SmallValueVector = SmallVector<Value *, nItems>;
template<size_t nItems>
using SmallTemporaryValueVector = SmallVector<Value, nItems>;
/**
* For functions where we do not expect deep recursion, we can use a sizable
* part of the stack a free allocation space.
*
* Note: this is expected to be multiplied by sizeof(Value), or about 24 bytes.
*/
constexpr size_t nonRecursiveStackReservation = 128;
/**
* Functions that maybe applied to self-similar inputs, such as concatMap on a
* tree, should reserve a smaller part of the stack for allocation.
*
* Note: this is expected to be multiplied by sizeof(Value), or about 24 bytes.
*/
constexpr size_t conservativeStackReservation = 16;
} // namespace nix
-1
View File
@@ -10,7 +10,6 @@ config_pub_h = configure_file(
headers = [ config_pub_h ] + files(
'attr-path.hh',
'attr-set.hh',
'counter.hh',
'eval-cache.hh',
'eval-error.hh',
'eval-gc.hh',
+49 -125
View File
@@ -2,17 +2,12 @@
///@file
#include <map>
#include <span>
#include <vector>
#include <memory_resource>
#include <algorithm>
#include "nix/expr/gc-small-vector.hh"
#include "nix/expr/value.hh"
#include "nix/expr/symbol-table.hh"
#include "nix/expr/eval-error.hh"
#include "nix/util/pos-idx.hh"
#include "nix/expr/counter.hh"
namespace nix {
@@ -81,20 +76,9 @@ struct AttrName
: expr(e) {};
};
static_assert(std::is_trivially_copy_constructible_v<AttrName>);
typedef std::vector<AttrName> AttrPath;
std::string showAttrPath(const SymbolTable & symbols, std::span<const AttrName> attrPath);
using UpdateQueue = SmallTemporaryValueVector<conservativeStackReservation>;
class Exprs
{
std::pmr::monotonic_buffer_resource buffer;
public:
std::pmr::polymorphic_allocator<char> alloc{&buffer};
};
std::string showAttrPath(const SymbolTable & symbols, const AttrPath & attrPath);
/* Abstract syntax of Nix expressions. */
@@ -105,7 +89,7 @@ struct Expr
Symbol sub, lessThan, mul, div, or_, findFile, nixPath, body;
};
static Counter nrExprs;
static unsigned long nrExprs;
Expr()
{
@@ -115,25 +99,8 @@ struct Expr
virtual ~Expr() {};
virtual void show(const SymbolTable & symbols, std::ostream & str) const;
virtual void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env);
/** Normal evaluation, implemented directly by all subclasses. */
virtual void eval(EvalState & state, Env & env, Value & v);
/**
* Create a thunk for the delayed computation of the given expression
* in the given environment. But if the expression is a variable,
* then look it up right away. This significantly reduces the number
* of thunks allocated.
*/
virtual Value * maybeThunk(EvalState & state, Env & env);
/**
* Only called when performing an attrset update: `//` or similar.
* Instead of writing to a Value &, this function writes to an UpdateQueue.
* This allows the expression to perform multiple updates in a delayed manner, gathering up all the updates before
* applying them.
*/
virtual void evalForUpdate(EvalState & state, Env & env, UpdateQueue & q, std::string_view errorCtx);
virtual void setName(Symbol name);
virtual void setDocComment(DocComment docComment) {};
@@ -191,7 +158,7 @@ struct ExprString : Expr
ExprString(std::string && s)
: s(std::move(s))
{
v.mkStringNoCopy(this->s.data());
v.mkString(this->s.data());
};
Value * maybeThunk(EvalState & state, Env & env) override;
@@ -201,16 +168,14 @@ struct ExprString : Expr
struct ExprPath : Expr
{
ref<SourceAccessor> accessor;
std::string s;
Value v;
ExprPath(std::pmr::polymorphic_allocator<char> & alloc, ref<SourceAccessor> accessor, std::string_view sv)
ExprPath(ref<SourceAccessor> accessor, std::string s)
: accessor(accessor)
, s(std::move(s))
{
auto len = sv.length();
char * s = alloc.allocate(len + 1);
sv.copy(s, len);
s[len] = '\0';
v.mkPath(&*accessor, s);
v.mkPath(&*accessor, this->s.c_str());
}
Value * maybeThunk(EvalState & state, Env & env) override;
@@ -277,33 +242,20 @@ struct ExprInheritFrom : ExprVar
struct ExprSelect : Expr
{
PosIdx pos;
uint32_t nAttrPath;
Expr *e, *def;
AttrName * attrPathStart;
ExprSelect(
std::pmr::polymorphic_allocator<char> & alloc,
const PosIdx & pos,
Expr * e,
std::span<const AttrName> attrPath,
Expr * def)
AttrPath attrPath;
ExprSelect(const PosIdx & pos, Expr * e, AttrPath attrPath, Expr * def)
: pos(pos)
, nAttrPath(attrPath.size())
, e(e)
, def(def)
, attrPathStart(alloc.allocate_object<AttrName>(nAttrPath))
{
std::ranges::copy(attrPath, attrPathStart);
};
, attrPath(std::move(attrPath)) {};
ExprSelect(std::pmr::polymorphic_allocator<char> & alloc, const PosIdx & pos, Expr * e, Symbol name)
ExprSelect(const PosIdx & pos, Expr * e, Symbol name)
: pos(pos)
, nAttrPath(1)
, e(e)
, def(0)
, attrPathStart((alloc.allocate_object<AttrName>()))
{
*attrPathStart = AttrName(name);
attrPath.push_back(AttrName(name));
};
PosIdx getPos() const override
@@ -311,11 +263,6 @@ struct ExprSelect : Expr
return pos;
}
std::span<const AttrName> getAttrPath() const
{
return {attrPathStart, nAttrPath};
}
/**
* Evaluate the `a.b.c` part of `a.b.c.d`. This exists mostly for the purpose of :doc in the repl.
*
@@ -333,14 +280,10 @@ struct ExprSelect : Expr
struct ExprOpHasAttr : Expr
{
Expr * e;
std::span<AttrName> attrPath;
ExprOpHasAttr(std::pmr::polymorphic_allocator<char> & alloc, Expr * e, std::vector<AttrName> attrPath)
AttrPath attrPath;
ExprOpHasAttr(Expr * e, AttrPath attrPath)
: e(e)
, attrPath({alloc.allocate_object<AttrName>(attrPath.size()), attrPath.size()})
{
std::ranges::copy(attrPath, this->attrPath.begin());
};
, attrPath(std::move(attrPath)) {};
PosIdx getPos() const override
{
@@ -622,61 +565,42 @@ struct ExprOpNot : Expr
COMMON_METHODS
};
#define MakeBinOpMembers(name, s) \
PosIdx pos; \
Expr *e1, *e2; \
name(Expr * e1, Expr * e2) \
: e1(e1) \
, e2(e2){}; \
name(const PosIdx & pos, Expr * e1, Expr * e2) \
: pos(pos) \
, e1(e1) \
, e2(e2){}; \
void show(const SymbolTable & symbols, std::ostream & str) const override \
{ \
str << "("; \
e1->show(symbols, str); \
str << " " s " "; \
e2->show(symbols, str); \
str << ")"; \
} \
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env) override \
{ \
e1->bindVars(es, env); \
e2->bindVars(es, env); \
} \
void eval(EvalState & state, Env & env, Value & v) override; \
PosIdx getPos() const override \
{ \
return pos; \
}
#define MakeBinOp(name, s) \
struct name : Expr \
{ \
PosIdx pos; \
Expr *e1, *e2; \
name(Expr * e1, Expr * e2) \
: e1(e1) \
, e2(e2) {}; \
name(const PosIdx & pos, Expr * e1, Expr * e2) \
: pos(pos) \
, e1(e1) \
, e2(e2) {}; \
void show(const SymbolTable & symbols, std::ostream & str) const override \
{ \
str << "("; \
e1->show(symbols, str); \
str << " " s " "; \
e2->show(symbols, str); \
str << ")"; \
} \
void bindVars(EvalState & es, const std::shared_ptr<const StaticEnv> & env) override \
{ \
e1->bindVars(es, env); \
e2->bindVars(es, env); \
} \
void eval(EvalState & state, Env & env, Value & v) override; \
PosIdx getPos() const override \
{ \
return pos; \
} \
};
#define MakeBinOp(name, s) \
struct name : Expr \
{ \
MakeBinOpMembers(name, s) \
}
MakeBinOp(ExprOpEq, "==") MakeBinOp(ExprOpNEq, "!=") MakeBinOp(ExprOpAnd, "&&") MakeBinOp(ExprOpOr, "||")
MakeBinOp(ExprOpImpl, "->") MakeBinOp(ExprOpUpdate, "//") MakeBinOp(ExprOpConcatLists, "++")
MakeBinOp(ExprOpEq, "==");
MakeBinOp(ExprOpNEq, "!=");
MakeBinOp(ExprOpAnd, "&&");
MakeBinOp(ExprOpOr, "||");
MakeBinOp(ExprOpImpl, "->");
MakeBinOp(ExprOpConcatLists, "++");
struct ExprOpUpdate : Expr
{
private:
/** Special case for merging of two attrsets. */
void eval(EvalState & state, Value & v, Value & v1, Value & v2);
void evalForUpdate(EvalState & state, Env & env, UpdateQueue & q);
public:
MakeBinOpMembers(ExprOpUpdate, "//");
virtual void evalForUpdate(EvalState & state, Env & env, UpdateQueue & q, std::string_view errorCtx) override;
};
struct ExprConcatStrings : Expr
struct ExprConcatStrings : Expr
{
PosIdx pos;
bool forceString;
+6 -3
View File
@@ -24,6 +24,7 @@ struct StringToken
}
};
// This type must be trivially copyable; see YYLTYPE_IS_TRIVIAL in parser.y.
struct ParserLocation
{
int beginOffset;
@@ -43,6 +44,9 @@ struct ParserLocation
beginOffset = stashedBeginOffset;
endOffset = stashedEndOffset;
}
/** Latest doc comment position, or 0. */
int doc_comment_first_column, doc_comment_last_column;
};
struct LexerState
@@ -67,7 +71,7 @@ struct LexerState
/**
* @brief Maps some positions to a DocComment, where the comment is relevant to the location.
*/
DocCommentMap & positionToDocComment;
std::unordered_map<PosIdx, DocComment> & positionToDocComment;
PosTable & positions;
PosTable::Origin origin;
@@ -78,14 +82,13 @@ struct LexerState
struct ParserState
{
const LexerState & lexerState;
std::pmr::polymorphic_allocator<char> & alloc;
SymbolTable & symbols;
PosTable & positions;
Expr * result;
SourcePath basePath;
PosTable::Origin origin;
const ref<SourceAccessor> rootFS;
static constexpr Expr::AstSymbols s = StaticEvalSymbols::create().exprSymbols;
const Expr::AstSymbols & s;
const EvalSettings & settings;
void dupAttr(const AttrPath & attrPath, const PosIdx pos, const PosIdx prevPos);
+21 -1
View File
@@ -8,11 +8,31 @@
namespace nix {
/**
* For functions where we do not expect deep recursion, we can use a sizable
* part of the stack a free allocation space.
*
* Note: this is expected to be multiplied by sizeof(Value), or about 24 bytes.
*/
constexpr size_t nonRecursiveStackReservation = 128;
/**
* Functions that maybe applied to self-similar inputs, such as concatMap on a
* tree, should reserve a smaller part of the stack for allocation.
*
* Note: this is expected to be multiplied by sizeof(Value), or about 24 bytes.
*/
constexpr size_t conservativeStackReservation = 16;
struct RegisterPrimOp
{
typedef std::vector<PrimOp> PrimOps;
static PrimOps & primOps();
static PrimOps & primOps()
{
static PrimOps primOps;
return primOps;
}
/**
* You can register a constant by passing an arity of 0. fun
@@ -110,7 +110,7 @@ struct PrintOptions
* `PrintOptions` for unknown and therefore potentially large values in error messages,
* to avoid printing "too much" output.
*/
static constexpr PrintOptions errorPrintOptions = PrintOptions{
static PrintOptions errorPrintOptions = PrintOptions{
.ansiColors = true,
.maxDepth = 10,
.maxAttrs = 10,
+11 -62
View File
@@ -28,8 +28,6 @@ public:
}
};
class StaticSymbolTable;
/**
* Symbols have the property that they can be compared efficiently
* (using an equality test), because the symbol table stores only one
@@ -39,38 +37,36 @@ class Symbol
{
friend class SymbolStr;
friend class SymbolTable;
friend class StaticSymbolTable;
private:
uint32_t id;
explicit constexpr Symbol(uint32_t id) noexcept
explicit Symbol(uint32_t id) noexcept
: id(id)
{
}
public:
constexpr Symbol() noexcept
Symbol() noexcept
: id(0)
{
}
[[gnu::always_inline]]
constexpr explicit operator bool() const noexcept
explicit operator bool() const noexcept
{
return id > 0;
}
/**
* The ID is a private implementation detail that should generally not be observed. However, we expose here just for
* sake of `switch...case`, which needs to dispatch on numbers. */
[[gnu::always_inline]]
constexpr uint32_t getId() const noexcept
auto operator<=>(const Symbol other) const noexcept
{
return id;
return id <=> other.id;
}
constexpr auto operator<=>(const Symbol & other) const noexcept = default;
bool operator==(const Symbol other) const noexcept
{
return id == other.id;
}
friend class std::hash<Symbol>;
};
@@ -122,12 +118,12 @@ public:
// for multi-threaded implementations: lock store and allocator here
const auto & [v, idx] = key.store.add(SymbolValue{});
if (size == 0) {
v.mkStringNoCopy("", nullptr);
v.mkString("", nullptr);
} else {
auto s = key.alloc.allocate(size + 1);
memcpy(s, key.s.data(), size);
s[size] = '\0';
v.mkStringNoCopy(s, nullptr);
v.mkString(s, nullptr);
}
v.size_ = size;
v.idx = idx;
@@ -214,39 +210,6 @@ public:
};
};
class SymbolTable;
/**
* Convenience class to statically assign symbol identifiers at compile-time.
*/
class StaticSymbolTable
{
static constexpr std::size_t maxSize = 1024;
struct StaticSymbolInfo
{
std::string_view str;
Symbol sym;
};
std::array<StaticSymbolInfo, maxSize> symbols;
std::size_t size = 0;
public:
constexpr StaticSymbolTable() = default;
constexpr Symbol create(std::string_view str)
{
/* No need to check bounds because out of bounds access is
a compilation error. */
auto sym = Symbol(size + 1); //< +1 because Symbol with id = 0 is reserved
symbols[size++] = {str, sym};
return sym;
}
void copyIntoSymbolTable(SymbolTable & symtab) const;
};
/**
* Symbol table used by the parser and evaluator to represent and look
* up identifiers and attributes efficiently.
@@ -269,10 +232,6 @@ private:
boost::unordered_flat_set<SymbolStr, SymbolStr::Hash, SymbolStr::Equal> symbols{SymbolStr::chunkSize};
public:
SymbolTable(const StaticSymbolTable & staticSymtab)
{
staticSymtab.copyIntoSymbolTable(*this);
}
/**
* Converts a string into a symbol.
@@ -317,16 +276,6 @@ public:
}
};
inline void StaticSymbolTable::copyIntoSymbolTable(SymbolTable & symtab) const
{
for (std::size_t i = 0; i < size; ++i) {
auto [str, staticSym] = symbols[i];
auto sym = symtab.create(str);
if (sym != staticSym) [[unlikely]]
unreachable();
}
}
} // namespace nix
template<>
+9 -47
View File
@@ -12,7 +12,6 @@
#include "nix/expr/print-options.hh"
#include "nix/util/checked-arithmetic.hh"
#include <boost/unordered/unordered_flat_map_fwd.hpp>
#include <nlohmann/json_fwd.hpp>
namespace nix {
@@ -155,7 +154,7 @@ class ListBuilder
Value * inlineElems[2] = {nullptr, nullptr};
public:
Value ** elems;
ListBuilder(size_t size);
ListBuilder(EvalState & state, size_t size);
// NOTE: Can be noexcept because we are just copying integral values and
// raw pointers.
@@ -370,7 +369,7 @@ namespace detail {
/* Whether to use a specialization of ValueStorage that does bitpacking into
alignment niches. */
template<std::size_t ptrSize>
inline constexpr bool useBitPackedValueStorage = (ptrSize == 8) && (__STDCPP_DEFAULT_NEW_ALIGNMENT__ >= 16);
inline constexpr bool useBitPackedValueStorage = (ptrSize == 8) && (__STDCPP_DEFAULT_NEW_ALIGNMENT__ >= 8);
} // namespace detail
@@ -379,8 +378,7 @@ inline constexpr bool useBitPackedValueStorage = (ptrSize == 8) && (__STDCPP_DEF
* Packs discriminator bits into the pointer alignment niches.
*/
template<std::size_t ptrSize>
class alignas(16) ValueStorage<ptrSize, std::enable_if_t<detail::useBitPackedValueStorage<ptrSize>>>
: public detail::ValueBase
class ValueStorage<ptrSize, std::enable_if_t<detail::useBitPackedValueStorage<ptrSize>>> : public detail::ValueBase
{
/* Needs a dependent type name in order for member functions (and
* potentially ill-formed bit casts) to be SFINAE'd out.
@@ -834,35 +832,6 @@ struct Value : public ValueStorage<sizeof(void *)>
{
friend std::string showType(const Value & v);
/**
* Empty list constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
static Value vEmptyList;
/**
* `null` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
static Value vNull;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
static Value vTrue;
/**
* `true` constant.
*
* This is _not_ a singleton. Pointer equality is _not_ sufficient.
*/
static Value vFalse;
private:
template<InternalType... discriminator>
bool isa() const noexcept
{
@@ -991,7 +960,7 @@ public:
setStorage(b);
}
void mkStringNoCopy(const char * s, const char ** context = 0) noexcept
inline void mkString(const char * s, const char ** context = 0) noexcept
{
setStorage(StringWithContext{.c_str = s, .context = context});
}
@@ -1003,6 +972,7 @@ public:
void mkStringMove(const char * s, const NixStringContext & context);
void mkPath(const SourcePath & path);
void mkPath(std::string_view path);
inline void mkPath(SourceAccessor * accessor, const char * path) noexcept
{
@@ -1023,20 +993,12 @@ public:
void mkList(const ListBuilder & builder) noexcept
{
switch (builder.size) {
case 0:
setStorage(List{.size = 0, .elems = nullptr});
break;
case 1:
if (builder.size == 1)
setStorage(std::array<Value *, 2>{builder.inlineElems[0], nullptr});
break;
case 2:
else if (builder.size == 2)
setStorage(std::array<Value *, 2>{builder.inlineElems[0], builder.inlineElems[1]});
break;
default:
else
setStorage(List{.size = builder.size, .elems = builder.elems});
break;
}
}
inline void mkThunk(Env * e, Expr * ex) noexcept
@@ -1196,7 +1158,7 @@ void Value::mkBlackhole()
}
typedef std::vector<Value *, traceable_allocator<Value *>> ValueVector;
typedef boost::unordered_flat_map<
typedef std::unordered_map<
Symbol,
Value *,
std::hash<Symbol>,
+2 -2
View File
@@ -1,11 +1,11 @@
#include "lexer-helpers.hh"
void nix::lexer::internal::initLoc(Parser::location_type * loc)
void nix::lexer::internal::initLoc(YYLTYPE * loc)
{
loc->beginOffset = loc->endOffset = 0;
}
void nix::lexer::internal::adjustLoc(yyscan_t yyscanner, Parser::location_type * loc, const char * s, size_t len)
void nix::lexer::internal::adjustLoc(yyscan_t yyscanner, YYLTYPE * loc, const char * s, size_t len)
{
loc->stash();
+7 -3
View File
@@ -2,12 +2,16 @@
#include <cstddef>
#include "parser-scanner-decls.hh"
// including the generated headers twice leads to errors
#ifndef BISON_HEADER
# include "lexer-tab.hh"
# include "parser-tab.hh"
#endif
namespace nix::lexer::internal {
void initLoc(Parser::location_type * loc);
void initLoc(YYLTYPE * loc);
void adjustLoc(yyscan_t yyscanner, Parser::location_type * loc, const char * s, size_t len);
void adjustLoc(yyscan_t yyscanner, YYLTYPE * loc, const char * s, size_t len);
} // namespace nix::lexer::internal
-4
View File
@@ -82,10 +82,6 @@ static void requireExperimentalFeature(const ExperimentalFeature & feature, cons
}
using enum nix::Parser::token::token_kind_type;
using YYSTYPE = nix::Parser::value_type;
using YYLTYPE = nix::Parser::location_type;
// yacc generates code that uses unannotated fallthrough.
#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
+1 -13
View File
@@ -53,12 +53,7 @@ deps_other += boost
nlohmann_json = dependency('nlohmann_json', version : '>= 3.9')
deps_public += nlohmann_json
bdw_gc_required = get_option('gc').disable_if(
'address' in get_option('b_sanitize'),
error_message : 'Building with Boehm GC and ASAN is not supported',
)
bdw_gc = dependency('bdw-gc', required : bdw_gc_required)
bdw_gc = dependency('bdw-gc', required : get_option('gc'))
if bdw_gc.found()
deps_public += bdw_gc
foreach funcspec : [
@@ -69,10 +64,6 @@ if bdw_gc.found()
define_value = cxx.has_function(funcspec).to_int()
configdata_priv.set(define_name, define_value)
endforeach
if host_machine.system() == 'cygwin'
# undefined reference to `__wrap__Znwm'
configdata_pub.set('GC_NO_INLINE_STD_NEW', 1)
endif
endif
# Used in public header. Affects ABI!
configdata_pub.set('NIX_USE_BOEHMGC', bdw_gc.found().to_int())
@@ -97,7 +88,6 @@ config_priv_h = configure_file(
)
subdir('nix-meson-build-support/common')
subdir('nix-meson-build-support/asan-options')
parser_tab = custom_target(
input : 'parser.y',
@@ -173,7 +163,6 @@ sources = files(
'search-path.cc',
'value-to-json.cc',
'value-to-xml.cc',
'value.cc',
'value/context.cc',
)
@@ -191,7 +180,6 @@ this_library = library(
parser_tab,
lexer_tab,
generated_headers,
soversion : nix_soversion,
dependencies : deps_public + deps_private + deps_other,
include_directories : include_dirs,
link_args : linker_export_flags,
+5 -5
View File
@@ -11,7 +11,7 @@
namespace nix {
Counter Expr::nrExprs;
unsigned long Expr::nrExprs = 0;
ExprBlackHole eBlackHole;
@@ -45,7 +45,7 @@ void ExprString::show(const SymbolTable & symbols, std::ostream & str) const
void ExprPath::show(const SymbolTable & symbols, std::ostream & str) const
{
str << v.pathStr();
str << s;
}
void ExprVar::show(const SymbolTable & symbols, std::ostream & str) const
@@ -57,7 +57,7 @@ void ExprSelect::show(const SymbolTable & symbols, std::ostream & str) const
{
str << "(";
e->show(symbols, str);
str << ")." << showAttrPath(symbols, getAttrPath());
str << ")." << showAttrPath(symbols, attrPath);
if (def) {
str << " or (";
def->show(symbols, str);
@@ -261,7 +261,7 @@ void ExprPos::show(const SymbolTable & symbols, std::ostream & str) const
str << "__curPos";
}
std::string showAttrPath(const SymbolTable & symbols, std::span<const AttrName> attrPath)
std::string showAttrPath(const SymbolTable & symbols, const AttrPath & attrPath)
{
std::ostringstream out;
bool first = true;
@@ -362,7 +362,7 @@ void ExprSelect::bindVars(EvalState & es, const std::shared_ptr<const StaticEnv>
e->bindVars(es, env);
if (def)
def->bindVars(es, env);
for (auto & i : getAttrPath())
for (auto & i : attrPath)
if (!i.symbol)
i.expr->bindVars(es, env);
}
-17
View File
@@ -1,17 +0,0 @@
#pragma once
#ifndef BISON_HEADER
# include "parser-tab.hh"
using YYSTYPE = nix::parser::BisonParser::value_type;
using YYLTYPE = nix::parser::BisonParser::location_type;
# include "lexer-tab.hh" // IWYU pragma: export
#endif
namespace nix {
class Parser : public parser::BisonParser
{
using BisonParser::BisonParser;
};
} // namespace nix
+37 -39
View File
@@ -1,7 +1,5 @@
%skeleton "lalr1.cc"
%define api.location.type { ::nix::ParserLocation }
%define api.namespace { ::nix::parser }
%define api.parser.class { BisonParser }
%define api.pure
%locations
%define parse.error verbose
%defines
@@ -28,12 +26,19 @@
#include "nix/expr/eval-settings.hh"
#include "nix/expr/parser-state.hh"
#define YY_DECL \
int yylex( \
nix::Parser::value_type * yylval_param, \
nix::Parser::location_type * yylloc_param, \
yyscan_t yyscanner, \
nix::ParserState * state)
// Bison seems to have difficulty growing the parser stack when using C++ with
// a custom location type. This undocumented macro tells Bison that our
// location type is "trivially copyable" in C++-ese, so it is safe to use the
// same memcpy macro it uses to grow the stack that it uses with its own
// default location type. Without this, we get "error: memory exhausted" when
// parsing some large Nix files. Our other options are to increase the initial
// stack size (200 by default) to be as large as we ever want to support (so
// that growing the stack is unnecessary), or redefine the stack-relocation
// macro ourselves (which is also undocumented).
#define YYLTYPE_IS_TRIVIAL 1
#define YY_DECL int yylex \
(YYSTYPE * yylval_param, YYLTYPE * yylloc_param, yyscan_t yyscanner, nix::ParserState * state)
// For efficiency, we only track offsets; not line,column coordinates
# define YYLLOC_DEFAULT(Current, Rhs, N) \
@@ -52,19 +57,19 @@
namespace nix {
typedef boost::unordered_flat_map<PosIdx, DocComment, std::hash<PosIdx>> DocCommentMap;
typedef std::unordered_map<PosIdx, DocComment> DocCommentMap;
Expr * parseExprFromBuf(
char * text,
size_t length,
Pos::Origin origin,
const SourcePath & basePath,
std::pmr::polymorphic_allocator<char> & alloc,
SymbolTable & symbols,
const EvalSettings & settings,
PosTable & positions,
DocCommentMap & docComments,
const ref<SourceAccessor> rootFS);
const ref<SourceAccessor> rootFS,
const Expr::AstSymbols & astSymbols);
}
@@ -74,30 +79,24 @@ Expr * parseExprFromBuf(
%{
/* The parser is very performance sensitive and loses out on a lot
of performance even with basic stdlib assertions. Since those don't
affect ABI we can disable those just for this file. */
#if defined(_GLIBCXX_ASSERTIONS) && !defined(_GLIBCXX_DEBUG)
#undef _GLIBCXX_ASSERTIONS
#endif
#include "parser-scanner-decls.hh"
#include "parser-tab.hh"
#include "lexer-tab.hh"
YY_DECL;
using namespace nix;
#define CUR_POS state->at(yylhs.location)
#define CUR_POS state->at(yyloc)
void parser::BisonParser::error(const location_type &loc_, const std::string &error)
void yyerror(YYLTYPE * loc, yyscan_t scanner, ParserState * state, const char * error)
{
auto loc = loc_;
if (std::string_view(error).starts_with("syntax error, unexpected end of file")) {
loc.beginOffset = loc.endOffset;
loc->beginOffset = loc->endOffset;
}
throw ParseError({
.msg = HintFmt(error),
.pos = state->positions[state->at(loc)]
.pos = state->positions[state->at(*loc)]
});
}
@@ -184,7 +183,7 @@ start: expr {
state->result = $1;
// This parser does not use yynerrs; suppress the warning.
(void) yynerrs_;
(void) yynerrs;
};
expr: expr_function;
@@ -259,7 +258,7 @@ expr_op
| expr_op OR expr_op { $$ = new ExprOpOr(state->at(@2), $1, $3); }
| expr_op IMPL expr_op { $$ = new ExprOpImpl(state->at(@2), $1, $3); }
| expr_op UPDATE expr_op { $$ = new ExprOpUpdate(state->at(@2), $1, $3); }
| expr_op '?' attrpath { $$ = new ExprOpHasAttr(state->alloc, $1, std::move(*$3)); delete $3; }
| expr_op '?' attrpath { $$ = new ExprOpHasAttr($1, std::move(*$3)); delete $3; }
| expr_op '+' expr_op
{ $$ = new ExprConcatStrings(state->at(@2), false, new std::vector<std::pair<PosIdx, Expr *> >({{state->at(@1), $1}, {state->at(@3), $3}})); }
| expr_op '-' expr_op { $$ = new ExprCall(state->at(@2), new ExprVar(state->s.sub), {$1, $3}); }
@@ -280,9 +279,9 @@ expr_app
expr_select
: expr_simple '.' attrpath
{ $$ = new ExprSelect(state->alloc, CUR_POS, $1, std::move(*$3), nullptr); delete $3; }
{ $$ = new ExprSelect(CUR_POS, $1, std::move(*$3), nullptr); delete $3; }
| expr_simple '.' attrpath OR_KW expr_select
{ $$ = new ExprSelect(state->alloc, CUR_POS, $1, std::move(*$3), $5); delete $3; $5->warnIfCursedOr(state->symbols, state->positions); }
{ $$ = new ExprSelect(CUR_POS, $1, std::move(*$3), $5); delete $3; $5->warnIfCursedOr(state->symbols, state->positions); }
| /* Backwards compatibility: because Nixpkgs has a function named ‘or’,
allow stuff like ‘map or [...]’. This production is problematic (see
https://github.com/NixOS/nix/issues/11118) and will be refactored in the
@@ -335,7 +334,7 @@ expr_simple
/* Let expressions `let {..., body = ...}' are just desugared
into `(rec {..., body = ...}).body'. */
| LET '{' binds '}'
{ $3->recursive = true; $3->pos = CUR_POS; $$ = new ExprSelect(state->alloc, noPos, $3, state->s.body); }
{ $3->recursive = true; $3->pos = CUR_POS; $$ = new ExprSelect(noPos, $3, state->s.body); }
| REC '{' binds '}'
{ $3->recursive = true; $3->pos = CUR_POS; $$ = $3; }
| '{' binds1 '}'
@@ -384,8 +383,8 @@ path_start
root filesystem accessor, rather than the accessor of the
current Nix expression. */
literal.front() == '/'
? new ExprPath(state->alloc, state->rootFS, path)
: new ExprPath(state->alloc, state->basePath.accessor, path);
? new ExprPath(state->rootFS, std::move(path))
: new ExprPath(state->basePath.accessor, std::move(path));
}
| HPATH {
if (state->settings.pureEval) {
@@ -395,7 +394,7 @@ path_start
);
}
Path path(getHome() + std::string($1.p + 1, $1.l - 1));
$$ = new ExprPath(state->alloc, ref<SourceAccessor>(state->rootFS), path);
$$ = new ExprPath(ref<SourceAccessor>(state->rootFS), std::move(path));
}
;
@@ -439,7 +438,7 @@ binds1
$accum->attrs.emplace(
i.symbol,
ExprAttrs::AttrDef(
new ExprSelect(state->alloc, iPos, from, i.symbol),
new ExprSelect(iPos, from, i.symbol),
iPos,
ExprAttrs::AttrDef::Kind::InheritedFrom));
}
@@ -539,12 +538,12 @@ Expr * parseExprFromBuf(
size_t length,
Pos::Origin origin,
const SourcePath & basePath,
std::pmr::polymorphic_allocator<char> & alloc,
SymbolTable & symbols,
const EvalSettings & settings,
PosTable & positions,
DocCommentMap & docComments,
const ref<SourceAccessor> rootFS)
const ref<SourceAccessor> rootFS,
const Expr::AstSymbols & astSymbols)
{
yyscan_t scanner;
LexerState lexerState {
@@ -554,12 +553,12 @@ Expr * parseExprFromBuf(
};
ParserState state {
.lexerState = lexerState,
.alloc = alloc,
.symbols = symbols,
.positions = positions,
.basePath = basePath,
.origin = lexerState.origin,
.rootFS = rootFS,
.s = astSymbols,
.settings = settings,
};
@@ -567,8 +566,7 @@ Expr * parseExprFromBuf(
Finally _destroy([&] { yylex_destroy(scanner); });
yy_scan_buffer(text, length, scanner);
Parser parser(scanner, &state);
parser.parse();
yyparse(scanner, &state);
return state.result;
}
-25
View File
@@ -1,7 +1,5 @@
#include "nix/store/store-api.hh"
#include "nix/expr/eval.hh"
#include "nix/util/mounted-source-accessor.hh"
#include "nix/fetchers/fetch-to-store.hh"
namespace nix {
@@ -20,27 +18,4 @@ SourcePath EvalState::storePath(const StorePath & path)
return {rootFS, CanonPath{store->printStorePath(path)}};
}
StorePath
EvalState::mountInput(fetchers::Input & input, const fetchers::Input & originalInput, ref<SourceAccessor> accessor)
{
auto storePath = fetchToStore(fetchSettings, *store, accessor, FetchMode::Copy, input.getName());
allowPath(storePath); // FIXME: should just whitelist the entire virtual store
storeFS->mount(CanonPath(store->printStorePath(storePath)), accessor);
auto narHash = store->queryPathInfo(storePath)->narHash;
input.attrs.insert_or_assign("narHash", narHash.to_string(HashFormat::SRI, true));
if (originalInput.getNarHash() && narHash != *originalInput.getNarHash())
throw Error(
(unsigned int) 102,
"NAR hash mismatch in input '%s', expected '%s' but got '%s'",
originalInput.to_string(),
narHash.to_string(HashFormat::SRI, true),
originalInput.getNarHash()->to_string(HashFormat::SRI, true));
return storePath;
}
} // namespace nix
+115 -177
View File
@@ -18,8 +18,6 @@
#include "nix/util/sort.hh"
#include <boost/container/small_vector.hpp>
#include <boost/unordered/concurrent_flat_map.hpp>
#include <boost/unordered/unordered_flat_map.hpp>
#include <nlohmann/json.hpp>
#include <sys/types.h>
@@ -39,12 +37,6 @@
namespace nix {
RegisterPrimOp::PrimOps & RegisterPrimOp::primOps()
{
static RegisterPrimOp::PrimOps primOps;
return primOps;
}
/*************************************************************
* Miscellaneous
*************************************************************/
@@ -222,20 +214,20 @@ void derivationToValue(
auto path2 = path.path.abs();
Derivation drv = state.store->readDerivation(storePath);
auto attrs = state.buildBindings(3 + drv.outputs.size());
attrs.alloc(state.s.drvPath)
attrs.alloc(state.sDrvPath)
.mkString(
path2,
{
NixStringContextElem::DrvDeep{.drvPath = storePath},
});
attrs.alloc(state.s.name).mkString(drv.env["name"]);
attrs.alloc(state.sName).mkString(drv.env["name"]);
auto list = state.buildList(drv.outputs.size());
for (const auto & [i, o] : enumerate(drv.outputs)) {
mkOutputString(state, attrs, storePath, o);
(list[i] = state.allocValue())->mkString(o.first);
}
attrs.alloc(state.s.outputs).mkList(list);
attrs.alloc(state.sOutputs).mkList(list);
auto w = state.allocValue();
w->mkAttrs(attrs);
@@ -268,7 +260,7 @@ static void scopedImport(EvalState & state, const PosIdx pos, SourcePath & path,
{
state.forceAttrs(*vScope, pos, "while evaluating the first argument passed to builtins.scopedImport");
Env * env = &state.mem.allocEnv(vScope->attrs()->size());
Env * env = &state.allocEnv(vScope->attrs()->size());
env->up = &state.baseEnv;
auto staticEnv = std::make_shared<StaticEnv>(nullptr, state.staticBaseEnv, vScope->attrs()->size());
@@ -491,40 +483,42 @@ void prim_exec(EvalState & state, const PosIdx pos, Value ** args, Value & v)
static void prim_typeOf(EvalState & state, const PosIdx pos, Value ** args, Value & v)
{
state.forceValue(*args[0], pos);
std::string t;
switch (args[0]->type()) {
case nInt:
v.mkStringNoCopy("int");
t = "int";
break;
case nBool:
v.mkStringNoCopy("bool");
t = "bool";
break;
case nString:
v.mkStringNoCopy("string");
t = "string";
break;
case nPath:
v.mkStringNoCopy("path");
t = "path";
break;
case nNull:
v.mkStringNoCopy("null");
t = "null";
break;
case nAttrs:
v.mkStringNoCopy("set");
t = "set";
break;
case nList:
v.mkStringNoCopy("list");
t = "list";
break;
case nFunction:
v.mkStringNoCopy("lambda");
t = "lambda";
break;
case nExternal:
v.mkString(args[0]->external()->typeOf());
t = args[0]->external()->typeOf();
break;
case nFloat:
v.mkStringNoCopy("float");
t = "float";
break;
case nThunk:
unreachable();
}
v.mkString(t);
}
static RegisterPrimOp primop_typeOf({
@@ -737,7 +731,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value ** ar
/* Get the start set. */
auto startSet = state.getAttr(
state.s.startSet, args[0]->attrs(), "in the attrset passed as argument to builtins.genericClosure");
state.sStartSet, args[0]->attrs(), "in the attrset passed as argument to builtins.genericClosure");
state.forceList(
*startSet->value,
@@ -755,7 +749,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value ** ar
/* Get the operator. */
auto op = state.getAttr(
state.s.operator_, args[0]->attrs(), "in the attrset passed as argument to builtins.genericClosure");
state.sOperator, args[0]->attrs(), "in the attrset passed as argument to builtins.genericClosure");
state.forceFunction(
*op->value, noPos, "while evaluating the 'operator' attribute passed as argument to builtins.genericClosure");
@@ -777,7 +771,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value ** ar
"while evaluating one of the elements generated by (or initially passed to) builtins.genericClosure");
auto key = state.getAttr(
state.s.key,
state.sKey,
e->attrs(),
"in one of the attrsets generated by (or initially passed to) builtins.genericClosure");
state.forceValue(*key->value, noPos);
@@ -1082,12 +1076,12 @@ static void prim_tryEval(EvalState & state, const PosIdx pos, Value ** args, Val
try {
state.forceValue(*args[0], pos);
attrs.insert(state.s.value, args[0]);
attrs.insert(state.symbols.create("success"), &Value::vTrue);
attrs.insert(state.sValue, args[0]);
attrs.insert(state.symbols.create("success"), &state.vTrue);
} catch (AssertionError & e) {
// `value = false;` is unfortunate but removing it is a breaking change.
attrs.insert(state.s.value, &Value::vFalse);
attrs.insert(state.symbols.create("success"), &Value::vFalse);
attrs.insert(state.sValue, &state.vFalse);
attrs.insert(state.symbols.create("success"), &state.vFalse);
}
// restore the debugRepl pointer if we saved it earlier.
@@ -1298,8 +1292,7 @@ static void prim_derivationStrict(EvalState & state, const PosIdx pos, Value **
auto attrs = args[0]->attrs();
/* Figure out the name first (for stack backtraces). */
auto nameAttr =
state.getAttr(state.s.name, attrs, "in the attrset passed as argument to builtins.derivationStrict");
auto nameAttr = state.getAttr(state.sName, attrs, "in the attrset passed as argument to builtins.derivationStrict");
std::string_view drvName;
try {
@@ -1373,8 +1366,8 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
using nlohmann::json;
std::optional<StructuredAttrs> jsonObject;
auto pos = v.determinePos(noPos);
auto attr = attrs->get(state.s.structuredAttrs);
if (attr
auto attr = attrs->find(state.sStructuredAttrs);
if (attr != attrs->end()
&& state.forceBool(
*attr->value,
pos,
@@ -1384,8 +1377,8 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
/* Check whether null attributes should be ignored. */
bool ignoreNulls = false;
attr = attrs->get(state.s.ignoreNulls);
if (attr)
attr = attrs->find(state.sIgnoreNulls);
if (attr != attrs->end())
ignoreNulls = state.forceBool(
*attr->value,
pos,
@@ -1408,7 +1401,7 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
outputs.insert("out");
for (auto & i : attrs->lexicographicOrder(state.symbols)) {
if (i->name == state.s.ignoreNulls)
if (i->name == state.sIgnoreNulls)
continue;
auto key = state.symbols[i->name];
vomit("processing attribute '%1%'", key);
@@ -1460,22 +1453,19 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
continue;
}
switch (i->name.getId()) {
case EvalState::s.contentAddressed.getId():
if (state.forceBool(*i->value, pos, context_below)) {
contentAddressed = true;
experimentalFeatureSettings.require(Xp::CaDerivations);
}
break;
case EvalState::s.impure.getId():
if (state.forceBool(*i->value, pos, context_below)) {
isImpure = true;
experimentalFeatureSettings.require(Xp::ImpureDerivations);
}
break;
if (i->name == state.sContentAddressed && state.forceBool(*i->value, pos, context_below)) {
contentAddressed = true;
experimentalFeatureSettings.require(Xp::CaDerivations);
}
else if (i->name == state.sImpure && state.forceBool(*i->value, pos, context_below)) {
isImpure = true;
experimentalFeatureSettings.require(Xp::ImpureDerivations);
}
/* The `args' attribute is special: it supplies the
command-line arguments to the builder. */
case EvalState::s.args.getId():
else if (i->name == state.sArgs) {
state.forceList(*i->value, pos, context_below);
for (auto elem : i->value->listView()) {
auto s = state
@@ -1484,116 +1474,86 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
.toOwned();
drv.args.push_back(s);
}
break;
}
/* All other attributes are passed to the builder through
the environment. */
default:
else {
if (jsonObject) {
if (i->name == state.s.structuredAttrs)
if (i->name == state.sStructuredAttrs)
continue;
jsonObject->structuredAttrs.emplace(key, printValueAsJSON(state, true, *i->value, pos, context));
switch (i->name.getId()) {
case EvalState::s.builder.getId():
if (i->name == state.sBuilder)
drv.builder = state.forceString(*i->value, context, pos, context_below);
break;
case EvalState::s.system.getId():
else if (i->name == state.sSystem)
drv.platform = state.forceStringNoCtx(*i->value, pos, context_below);
break;
case EvalState::s.outputHash.getId():
else if (i->name == state.sOutputHash)
outputHash = state.forceStringNoCtx(*i->value, pos, context_below);
break;
case EvalState::s.outputHashAlgo.getId():
else if (i->name == state.sOutputHashAlgo)
outputHashAlgo = parseHashAlgoOpt(state.forceStringNoCtx(*i->value, pos, context_below));
break;
case EvalState::s.outputHashMode.getId():
else if (i->name == state.sOutputHashMode)
handleHashMode(state.forceStringNoCtx(*i->value, pos, context_below));
break;
case EvalState::s.outputs.getId(): {
/* Require 'outputs' to be a list of strings. */
else if (i->name == state.sOutputs) {
/* Require ‘outputs’ to be a list of strings. */
state.forceList(*i->value, pos, context_below);
Strings ss;
for (auto elem : i->value->listView())
ss.emplace_back(state.forceStringNoCtx(*elem, pos, context_below));
handleOutputs(ss);
break;
}
default:
break;
}
switch (i->name.getId()) {
case EvalState::s.allowedReferences.getId():
if (i->name == state.sAllowedReferences)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'allowedReferences'; use 'outputChecks.<output>.allowedReferences' instead",
drvName);
break;
case EvalState::s.allowedRequisites.getId():
if (i->name == state.sAllowedRequisites)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'allowedRequisites'; use 'outputChecks.<output>.allowedRequisites' instead",
drvName);
break;
case EvalState::s.disallowedReferences.getId():
if (i->name == state.sDisallowedReferences)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'disallowedReferences'; use 'outputChecks.<output>.disallowedReferences' instead",
drvName);
break;
case EvalState::s.disallowedRequisites.getId():
if (i->name == state.sDisallowedRequisites)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'disallowedRequisites'; use 'outputChecks.<output>.disallowedRequisites' instead",
drvName);
break;
case EvalState::s.maxSize.getId():
if (i->name == state.sMaxSize)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'maxSize'; use 'outputChecks.<output>.maxSize' instead",
drvName);
break;
case EvalState::s.maxClosureSize.getId():
if (i->name == state.sMaxClosureSize)
warn(
"In a derivation named '%s', 'structuredAttrs' disables the effect of the derivation attribute 'maxClosureSize'; use 'outputChecks.<output>.maxClosureSize' instead",
drvName);
break;
default:
break;
}
} else {
auto s = state.coerceToString(pos, *i->value, context, context_below, true).toOwned();
if (i->name == state.s.json) {
if (i->name == state.sJson) {
warn(
"In derivation '%s': setting structured attributes via '__json' is deprecated, and may be disallowed in future versions of Nix. Set '__structuredAttrs = true' instead.",
drvName);
drv.structuredAttrs = StructuredAttrs::parse(s);
} else {
drv.env.emplace(key, s);
switch (i->name.getId()) {
case EvalState::s.builder.getId():
if (i->name == state.sBuilder)
drv.builder = std::move(s);
break;
case EvalState::s.system.getId():
else if (i->name == state.sSystem)
drv.platform = std::move(s);
break;
case EvalState::s.outputHash.getId():
else if (i->name == state.sOutputHash)
outputHash = std::move(s);
break;
case EvalState::s.outputHashAlgo.getId():
else if (i->name == state.sOutputHashAlgo)
outputHashAlgo = parseHashAlgoOpt(s);
break;
case EvalState::s.outputHashMode.getId():
else if (i->name == state.sOutputHashMode)
handleHashMode(s);
break;
case EvalState::s.outputs.getId():
else if (i->name == state.sOutputs)
handleOutputs(tokenizeString<Strings>(s));
break;
default:
break;
}
}
}
break;
}
} catch (Error & e) {
@@ -1758,11 +1718,11 @@ static void derivationStrictInternal(EvalState & state, std::string_view drvName
read them later. */
{
auto h = hashDerivationModulo(*state.store, drv, false);
drvHashes.insert_or_assign(drvPath, std::move(h));
drvHashes.lock()->insert_or_assign(drvPath, h);
}
auto result = state.buildBindings(1 + drv.outputs.size());
result.alloc(state.s.drvPath)
result.alloc(state.sDrvPath)
.mkString(
drvPathS,
{
@@ -2051,14 +2011,14 @@ static void prim_findFile(EvalState & state, const PosIdx pos, Value ** args, Va
state.forceAttrs(*v2, pos, "while evaluating an element of the list passed to builtins.findFile");
std::string prefix;
auto i = v2->attrs()->get(state.s.prefix);
if (i)
auto i = v2->attrs()->find(state.sPrefix);
if (i != v2->attrs()->end())
prefix = state.forceStringNoCtx(
*i->value,
pos,
"while evaluating the `prefix` attribute of an element of the list passed to builtins.findFile");
i = state.getAttr(state.s.path, v2->attrs(), "in an element of the __nixPath");
i = state.getAttr(state.sPath, v2->attrs(), "in an element of the __nixPath");
NixStringContext context;
auto path =
@@ -2254,45 +2214,19 @@ static RegisterPrimOp primop_hashFile({
.fun = prim_hashFile,
});
static const Value & fileTypeToString(EvalState & state, SourceAccessor::Type type)
static Value * fileTypeToString(EvalState & state, SourceAccessor::Type type)
{
struct Constants
{
Value regular;
Value directory;
Value symlink;
Value unknown;
};
static const Constants stringValues = []() {
Constants res;
res.regular.mkStringNoCopy("regular");
res.directory.mkStringNoCopy("directory");
res.symlink.mkStringNoCopy("symlink");
res.unknown.mkStringNoCopy("unknown");
return res;
}();
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wswitch-enum"
using enum SourceAccessor::Type;
switch (type) {
case tRegular:
return stringValues.regular;
case tDirectory:
return stringValues.directory;
case tSymlink:
return stringValues.symlink;
default:
return stringValues.unknown;
}
return type == SourceAccessor::Type::tRegular ? &state.vStringRegular
: type == SourceAccessor::Type::tDirectory ? &state.vStringDirectory
: type == SourceAccessor::Type::tSymlink ? &state.vStringSymlink
: &state.vStringUnknown;
}
static void prim_readFileType(EvalState & state, const PosIdx pos, Value ** args, Value & v)
{
auto path = realisePath(state, pos, *args[0], std::nullopt);
/* Retrieve the directory entry type and stringize it. */
v = fileTypeToString(state, path.lstat().type);
v = *fileTypeToString(state, path.lstat().type);
}
static RegisterPrimOp primop_readFileType({
@@ -2336,9 +2270,7 @@ static void prim_readDir(EvalState & state, const PosIdx pos, Value ** args, Val
} else {
// This branch of the conditional is much more likely.
// Here we just stringize the directory entry type.
// N.B. const_cast here is ok, because these values will never be modified, since
// only thunks are mutable - other types do not change once constructed.
attrs.insert(state.symbols.create(name), const_cast<Value *>(&fileTypeToString(state, *type)));
attrs.insert(state.symbols.create(name), fileTypeToString(state, *type));
}
}
@@ -2713,7 +2645,7 @@ bool EvalState::callPathFilter(Value * filterFun, const SourcePath & path, PosId
arg1.mkString(path.path.abs());
// assert that type is not "unknown"
Value * args[]{&arg1, const_cast<Value *>(&fileTypeToString(*this, st.type))};
Value * args[]{&arg1, fileTypeToString(*this, st.type)};
Value res;
callFunction(*filterFun, args, res, pos);
@@ -2859,7 +2791,7 @@ static void prim_path(EvalState & state, const PosIdx pos, Value ** args, Value
if (n == "path")
path.emplace(state.coerceToPath(
attr.pos, *attr.value, context, "while evaluating the 'path' attribute passed to 'builtins.path'"));
else if (attr.name == state.s.name)
else if (attr.name == state.sName)
name = state.forceStringNoCtx(
*attr.value, attr.pos, "while evaluating the `name` attribute passed to builtins.path");
else if (n == "filter")
@@ -3019,8 +2951,8 @@ static void prim_unsafeGetAttrPos(EvalState & state, const PosIdx pos, Value **
auto attr = state.forceStringNoCtx(
*args[0], pos, "while evaluating the first argument passed to builtins.unsafeGetAttrPos");
state.forceAttrs(*args[1], pos, "while evaluating the second argument passed to builtins.unsafeGetAttrPos");
auto i = args[1]->attrs()->get(state.symbols.create(attr));
if (!i)
auto i = args[1]->attrs()->find(state.symbols.create(attr));
if (i == args[1]->attrs()->end())
v.mkNull();
else
state.mkPos(v, i->pos);
@@ -3087,7 +3019,7 @@ static void prim_hasAttr(EvalState & state, const PosIdx pos, Value ** args, Val
{
auto attr = state.forceStringNoCtx(*args[0], pos, "while evaluating the first argument passed to builtins.hasAttr");
state.forceAttrs(*args[1], pos, "while evaluating the second argument passed to builtins.hasAttr");
v.mkBool(args[1]->attrs()->get(state.symbols.create(attr)));
v.mkBool(args[1]->attrs()->find(state.symbols.create(attr)) != args[1]->attrs()->end());
}
static RegisterPrimOp primop_hasAttr({
@@ -3172,13 +3104,13 @@ static void prim_listToAttrs(EvalState & state, const PosIdx pos, Value ** args,
// Step 1. Sort the name-value attrsets in place using the memory we allocate for the result
auto listView = args[0]->listView();
size_t listSize = listView.size();
auto & bindings = *state.mem.allocBindings(listSize);
auto & bindings = *state.allocBindings(listSize);
using ElemPtr = decltype(&bindings[0].value);
for (const auto & [n, v2] : enumerate(listView)) {
state.forceAttrs(*v2, pos, "while evaluating an element of the list passed to builtins.listToAttrs");
auto j = state.getAttr(state.s.name, v2->attrs(), "in a {name=...; value=...;} pair");
auto j = state.getAttr(state.sName, v2->attrs(), "in a {name=...; value=...;} pair");
auto name = state.forceStringNoCtx(
*j->value,
@@ -3205,7 +3137,7 @@ static void prim_listToAttrs(EvalState & state, const PosIdx pos, Value ** args,
// Note that .value is actually a Value * *; see earlier comments
Value * v2 = *std::bit_cast<ElemPtr>(attr.value);
auto j = state.getAttr(state.s.value, v2->attrs(), "in a {name=...; value=...;} pair");
auto j = state.getAttr(state.sValue, v2->attrs(), "in a {name=...; value=...;} pair");
prev = attr.name;
bindings.push_back({prev, j->value, j->pos});
}
@@ -3297,14 +3229,14 @@ static void prim_intersectAttrs(EvalState & state, const PosIdx pos, Value ** ar
if (left.size() < right.size()) {
for (auto & l : left) {
auto r = right.get(l.name);
if (r)
auto r = right.find(l.name);
if (r != right.end())
attrs.insert(*r);
}
} else {
for (auto & r : right) {
auto l = left.get(r.name);
if (l)
auto l = left.find(r.name);
if (l != left.end())
attrs.insert(r);
}
}
@@ -3367,14 +3299,14 @@ static void prim_functionArgs(EvalState & state, const PosIdx pos, Value ** args
{
state.forceValue(*args[0], pos);
if (args[0]->isPrimOpApp() || args[0]->isPrimOp()) {
v.mkAttrs(&Bindings::emptyBindings);
v.mkAttrs(&state.emptyBindings);
return;
}
if (!args[0]->isLambda())
state.error<TypeError>("'functionArgs' requires a function").atPos(pos).debugThrow();
if (!args[0]->lambda().fun->hasFormals()) {
v.mkAttrs(&Bindings::emptyBindings);
v.mkAttrs(&state.emptyBindings);
return;
}
@@ -4021,13 +3953,13 @@ static void prim_partition(EvalState & state, const PosIdx pos, Value ** args, V
auto rlist = state.buildList(rsize);
if (rsize)
memcpy(rlist.elems, right.data(), sizeof(Value *) * rsize);
attrs.alloc(state.s.right).mkList(rlist);
attrs.alloc(state.sRight).mkList(rlist);
auto wsize = wrong.size();
auto wlist = state.buildList(wsize);
if (wsize)
memcpy(wlist.elems, wrong.data(), sizeof(Value *) * wsize);
attrs.alloc(state.s.wrong).mkList(wlist);
attrs.alloc(state.sWrong).mkList(wlist);
v.mkAttrs(attrs);
}
@@ -4068,7 +4000,7 @@ static void prim_groupBy(EvalState & state, const PosIdx pos, Value ** args, Val
auto name = state.forceStringNoCtx(
res, pos, "while evaluating the return value of the grouping function passed to builtins.groupBy");
auto sym = state.symbols.create(name);
auto vector = attrs.try_emplace<ValueVector>(sym, {}).first;
auto vector = attrs.try_emplace(sym, ValueVector()).first;
vector->second.push_back(vElem);
}
@@ -4421,7 +4353,7 @@ static void prim_substring(EvalState & state, const PosIdx pos, Value ** args, V
if (len == 0) {
state.forceValue(*args[2], pos);
if (args[2]->type() == nString) {
v.mkStringNoCopy("", args[2]->context());
v.mkString("", args[2]->context());
return;
}
}
@@ -4603,21 +4535,27 @@ static RegisterPrimOp primop_convertHash({
struct RegexCache
{
boost::concurrent_flat_map<std::string, std::regex, StringViewHash, std::equal_to<>> cache;
struct State
{
std::unordered_map<std::string, std::regex, StringViewHash, std::equal_to<>> cache;
};
Sync<State> state_;
std::regex get(std::string_view re)
{
std::regex regex;
auto state(state_.lock());
auto it = state->cache.find(re);
if (it != state->cache.end())
return it->second;
/* No std::regex constructor overload from std::string_view, but can be constructed
from a pointer + size or an iterator range. */
cache.try_emplace_and_cvisit(
re,
/*s=*/re.data(),
/*count=*/re.size(),
std::regex::extended,
[&regex](const auto & kv) { regex = kv.second; },
[&regex](const auto & kv) { regex = kv.second; });
return regex;
return state->cache
.emplace(
std::piecewise_construct,
std::forward_as_tuple(re),
std::forward_as_tuple(/*s=*/re.data(), /*count=*/re.size(), std::regex::extended))
.first->second;
}
};
@@ -4648,7 +4586,7 @@ void prim_match(EvalState & state, const PosIdx pos, Value ** args, Value & v)
auto list = state.buildList(match.size() - 1);
for (const auto & [i, v2] : enumerate(list))
if (!match[i + 1].matched)
v2 = &Value::vNull;
v2 = &state.vNull;
else
v2 = mkString(state, match[i + 1]);
v.mkList(list);
@@ -4740,7 +4678,7 @@ void prim_split(EvalState & state, const PosIdx pos, Value ** args, Value & v)
auto list2 = state.buildList(slen);
for (const auto & [si, v2] : enumerate(list2)) {
if (!match[si + 1].matched)
v2 = &Value::vNull;
v2 = &state.vNull;
else
v2 = mkString(state, match[si + 1]);
}
@@ -4861,7 +4799,7 @@ static void prim_replaceStrings(EvalState & state, const PosIdx pos, Value ** ar
from.emplace_back(state.forceString(
*elem, pos, "while evaluating one of the strings to replace passed to builtins.replaceStrings"));
boost::unordered_flat_map<size_t, std::string_view> cache;
std::unordered_map<size_t, std::string_view> cache;
auto to = args[1]->listView();
NixStringContext context;
@@ -4940,7 +4878,7 @@ static void prim_parseDrvName(EvalState & state, const PosIdx pos, Value ** args
state.forceStringNoCtx(*args[0], pos, "while evaluating the first argument passed to builtins.parseDrvName");
DrvName parsed(name);
auto attrs = state.buildBindings(2);
attrs.alloc(state.s.name).mkString(parsed.name);
attrs.alloc(state.sName).mkString(parsed.name);
attrs.alloc("version").mkString(parsed.version);
v.mkAttrs(attrs);
}
@@ -5094,7 +5032,7 @@ void EvalState::createBaseEnv(const EvalSettings & evalSettings)
addConstant(
"null",
&Value::vNull,
&vNull,
{
.type = nNull,
.doc = R"(
+2 -2
View File
@@ -219,7 +219,7 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value ** args,
auto list = state.buildList(info.second.outputs.size());
for (const auto & [i, output] : enumerate(info.second.outputs))
(list[i] = state.allocValue())->mkString(output);
infoAttrs.alloc(state.s.outputs).mkList(list);
infoAttrs.alloc(state.sOutputs).mkList(list);
}
attrs.alloc(state.store->printStorePath(info.first)).mkAttrs(infoAttrs);
}
@@ -300,7 +300,7 @@ static void prim_appendContext(EvalState & state, const PosIdx pos, Value ** arg
}
}
if (auto attr = i.value->attrs()->get(state.s.outputs)) {
if (auto attr = i.value->attrs()->get(state.sOutputs)) {
state.forceList(*attr->value, attr->pos, "while evaluating the `outputs` attribute of a string context");
if (attr->value->listSize() && !isDerivation(name)) {
state
+1 -1
View File
@@ -84,7 +84,7 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value ** ar
auto [storePath, input2] = input.fetchToStore(state.store);
auto attrs2 = state.buildBindings(8);
state.mkStorePathString(storePath, attrs2.alloc(state.s.outPath));
state.mkStorePathString(storePath, attrs2.alloc(state.sOutPath));
if (input2.getRef())
attrs2.alloc("branch").mkString(*input2.getRef());
// Backward compatibility: set 'rev' to
Loaded 100 of 551 files, more files were not shown because too many files have changed in this diff. Show more