Compare commits

..
Author SHA1 Message Date
Valentin GagarinandRobert Hensing 7834e3222d show Nix logo in the manual (#10443)
the location of files is hard-coded by mdBook.
there is also seems to be no way to define custom templates, therefore
all styling has to be done in the CSS override.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-04-09 09:19:37 +00:00
Eelco Dolstra d9c204fafc Bump version 2024-03-15 21:12:09 +01:00
Eelco Dolstra bef56c43c0 Merge pull request #10250 from NixOS/backport-10233-to-2.16-maintenance
[Backport 2.16-maintenance] Manual backport of f8170ce9f1 to fix CVE-2024-27297
2024-03-15 19:19:51 +01:00
Nick Hu 1ba10f66cb Manual backport of f8170ce9f1 to fix CVE-2024-27297
(cherry picked from commit 3e99257053)
2024-03-15 17:14:39 +00:00
Robert Hensing e8334d110c Merge pull request #10183 from johnrichardrinehart/jrinehart/2_16-faster-flake-lock-parsing
2.16 faster flake lock parsing
2024-03-07 20:21:27 +01:00
Graham Dennis f94fb636c0 Faster flake.lock parsing
This PR reduces the creation of short-lived basic_json objects while
parsing flake.lock files. For large flake.lock files (~1.5MB) I was
observing ~60s being spent for trivial nix build operations while
after this change it is now taking ~1.6s.

(cherry picked from commit 7fd0de38c6)
2024-03-07 10:20:15 -08:00
github-actions[bot] 0a9cf6328e fix location of _redirects file (#9954) 2024-02-07 11:44:22 +01:00
github-actions[bot]andValentin Gagarin f7d9cae1e0 fix an old lost direct (#9715)
this part must have been moved quite a while ago, but apparently so far
no one noticed

(cherry picked from commit 6db805b3d1)

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-01-08 20:57:20 +01:00
Robert Hensing d16050d434 Merge pull request #9469 from NixOS/backport-9459-to-2.16-maintenance
[Backport 2.16-maintenance] add path based redirects
2023-11-27 15:15:41 +01:00
Valentin Gagarin f1eaf918d9 add path based redirects
up to now, those were managed outside of this repo, which as
unsurprisingly a real hassle to deal with if one wanted to prevent URLs
from breaking when moving pages around. this change removes a large part
of the friction involved in moving content in the Nix manual.

possible next steps for further automation:
- check for content that moved and warn if it's not reachable from
  links that were valid prior to a change
- create redirect rules automatically based on this information

(cherry picked from commit 2b7016cc56)
2023-11-27 13:28:53 +00:00
John Ericson c86080d308 Merge pull request #9434 from NixOS/backport-9431-to-2.16-maintenance
[Backport 2.16-maintenance] Add missing `-lrapidcheck` fixing build with shared lib
2023-11-22 09:35:56 -05:00
John Ericson 45ac34e90c Add missing -lrapidcheck fixing build with shared lib
https://github.com/NixOS/nixpkgs/pull/269064 makes rapidcheck be build
as a shared lib, but that broke Nix because the `-lrapidcheck` was
missing. This fixes that (and doesn't break Nix what the library is a
static archive as today).

(cherry picked from commit 46131567da)
2023-11-22 04:48:04 +00:00
Théophane Hufschmitt 40cb457496 Merge pull request #9404 from NixOS/backport-9390-to-2.16-maintenance
[Backport 2.16-maintenance] fetchTree: clarify docs for shallow flag
2023-11-20 15:38:17 +01:00
DavHau e7c9ce0b70 fetchTree: clarify docs for shallow flag
(cherry picked from commit 796a7eb92d)
2023-11-20 14:15:55 +00:00
Robert Hensing 5ab9384b33 Merge pull request #9120 from NixOS/backport-9095-to-2.16-maintenance
[Backport 2.16-maintenance] StorePath: reject names starting with '.'
2023-10-23 22:44:44 +02:00
Robert Hensing 49dfdd6d88 Merge pull request #9211 from NixOS/backport-8534-to-2.16-maintenance
[Backport 2.16-maintenance] ci: bump install-nix-action, don't fail fast
2023-10-23 17:19:29 +02:00
Naïm Favier ade3bffad3 ci: bump install-nix-action, don't fail fast
(cherry picked from commit b931d83550)
2023-10-23 13:27:50 +00:00
Eelco Dolstra a5a01a7674 Bump version 2023-10-19 21:35:14 +02:00
Eelco Dolstra 0201a31342 Fix C++17 build
(cherry picked from commit 2cf2bb45fc)
(cherry picked from commit 83a035aafe)
2023-10-19 19:00:28 +02:00
Eelco Dolstra 8c9e4668b2 Tarball trees: Propagate lastModified
This makes them behave consistently with GitHub/GitLab flakes.

(cherry picked from commit 81045f243f)
2023-10-19 19:00:24 +02:00
Eelco Dolstra a0f07ae3fa Document that redirected tarball flakerefs can specify lastModified
(cherry picked from commit 56763ff918)
2023-10-19 19:00:19 +02:00
Eelco Dolstra ec2baaaaac Add docs
(cherry picked from commit cab03fb779)
2023-10-19 19:00:13 +02:00
Eelco Dolstra 114ba0e363 Allow tarball URLs to redirect to a lockable immutable URL
Previously, for tarball flakes, we recorded the original URL of the
tarball flake, rather than the URL to which it ultimately
redirects. Thus, a flake URL like
http://example.org/patchelf-latest.tar that redirects to
http://example.org/patchelf-<revision>.tar was not really usable. We
couldn't record the redirected URL, because sites like GitHub redirect
to CDN URLs that we can't rely on to be stable.

So now we use the redirected URL only if the server returns the
`x-nix-is-immutable` or `x-amz-meta-nix-is-immutable` headers in its
response.

(cherry picked from commit 1ad3328c5e)
2023-10-19 18:59:57 +02:00
Eelco Dolstra 8ce966ee00 Add a generic check for rev attribute mismatches
(cherry picked from commit 3402b650cd)
2023-10-19 18:59:47 +02:00
edef 10b38a55cf StorePath: reject names starting with '.'
This has been the behaviour before Nix 2.4. It was dropped in a rewrite
in 759947bf72, allowing the creation of
store paths that aren't considered valid by older Nix versions or other
Nix tooling.

Nix 2.4 didn't ship in NixOS until 22.05, and stdenv.mkDerivation in
nixpkgs drops leading periods since April 2022, so it's unlikely anyone
is relying on the current lax behaviour.

Closes #9091.

Change-Id: I4a57bd9899e1b0dba56870ae5a1b680918a18ce9
(cherry picked from commit 24bda0c7b3)
2023-10-08 12:21:13 +00:00
Eelco Dolstra 15a3e6e282 Merge pull request #9083 from NixOS/backport-9079-to-2.16-maintenance
[Backport 2.16-maintenance] [Backport 2.18-maintenance] pathExists: isDir when endswith /.
2023-10-02 14:09:32 +02:00
Robert Hensing 7cebcde9cc pathExists: isDir when endswith /.
(cherry picked from commit f8a3893e8d)
(cherry picked from commit 860f64c345)
2023-10-02 12:08:45 +00:00
Eelco Dolstra 0529a89988 Merge pull request #8896 from NixOS/backport-8869-to-2.16-maintenance
[Backport 2.16-maintenance] Fix #8838, pathExists: isDir when ends with `/ `
2023-09-01 14:57:02 +02:00
Robert Hensing 6305ea6ad2 pathExists: isDir when endswith /
Fixes https://github.com/NixOS/nix/issues/8838

(cherry picked from commit 1e08e12d81)
2023-09-01 11:16:18 +00:00
Robert Hensing 9fcf142ee0 tests/lang/eval-okay-pathexists: Add cases
(cherry picked from commit d2e6cfa075)
2023-09-01 11:16:18 +00:00
Eelco Dolstra 24dd90e988 Merge pull request #8714 from NixOS/backport-8506-to-2.16-maintenance
[Backport 2.16-maintenance] Pass NIX_SSHOPTS when checking for an ssh master connection.
2023-07-20 17:15:48 +02:00
Théophane Hufschmitt 681c322f25 Merge pull request #8712 from NixOS/backport-8483-to-2.16-maintenance
[Backport 2.16-maintenance] restoreMountNamespace(): Restore the original root directory
2023-07-19 09:39:27 +02:00
David McFarland 52ddb32f4a Pass common ssh options in isMasterRunning
(cherry picked from commit d5e1eb20a2)
2023-07-18 13:48:57 +00:00
David McFarland ba0400f918 Add test of explicit ssh control path in nix-copy test
This highlights a problem caused by SSHMaster::isMasterRunning returning
false when NIX_SSHOPTS contains -oControlPath.

(cherry picked from commit 5454fdcceb)
2023-07-18 13:48:57 +00:00
Eelco Dolstra 92e198f1db restoreMountNamespace(): Restore the original root directory
This is necessary when we're in a chroot environment, where the
process root is not the same as the root of the mount namespace
(e.g. in nixos-enter).

Fixes #7602.

(cherry picked from commit e54538c461)
2023-07-17 20:25:23 +00:00
Eelco Dolstra 96aae7ac03 Merge pull request #8555 from NixOS/backport-8552-to-2.16-maintenance
[Backport 2.16-maintenance] GC server: Clear O_NONBLOCK on the right file descriptor
2023-06-20 13:00:38 +02:00
github-actions[bot]andTravis A. Everett afd1c095c8 redirect old platform uninstall instruction links (#8553)
Uninstall instructions were moved to their own page in #8267. The
overall section link was redirected in #8286, but platform-specific
links (which I give out frequently when I triage installer trouble)
weren't included.

(cherry picked from commit 33d3889831)

Co-authored-by: Travis A. Everett <travis.a.everett@gmail.com>
2023-06-20 12:14:56 +02:00
Eelco Dolstra ac32609ed9 GC server: Clear O_NONBLOCK on the right file descriptor
The bug fix in 6d30f9e6fe erroneously
cleared O_NONBLOCK on the server rather than client FD (leaving both
in an incorrect state).

Fixes #8551.

(cherry picked from commit a6a75ecad8)
2023-06-20 10:13:13 +00:00
Eelco Dolstra 6906d1185e Bump version 2023-06-06 15:47:39 +02:00
Eelco Dolstra 84050709ea Merge pull request #8460 from NixOS/backport-8391-to-2.16-maintenance
[Backport 2.16-maintenance] Remove old default from docs for `hashed-mirrors`
2023-06-06 12:55:23 +02:00
Aneesh Agrawal 567a5f0146 Remove old default from docs for hashed-mirrors
The `hashed-mirrors` option did use to have this default value,
but it was removed and re-added with an empty default value.
As the autogenerated docs show the (actual) default values from code,
remove this incorrect reference from the docs.

I was updating my nix.conf settings after a few years and noticed this.

(cherry picked from commit c694f1a2f3)
2023-06-06 10:54:46 +00:00
Eelco Dolstra 5ba5bdde36 Merge pull request #8458 from NixOS/backport-8452-to-2.16-maintenance
[Backport 2.16-maintenance] SourcePath::resolveSymlinks(): Fix handling of symlinks that start with '..'
2023-06-06 12:39:10 +02:00
Eelco Dolstra 5b1b25695c Fix SourcePath::resolveSymlinks()
This fixes handling of symlinks that start with '..', and symlink
targets that contain symlinks themselves.

(cherry picked from commit f5c6b29940)
2023-06-06 10:06:42 +00:00
Eelco Dolstra 39018fd42e Bump version 2023-05-31 17:08:37 +02:00
Eelco Dolstra 1ac5f9eac2 Mark official release 2023-05-31 14:06:00 +02:00
277 changed files with 2356 additions and 6174 deletions

No files matched your search

@@ -11,10 +11,6 @@ assignees: ''
<!-- describe your problem -->
## Proposal
<!-- propose a solution -->
## Checklist
<!-- make sure this issue is not redundant or obsolete -->
@@ -26,6 +22,10 @@ assignees: ''
[source]: https://github.com/NixOS/nix/tree/master/doc/manual/src
[open documentation issues and pull requests]: https://github.com/NixOS/nix/labels/documentation
## Proposal
<!-- propose a solution -->
## Priorities
Add :+1: to [issues you find important](https://github.com/NixOS/nix/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc).
+1 -1
View File
@@ -16,7 +16,7 @@
"new-cli":
- src/nix/**/*
"with-tests":
"tests":
# Unit tests
- src/*/tests/**/*
# Functional and integration tests
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
fetch-depth: 0
- name: Create backport PRs
# should be kept in sync with `version`
uses: zeebe-io/backport-action@v1.3.1
uses: zeebe-io/backport-action@v1.3.0
with:
# Config README: https://github.com/zeebe-io/backport-action#backport-action
github_token: ${{ secrets.GITHUB_TOKEN }}
+1 -4
View File
@@ -18,7 +18,7 @@ perl/Makefile.config
/doc/manual/generated/*
/doc/manual/nix.json
/doc/manual/conf-file.json
/doc/manual/language.json
/doc/manual/builtins.json
/doc/manual/xp-features.json
/doc/manual/src/SUMMARY.md
/doc/manual/src/command-ref/new-cli
@@ -26,7 +26,6 @@ perl/Makefile.config
/doc/manual/src/command-ref/experimental-features-shortlist.md
/doc/manual/src/contributing/experimental-feature-descriptions.md
/doc/manual/src/language/builtins.md
/doc/manual/src/language/builtin-constants.md
# /scripts/
/scripts/nix-profile.sh
@@ -90,12 +89,10 @@ perl/Makefile.config
/tests/ca/config.nix
/tests/dyn-drv/config.nix
/tests/repl-result-out
/tests/test-libstoreconsumer/test-libstoreconsumer
# /tests/lang/
/tests/lang/*.out
/tests/lang/*.out.xml
/tests/lang/*.err
/tests/lang/*.ast
/perl/lib/Nix/Config.pm
+1 -1
View File
@@ -1 +1 @@
2.17.2
2.16.4
+1 -2
View File
@@ -5,6 +5,7 @@ We appreciate your support.
Reading and following these guidelines will help us make the contribution process easy and effective for everyone involved.
## Report a bug
1. Check on the [GitHub issue tracker](https://github.com/NixOS/nix/issues) if your bug was already reported.
@@ -29,8 +30,6 @@ Check out the [security policy](https://github.com/NixOS/nix/security/policy).
You can use [labels](https://github.com/NixOS/nix/labels) to filter for relevant topics.
2. Search for related issues that cover what you're going to work on. It could help to mention there that you will work on the issue.
Issues labeled ["good first issue"](https://github.com/NixOS/nix/labels/good-first-issue) should be relatively easy to fix and are likely to get merged quickly.
Pull requests addressing issues labeled ["idea approved"](https://github.com/NixOS/nix/labels/idea%20approved) are especially welcomed by maintainers and will receive prioritised review.
3. Check the [Nix reference manual](https://nixos.org/manual/nix/unstable/contributing/hacking.html) for information on building Nix and running its tests.
-3
View File
@@ -27,9 +27,6 @@ makefiles += \
src/libstore/tests/local.mk \
src/libexpr/tests/local.mk \
tests/local.mk \
tests/ca/local.mk \
tests/dyn-drv/local.mk \
tests/test-libstoreconsumer/local.mk \
tests/plugins/local.mk
else
makefiles += \
+1 -8
View File
@@ -5,14 +5,7 @@ AC_CONFIG_AUX_DIR(config)
AC_PROG_SED
# Construct a Nix system name (like "i686-linux"):
# https://www.gnu.org/software/autoconf/manual/html_node/Canonicalizing.html#index-AC_005fCANONICAL_005fHOST-1
# The inital value is produced by the `config/config.guess` script:
# upstream: https://git.savannah.gnu.org/cgit/config.git/tree/config.guess
# It has the following form, which is not documented anywhere:
# <cpu>-<vendor>-<os>[<version>][-<abi>]
# If `./configure` is passed any of the `--host`, `--build`, `--target` options, the value comes from `config/config.sub` instead:
# upstream: https://git.savannah.gnu.org/cgit/config.git/tree/config.sub
# Construct a Nix system name (like "i686-linux").
AC_CANONICAL_HOST
AC_MSG_CHECKING([for the canonical Nix system name])
+22
View File
@@ -1,3 +1,25 @@
:root {
--sidebar-width: 23em;
}
h1.menu-title::before {
content: "";
background-image: url("./favicon.svg");
padding: 1.25em;
background-position: center center;
background-size: 2em;
background-repeat: no-repeat;
}
h1.menu-title {
padding: 0.5em;
}
.sidebar .sidebar-scrollbox {
padding: 1em;
}
h1:not(:first-of-type) {
margin-top: 1.3em;
}
-31
View File
@@ -1,31 +0,0 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
in
builtinsInfo:
let
showBuiltin = name: { doc, type, impure-only }:
let
type' = optionalString (type != null) " (${type})";
impureNotice = optionalString impure-only ''
> **Note**
>
> Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
'';
in
squash ''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name}</code></a>${type'}
</dt>
<dd>
${doc}
${impureNotice}
</dd>
'';
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
+7 -11
View File
@@ -1,28 +1,24 @@
let
inherit (builtins) concatStringsSep attrValues mapAttrs;
inherit (import ./utils.nix) optionalString squash;
inherit (builtins) concatStringsSep attrNames;
in
builtinsInfo:
let
showBuiltin = name: { doc, args, arity, experimental-feature }:
showBuiltin = name:
let
experimentalNotice = optionalString (experimental-feature != null) ''
This function is only available if the [${experimental-feature}](@docroot@/contributing/experimental-features.md#xp-feature-${experimental-feature}) experimental feature is enabled.
'';
inherit (builtinsInfo.${name}) doc args;
in
squash ''
''
<dt id="builtins-${name}">
<a href="#builtins-${name}"><code>${name} ${listArgs args}</code></a>
</dt>
<dd>
${doc}
${experimentalNotice}
${doc}
</dd>
'';
listArgs = args: concatStringsSep " " (map (s: "<var>${s}</var>") args);
in
concatStringsSep "\n" (attrValues (mapAttrs showBuiltin builtinsInfo))
concatStringsSep "\n" (map showBuiltin (attrNames builtinsInfo))
+5 -11
View File
@@ -128,20 +128,14 @@ $(d)/xp-features.json: $(bindir)/nix
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-xp-features > $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtins.md: $(d)/language.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(bindir)/nix
$(d)/src/language/builtins.md: $(d)/builtins.json $(d)/generate-builtins.nix $(d)/src/language/builtins-prefix.md $(bindir)/nix
@cat doc/manual/src/language/builtins-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<)).builtins' >> $@.tmp;
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtins.nix (builtins.fromJSON (builtins.readFile $<))' >> $@.tmp;
@cat doc/manual/src/language/builtins-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/src/language/builtin-constants.md: $(d)/language.json $(d)/generate-builtin-constants.nix $(d)/src/language/builtin-constants-prefix.md $(bindir)/nix
@cat doc/manual/src/language/builtin-constants-prefix.md > $@.tmp
$(trace-gen) $(nix-eval) --expr 'import doc/manual/generate-builtin-constants.nix (builtins.fromJSON (builtins.readFile $<)).constants' >> $@.tmp;
@cat doc/manual/src/language/builtin-constants-suffix.md >> $@.tmp
@mv $@.tmp $@
$(d)/language.json: $(bindir)/nix
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-language > $@.tmp
$(d)/builtins.json: $(bindir)/nix
$(trace-gen) $(dummy-env) NIX_PATH=nix/corepkgs=corepkgs $(bindir)/nix __dump-builtins > $@.tmp
@mv $@.tmp $@
# Generate the HTML manual.
@@ -173,7 +167,7 @@ doc/manual/generated/man1/nix3-manpages: $(d)/src/command-ref/new-cli
done
@touch $@
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/language/builtin-constants.md
$(docdir)/manual/index.html: $(MANUAL_SRCS) $(d)/book.toml $(d)/anchors.jq $(d)/custom.css $(d)/src/SUMMARY.md $(d)/src/command-ref/new-cli $(d)/src/contributing/experimental-feature-descriptions.md $(d)/src/command-ref/conf-file.md $(d)/src/language/builtins.md $(d)/src/favicon.png $(d)/src/favicon.svg
$(trace-gen) \
tmp="$$(mktemp -d)"; \
cp -r doc/manual "$$tmp"; \
+4 -16
View File
@@ -284,7 +284,7 @@ const redirects = {
"chap-introduction": "introduction.html",
"ch-basic-package-mgmt": "package-management/basic-package-mgmt.html",
"ssec-binary-cache-substituter": "package-management/binary-cache-substituter.html",
"sec-channels": "command-ref/nix-channel.html",
"sec-channels": "package-management/channels.html",
"ssec-copy-closure": "package-management/copy-closure.html",
"sec-garbage-collection": "package-management/garbage-collection.html",
"ssec-gc-roots": "package-management/garbage-collector-roots.html",
@@ -333,31 +333,19 @@ const redirects = {
"ssec-relnotes-2.0": "release-notes/rl-2.0.html",
"ssec-relnotes-2.1": "release-notes/rl-2.1.html",
"ssec-relnotes-2.2": "release-notes/rl-2.2.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html",
"ssec-relnotes-2.3": "release-notes/rl-2.3.html"
},
"language/values.html": {
"simple-values": "#primitives",
"lists": "#list",
"strings": "#string",
"lists": "#list",
"attribute-sets": "#attribute-set",
"attribute-sets": "#attribute-set"
},
"installation/installing-binary.html": {
"linux": "uninstall.html#linux",
"macos": "uninstall.html#macos",
"uninstalling": "uninstall.html",
}
"contributing/hacking.html": {
"nix-with-flakes": "#building-nix-with-flakes",
"classic-nix": "#building-nix",
"running-tests": "testing.html#running-tests",
"unit-tests": "testing.html#unit-tests",
"functional-tests": "testing.html#functional-tests",
"debugging-failing-functional-tests": "testing.html#debugging-failing-functional-tests",
"integration-tests": "testing.html#integration-tests",
"installer-tests": "testing.html#installer-tests",
"one-time-setup": "testing.html#one-time-setup",
"using-the-ci-generated-installer-for-manual-testing": "testing.html#using-the-ci-generated-installer-for-manual-testing",
"uninstalling": "uninstall.html"
}
};
+2 -5
View File
@@ -21,6 +21,7 @@
- [Profiles](package-management/profiles.md)
- [Garbage Collection](package-management/garbage-collection.md)
- [Garbage Collector Roots](package-management/garbage-collector-roots.md)
- [Channels](package-management/channels.md)
- [Sharing Packages Between Machines](package-management/sharing-packages.md)
- [Serving a Nix store via HTTP](package-management/binary-cache-substituter.md)
- [Copying Closures via SSH](package-management/copy-closure.md)
@@ -96,19 +97,15 @@
- [manifest.json](command-ref/files/manifest.json.md)
- [Channels](command-ref/files/channels.md)
- [Default Nix expression](command-ref/files/default-nix-expression.md)
- [Architecture and Design](architecture/architecture.md)
- [File System Object](architecture/file-system-object.md)
- [Architecture](architecture/architecture.md)
- [Protocols](protocols/protocols.md)
- [Serving Tarball Flakes](protocols/tarball-fetcher.md)
- [Glossary](glossary.md)
- [Contributing](contributing/contributing.md)
- [Hacking](contributing/hacking.md)
- [Testing](contributing/testing.md)
- [Experimental Features](contributing/experimental-features.md)
- [CLI guideline](contributing/cli-guideline.md)
- [C++ style guide](contributing/cxx.md)
- [Release Notes](release-notes/release-notes.md)
- [Release 2.17 (2023-07-24)](release-notes/rl-2.17.md)
- [Release 2.16 (2023-05-31)](release-notes/rl-2.16.md)
- [Release 2.15 (2023-04-11)](release-notes/rl-2.15.md)
- [Release 2.14 (2023-02-28)](release-notes/rl-2.14.md)
+3 -3
View File
@@ -7,11 +7,11 @@ It should help users understand why Nix behaves as it does, and it should help d
Nix consists of [hierarchical layers].
[hierarchical layers]: https://en.wikipedia.org/wiki/Multitier_architecture#Layers
[hierarchical layers]: https://en.m.wikipedia.org/wiki/Multitier_architecture#Layers
The following [concept map] shows its main components (rectangles), the objects they operate on (rounded rectangles), and their interactions (connecting phrases):
[concept map]: https://en.wikipedia.org/wiki/Concept_map
[concept map]: https://en.m.wikipedia.org/wiki/Concept_map
```
@@ -76,7 +76,7 @@ The result of a build task can be input to another build task.
The following [data flow diagram] shows a build plan for illustration.
Build inputs used as instructions to a build task are marked accordingly:
[data flow diagram]: https://en.wikipedia.org/wiki/Data-flow_diagram
[data flow diagram]: https://en.m.wikipedia.org/wiki/Data-flow_diagram
```
+--------------------------------------------------------------------+
@@ -1,64 +0,0 @@
# File System Object
Nix uses a simplified model of the file system, which consists of file system objects.
Every file system object is one of the following:
- File
- A possibly empty sequence of bytes for contents
- A single boolean representing the [executable](https://en.m.wikipedia.org/wiki/File-system_permissions#Permissions) permission
- Directory
Mapping of names to child file system objects
- [Symbolic link](https://en.m.wikipedia.org/wiki/Symbolic_link)
An arbitrary string.
Nix does not assign any semantics to symbolic links.
File system objects and their children form a tree.
A bare file or symlink can be a root file system object.
Nix does not encode any other file system notions such as [hard links](https://en.m.wikipedia.org/wiki/Hard_link), [permissions](https://en.m.wikipedia.org/wiki/File-system_permissions), timestamps, or other metadata.
## Examples of file system objects
A plain file:
```
50 B, executable: false
```
An executable file:
```
122 KB, executable: true
```
A symlink:
```
-> /usr/bin/sh
```
A directory with contents:
```
├── bin
│   └── hello: 35 KB, executable: true
└── share
├── info
│   └── hello.info: 36 KB, executable: false
└── man
└── man1
└── hello.1.gz: 790 B, executable: false
```
A directory that contains a symlink and other directories:
```
├── bin -> share/go/bin
├── nix-support/
└── share/
```
+34 -52
View File
@@ -4,67 +4,49 @@
# Description
Nix supports a variety of configuration settings, which are read from configuration files or taken as command line flags.
By default Nix reads settings from the following places:
## Configuration file
- The system-wide configuration file `sysconfdir/nix/nix.conf` (i.e.
`/etc/nix/nix.conf` on most systems), or `$NIX_CONF_DIR/nix.conf` if
`NIX_CONF_DIR` is set. Values loaded in this file are not forwarded
to the Nix daemon. The client assumes that the daemon has already
loaded them.
By default Nix reads settings from the following places, in that order:
- If `NIX_USER_CONF_FILES` is set, then each path separated by `:`
will be loaded in reverse order.
1. The system-wide configuration file `sysconfdir/nix/nix.conf` (i.e. `/etc/nix/nix.conf` on most systems), or `$NIX_CONF_DIR/nix.conf` if [`NIX_CONF_DIR`](./env-common.md#env-NIX_CONF_DIR) is set.
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS`
and `XDG_CONFIG_HOME`. If unset, `XDG_CONFIG_DIRS` defaults to
`/etc/xdg`, and `XDG_CONFIG_HOME` defaults to `$HOME/.config`
as per [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
Values loaded in this file are not forwarded to the Nix daemon.
The client assumes that the daemon has already loaded them.
- If `NIX_CONFIG` is set, its contents is treated as the contents of
a configuration file.
1. If [`NIX_USER_CONF_FILES`](./env-common.md#env-NIX_USER_CONF_FILES) is set, then each path separated by `:` will be loaded in reverse order.
The configuration files consist of `name = value` pairs, one per
line. Other files can be included with a line like `include path`,
where *path* is interpreted relative to the current conf file and a
missing file is an error unless `!include` is used instead. Comments
start with a `#` character. Here is an example configuration file:
Otherwise it will look for `nix/nix.conf` files in `XDG_CONFIG_DIRS` and [`XDG_CONFIG_HOME`](./env-common.md#env-XDG_CONFIG_HOME).
If unset, `XDG_CONFIG_DIRS` defaults to `/etc/xdg`, and `XDG_CONFIG_HOME` defaults to `$HOME/.config` as per [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
keep-outputs = true # Nice for developers
keep-derivations = true # Idem
1. If [`NIX_CONFIG`](./env-common.md#env-NIX_CONFIG) is set, its contents are treated as the contents of a configuration file.
You can override settings on the command line using the `--option`
flag, e.g. `--option keep-outputs false`. Every configuration setting
also has a corresponding command line flag, e.g. `--max-jobs 16`; for
Boolean settings, there are two flags to enable or disable the setting
(e.g. `--keep-failed` and `--no-keep-failed`).
### File format
A configuration setting usually overrides any previous value. However,
you can prefix the name of the setting by `extra-` to *append* to the
previous value. For instance,
Configuration files consist of `name = value` pairs, one per line.
Comments start with a `#` character.
substituters = a b
extra-substituters = c d
Example:
defines the `substituters` setting to be `a b c d`. This is also
available as a command line flag (e.g. `--extra-substituters`).
```
keep-outputs = true # Nice for developers
keep-derivations = true # Idem
```
Other files can be included with a line like `include <path>`, where `<path>` is interpreted relative to the current configuration file.
A missing file is an error unless `!include` is used instead.
A configuration setting usually overrides any previous value.
However, for settings that take a list of items, you can prefix the name of the setting by `extra-` to *append* to the previous value.
For instance,
```
substituters = a b
extra-substituters = c d
```
defines the `substituters` setting to be `a b c d`.
Unknown option names are not an error, and are simply ignored with a warning.
## Command line flags
Configuration options can be set on the command line, overriding the values set in the [configuration file](#configuration-file):
- Every configuration setting has corresponding command line flag (e.g. `--max-jobs 16`).
Boolean settings do not need an argument, and can be explicitly disabled with the `no-` prefix (e.g. `--keep-failed` and `--no-keep-failed`).
Unknown option names are invalid flags (unless there is already a flag with that name), and are rejected with an error.
- The flag `--option <name> <value>` is interpreted exactly like a `<name> = <value>` in a setting file.
Unknown option names are ignored with a warning.
The `extra-` prefix is supported for settings that take a list of items (e.g. `--extra-trusted users alice` or `--option extra-trusted-users alice`).
# Available settings
The following settings are currently available:
+2 -5
View File
@@ -51,9 +51,8 @@ derivation).
# Options
All options not listed here are passed to
[`nix-store --realise`](nix-store/realise.md),
except for `--arg` and `--attr` / `-A` which are passed to [`nix-instantiate`](nix-instantiate.md).
All options not listed here are passed to `nix-store --realise`,
except for `--arg` and `--attr` / `-A` which are passed to `nix-instantiate`.
- <span id="opt-no-out-link">[`--no-out-link`](#opt-no-out-link)<span>
@@ -70,8 +69,6 @@ except for `--arg` and `--attr` / `-A` which are passed to [`nix-instantiate`](n
Change the name of the symlink to the output path created from
`result` to *outlink*.
{{#include ./status-build-failure.md}}
{{#include ./opt-common.md}}
{{#include ./env-common.md}}
+31 -55
View File
@@ -4,63 +4,49 @@
# Synopsis
`nix-channel` {`--add` url [*name*] | `--remove` *name* | `--list` | `--update` [*names…*] | `--list-generations` | `--rollback` [*generation*] }
`nix-channel` {`--add` url [*name*] | `--remove` *name* | `--list` | `--update` [*names…*] | `--rollback` [*generation*] }
# Description
Channels are a mechanism for referencing remote Nix expressions and conveniently retrieving their latest version.
A Nix channel is a mechanism that allows you to automatically stay
up-to-date with a set of pre-built Nix expressions. A Nix channel is
just a URL that points to a place containing a set of Nix expressions.
The moving parts of channels are:
- The official channels listed at <https://nixos.org/channels>
- The user-specific list of [subscribed channels](#subscribed-channels)
- The [downloaded channel contents](#channels)
- The [Nix expression search path](@docroot@/command-ref/conf-file.md#conf-nix-path), set with the [`-I` option](#opt-i) or the [`NIX_PATH` environment variable](#env-NIX_PATH)
> **Note**
>
> The state of a subscribed channel is external to the Nix expressions relying on it.
> This may limit reproducibility.
>
> Dependencies on other Nix expressions can be declared explicitly with:
> - [`fetchurl`](@docroot@/language/builtins.md#builtins-fetchurl), [`fetchTarball`](@docroot@/language/builtins.md#builtins-fetchTarball), or [`fetchGit`](@docroot@/language/builtins.md#builtins-fetchGit) in Nix expressions
> - the [`-I` option](@docroot@/command-ref/opt-common.md#opt-I) in command line invocations
To see the list of official NixOS channels, visit
<https://nixos.org/channels>.
This command has the following operations:
- `--add` *url* \[*name*\]\
Add a channel *name* located at *url* to the list of subscribed channels.
If *name* is omitted, default to the last component of *url*, with the suffixes `-stable` or `-unstable` removed.
> **Note**
>
> `--add` does not automatically perform an update.
> Use `--update` explicitly.
Adds a channel named *name* with URL *url* to the list of subscribed
channels. If *name* is omitted, it defaults to the last component of
*url*, with the suffixes `-stable` or `-unstable` removed.
A channel URL must point to a directory containing a file `nixexprs.tar.gz`.
At the top level, that tarball must contain a single directory with a `default.nix` file that serves as the channel’s entry point.
- `--remove` *name*\
Remove the channel *name* from the list of subscribed channels.
Removes the channel named *name* from the list of subscribed
channels.
- `--list`\
Print the names and URLs of all subscribed channels on standard output.
Prints the names and URLs of all subscribed channels on standard
output.
- `--update` \[*names*…\]\
Download the Nix expressions of subscribed channels and create a new generation.
Update all channels if none is specified, and only those included in *names* otherwise.
- `--list-generations`\
Prints a list of all the current existing generations for the
channel profile.
Works the same way as
```
nix-env --profile /nix/var/nix/profiles/per-user/$USER/channels --list-generations
```
Downloads the Nix expressions of all subscribed channels (or only
those included in *names* if specified) and makes them the default
for `nix-env` operations (by symlinking them from the directory
`~/.nix-defexpr`).
- `--rollback` \[*generation*\]\
Revert channels to the state before the last call to `nix-channel --update`.
Optionally, you can specify a specific channel *generation* number to restore.
Reverts the previous call to `nix-channel
--update`. Optionally, you can specify a specific channel generation
number to restore.
Note that `--add` does not automatically perform an update.
The list of subscribed channels is stored in `~/.nix-channels`.
{{#include ./opt-common.md}}
@@ -74,33 +60,23 @@ This command has the following operations:
# Examples
Subscribe to the Nixpkgs channel and run `hello` from the GNU Hello package:
To subscribe to the Nixpkgs channel and install the GNU Hello package:
```console
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
$ nix-channel --list
nixpkgs https://nixos.org/channels/nixpkgs
$ nix-channel --update
$ nix-shell -p hello --run hello
hello
$ nix-env --install --attr nixpkgs.hello
```
Revert channel updates using `--rollback`:
You can revert channel updates using `--rollback`:
```console
$ nix-instantiate --eval '<nixpkgs>' --attr lib.version
"22.11pre296212.530a53dcbc9"
$ nix-instantiate --eval --expr '(import <nixpkgs> {}).lib.version'
"14.04.527.0e935f1"
$ nix-channel --rollback
switching from generation 483 to 482
$ nix-instantiate --eval '<nixpkgs>' --attr lib.version
"22.11pre281526.d0419badfad"
```
Remove a channel:
```console
$ nix-channel --remove nixpkgs
$ nix-channel --list
$ nix-instantiate --eval --expr '(import <nixpkgs> {}).lib.version'
"14.04.526.dbadfad"
```
@@ -1,6 +1,6 @@
# Name
`nix-collect-garbage` - delete unreachable [store objects]
`nix-collect-garbage` - delete unreachable store paths
# Synopsis
@@ -8,57 +8,17 @@
# Description
The command `nix-collect-garbage` is mostly an alias of [`nix-store --gc`](@docroot@/command-ref/nix-store/gc.md).
That is, it deletes all unreachable [store objects] in the Nix store to clean up your system.
However, it provides two additional options,
[`--delete-old`](#opt-delete-old) and [`--delete-older-than`](#opt-delete-older-than),
which also delete old [profiles], allowing potentially more [store objects] to be deleted because profiles are also garbage collection roots.
These options are the equivalent of running
[`nix-env --delete-generations`](@docroot@/command-ref/nix-env/delete-generations.md)
with various augments on multiple profiles,
prior to running `nix-collect-garbage` (or just `nix-store --gc`) without any flags.
> **Note**
>
> Deleting previous configurations makes rollbacks to them impossible.
These flags should be used with care, because they potentially delete generations of profiles used by other users on the system.
## Locations searched for profiles
`nix-collect-garbage` cannot know about all profiles; that information doesn't exist.
Instead, it looks in a few locations, and acts on all profiles it finds there:
1. The default profile locations as specified in the [profiles] section of the manual.
2. > **NOTE**
>
> Not stable; subject to change
>
> Do not rely on this functionality; it just exists for migration purposes and is may change in the future.
> These deprecated paths remain a private implementation detail of Nix.
`$NIX_STATE_DIR/profiles` and `$NIX_STATE_DIR/profiles/per-user`.
With the exception of `$NIX_STATE_DIR/profiles/per-user/root` and `$NIX_STATE_DIR/profiles/default`, these directories are no longer used by other commands.
`nix-collect-garbage` looks there anyways in order to clean up profiles from older versions of Nix.
# Options
These options are for deleting old [profiles] prior to deleting unreachable [store objects].
- <span id="opt-delete-old">[`--delete-old`](#opt-delete-old)</span> / `-d`\
Delete all old generations of profiles.
This is the equivalent of invoking `nix-env --delete-generations old` on each found profile.
- <span id="opt-delete-older-than">[`--delete-older-than`](#opt-delete-older-than)</span> *period*\
Delete all generations of profiles older than the specified amount (except for the generations that were active at that point in time).
*period* is a value such as `30d`, which would mean 30 days.
This is the equivalent of invoking [`nix-env --delete-generations <period>`](@docroot@/command-ref/nix-env/delete-generations.md#generations-time) on each found profile.
See the documentation of that command for additional information about the *period* argument.
The command `nix-collect-garbage` is mostly an alias of [`nix-store
--gc`](@docroot@/command-ref/nix-store/gc.md), that is, it deletes all
unreachable paths in the Nix store to clean up your system. However,
it provides two additional options: `-d` (`--delete-old`), which
deletes all old generations of all profiles in `/nix/var/nix/profiles`
by invoking `nix-env --delete-generations old` on all profiles (of
course, this makes rollbacks to previous configurations impossible);
and `--delete-older-than` *period*, where period is a value such as
`30d`, which deletes all generations older than the specified number
of days in all profiles in `/nix/var/nix/profiles` (except for the
generations that were active at that point in time).
{{#include ./opt-common.md}}
@@ -72,6 +32,3 @@ generations of each profile, do
```console
$ nix-collect-garbage -d
```
[profiles]: @docroot@/command-ref/files/profiles.md
[store objects]: @docroot@/glossary.md#gloss-store-object
@@ -9,47 +9,14 @@
# Description
This operation deletes the specified generations of the current profile.
*generations* can be a one of the following:
- <span id="generations-list">`<number>...`</span>:\
A list of generation numbers, each one a separate command-line argument.
Delete exactly the profile generations given by their generation number.
Deleting the current generation is not allowed.
- The special value <span id="generations-old">`old`</span>
Delete all generations except the current one.
> **WARNING**
>
> Older *and newer* generations will be deleted by this operation.
>
> One might expect this to just delete older generations than the curent one, but that is only true if the current generation is also the latest.
> Because one can roll back to a previous generation, it is possible to have generations newer than the current one.
> They will also be deleted.
- <span id="generations-time">`<number>d`</span>:\
The last *number* days
*Example*: `30d`
Delete all generations created more than *number* days ago, except the most recent one of them.
This allows rolling back to generations that were available within the specified period.
- <span id="generations-count">`+<number>`</span>:\
The last *number* generations up to the present
*Example*: `+5`
Keep the last *number* generations, along with any newer than current.
Periodically deleting old generations is important to make garbage collection
effective.
The is because profiles are also garbage collection roots — any [store object] reachable from a profile is "alive" and ineligible for deletion.
[store object]: @docroot@/glossary.md#gloss-store-object
The generations can be a list of generation numbers, the special value
`old` to delete all non-current generations, a value such as `30d` to
delete all generations older than the specified number of days (except
for the generation that was active at that point in time), or a value
such as `+5` to keep the last `5` generations ignoring any newer than
current, e.g., if `30` is the current generation `+5` will delete
generation `25` and all older generations. Periodically deleting old
generations is important to make garbage collection effective.
{{#include ./opt-common.md}}
@@ -61,35 +28,19 @@ The is because profiles are also garbage collection roots — any [store object]
# Examples
## Delete explicit generation numbers
```console
$ nix-env --delete-generations 3 4 8
```
Delete the generations numbered 3, 4, and 8, so long as the current active generation is not any of those.
## Keep most-recent by count (number of generations)
```console
$ nix-env --delete-generations +5
```
Suppose `30` is the current generation, and we currently have generations numbered `20` through `32`.
Then this command will delete generations `20` through `25` (`<= 30 - 5`),
and keep generations `26` through `31` (`> 30 - 5`).
## Keep most-recent by time (number of days)
```console
$ nix-env --delete-generations 30d
```
This command will delete all generations older than 30 days, except for the generation that was active 30 days ago (if it currently exists).
## Delete all older
```console
$ nix-env --profile other_profile --delete-generations old
```
@@ -54,7 +54,36 @@ The following flags are available:
previous build, the new output path is left in
`/nix/store/name.check.`
{{#include ../status-build-failure.md}}
Special exit codes:
- `100`\
Generic build failure, the builder process returned with a non-zero
exit code.
- `101`\
Build timeout, the build was aborted because it did not complete
within the specified `timeout`.
- `102`\
Hash mismatch, the build output was rejected because it does not
match the [`outputHash` attribute of the
derivation](@docroot@/language/advanced-attributes.md).
- `104`\
Not deterministic, the build succeeded in check mode but the
resulting output is not binary reproducible.
With the `--keep-going` flag it's possible for multiple failures to
occur, in this case the 1xx status codes are or combined using binary
or.
1100100
^^^^
|||`- timeout
||`-- output hash mismatch
|`--- build failure
`---- not deterministic
{{#include ./opt-common.md}}
@@ -1,34 +0,0 @@
# Special exit codes for build failure
1xx status codes are used when requested builds failed.
The following codes are in use:
- `100` Generic build failure
The builder process returned with a non-zero exit code.
- `101` Build timeout
The build was aborted because it did not complete within the specified `timeout`.
- `102` Hash mismatch
The build output was rejected because it does not match the
[`outputHash` attribute of the derivation](@docroot@/language/advanced-attributes.md).
- `104` Not deterministic
The build succeeded in check mode but the resulting output is not binary reproducible.
With the `--keep-going` flag it's possible for multiple failures to occur.
In this case the 1xx status codes are or combined using
[bitwise OR](https://en.wikipedia.org/wiki/Bitwise_operation#OR).
```
0b1100100
^^^^
|||`- timeout
||`-- output hash mismatch
|`--- build failure
`---- not deterministic
```
-28
View File
@@ -1,28 +0,0 @@
# C++ style guide
Some miscellaneous notes on how we write C++.
Formatting we hope to eventually normalize automatically, so this section is free to just discuss higher-level concerns.
## The `*-impl.hh` pattern
Let's start with some background info first.
Headers, are supposed to contain declarations, not definitions.
This allows us to change a definition without changing the declaration, and have a very small rebuild during development.
Templates, however, need to be specialized to use-sites.
Absent fancier techniques, templates require that the definition, not just mere declaration, must be available at use-sites in order to make that specialization on the fly as part of compiling those use-sites.
Making definitions available like that means putting them in headers, but that is unfortunately means we get all the extra rebuilds we want to avoid by just putting declarations there as described above.
The `*-impl.hh` pattern is a ham-fisted partial solution to this problem.
It constitutes:
- Declaring items only in the main `foo.hh`, including templates.
- Putting template definitions in a companion `foo-impl.hh` header.
Most C++ developers would accompany this by having `foo.hh` include `foo-impl.hh`, to ensure any file getting the template declarations also got the template definitions.
But we've found not doing this has some benefits and fewer than imagined downsides.
The fact remains that headers are rarely as minimal as they could be;
there is often code that needs declarations from the headers but not the templates within them.
With our pattern where `foo.hh` doesn't include `foo-impl.hh`, that means they can just include `foo.hh`
Code that needs both just includes `foo.hh` and `foo-impl.hh`.
This does make linking error possible where something forgets to include `foo-impl.hh` that needs it, but those are build-time only as easy to fix.
+197 -61
View File
@@ -12,15 +12,14 @@ The following instructions assume you already have some version of Nix installed
[installation instructions]: ../installation/installation.md
## Building Nix with flakes
## Nix with flakes
This section assumes you are using Nix with the [`flakes`] and [`nix-command`] experimental features enabled.
See the [Building Nix](#building-nix) section for equivalent instructions using stable Nix interfaces.
This section assumes you are using Nix with [flakes] enabled. See the [next section](#classic-nix) for equivalent instructions which don't require flakes.
[`flakes`]: @docroot@/contributing/experimental-features.md#xp-feature-flakes
[`nix-command`]: @docroot@/contributing/experimental-features.md#xp-nix-command
[flakes]: ../command-ref/new-cli/nix3-flake.md#description
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
To build all dependencies and start a shell in which all environment
variables are set up so that those dependencies can be found:
```console
$ nix develop
@@ -56,17 +55,20 @@ To install it in `$(pwd)/outputs` and test it:
nix (Nix) 2.12
```
To build a release version of Nix for the current operating system and CPU architecture:
To build a release version of Nix:
```console
$ nix build
```
You can also build Nix for one of the [supported platforms](#platforms).
You can also build Nix for one of the [supported target platforms](#target-platforms).
## Building Nix
## Classic Nix
To build all dependencies and start a shell in which all environment variables are set up so that those dependencies can be found:
This section is for Nix without [flakes].
To build all dependencies and start a shell in which all environment
variables are set up so that those dependencies can be found:
```console
$ nix-shell
@@ -100,82 +102,51 @@ To install it in `$(pwd)/outputs` and test it:
nix (Nix) 2.12
```
To build a release version of Nix for the current operating system and CPU architecture:
To build Nix for the current operating system and CPU architecture use
```console
$ nix-build
```
You can also build Nix for one of the [supported platforms](#platforms).
You can also build Nix for one of the [supported target platforms](#target-platforms).
## Platforms
Nix can be built for various platforms, as specified in [`flake.nix`]:
As specified in [`flake.nix`], Nix can be built for various platforms:
- `aarch64-linux`
- `i686-linux`
- `x86_64-darwin`
- `x86_64-linux`
[`flake.nix`]: https://github.com/nixos/nix/blob/master/flake.nix
- `x86_64-linux`
- `x86_64-darwin`
- `i686-linux`
- `aarch64-linux`
- `aarch64-darwin`
- `armv6l-linux`
- `armv7l-linux`
In order to build Nix for a different platform than the one you're currently
on, you need a way for your current Nix installation to build code for that
platform. Common solutions include [remote builders] and [binary format emulation]
on, you need to have some way for your system Nix to build code for that
platform. Common solutions include [remote builders] and [binfmt emulation]
(only supported on NixOS).
[remote builders]: ../advanced-topics/distributed-builds.md
[binary format emulation]: https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems
[binfmt emulation]: https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems
Given such a setup, executing the build only requires selecting the respective attribute.
For example, to compile for `aarch64-linux`:
```console
$ nix-build --attr packages.aarch64-linux.default
```
or for Nix with the [`flakes`] and [`nix-command`] experimental features enabled:
These solutions let Nix perform builds as if you're on the native platform, so
executing the build is as simple as
```console
$ nix build .#packages.aarch64-linux.default
```
Cross-compiled builds are available for ARMv6 (`armv6l-linux`) and ARMv7 (`armv7l-linux`).
Add more [system types](#system-type) to `crossSystems` in `flake.nix` to bootstrap Nix on unsupported platforms.
for flake-enabled Nix, or
## System type
Nix uses a string with he following format to identify the *system type* or *platform* it runs on:
```
<cpu>-<os>[-<abi>]
```console
$ nix-build --attr packages.aarch64-linux.default
```
It is set when Nix is compiled for the given system, and based on the output of [`config.guess`](https://github.com/nixos/nix/blob/master/config/config.guess) ([upstream](https://git.savannah.gnu.org/cgit/config.git/tree/config.guess)):
for classic Nix.
```
<cpu>-<vendor>-<os>[<version>][-<abi>]
```
You can use any of the other supported platforms in place of `aarch64-linux`.
When Nix is built such that `./configure` is passed any of the `--host`, `--build`, `--target` options, the value is based on the output of [`config.sub`](https://github.com/nixos/nix/blob/master/config/config.sub) ([upstream](https://git.savannah.gnu.org/cgit/config.git/tree/config.sub)):
```
<cpu>-<vendor>[-<kernel>]-<os>
```
For historic reasons and backward-compatibility, some CPU and OS identifiers are translated from the GNU Autotools naming convention in [`configure.ac`](https://github.com/nixos/nix/blob/master/configure.ac) as follows:
| `config.guess` | Nix |
|----------------------------|---------------------|
| `amd64` | `x86_64` |
| `i*86` | `i686` |
| `arm6` | `arm6l` |
| `arm7` | `arm7l` |
| `linux-gnu*` | `linux` |
| `linux-musl*` | `linux` |
Cross-compiled builds are available for ARMv6 and ARMv7, and Nix on unsupported platforms can be bootstrapped by adding more `crossSystems` in `flake.nix`.
## Compilation environments
@@ -221,6 +192,171 @@ Configure your editor to use the `clangd` from the shell, either by running it i
> Some other editors (e.g. Emacs, Vim) need a plugin to support LSP servers in general (e.g. [lsp-mode](https://github.com/emacs-lsp/lsp-mode) for Emacs and [vim-lsp](https://github.com/prabirshrestha/vim-lsp) for vim).
> Editor-specific setup is typically opinionated, so we will not cover it here in more detail.
## Running tests
### Unit-tests
The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined
under `src/{library_name}/tests` using the
[googletest](https://google.github.io/googletest/) and
[rapidcheck](https://github.com/emil-e/rapidcheck) frameworks.
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option.
### Functional tests
The functional tests reside under the `tests` directory and are listed in `tests/local.mk`.
Each test is a bash script.
The whole test suite can be run with:
```shell-session
$ make install && make installcheck
ran test tests/foo.sh... [PASS]
ran test tests/bar.sh... [PASS]
...
```
Individual tests can be run with `make`:
```shell-session
$ make tests/${testName}.sh.test
ran test tests/${testName}.sh... [PASS]
```
or without `make`:
```shell-session
$ ./mk/run-test.sh tests/${testName}.sh
ran test tests/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
+ foo
output from foo
+ bar
output from bar
...
```
The test script will then be traced with `set -x` and the output displayed as it happens, regardless of whether the test succeeds or fails.
#### Debugging failing functional tests
When a functional test fails, it usually does so somewhere in the middle of the script.
To figure out what's wrong, it is convenient to run the test regularly up to the failing `nix` command, and then run that command with a debugger like GDB.
For example, if the script looks like:
```bash
foo
nix blah blub
bar
```
edit it like so:
```diff
foo
-nix blah blub
+gdb --args nix blah blub
bar
```
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
...
+ gdb blash blub
GNU gdb (GDB) 12.1
...
(gdb)
```
One can debug the Nix invocation in all the usual ways.
For example, enter `run` to start the Nix invocation.
### Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
### Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
- `armv6l-linux`
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
#### One-time setup
1. Have a GitHub account with a fork of the [Nix repository](https://github.com/NixOS/nix).
2. At cachix.org:
- Create or log in to an account.
- Create a Cachix cache using the format `<github-username>-nix-install-tests`.
- Navigate to the new cache > Settings > Auth Tokens.
- Generate a new Cachix auth token and copy the generated value.
3. At github.com:
- Navigate to your Nix fork > Settings > Secrets > Actions > New repository secret.
- Name the secret `CACHIX_AUTH_TOKEN`.
- Paste the copied value of the Cachix cache auth token.
#### Using the CI-generated installer for manual testing
After the CI run completes, you can check the output to extract the installer URL:
1. Click into the detailed view of the CI run.
2. Click into any `installer_test` run (the URL you're here to extract will be the same in all of them).
3. Click into the `Run cachix/install-nix-action@v...` step and click the detail triangle next to the first log line (it will also be `Run cachix/install-nix-action@v...`)
4. Copy the value of `install_url`
5. To generate an install command, plug this `install_url` and your GitHub username into this template:
```console
curl -L <install_url> | sh -s -- --tarball-url-prefix https://<github-username>-nix-install-tests.cachix.org/serve
```
<!-- #### Manually generating test installers
There's obviously a manual way to do this, and it's still the only way for
platforms that lack GA runners.
I did do this back in Fall 2020 (before the GA approach encouraged here). I'll
sketch what I recall in case it encourages someone to fill in detail, but: I
didn't know what I was doing at the time and had to fumble/ask around a lot--
so I don't want to uphold any of it as "right". It may have been dumb or
the _hard_ way from the getgo. Fundamentals may have changed since.
Here's the build command I used to do this on and for x86_64-darwin:
nix build --out-link /tmp/foo ".#checks.x86_64-darwin.binaryTarball"
I used the stable out-link to make it easier to script the next steps:
link=$(readlink /tmp/foo)
cp $link/*-darwin.tar.xz ~/somewheres
I've lost the last steps and am just going from memory:
From here, I think I had to extract and modify the `install` script to point
it at this tarball (which I scped to my own site, but it might make more sense
to just share them locally). I extracted this script once and then just
search/replaced in it for each new build.
The installer now supports a `--tarball-url-prefix` flag which _may_ have
solved this need?
-->
### Checking links in the manual
The build checks for broken internal links.
@@ -242,7 +378,7 @@ rm $(git ls-files doc/manual/ -o | grep -F '.md') && rmdir doc/manual/src/comman
[`mdbook-linkcheck`] does not implement checking [URI fragments] yet.
[`mdbook-linkcheck`]: https://github.com/Michael-F-Bryan/mdbook-linkcheck
[URI fragments]: https://en.wikipedia.org/wiki/URI_fragment
[URI fragments]: https://en.m.wikipedia.org/wiki/URI_fragment
#### `@docroot@` variable
-221
View File
@@ -1,221 +0,0 @@
# Running tests
## Unit-tests
The unit-tests for each Nix library (`libexpr`, `libstore`, etc..) are defined
under `src/{library_name}/tests` using the
[googletest](https://google.github.io/googletest/) and
[rapidcheck](https://github.com/emil-e/rapidcheck) frameworks.
You can run the whole testsuite with `make check`, or the tests for a specific component with `make libfoo-tests_RUN`. Finer-grained filtering is also possible using the [--gtest_filter](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests) command-line option.
## Functional tests
The functional tests reside under the `tests` directory and are listed in `tests/local.mk`.
Each test is a bash script.
### Running the whole test suite
The whole test suite can be run with:
```shell-session
$ make install && make installcheck
ran test tests/foo.sh... [PASS]
ran test tests/bar.sh... [PASS]
...
```
### Grouping tests
Sometimes it is useful to group related tests so they can be easily run together without running the entire test suite.
Each test group is in a subdirectory of `tests`.
For example, `tests/ca/local.mk` defines a `ca` test group for content-addressed derivation outputs.
That test group can be run like this:
```shell-session
$ make ca.test-group -j50
ran test tests/ca/nix-run.sh... [PASS]
ran test tests/ca/import-derivation.sh... [PASS]
...
```
The test group is defined in Make like this:
```makefile
$(test-group-name)-tests := \
$(d)/test0.sh \
$(d)/test1.sh \
...
install-tests-groups += $(test-group-name)
```
### Running individual tests
Individual tests can be run with `make`:
```shell-session
$ make tests/${testName}.sh.test
ran test tests/${testName}.sh... [PASS]
```
or without `make`:
```shell-session
$ ./mk/run-test.sh tests/${testName}.sh
ran test tests/${testName}.sh... [PASS]
```
To see the complete output, one can also run:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
+ foo
output from foo
+ bar
output from bar
...
```
The test script will then be traced with `set -x` and the output displayed as it happens, regardless of whether the test succeeds or fails.
### Debugging failing functional tests
When a functional test fails, it usually does so somewhere in the middle of the script.
To figure out what's wrong, it is convenient to run the test regularly up to the failing `nix` command, and then run that command with a debugger like GDB.
For example, if the script looks like:
```bash
foo
nix blah blub
bar
```
edit it like so:
```diff
foo
-nix blah blub
+gdb --args nix blah blub
bar
```
Then, running the test with `./mk/debug-test.sh` will drop you into GDB once the script reaches that point:
```shell-session
$ ./mk/debug-test.sh tests/${testName}.sh
...
+ gdb blash blub
GNU gdb (GDB) 12.1
...
(gdb)
```
One can debug the Nix invocation in all the usual ways.
For example, enter `run` to start the Nix invocation.
### Characterization testing
Occasionally, Nix utilizes a technique called [Characterization Testing](https://en.wikipedia.org/wiki/Characterization_test) as part of the functional tests.
This technique is to include the exact output/behavior of a former version of Nix in a test in order to check that Nix continues to produce the same behavior going forward.
For example, this technique is used for the language tests, to check both the printed final value if evaluation was successful, and any errors and warnings encountered.
It is frequently useful to regenerate the expected output.
To do that, rerun the failed test with `_NIX_TEST_ACCEPT=1`.
(At least, this is the convention we've used for `tests/lang.sh`.
If we add more characterization testing we should always strive to be consistent.)
An interesting situation to document is the case when these tests are "overfitted".
The language tests are, again, an example of this.
The expected successful output of evaluation is supposed to be highly stable – we do not intend to make breaking changes to (the stable parts of) the Nix language.
However, the errors and warnings during evaluation (successful or not) are not stable in this way.
We are free to change how they are displayed at any time.
It may be surprising that we would test non-normative behavior like diagnostic outputs.
Diagnostic outputs are indeed not a stable interface, but they still are important to users.
By recording the expected output, the test suite guards against accidental changes, and ensure the *result* (not just the code that implements it) of the diagnostic code paths are under code review.
Regressions are caught, and improvements always show up in code review.
To ensure that characterization testing doesn't make it harder to intentionally change these interfaces, there always must be an easy way to regenerate the expected output, as we do with `_NIX_TEST_ACCEPT=1`.
## Integration tests
The integration tests are defined in the Nix flake under the `hydraJobs.tests` attribute.
These tests include everything that needs to interact with external services or run Nix in a non-trivial distributed setup.
Because these tests are expensive and require more than what the standard github-actions setup provides, they only run on the master branch (on <https://hydra.nixos.org/jobset/nix/master>).
You can run them manually with `nix build .#hydraJobs.tests.{testName}` or `nix-build -A hydraJobs.tests.{testName}`
## Installer tests
After a one-time setup, the Nix repository's GitHub Actions continuous integration (CI) workflow can test the installer each time you push to a branch.
Creating a Cachix cache for your installer tests and adding its authorization token to GitHub enables [two installer-specific jobs in the CI workflow](https://github.com/NixOS/nix/blob/88a45d6149c0e304f6eb2efcc2d7a4d0d569f8af/.github/workflows/ci.yml#L50-L91):
- The `installer` job generates installers for the platforms below and uploads them to your Cachix cache:
- `x86_64-linux`
- `armv6l-linux`
- `armv7l-linux`
- `x86_64-darwin`
- The `installer_test` job (which runs on `ubuntu-latest` and `macos-latest`) will try to install Nix with the cached installer and run a trivial Nix command.
### One-time setup
1. Have a GitHub account with a fork of the [Nix repository](https://github.com/NixOS/nix).
2. At cachix.org:
- Create or log in to an account.
- Create a Cachix cache using the format `<github-username>-nix-install-tests`.
- Navigate to the new cache > Settings > Auth Tokens.
- Generate a new Cachix auth token and copy the generated value.
3. At github.com:
- Navigate to your Nix fork > Settings > Secrets > Actions > New repository secret.
- Name the secret `CACHIX_AUTH_TOKEN`.
- Paste the copied value of the Cachix cache auth token.
## Working on documentation
### Using the CI-generated installer for manual testing
After the CI run completes, you can check the output to extract the installer URL:
1. Click into the detailed view of the CI run.
2. Click into any `installer_test` run (the URL you're here to extract will be the same in all of them).
3. Click into the `Run cachix/install-nix-action@v...` step and click the detail triangle next to the first log line (it will also be `Run cachix/install-nix-action@v...`)
4. Copy the value of `install_url`
5. To generate an install command, plug this `install_url` and your GitHub username into this template:
```console
curl -L <install_url> | sh -s -- --tarball-url-prefix https://<github-username>-nix-install-tests.cachix.org/serve
```
<!-- #### Manually generating test installers
There's obviously a manual way to do this, and it's still the only way for
platforms that lack GA runners.
I did do this back in Fall 2020 (before the GA approach encouraged here). I'll
sketch what I recall in case it encourages someone to fill in detail, but: I
didn't know what I was doing at the time and had to fumble/ask around a lot--
so I don't want to uphold any of it as "right". It may have been dumb or
the _hard_ way from the getgo. Fundamentals may have changed since.
Here's the build command I used to do this on and for x86_64-darwin:
nix build --out-link /tmp/foo ".#checks.x86_64-darwin.binaryTarball"
I used the stable out-link to make it easier to script the next steps:
link=$(readlink /tmp/foo)
cp $link/*-darwin.tar.xz ~/somewheres
I've lost the last steps and am just going from memory:
From here, I think I had to extract and modify the `install` script to point
it at this tarball (which I scped to my own site, but it might make more sense
to just share them locally). I extracted this script once and then just
search/replaced in it for each new build.
The installer now supports a `--tarball-url-prefix` flag which _may_ have
solved this need?
-->
Binary file not shown.

After

Width:  |  Height:  |  Size: 1.2 KiB

+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="587.11" height="516.604" viewBox="0 0 550.416 484.317"><defs><linearGradient id="a"><stop offset="0" style="stop-color:#699ad7;stop-opacity:1"/><stop offset=".243" style="stop-color:#7eb1dd;stop-opacity:1"/><stop offset="1" style="stop-color:#7ebae4;stop-opacity:1"/></linearGradient><linearGradient id="b"><stop offset="0" style="stop-color:#415e9a;stop-opacity:1"/><stop offset=".232" style="stop-color:#4a6baf;stop-opacity:1"/><stop offset="1" style="stop-color:#5277c3;stop-opacity:1"/></linearGradient><linearGradient xlink:href="#a" id="c" x1="200.597" x2="290.087" y1="351.411" y2="506.188" gradientTransform="translate(70.65 -1055.151)" gradientUnits="userSpaceOnUse"/><linearGradient xlink:href="#b" id="e" x1="-584.199" x2="-496.297" y1="782.336" y2="937.714" gradientTransform="translate(864.696 -1491.34)" gradientUnits="userSpaceOnUse"/></defs><g style="display:inline;opacity:1" transform="translate(-132.651 958.04)"><path id="d" d="m309.549-710.388 122.197 211.675-56.157.527-32.624-56.87-32.856 56.566-27.903-.011-14.29-24.69 46.81-80.49-33.23-57.826z" style="opacity:1;fill:url(#c);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:3;stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(60 407.112 -715.787)"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(-60 407.312 -715.7)"/><use xlink:href="#d" width="100%" height="100%" transform="rotate(180 407.419 -715.756)"/><path id="f" d="m309.549-710.388 122.197 211.675-56.157.527-32.624-56.87-32.856 56.566-27.903-.011-14.29-24.69 46.81-80.49-33.23-57.826z" style="color:#000;clip-rule:nonzero;display:inline;overflow:visible;visibility:visible;opacity:1;isolation:auto;mix-blend-mode:normal;color-interpolation:sRGB;color-interpolation-filters:linearRGB;solid-color:#000;solid-opacity:1;fill:url(#e);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:3;stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;color-rendering:auto;image-rendering:auto;shape-rendering:auto;text-rendering:auto;enable-background:accumulate"/><use xlink:href="#f" width="100%" height="100%" style="display:inline" transform="rotate(120 407.34 -716.084)"/><use xlink:href="#f" width="100%" height="100%" style="display:inline" transform="rotate(-120 407.288 -715.87)"/></g></svg>

After

Width:  |  Height:  |  Size: 2.5 KiB

+8 -14
View File
@@ -85,17 +85,12 @@
[store path]: #gloss-store-path
- [file system object]{#gloss-store-object}\
The Nix data model for representing simplified file system data.
See [File System Object](@docroot@/architecture/file-system-object.md) for details.
[file system object]: #gloss-file-system-object
- [store object]{#gloss-store-object}\
A store object consists of a [file system object], [reference]s to other store objects, and other metadata.
It can be referred to by a [store path].
A file that is an immediate child of the Nix store directory. These
can be regular files, but also entire directory trees. Store objects
can be sources (objects copied from outside of the store),
derivation outputs (objects produced by running a build task), or
derivations (files describing a build task).
[store object]: #gloss-store-object
@@ -117,10 +112,9 @@
from some server.
- [substituter]{#gloss-substituter}\
An additional [store]{#gloss-store} from which Nix can obtain store objects instead of building them.
Often the substituter is a [binary cache](#gloss-binary-cache), but any store can serve as substituter.
See the [`substituters` configuration option](./command-ref/conf-file.md#conf-substituters) for details.
A *substituter* is an additional store from which Nix will
copy store objects it doesn't have. For details, see the
[`substituters` option](./command-ref/conf-file.md#conf-substituters).
[substituter]: #gloss-substituter
@@ -10,7 +10,7 @@
- Bash Shell. The `./configure` script relies on bashisms, so Bash is
required.
- A version of GCC or Clang that supports C++20.
- A version of GCC or Clang that supports C++17.
- `pkg-config` to locate dependencies. If your distribution does not
provide it, you can get it from
@@ -1,5 +0,0 @@
# Built-in Constants
These constants are built into the Nix language evaluator:
<dl>
@@ -1 +0,0 @@
</dl>
@@ -0,0 +1,43 @@
# Built-in Constants
These constants are built into the Nix language evaluator:
- [`builtins`]{#builtins-builtins} (attribute set)
Contains all the [built-in functions](./builtins.md) and values, in order to avoid polluting the global scope.
Since built-in functions were added over time, [testing for attributes](./operators.md#has-attribute) in `builtins` can be used for graceful fallback on older Nix installations:
```nix
if builtins ? getEnv then builtins.getEnv "PATH" else ""
```
- [`builtins.currentSystem`]{#builtins-currentSystem} (string)
The built-in value `currentSystem` evaluates to the Nix platform
identifier for the Nix installation on which the expression is being
evaluated, such as `"i686-linux"` or `"x86_64-darwin"`.
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
- [`builtins.currentTime`]{#builtins-currentTime} (integer)
Return the [Unix time](https://en.wikipedia.org/wiki/Unix_time) at first evaluation.
Repeated references to that name will re-use the initially obtained value.
Example:
```console
$ nix repl
Welcome to Nix 2.15.1 Type :? for help.
nix-repl> builtins.currentTime
1683705525
nix-repl> builtins.currentTime
1683705525
```
The [store path](@docroot@/glossary.md#gloss-store-path) of a derivation depending on `currentTime` will differ for each evaluation.
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
+1 -1
View File
@@ -3,7 +3,7 @@
This section lists the functions built into the Nix language evaluator.
All built-in functions are available through the global [`builtins`](./builtin-constants.md#builtins-builtins) constant.
For convenience, some built-ins can be accessed directly:
For convenience, some built-ins are can be accessed directly:
- [`derivation`](#builtins-derivation)
- [`import`](#builtins-import)
+35 -63
View File
@@ -2,11 +2,8 @@
## Recursive sets
Recursive sets are like normal [attribute sets](./values.md#attribute-set), but the attributes can refer to each other.
> *rec-attrset* = `rec {` [ *name* `=` *expr* `;` `]`... `}`
Example:
Recursive sets are just normal sets, but the attributes can refer to
each other. For example,
```nix
rec {
@@ -15,9 +12,7 @@ rec {
}.x
```
This evaluates to `123`.
Note that without `rec` the binding `x = y;` would
evaluates to `123`. Note that without `rec` the binding `x = y;` would
refer to the variable `y` in the surrounding scope, if one exists, and
would be invalid if no such variable exists. That is, in a normal
(non-recursive) set, attributes are not added to the lexical scope; in a
@@ -38,10 +33,7 @@ will crash with an `infinite recursion encountered` error message.
## Let-expressions
A let-expression allows you to define local variables for an expression.
> *let-in* = `let` [ *identifier* = *expr* ]... `in` *expr*
Example:
For instance,
```nix
let
@@ -50,19 +42,18 @@ let
in x + y
```
This evaluates to `"foobar"`.
evaluates to `"foobar"`.
## Inheriting attributes
When defining an [attribute set](./values.md#attribute-set) or in a [let-expression](#let-expressions) it is often convenient to copy variables from the surrounding lexical scope (e.g., when you want to propagate attributes).
This can be shortened using the `inherit` keyword.
Example:
When defining a set or in a let-expression it is often convenient to
copy variables from the surrounding lexical scope (e.g., when you want
to propagate attributes). This can be shortened using the `inherit`
keyword. For instance,
```nix
let x = 123; in
{
inherit x;
{ inherit x;
y = 456;
}
```
@@ -71,31 +62,23 @@ is equivalent to
```nix
let x = 123; in
{
x = x;
{ x = x;
y = 456;
}
```
and both evaluate to `{ x = 123; y = 456; }`.
> **Note**
>
> This works because `x` is added to the lexical scope by the `let` construct.
It is also possible to inherit attributes from another attribute set.
Example:
In this fragment from `all-packages.nix`,
and both evaluate to `{ x = 123; y = 456; }`. (Note that this works
because `x` is added to the lexical scope by the `let` construct.) It is
also possible to inherit attributes from another set. For instance, in
this fragment from `all-packages.nix`,
```nix
graphviz = (import ../tools/graphics/graphviz) {
inherit fetchurl stdenv libpng libjpeg expat x11 yacc;
inherit (xorg) libXaw;
inherit (xlibs) libXaw;
};
xorg = {
xlibs = {
libX11 = ...;
libXaw = ...;
...
@@ -109,7 +92,7 @@ libjpg = ...;
the set used in the function call to the function defined in
`../tools/graphics/graphviz` inherits a number of variables from the
surrounding scope (`fetchurl` ... `yacc`), but also inherits `libXaw`
(the X Athena Widgets) from the `xorg` set.
(the X Athena Widgets) from the `xlibs` (X11 client-side libraries) set.
Summarizing the fragment
@@ -208,41 +191,30 @@ three kinds of patterns:
```nix
{ x, y, z, ... } @ args: z + y + x + args.a
```
Here `args` is bound to the argument *as passed*, which is further
matched against the pattern `{ x, y, z, ... }`.
The `@`-pattern makes mainly sense with an ellipsis(`...`) as
Here `args` is bound to the entire argument, which is further
matched against the pattern `{ x, y, z,
... }`. `@`-pattern makes mainly sense with an ellipsis(`...`) as
you can access attribute names as `a`, using `args.a`, which was
given as an additional attribute to the function.
> **Warning**
>
> `args@` binds the name `args` to the attribute set that is passed to the function.
> In particular, `args` does *not* include any default values specified with `?` in the function's set pattern.
>
>
> The `args@` expression is bound to the argument passed to the
> function which means that attributes with defaults that aren't
> explicitly specified in the function call won't cause an
> evaluation error, but won't exist in `args`.
>
> For instance
>
>
> ```nix
> let
> f = args@{ a ? 23, ... }: [ a args ];
> function = args@{ a ? 23, ... }: args;
> in
> f {}
> ```
>
> is equivalent to
>
> ```nix
> let
> f = args @ { ... }: [ (args.a or 23) args ];
> in
> f {}
> ```
>
> and both expressions will evaluate to:
>
> ```nix
> [ 23 {} ]
> ```
> function {}
> ````
>
> will evaluate to an empty attribute set.
Note that functions do not have names. If you want to give them a name,
you can bind them to an attribute, e.g.,
+5 -4
View File
@@ -1,11 +1,12 @@
# Nix Language
The Nix language is designed for conveniently creating and composing *derivations* – precise descriptions of how contents of existing files are used to derive new files.
It is:
The Nix language is
- *domain-specific*
It comes with [built-in functions](@docroot@/language/builtins.md) to integrate with the Nix store, which manages files and performs the derivations declared in the Nix language.
It only exists for the Nix package manager:
to describe packages and configurations as well as their variants and compositions.
It is not intended for general purpose use.
- *declarative*
@@ -24,7 +25,7 @@ It is:
- *lazy*
Values are only computed when they are needed.
Expressions are only evaluated when their value is needed.
- *dynamically typed*
+7 -4
View File
@@ -35,14 +35,17 @@
## Attribute selection
> *attrset* `.` *attrpath* \[ `or` *expr* \]
Select the attribute denoted by attribute path *attrpath* from [attribute set] *attrset*.
If the attribute doesn’t exist, return the *expr* after `or` if provided, otherwise abort evaluation.
An attribute path is a dot-separated list of [attribute names](./values.md#attribute-set).
<!-- FIXME: the following should to into its own language syntax section, but that needs more work to fit in well -->
> *attrpath* = *name* [ `.` *name* ]...
An attribute path is a dot-separated list of attribute names.
An attribute name can be an identifier or a string.
> *attrpath* = *name* [ `.` *name* ]... \
> *name* = *identifier* | *string* \
> *identifier* ~ `[a-zA-Z_][a-zA-Z0-9_'-]*`
[Attribute selection]: #attribute-selection
+4 -16
View File
@@ -164,17 +164,9 @@ Note that lists are only lazy in values, and they are strict in length.
An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
An attribute name can be an identifier or a [string](#string).
An identifier must start with a letter (`a-z`, `A-Z`) or underscore (`_`), and can otherwise contain letters (`a-z`, `A-Z`), numbers (`0-9`), underscores (`_`), apostrophes (`'`), or dashes (`-`).
> *name* = *identifier* | *string* \
> *identifier* ~ `[a-zA-Z_][a-zA-Z0-9_'-]*`
Names and values are separated by an equal sign (`=`).
Each value is an arbitrary expression terminated by a semicolon (`;`).
> *attrset* = `{` [ *name* `=` *expr* `;` `]`... `}`
Attributes can appear in any order.
An attribute name may only occur once.
@@ -190,19 +182,15 @@ Example:
This defines a set with attributes named `x`, `text`, `y`.
Attributes can be accessed with the [`.` operator](./operators.md#attribute-selection).
Example:
Attributes can be selected from a set using the `.` operator. For
instance,
```nix
{ a = "Foo"; b = "Bar"; }.a
```
This evaluates to `"Foo"`.
It is possible to provide a default value in an attribute selection using the `or` keyword.
Example:
evaluates to `"Foo"`. It is possible to provide a default value in an
attribute selection using the `or` keyword:
```nix
{ a = "Foo"; b = "Bar"; }.c or "Xyzzy"
@@ -25,7 +25,7 @@ or completely new ones.)
You can manually download the latest version of Nixpkgs from
<https://github.com/NixOS/nixpkgs>. However, it’s much more
convenient to use the Nixpkgs [*channel*](../command-ref/nix-channel.md), since it makes
convenient to use the Nixpkgs [*channel*](channels.md), since it makes
it easy to stay up to date with new versions of Nixpkgs. Nixpkgs is
automatically added to your list of “subscribed” channels when you
install Nix. If this is not the case for some reason, you can add it
@@ -0,0 +1,50 @@
# Channels
If you want to stay up to date with a set of packages, it’s not very
convenient to manually download the latest set of Nix expressions for
those packages and upgrade using `nix-env`. Fortunately, there’s a
better way: *Nix channels*.
A Nix channel is just a URL that points to a place that contains a set
of Nix expressions and a manifest. Using the command
[`nix-channel`](../command-ref/nix-channel.md) you can automatically
stay up to date with whatever is available at that URL.
To see the list of official NixOS channels, visit
<https://nixos.org/channels>.
You can “subscribe” to a channel using `nix-channel --add`, e.g.,
```console
$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
```
subscribes you to a channel that always contains that latest version of
the Nix Packages collection. (Subscribing really just means that the URL
is added to the file `~/.nix-channels`, where it is read by subsequent
calls to `nix-channel
--update`.) You can “unsubscribe” using `nix-channel
--remove`:
```console
$ nix-channel --remove nixpkgs
```
To obtain the latest Nix expressions available in a channel, do
```console
$ nix-channel --update
```
This downloads and unpacks the Nix expressions in every channel
(downloaded from `url/nixexprs.tar.bz2`). It also makes the union of
each channel’s Nix expressions available by default to `nix-env`
operations (via the symlink `~/.nix-defexpr/channels`). Consequently,
you can then say
```console
$ nix-env --upgrade
```
to upgrade all packages in your profile to the latest versions available
in the subscribed channels.
+5 -7
View File
@@ -1,17 +1,15 @@
# Lockable HTTP Tarball Protocol
# Serving Tarball Flakes
Tarball flakes can be served as regular tarballs via HTTP or the file
system (for `file://` URLs). Unless the server implements the Lockable
HTTP Tarball protocol, it is the responsibility of the user to make sure that
the URL always produces the same tarball contents.
Tarball flakes are served as regular tarballs via HTTP or the file
system (for `file://` URLs).
An HTTP server can return an "immutable" HTTP URL appropriate for lock
An HTTP server can return an "immutable" flakeref appropriate for lock
files. This allows users to specify a tarball flake input in
`flake.nix` that requests the latest version of a flake
(e.g. `https://example.org/hello/latest.tar.gz`), while `flake.lock`
will record a URL whose contents will not change
(e.g. `https://example.org/hello/<revision>.tar.gz`). To do so, the
server must return an [HTTP `Link` header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Link) with the `rel` attribute set to
server must return a `Link` header with the `rel` attribute set to
`immutable`, as follows:
```
-42
View File
@@ -1,42 +0,0 @@
# Release 2.17 (2023-07-24)
* [`nix-channel`](../command-ref/nix-channel.md) now supports a `--list-generations` subcommand.
* The function [`builtins.fetchClosure`](../language/builtins.md#builtins-fetchClosure) can now fetch input-addressed paths in [pure evaluation mode](../command-ref/conf-file.md#conf-pure-eval), as those are not impure.
* Nix now allows unprivileged/[`allowed-users`](../command-ref/conf-file.md#conf-allowed-users) to sign paths.
Previously, only [`trusted-users`](../command-ref/conf-file.md#conf-trusted-users) users could sign paths.
* Nested dynamic attributes are now merged correctly by the parser. For example:
```nix
{
nested = {
foo = 1;
};
nested = {
${"ba" + "r"} = 2;
};
}
```
This used to silently discard `nested.bar`, but now behaves as one would expect and evaluates to:
```nix
{ nested = { bar = 2; foo = 1; }; }
```
Note that the feature of merging multiple *full declarations* of attribute sets like `nested` in the example is of questionable value.
It allows writing expressions that are very hard to read, for instance when there are many lines of code between two declarations of the same attribute.
This has been around for a long time and is therefore supported for backwards compatibility, but should not be relied upon.
Instead, consider using the *nested attribute path* syntax:
```nix
{
nested.foo = 1;
nested.${"ba" + "r"} = 2;
}
```
* Tarball flakes can now redirect to an "immutable" URL that will be recorded in lock files. This allows the use of "mutable" tarball URLs like `https://example.org/hello/latest.tar.gz` in flakes. See the [tarball fetcher](../protocols/tarball-fetcher.md) for details.
+1
View File
@@ -1 +1,2 @@
# Release X.Y (202?-??-??)
-1
View File
@@ -117,7 +117,6 @@ Pull requests in this column are reviewed together during work meetings.
This is both for spreading implementation knowledge and for establishing common values in code reviews.
When the overall direction is agreed upon, even when further changes are required, the pull request is assigned to one team member.
If significant changes are requested or reviewers cannot come to a conclusion in reasonable time, the pull request is [marked as draft](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request#converting-a-pull-request-to-a-draft).
### Assigned
+3 -2
View File
@@ -119,7 +119,8 @@ release:
TODO: This script requires the right AWS credentials. Document.
TODO: This script currently requires a
`/home/eelco/Dev/nix-pristine`.
`/home/eelco/Dev/nix-pristine` and
`/home/eelco/Dev/nixpkgs-pristine`.
TODO: trigger nixos.org netlify: https://docs.netlify.com/configure-builds/build-hooks/
@@ -140,7 +141,7 @@ release:
$ git checkout master
$ git pull
$ NEW_VERSION=2.13.0
$ echo $NEW_VERSION > .version
$ echo -n $NEW_VERSION > .version
$ git checkout -b bump-$NEW_VERSION
$ git commit -a -m 'Bump version'
$ git push --set-upstream origin bump-$NEW_VERSION
+20 -44
View File
@@ -80,38 +80,6 @@ my $s3_us = Net::Amazon::S3->new(
my $channelsBucket = $s3_us->bucket($channelsBucketName) or die;
sub getStorePath {
my ($jobName, $output) = @_;
my $buildInfo = decode_json(fetch("$evalUrl/job/$jobName", 'application/json'));
return $buildInfo->{buildoutputs}->{$output or "out"}->{path} or die "cannot get store path for '$jobName'";
}
sub copyManual {
my $manual = getStorePath("build.x86_64-linux", "doc");
print "$manual\n";
my $manualNar = "$tmpDir/$releaseName-manual.nar.xz";
print "$manualNar\n";
unless (-e $manualNar) {
system("NIX_REMOTE=$binaryCache nix store dump-path '$manual' | xz > '$manualNar'.tmp") == 0
or die "unable to fetch $manual\n";
rename("$manualNar.tmp", $manualNar) or die;
}
unless (-e "$tmpDir/manual") {
system("xz -d < '$manualNar' | nix-store --restore $tmpDir/manual.tmp") == 0
or die "unable to unpack $manualNar\n";
rename("$tmpDir/manual.tmp/share/doc/nix/manual", "$tmpDir/manual") or die;
system("rm -rf '$tmpDir/manual.tmp'") == 0 or die;
}
system("aws s3 sync '$tmpDir/manual' s3://$releasesBucketName/$releaseDir/manual") == 0
or die "syncing manual to S3\n";
}
copyManual;
sub downloadFile {
my ($jobName, $productNr, $dstName) = @_;
@@ -211,20 +179,9 @@ if ($isLatest) {
system("docker manifest push nixos/nix:latest") == 0 or die;
}
# Upload nix-fallback-paths.nix.
write_file("$tmpDir/fallback-paths.nix",
"{\n" .
" x86_64-linux = \"" . getStorePath("build.x86_64-linux") . "\";\n" .
" i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" .
" aarch64-darwin = \"" . getStorePath("build.aarch64-darwin") . "\";\n" .
"}\n");
# Upload release files to S3.
for my $fn (glob "$tmpDir/*") {
my $name = basename($fn);
next if $name eq "manual";
my $dstKey = "$releaseDir/" . $name;
unless (defined $releasesBucket->head_key($dstKey)) {
print STDERR "uploading $fn to s3://$releasesBucketName/$dstKey...\n";
@@ -232,7 +189,8 @@ for my $fn (glob "$tmpDir/*") {
my $configuration = ();
$configuration->{content_type} = "application/octet-stream";
if ($fn =~ /.sha256|install|\.nix$/) {
if ($fn =~ /.sha256|install/) {
# Text files
$configuration->{content_type} = "text/plain";
}
@@ -241,6 +199,24 @@ for my $fn (glob "$tmpDir/*") {
}
}
# Print new nix-fallback-paths.nix.
if ($isLatest) {
sub getStorePath {
my ($jobName) = @_;
my $buildInfo = decode_json(fetch("$evalUrl/job/$jobName", 'application/json'));
return $buildInfo->{buildoutputs}->{out}->{path} or die "cannot get store path for '$jobName'";
}
print STDERR "nixos/modules/installer/tools/nix-fallback-paths.nix:\n" .
"{\n" .
" x86_64-linux = \"" . getStorePath("build.x86_64-linux") . "\";\n" .
" i686-linux = \"" . getStorePath("build.i686-linux") . "\";\n" .
" aarch64-linux = \"" . getStorePath("build.aarch64-linux") . "\";\n" .
" x86_64-darwin = \"" . getStorePath("build.x86_64-darwin") . "\";\n" .
" aarch64-darwin = \"" . getStorePath("build.aarch64-darwin") . "\";\n" .
"}\n";
}
# Update the "latest" symlink.
$channelsBucket->add_key(
"nix-latest/install", "",
-1
View File
@@ -10,7 +10,6 @@ ConditionPathIsReadWrite=@localstatedir@/nix/daemon-socket
ExecStart=@@bindir@/nix-daemon nix-daemon --daemon
KillMode=process
LimitNOFILE=1048576
TasksMax=1048576
[Install]
WantedBy=multi-user.target
+1 -19
View File
@@ -10,7 +10,6 @@ bin-scripts :=
noinst-scripts :=
man-pages :=
install-tests :=
install-tests-groups :=
ifdef HOST_OS
HOST_KERNEL = $(firstword $(subst -, ,$(HOST_OS)))
@@ -122,16 +121,7 @@ $(foreach script, $(bin-scripts), $(eval $(call install-program-in,$(script),$(b
$(foreach script, $(bin-scripts), $(eval programs-list += $(script)))
$(foreach script, $(noinst-scripts), $(eval programs-list += $(script)))
$(foreach template, $(template-files), $(eval $(call instantiate-template,$(template))))
$(foreach test, $(install-tests), \
$(eval $(call run-install-test,$(test))) \
$(eval installcheck: $(test).test))
$(foreach test-group, $(install-tests-groups), \
$(eval $(call run-install-test-group,$(test-group))) \
$(eval installcheck: $(test-group).test-group) \
$(foreach test, $($(test-group)-tests), \
$(eval $(call run-install-test,$(test))) \
$(eval $(test-group).test-group: $(test).test)))
$(foreach test, $(install-tests), $(eval $(call run-install-test,$(test))))
$(foreach file, $(man-pages), $(eval $(call install-data-in, $(file), $(mandir)/man$(patsubst .%,%,$(suffix $(file))))))
@@ -161,14 +151,6 @@ ifdef libs-list
@echo "The following libraries can be built:"
@echo ""
@for i in $(libs-list); do echo " $$i"; done
endif
ifdef install-tests-groups
@echo ""
@echo "The following groups of functional tests can be run:"
@echo ""
@for i in $(install-tests-groups); do echo " $$i.test-group"; done
@echo ""
@echo "(installcheck includes tests in test groups too.)"
endif
@echo ""
@echo "The following variables control the build:"
+2 -6
View File
@@ -4,6 +4,8 @@ test-deps =
define run-install-test
installcheck: $1.test
.PHONY: $1.test
$1.test: $1 $(test-deps)
@env BASH=$(bash) $(bash) mk/run-test.sh $1 < /dev/null
@@ -14,12 +16,6 @@ define run-install-test
endef
define run-install-test-group
.PHONY: $1.test-group
endef
.PHONY: check installcheck
print-top-help += \
+4 -2
View File
@@ -294,8 +294,10 @@ SV * makeFixedOutputPath(int recursive, char * algo, char * hash, char * name)
auto h = Hash::parseAny(hash, parseHashType(algo));
auto method = recursive ? FileIngestionMethod::Recursive : FileIngestionMethod::Flat;
auto path = store()->makeFixedOutputPath(name, FixedOutputInfo {
.method = method,
.hash = h,
.hash = {
.method = method,
.hash = h,
},
.references = {},
});
XPUSHs(sv_2mortal(newSVpv(store()->printStorePath(path).c_str(), 0)));
+1 -1
View File
@@ -100,7 +100,7 @@ poly_extra_try_me_commands() {
poly_configure_nix_daemon_service() {
task "Setting up the nix-daemon LaunchDaemon"
_sudo "to set up the nix-daemon as a LaunchDaemon" \
/usr/bin/install -m -rw-r--r-- "/nix/var/nix/profiles/default$NIX_DAEMON_DEST" "$NIX_DAEMON_DEST"
/bin/cp -f "/nix/var/nix/profiles/default$NIX_DAEMON_DEST" "$NIX_DAEMON_DEST"
_sudo "to load the LaunchDaemon plist for nix-daemon" \
launchctl load /Library/LaunchDaemons/org.nixos.nix-daemon.plist
+3 -7
View File
@@ -700,10 +700,6 @@ EOF
}
welcome_to_nix() {
local -r NIX_UID_RANGES="${NIX_FIRST_BUILD_UID}..$((NIX_FIRST_BUILD_UID + NIX_USER_COUNT - 1))"
local -r RANGE_TEXT=$(echo -ne "${BLUE}(uids [${NIX_UID_RANGES}])${ESC}")
local -r GROUP_TEXT=$(echo -ne "${BLUE}(gid ${NIX_BUILD_GROUP_ID})${ESC}")
ok "Welcome to the Multi-User Nix Installation"
cat <<EOF
@@ -717,8 +713,8 @@ manager. This will happen in a few stages:
2. Show you what I am going to install and where. Then I will ask
if you are ready to continue.
3. Create the system users ${RANGE_TEXT} and groups ${GROUP_TEXT}
that the Nix daemon uses to run builds.
3. Create the system users and groups that the Nix daemon uses to run
builds.
4. Perform the basic installation of the Nix files daemon.
@@ -884,7 +880,7 @@ configure_shell_profile() {
fi
done
task "Setting up shell profiles for Fish with ${PROFILE_FISH_SUFFIX} inside ${PROFILE_FISH_PREFIXES[*]}"
task "Setting up shell profiles for Fish with with ${PROFILE_FISH_SUFFIX} inside ${PROFILE_FISH_PREFIXES[*]}"
for fish_prefix in "${PROFILE_FISH_PREFIXES[@]}"; do
if [ ! -d "$fish_prefix" ]; then
# this specific prefix (ie: /etc/fish) is very likely to exist
+1 -1
View File
@@ -299,7 +299,7 @@ connected:
!trusted || *trusted;
});
// See the very large comment in `case WorkerProto::Op::BuildDerivation:` in
// See the very large comment in `case wopBuildDerivation:` in
// `src/libstore/daemon.cc` that explains the trust model here.
//
// This condition mirrors that: that code enforces the "rules" outlined there;
-67
View File
@@ -1,67 +0,0 @@
#include "built-path.hh"
#include "derivations.hh"
#include "store-api.hh"
#include <nlohmann/json.hpp>
#include <optional>
namespace nix {
nlohmann::json BuiltPath::Built::toJSON(ref<Store> store) const {
nlohmann::json res;
res["drvPath"] = store->printStorePath(drvPath);
for (const auto& [output, path] : outputs) {
res["outputs"][output] = store->printStorePath(path);
}
return res;
}
StorePathSet BuiltPath::outPaths() const
{
return std::visit(
overloaded{
[](const BuiltPath::Opaque & p) { return StorePathSet{p.path}; },
[](const BuiltPath::Built & b) {
StorePathSet res;
for (auto & [_, path] : b.outputs)
res.insert(path);
return res;
},
}, raw()
);
}
RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
{
RealisedPath::Set res;
std::visit(
overloaded{
[&](const BuiltPath::Opaque & p) { res.insert(p.path); },
[&](const BuiltPath::Built & p) {
auto drvHashes =
staticOutputHashes(store, store.readDerivation(p.drvPath));
for (auto& [outputName, outputPath] : p.outputs) {
if (experimentalFeatureSettings.isEnabled(
Xp::CaDerivations)) {
auto drvOutput = get(drvHashes, outputName);
if (!drvOutput)
throw Error(
"the derivation '%s' has unrealised output '%s' (derived-path.cc/toRealisedPaths)",
store.printStorePath(p.drvPath), outputName);
auto thisRealisation = store.queryRealisation(
DrvOutput{*drvOutput, outputName});
assert(thisRealisation); // We’ve built it, so we must
// have the realisation
res.insert(*thisRealisation);
} else {
res.insert(outputPath);
}
}
},
},
raw());
return res;
}
}
-48
View File
@@ -1,48 +0,0 @@
#include "derived-path.hh"
#include "realisation.hh"
namespace nix {
/**
* A built derived path with hints in the form of optional concrete output paths.
*
* See 'BuiltPath' for more an explanation.
*/
struct BuiltPathBuilt {
StorePath drvPath;
std::map<std::string, StorePath> outputs;
nlohmann::json toJSON(ref<Store> store) const;
static BuiltPathBuilt parse(const Store & store, std::string_view);
GENERATE_CMP(BuiltPathBuilt, me->drvPath, me->outputs);
};
using _BuiltPathRaw = std::variant<
DerivedPath::Opaque,
BuiltPathBuilt
>;
/**
* A built path. Similar to a DerivedPath, but enriched with the corresponding
* output path(s).
*/
struct BuiltPath : _BuiltPathRaw {
using Raw = _BuiltPathRaw;
using Raw::Raw;
using Opaque = DerivedPathOpaque;
using Built = BuiltPathBuilt;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
StorePathSet outPaths() const;
RealisedPath::Set toRealisedPaths(Store & store) const;
};
typedef std::vector<BuiltPath> BuiltPaths;
}
+3 -1
View File
@@ -239,7 +239,9 @@ void MixProfile::updateProfile(const StorePath & storePath)
if (!store) throw Error("'--profile' is not supported for this Nix store");
auto profile2 = absPath(*profile);
switchLink(profile2,
createGeneration(*store, profile2, storePath));
createGeneration(
ref<LocalFSStore>(store),
profile2, storePath));
}
void MixProfile::updateProfile(const BuiltPaths & buildables)
+1 -3
View File
@@ -105,9 +105,7 @@ MixEvalArgs::MixEvalArgs()
)",
.category = category,
.labels = {"path"},
.handler = {[&](std::string s) {
searchPath.elements.emplace_back(SearchPath::Elem::parse(s));
}}
.handler = {[&](std::string s) { searchPath.push_back(s); }}
});
addFlag({
+1 -2
View File
@@ -3,7 +3,6 @@
#include "args.hh"
#include "common-args.hh"
#include "search-path.hh"
namespace nix {
@@ -20,7 +19,7 @@ struct MixEvalArgs : virtual Args, virtual MixRepair
Bindings * getAutoArgs(EvalState & state);
SearchPath searchPath;
Strings searchPath;
std::optional<std::string> evalStoreUrl;
+1 -1
View File
@@ -151,7 +151,7 @@ DerivedPathsWithInfo InstallableFlake::toDerivedPaths()
},
ExtraPathInfoFlake::Flake {
.originalRef = flakeRef,
.lockedRef = getLockedFlake()->flake.lockedRef,
.resolvedRef = getLockedFlake()->flake.lockedRef,
}),
}};
}
+1 -1
View File
@@ -19,7 +19,7 @@ struct ExtraPathInfoFlake : ExtraPathInfoValue
*/
struct Flake {
FlakeRef originalRef;
FlakeRef lockedRef;
FlakeRef resolvedRef;
};
Flake flake;
+3 -3
View File
@@ -701,7 +701,7 @@ RawInstallablesCommand::RawInstallablesCommand()
{
addFlag({
.longName = "stdin",
.description = "Read installables from the standard input. No default installable applied.",
.description = "Read installables from the standard input.",
.handler = {&readFromStdIn, true}
});
@@ -730,9 +730,9 @@ void RawInstallablesCommand::run(ref<Store> store)
while (std::cin >> word) {
rawInstallables.emplace_back(std::move(word));
}
} else {
applyDefaultInstallables(rawInstallables);
}
applyDefaultInstallables(rawInstallables);
run(store, std::move(rawInstallables));
}
-1
View File
@@ -5,7 +5,6 @@
#include "path.hh"
#include "outputs-spec.hh"
#include "derived-path.hh"
#include "built-path.hh"
#include "store-api.hh"
#include "build-result.hh"
+4 -4
View File
@@ -68,7 +68,7 @@ struct NixRepl
const Path historyFile;
NixRepl(const SearchPath & searchPath, nix::ref<Store> store,ref<EvalState> state,
NixRepl(const Strings & searchPath, nix::ref<Store> store,ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
virtual ~NixRepl();
@@ -104,7 +104,7 @@ std::string removeWhitespace(std::string s)
}
NixRepl::NixRepl(const SearchPath & searchPath, nix::ref<Store> store, ref<EvalState> state,
NixRepl::NixRepl(const Strings & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<NixRepl::AnnotatedValues()> getValues)
: AbstractNixRepl(state)
, debugTraceIndex(0)
@@ -1024,7 +1024,7 @@ std::ostream & NixRepl::printValue(std::ostream & str, Value & v, unsigned int m
std::unique_ptr<AbstractNixRepl> AbstractNixRepl::create(
const SearchPath & searchPath, nix::ref<Store> store, ref<EvalState> state,
const Strings & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues)
{
return std::make_unique<NixRepl>(
@@ -1044,7 +1044,7 @@ void AbstractNixRepl::runSimple(
NixRepl::AnnotatedValues values;
return values;
};
SearchPath searchPath = {};
const Strings & searchPath = {};
auto repl = std::make_unique<NixRepl>(
searchPath,
openStore(),
+1 -1
View File
@@ -25,7 +25,7 @@ struct AbstractNixRepl
typedef std::vector<std::pair<Value*,std::string>> AnnotatedValues;
static std::unique_ptr<AbstractNixRepl> create(
const SearchPath & searchPath, nix::ref<Store> store, ref<EvalState> state,
const Strings & searchPath, nix::ref<Store> store, ref<EvalState> state,
std::function<AnnotatedValues()> getValues);
static void runSimple(
+44 -60
View File
@@ -95,16 +95,11 @@ RootValue allocRootValue(Value * v)
#endif
}
void Value::print(const SymbolTable &symbols, std::ostream &str,
std::set<const void *> *seen, int depth) const
void Value::print(const SymbolTable & symbols, std::ostream & str,
std::set<const void *> * seen) const
{
checkInterrupt();
if (depth <= 0) {
str << "«too deep»";
return;
}
switch (internalType) {
case tInt:
str << integer;
@@ -128,7 +123,7 @@ void Value::print(const SymbolTable &symbols, std::ostream &str,
str << "{ ";
for (auto & i : attrs->lexicographicOrder(symbols)) {
str << symbols[i->name] << " = ";
i->value->print(symbols, str, seen, depth - 1);
i->value->print(symbols, str, seen);
str << "; ";
}
str << "}";
@@ -144,7 +139,7 @@ void Value::print(const SymbolTable &symbols, std::ostream &str,
str << "[ ";
for (auto v2 : listItems()) {
if (v2)
v2->print(symbols, str, seen, depth - 1);
v2->print(symbols, str, seen);
else
str << "(nullptr)";
str << " ";
@@ -186,10 +181,11 @@ void Value::print(const SymbolTable &symbols, std::ostream &str,
}
}
void Value::print(const SymbolTable &symbols, std::ostream &str,
bool showRepeated, int depth) const {
void Value::print(const SymbolTable & symbols, std::ostream & str, bool showRepeated) const
{
std::set<const void *> seen;
print(symbols, str, showRepeated ? nullptr : &seen, depth);
print(symbols, str, showRepeated ? nullptr : &seen);
}
// Pretty print types for assertion errors
@@ -215,21 +211,20 @@ const Value * getPrimOp(const Value &v) {
return primOp;
}
std::string_view showType(ValueType type, bool withArticle)
std::string_view showType(ValueType type)
{
#define WA(a, w) withArticle ? a " " w : w
switch (type) {
case nInt: return WA("an", "integer");
case nBool: return WA("a", "Boolean");
case nString: return WA("a", "string");
case nPath: return WA("a", "path");
case nInt: return "an integer";
case nBool: return "a Boolean";
case nString: return "a string";
case nPath: return "a path";
case nNull: return "null";
case nAttrs: return WA("a", "set");
case nList: return WA("a", "list");
case nFunction: return WA("a", "function");
case nExternal: return WA("an", "external value");
case nFloat: return WA("a", "float");
case nThunk: return WA("a", "thunk");
case nAttrs: return "a set";
case nList: return "a list";
case nFunction: return "a function";
case nExternal: return "an external value";
case nFloat: return "a float";
case nThunk: return "a thunk";
}
abort();
}
@@ -498,7 +493,7 @@ ErrorBuilder & ErrorBuilder::withFrame(const Env & env, const Expr & expr)
EvalState::EvalState(
const SearchPath & _searchPath,
const Strings & _searchPath,
ref<Store> store,
std::shared_ptr<Store> buildStore)
: sWith(symbols.create("<with>"))
@@ -563,32 +558,30 @@ EvalState::EvalState(
/* Initialise the Nix expression search path. */
if (!evalSettings.pureEval) {
for (auto & i : _searchPath.elements)
addToSearchPath(SearchPath::Elem {i});
for (auto & i : evalSettings.nixPath.get())
addToSearchPath(SearchPath::Elem::parse(i));
for (auto & i : _searchPath) addToSearchPath(i);
for (auto & i : evalSettings.nixPath.get()) addToSearchPath(i);
}
if (evalSettings.restrictEval || evalSettings.pureEval) {
allowedPaths = PathSet();
for (auto & i : searchPath.elements) {
auto r = resolveSearchPathPath(i.path);
if (!r) continue;
for (auto & i : searchPath) {
auto r = resolveSearchPathElem(i);
if (!r.first) continue;
auto path = *std::move(r);
auto path = r.second;
if (store->isInStore(path)) {
if (store->isInStore(r.second)) {
try {
StorePathSet closure;
store->computeFSClosure(store->toStorePath(path).first, closure);
store->computeFSClosure(store->toStorePath(r.second).first, closure);
for (auto & path : closure)
allowPath(path);
} catch (InvalidPath &) {
allowPath(path);
allowPath(r.second);
}
} else
allowPath(path);
allowPath(r.second);
}
}
@@ -709,34 +702,28 @@ Path EvalState::toRealPath(const Path & path, const NixStringContext & context)
}
Value * EvalState::addConstant(const std::string & name, Value & v, Constant info)
Value * EvalState::addConstant(const std::string & name, Value & v)
{
Value * v2 = allocValue();
*v2 = v;
addConstant(name, v2, info);
addConstant(name, v2);
return v2;
}
void EvalState::addConstant(const std::string & name, Value * v, Constant info)
void EvalState::addConstant(const std::string & name, Value * v)
{
staticBaseEnv->vars.emplace_back(symbols.create(name), baseEnvDispl);
baseEnv.values[baseEnvDispl++] = v;
auto name2 = name.substr(0, 2) == "__" ? name.substr(2) : name;
baseEnv.values[0]->attrs->push_back(Attr(symbols.create(name2), v));
}
constantInfos.push_back({name2, info});
if (!(evalSettings.pureEval && info.impureOnly)) {
/* Check the type, if possible.
We might know the type of a thunk in advance, so be allowed
to just write it down in that case. */
if (auto gotType = v->type(true); gotType != nThunk)
assert(info.type == gotType);
/* Install value the base environment. */
staticBaseEnv->vars.emplace_back(symbols.create(name), baseEnvDispl);
baseEnv.values[baseEnvDispl++] = v;
baseEnv.values[0]->attrs->push_back(Attr(symbols.create(name2), v));
}
Value * EvalState::addPrimOp(const std::string & name,
size_t arity, PrimOpFun primOp)
{
return addPrimOp(PrimOp { .fun = primOp, .arity = arity, .name = name });
}
@@ -750,10 +737,7 @@ Value * EvalState::addPrimOp(PrimOp && primOp)
vPrimOp->mkPrimOp(new PrimOp(primOp));
Value v;
v.mkApp(vPrimOp, vPrimOp);
return addConstant(primOp.name, v, {
.type = nThunk, // FIXME
.doc = primOp.doc,
});
return addConstant(primOp.name, v);
}
auto envName = symbols.create(primOp.name);
@@ -779,13 +763,13 @@ std::optional<EvalState::Doc> EvalState::getDoc(Value & v)
{
if (v.isPrimOp()) {
auto v2 = &v;
if (auto * doc = v2->primOp->doc)
if (v2->primOp->doc)
return Doc {
.pos = {},
.name = v2->primOp->name,
.arity = v2->primOp->arity,
.args = v2->primOp->args,
.doc = doc,
.doc = v2->primOp->doc,
};
}
return {};
+21 -96
View File
@@ -9,7 +9,6 @@
#include "config.hh"
#include "experimental-features.hh"
#include "input-accessor.hh"
#include "search-path.hh"
#include <map>
#include <optional>
@@ -26,72 +25,15 @@ struct DerivedPath;
enum RepairFlag : bool;
/**
* Function that implements a primop.
*/
typedef void (* PrimOpFun) (EvalState & state, const PosIdx pos, Value * * args, Value & v);
/**
* Info about a primitive operation, and its implementation
*/
struct PrimOp
{
/**
* Name of the primop. `__` prefix is treated specially.
*/
std::string name;
/**
* Names of the parameters of a primop, for primops that take a
* fixed number of arguments to be substituted for these parameters.
*/
std::vector<std::string> args;
/**
* Aritiy of the primop.
*
* If `args` is not empty, this field will be computed from that
* field instead, so it doesn't need to be manually set.
*/
size_t arity = 0;
/**
* Optional free-form documentation about the primop.
*/
const char * doc = nullptr;
/**
* Implementation of the primop.
*/
PrimOpFun fun;
/**
* Optional experimental for this to be gated on.
*/
std::optional<ExperimentalFeature> experimentalFeature;
};
/**
* Info about a constant
*/
struct Constant
{
/**
* Optional type of the constant (known since it is a fixed value).
*
* @todo we should use an enum for this.
*/
ValueType type = nThunk;
/**
* Optional free-form documentation about the constant.
*/
size_t arity;
std::string name;
std::vector<std::string> args;
const char * doc = nullptr;
/**
* Whether the constant is impure, and not available in pure mode.
*/
bool impureOnly = false;
};
#if HAVE_BOEHMGC
@@ -123,6 +65,11 @@ std::string printValue(const EvalState & state, const Value & v);
std::ostream & operator << (std::ostream & os, const ValueType t);
// FIXME: maybe change this to an std::variant<SourcePath, URL>.
typedef std::pair<std::string, std::string> SearchPathElem;
typedef std::list<SearchPathElem> SearchPath;
/**
* Initialise the Boehm GC, if applicable.
*/
@@ -309,7 +256,7 @@ private:
SearchPath searchPath;
std::map<std::string, std::optional<std::string>> searchPathResolved;
std::map<std::string, std::pair<bool, std::string>> searchPathResolved;
/**
* Cache used by checkSourcePath().
@@ -336,12 +283,12 @@ private:
public:
EvalState(
const SearchPath & _searchPath,
const Strings & _searchPath,
ref<Store> store,
std::shared_ptr<Store> buildStore = nullptr);
~EvalState();
void addToSearchPath(SearchPath::Elem && elem);
void addToSearchPath(const std::string & s);
SearchPath getSearchPath() { return searchPath; }
@@ -423,16 +370,12 @@ public:
* Look up a file in the search path.
*/
SourcePath findFile(const std::string_view path);
SourcePath findFile(const SearchPath & searchPath, const std::string_view path, const PosIdx pos = noPos);
SourcePath findFile(SearchPath & searchPath, const std::string_view path, const PosIdx pos = noPos);
/**
* Try to resolve a search path value (not the optinal key part)
*
* If the specified search path element is a URI, download it.
*
* If it is not found, return `std::nullopt`
*/
std::optional<std::string> resolveSearchPathPath(const SearchPath::Path & path);
std::pair<bool, std::string> resolveSearchPathElem(const SearchPathElem & elem);
/**
* Evaluate an expression to normal form
@@ -566,23 +509,18 @@ public:
*/
std::shared_ptr<StaticEnv> staticBaseEnv; // !!! should be private
/**
* Name and documentation about every constant.
*
* Constants from primops are hard to crawl, and their docs will go
* here too.
*/
std::vector<std::pair<std::string, Constant>> constantInfos;
private:
unsigned int baseEnvDispl = 0;
void createBaseEnv();
Value * addConstant(const std::string & name, Value & v, Constant info);
Value * addConstant(const std::string & name, Value & v);
void addConstant(const std::string & name, Value * v, Constant info);
void addConstant(const std::string & name, Value * v);
Value * addPrimOp(const std::string & name,
size_t arity, PrimOpFun primOp);
Value * addPrimOp(PrimOp && primOp);
@@ -596,10 +534,6 @@ public:
std::optional<std::string> name;
size_t arity;
std::vector<std::string> args;
/**
* Unlike the other `doc` fields in this file, this one should never be
* `null`.
*/
const char * doc;
};
@@ -766,11 +700,8 @@ struct DebugTraceStacker {
/**
* @return A string representing the type of the value `v`.
*
* @param withArticle Whether to begin with an english article, e.g. "an
* integer" vs "integer".
*/
std::string_view showType(ValueType type, bool withArticle = true);
std::string_view showType(ValueType type);
std::string showType(const Value & v);
/**
@@ -802,12 +733,7 @@ struct EvalSettings : Config
Setting<Strings> nixPath{
this, getDefaultNixPath(), "nix-path",
R"(
List of directories to be searched for `<...>` file references
In particular, outside of [pure evaluation mode](#conf-pure-evaluation), this determines the value of
[`builtins.nixPath`](@docroot@/language/builtin-constants.md#builtins-nixPath).
)"};
"List of directories to be searched for `<...>` file references."};
Setting<bool> restrictEval{
this, false, "restrict-eval",
@@ -815,8 +741,7 @@ struct EvalSettings : Config
If set to `true`, the Nix evaluator will not allow access to any
files outside of the Nix search path (as set via the `NIX_PATH`
environment variable or the `-I` option), or to URIs outside of
[`allowed-uris`](../command-ref/conf-file.md#conf-allowed-uris).
The default is `false`.
`allowed-uri`. The default is `false`.
)"};
Setting<bool> pureEval{this, false, "pure-eval",
+3
View File
@@ -788,6 +788,9 @@ static RegisterPrimOp r2({
```nix
(builtins.getFlake "github:edolstra/dwarffs").rev
```
This function is only available if you enable the experimental feature
`flakes`.
)",
.fun = prim_getFlake,
.experimentalFeature = Xp::Flakes,
+1 -1
View File
@@ -36,7 +36,7 @@ static inline PosIdx makeCurPos(const YYLTYPE & loc, ParseData * data)
#define CUR_POS makeCurPos(*yylloc, data)
// backup to recover from yyless(0)
thread_local YYLTYPE prev_yylloc;
YYLTYPE prev_yylloc;
static void initLoc(YYLTYPE * loc)
{
+48 -45
View File
@@ -137,7 +137,6 @@ static void addAttr(ExprAttrs * attrs, AttrPath && attrPath,
dupAttr(state, ad.first, j2->second.pos, ad.second.pos);
jAttrs->attrs.emplace(ad.first, ad.second);
}
jAttrs->dynamicAttrs.insert(jAttrs->dynamicAttrs.end(), ae->dynamicAttrs.begin(), ae->dynamicAttrs.end());
} else {
dupAttr(state, attrPath, pos, j->second.pos);
}
@@ -276,12 +275,7 @@ static Expr * stripIndentation(const PosIdx pos, SymbolTable & symbols,
}
/* If this is a single string, then don't do a concatenation. */
if (es2->size() == 1 && dynamic_cast<ExprString *>((*es2)[0].second)) {
auto *const result = (*es2)[0].second;
delete es2;
return result;
}
return new ExprConcatStrings(pos, true, es2);
return es2->size() == 1 && dynamic_cast<ExprString *>((*es2)[0].second) ? (*es2)[0].second : new ExprConcatStrings(pos, true, es2);
}
@@ -336,7 +330,7 @@ void yyerror(YYLTYPE * loc, yyscan_t scanner, ParseData * data, const char * err
%type <ind_string_parts> ind_string_parts
%type <e> path_start string_parts string_attr
%type <id> attr
%token <id> ID
%token <id> ID ATTRPATH
%token <str> STR IND_STR
%token <n> INT
%token <nf> FLOAT
@@ -664,7 +658,7 @@ Expr * EvalState::parse(
ParseData data {
.state = *this,
.symbols = symbols,
.basePath = basePath,
.basePath = std::move(basePath),
.origin = {origin},
};
@@ -735,9 +729,19 @@ Expr * EvalState::parseStdin()
}
void EvalState::addToSearchPath(SearchPath::Elem && elem)
void EvalState::addToSearchPath(const std::string & s)
{
searchPath.elements.emplace_back(std::move(elem));
size_t pos = s.find('=');
std::string prefix;
Path path;
if (pos == std::string::npos) {
path = s;
} else {
prefix = std::string(s, 0, pos);
path = std::string(s, pos + 1);
}
searchPath.emplace_back(prefix, path);
}
@@ -747,19 +751,22 @@ SourcePath EvalState::findFile(const std::string_view path)
}
SourcePath EvalState::findFile(const SearchPath & searchPath, const std::string_view path, const PosIdx pos)
SourcePath EvalState::findFile(SearchPath & searchPath, const std::string_view path, const PosIdx pos)
{
for (auto & i : searchPath.elements) {
auto suffixOpt = i.prefix.suffixIfPotentialMatch(path);
if (!suffixOpt) continue;
auto suffix = *suffixOpt;
auto rOpt = resolveSearchPathPath(i.path);
if (!rOpt) continue;
auto r = *rOpt;
Path res = suffix == "" ? r : concatStrings(r, "/", suffix);
for (auto & i : searchPath) {
std::string suffix;
if (i.first.empty())
suffix = concatStrings("/", path);
else {
auto s = i.first.size();
if (path.compare(0, s, i.first) != 0 ||
(path.size() > s && path[s] != '/'))
continue;
suffix = path.size() == s ? "" : concatStrings("/", path.substr(s));
}
auto r = resolveSearchPathElem(i);
if (!r.first) continue;
Path res = r.second + suffix;
if (pathExists(res)) return CanonPath(canonPath(res));
}
@@ -776,53 +783,49 @@ SourcePath EvalState::findFile(const SearchPath & searchPath, const std::string_
}
std::optional<std::string> EvalState::resolveSearchPathPath(const SearchPath::Path & value0)
std::pair<bool, std::string> EvalState::resolveSearchPathElem(const SearchPathElem & elem)
{
auto & value = value0.s;
auto i = searchPathResolved.find(value);
auto i = searchPathResolved.find(elem.second);
if (i != searchPathResolved.end()) return i->second;
std::optional<std::string> res;
std::pair<bool, std::string> res;
if (EvalSettings::isPseudoUrl(value)) {
if (EvalSettings::isPseudoUrl(elem.second)) {
try {
auto storePath = fetchers::downloadTarball(
store, EvalSettings::resolvePseudoUrl(value), "source", false).tree.storePath;
res = { store->toRealPath(storePath) };
store, EvalSettings::resolvePseudoUrl(elem.second), "source", false).tree.storePath;
res = { true, store->toRealPath(storePath) };
} catch (FileTransferError & e) {
logWarning({
.msg = hintfmt("Nix search path entry '%1%' cannot be downloaded, ignoring", value)
.msg = hintfmt("Nix search path entry '%1%' cannot be downloaded, ignoring", elem.second)
});
res = std::nullopt;
res = { false, "" };
}
}
else if (hasPrefix(value, "flake:")) {
else if (hasPrefix(elem.second, "flake:")) {
experimentalFeatureSettings.require(Xp::Flakes);
auto flakeRef = parseFlakeRef(value.substr(6), {}, true, false);
debug("fetching flake search path element '%s''", value);
auto flakeRef = parseFlakeRef(elem.second.substr(6), {}, true, false);
debug("fetching flake search path element '%s''", elem.second);
auto storePath = flakeRef.resolve(store).fetchTree(store).first.storePath;
res = { store->toRealPath(storePath) };
res = { true, store->toRealPath(storePath) };
}
else {
auto path = absPath(value);
auto path = absPath(elem.second);
if (pathExists(path))
res = { path };
res = { true, path };
else {
logWarning({
.msg = hintfmt("Nix search path entry '%1%' does not exist, ignoring", value)
.msg = hintfmt("Nix search path entry '%1%' does not exist, ignoring", elem.second)
});
res = std::nullopt;
res = { false, "" };
}
}
if (res)
debug("resolved search path element '%s' to '%s'", value, *res);
else
debug("failed to resolve search path element '%s'", value);
debug("resolved search path element '%s' to '%s'", elem.second, res.second);
searchPathResolved[value] = res;
searchPathResolved[elem.second] = res;
return res;
}
+59 -262
View File
@@ -6,7 +6,7 @@
#include "globals.hh"
#include "json-to-value.hh"
#include "names.hh"
#include "path-references.hh"
#include "references.hh"
#include "store-api.hh"
#include "util.hh"
#include "value-to-json.hh"
@@ -238,7 +238,7 @@ static void import(EvalState & state, const PosIdx pos, Value & vPath, Value * v
}
}
static RegisterPrimOp primop_scopedImport(PrimOp {
static RegisterPrimOp primop_scopedImport(RegisterPrimOp::Info {
.name = "scopedImport",
.arity = 2,
.fun = [](EvalState & state, const PosIdx pos, Value * * args, Value & v)
@@ -692,7 +692,7 @@ static void prim_genericClosure(EvalState & state, const PosIdx pos, Value * * a
v.listElems()[n++] = i;
}
static RegisterPrimOp primop_genericClosure(PrimOp {
static RegisterPrimOp primop_genericClosure(RegisterPrimOp::Info {
.name = "__genericClosure",
.args = {"attrset"},
.arity = 1,
@@ -809,7 +809,7 @@ static void prim_addErrorContext(EvalState & state, const PosIdx pos, Value * *
}
}
static RegisterPrimOp primop_addErrorContext(PrimOp {
static RegisterPrimOp primop_addErrorContext(RegisterPrimOp::Info {
.name = "__addErrorContext",
.arity = 2,
.fun = prim_addErrorContext,
@@ -1300,10 +1300,9 @@ drvName, Bindings * attrs, Value & v)
auto method = ingestionMethod.value_or(FileIngestionMethod::Flat);
DerivationOutput::CAFixed dof {
.ca = ContentAddress {
.method = std::move(method),
.hash = std::move(h),
},
.ca = ContentAddress::fromParts(
std::move(method),
std::move(h)),
};
drv.env["out"] = state.store->printStorePath(dof.path(*state.store, drvName, "out"));
@@ -1401,7 +1400,7 @@ drvName, Bindings * attrs, Value & v)
v.mkAttrs(result);
}
static RegisterPrimOp primop_derivationStrict(PrimOp {
static RegisterPrimOp primop_derivationStrict(RegisterPrimOp::Info {
.name = "derivationStrict",
.arity = 1,
.fun = prim_derivationStrict,
@@ -1503,8 +1502,6 @@ static RegisterPrimOp primop_storePath({
in a new path (e.g. `/nix/store/ld01dnzc…-source-source`).
Not available in [pure evaluation mode](@docroot@/command-ref/conf-file.md#conf-pure-eval).
See also [`builtins.fetchClosure`](#builtins-fetchClosure).
)",
.fun = prim_storePath,
});
@@ -1671,10 +1668,7 @@ static void prim_findFile(EvalState & state, const PosIdx pos, Value * * args, V
}));
}
searchPath.elements.emplace_back(SearchPath::Elem {
.prefix = SearchPath::Prefix { .s = prefix },
.path = SearchPath::Path { .s = path },
});
searchPath.emplace_back(prefix, path);
}
auto path = state.forceStringNoCtx(*args[1], pos, "while evaluating the second argument passed to builtins.findFile");
@@ -1682,52 +1676,9 @@ static void prim_findFile(EvalState & state, const PosIdx pos, Value * * args, V
v.mkPath(state.checkSourcePath(state.findFile(searchPath, path, pos)));
}
static RegisterPrimOp primop_findFile(PrimOp {
static RegisterPrimOp primop_findFile(RegisterPrimOp::Info {
.name = "__findFile",
.args = {"search path", "lookup path"},
.doc = R"(
Look up the given path with the given search path.
A search path is represented list of [attribute sets](./values.md#attribute-set) with two attributes, `prefix`, and `path`.
`prefix` is a relative path.
`path` denotes a file system location; the exact syntax depends on the command line interface.
Examples of search path attribute sets:
- ```
{
prefix = "nixos-config";
path = "/etc/nixos/configuration.nix";
}
```
- ```
{
prefix = "";
path = "/nix/var/nix/profiles/per-user/root/channels";
}
```
The lookup algorithm checks each entry until a match is found, returning a [path value](@docroot@/language/values.html#type-path) of the match.
This is the process for each entry:
If the lookup path matches `prefix`, then the remainder of the lookup path (the "suffix") is searched for within the directory denoted by `patch`.
Note that the `path` may need to be downloaded at this point to look inside.
If the suffix is found inside that directory, then the entry is a match;
the combined absolute path of the directory (now downloaded if need be) and the suffix is returned.
The syntax
```nix
<nixpkgs>
```
is equivalent to:
```nix
builtins.findFile builtins.nixPath "nixpkgs"
```
)",
.arity = 2,
.fun = prim_findFile,
});
@@ -2177,8 +2128,10 @@ static void addPath(
std::optional<StorePath> expectedStorePath;
if (expectedHash)
expectedStorePath = state.store->makeFixedOutputPath(name, FixedOutputInfo {
.method = method,
.hash = *expectedHash,
.hash = {
.method = method,
.hash = *expectedHash,
},
.references = {},
});
@@ -2444,7 +2397,7 @@ static void prim_unsafeGetAttrPos(EvalState & state, const PosIdx pos, Value * *
state.mkPos(v, i->pos);
}
static RegisterPrimOp primop_unsafeGetAttrPos(PrimOp {
static RegisterPrimOp primop_unsafeGetAttrPos(RegisterPrimOp::Info {
.name = "__unsafeGetAttrPos",
.arity = 2,
.fun = prim_unsafeGetAttrPos,
@@ -4117,10 +4070,22 @@ static RegisterPrimOp primop_splitVersion({
RegisterPrimOp::PrimOps * RegisterPrimOp::primOps;
RegisterPrimOp::RegisterPrimOp(PrimOp && primOp)
RegisterPrimOp::RegisterPrimOp(std::string name, size_t arity, PrimOpFun fun)
{
if (!primOps) primOps = new PrimOps;
primOps->push_back(std::move(primOp));
primOps->push_back({
.name = name,
.args = {},
.arity = arity,
.fun = fun,
});
}
RegisterPrimOp::RegisterPrimOp(Info && info)
{
if (!primOps) primOps = new PrimOps;
primOps->push_back(std::move(info));
}
@@ -4133,253 +4098,85 @@ void EvalState::createBaseEnv()
/* `builtins' must be first! */
v.mkAttrs(buildBindings(128).finish());
addConstant("builtins", v, {
.type = nAttrs,
.doc = R"(
Contains all the [built-in functions](@docroot@/language/builtins.md) and values.
Since built-in functions were added over time, [testing for attributes](./operators.md#has-attribute) in `builtins` can be used for graceful fallback on older Nix installations:
```nix
# if hasContext is not available, we assume `s` has a context
if builtins ? hasContext then builtins.hasContext s else true
```
)",
});
addConstant("builtins", v);
v.mkBool(true);
addConstant("true", v, {
.type = nBool,
.doc = R"(
Primitive value.
It can be returned by
[comparison operators](@docroot@/language/operators.md#Comparison)
and used in
[conditional expressions](@docroot@/language/constructs.md#Conditionals).
The name `true` is not special, and can be shadowed:
```nix-repl
nix-repl> let true = 1; in true
1
```
)",
});
addConstant("true", v);
v.mkBool(false);
addConstant("false", v, {
.type = nBool,
.doc = R"(
Primitive value.
It can be returned by
[comparison operators](@docroot@/language/operators.md#Comparison)
and used in
[conditional expressions](@docroot@/language/constructs.md#Conditionals).
The name `false` is not special, and can be shadowed:
```nix-repl
nix-repl> let false = 1; in false
1
```
)",
});
addConstant("false", v);
v.mkNull();
addConstant("null", v, {
.type = nNull,
.doc = R"(
Primitive value.
The name `null` is not special, and can be shadowed:
```nix-repl
nix-repl> let null = 1; in null
1
```
)",
});
addConstant("null", v);
if (!evalSettings.pureEval) {
v.mkInt(time(0));
}
addConstant("__currentTime", v, {
.type = nInt,
.doc = R"(
Return the [Unix time](https://en.wikipedia.org/wiki/Unix_time) at first evaluation.
Repeated references to that name will re-use the initially obtained value.
addConstant("__currentTime", v);
Example:
```console
$ nix repl
Welcome to Nix 2.15.1 Type :? for help.
nix-repl> builtins.currentTime
1683705525
nix-repl> builtins.currentTime
1683705525
```
The [store path](@docroot@/glossary.md#gloss-store-path) of a derivation depending on `currentTime` will differ for each evaluation, unless both evaluate `builtins.currentTime` in the same second.
)",
.impureOnly = true,
});
if (!evalSettings.pureEval) {
v.mkString(settings.thisSystem.get());
addConstant("__currentSystem", v);
}
addConstant("__currentSystem", v, {
.type = nString,
.doc = R"(
The value of the [`system` configuration option](@docroot@/command-ref/conf-file.md#conf-pure-eval).
It can be used to set the `system` attribute for [`builtins.derivation`](@docroot@/language/derivations.md) such that the resulting derivation can be built on the same system that evaluates the Nix expression:
```nix
builtins.derivation {
# ...
system = builtins.currentSystem;
}
```
It can be overridden in order to create derivations for different system than the current one:
```console
$ nix-instantiate --system "mips64-linux" --eval --expr 'builtins.currentSystem'
"mips64-linux"
```
)",
.impureOnly = true,
});
v.mkString(nixVersion);
addConstant("__nixVersion", v, {
.type = nString,
.doc = R"(
The version of Nix.
For example, where the command line returns the current Nix version,
```shell-session
$ nix --version
nix (Nix) 2.16.0
```
the Nix language evaluator returns the same value:
```nix-repl
nix-repl> builtins.nixVersion
"2.16.0"
```
)",
});
addConstant("__nixVersion", v);
v.mkString(store->storeDir);
addConstant("__storeDir", v, {
.type = nString,
.doc = R"(
Logical file system location of the [Nix store](@docroot@/glossary.md#gloss-store) currently in use.
This value is determined by the `store` parameter in [Store URLs](@docroot@/command-ref/new-cli/nix3-help-stores.md):
```shell-session
$ nix-instantiate --store 'dummy://?store=/blah' --eval --expr builtins.storeDir
"/blah"
```
)",
});
addConstant("__storeDir", v);
/* Language version. This should be increased every time a new
language feature gets added. It's not necessary to increase it
when primops get added, because you can just use `builtins ?
primOp' to check. */
v.mkInt(6);
addConstant("__langVersion", v, {
.type = nInt,
.doc = R"(
The current version of the Nix language.
)",
});
addConstant("__langVersion", v);
// Miscellaneous
if (evalSettings.enableNativeCode) {
addPrimOp({
.name = "__importNative",
.arity = 2,
.fun = prim_importNative,
});
addPrimOp({
.name = "__exec",
.arity = 1,
.fun = prim_exec,
});
addPrimOp("__importNative", 2, prim_importNative);
addPrimOp("__exec", 1, prim_exec);
}
addPrimOp({
.fun = evalSettings.traceVerbose ? prim_trace : prim_second,
.arity = 2,
.name = "__traceVerbose",
.args = { "e1", "e2" },
.arity = 2,
.doc = R"(
Evaluate *e1* and print its abstract syntax representation on standard
error if `--trace-verbose` is enabled. Then return *e2*. This function
is useful for debugging.
)",
.fun = evalSettings.traceVerbose ? prim_trace : prim_second,
});
/* Add a value containing the current Nix expression search path. */
mkList(v, searchPath.elements.size());
mkList(v, searchPath.size());
int n = 0;
for (auto & i : searchPath.elements) {
for (auto & i : searchPath) {
auto attrs = buildBindings(2);
attrs.alloc("path").mkString(i.path.s);
attrs.alloc("prefix").mkString(i.prefix.s);
attrs.alloc("path").mkString(i.second);
attrs.alloc("prefix").mkString(i.first);
(v.listElems()[n++] = allocValue())->mkAttrs(attrs);
}
addConstant("__nixPath", v, {
.type = nList,
.doc = R"(
The search path used to resolve angle bracket path lookups.
Angle bracket expressions can be
[desugared](https://en.wikipedia.org/wiki/Syntactic_sugar)
using this and
[`builtins.findFile`](./builtins.html#builtins-findFile):
```nix
<nixpkgs>
```
is equivalent to:
```nix
builtins.findFile builtins.nixPath "nixpkgs"
```
)",
});
addConstant("__nixPath", v);
if (RegisterPrimOp::primOps)
for (auto & primOp : *RegisterPrimOp::primOps)
if (experimentalFeatureSettings.isEnabled(primOp.experimentalFeature))
if (!primOp.experimentalFeature
|| experimentalFeatureSettings.isEnabled(*primOp.experimentalFeature))
{
auto primOpAdjusted = primOp;
primOpAdjusted.arity = std::max(primOp.args.size(), primOp.arity);
addPrimOp(std::move(primOpAdjusted));
addPrimOp({
.fun = primOp.fun,
.arity = std::max(primOp.args.size(), primOp.arity),
.name = primOp.name,
.args = primOp.args,
.doc = primOp.doc,
});
}
/* Add a wrapper around the derivation primop that computes the
`drvPath' and `outPath' attributes lazily.
Null docs because it is documented separately.
*/
`drvPath' and `outPath' attributes lazily. */
auto vDerivation = allocValue();
addConstant("derivation", vDerivation, {
.type = nFunction,
});
addConstant("derivation", vDerivation);
/* Now that we've added all primops, sort the `builtins' set,
because attribute lookups expect it to be sorted. */
+17 -2
View File
@@ -10,7 +10,17 @@ namespace nix {
struct RegisterPrimOp
{
typedef std::vector<PrimOp> PrimOps;
struct Info
{
std::string name;
std::vector<std::string> args;
size_t arity = 0;
const char * doc;
PrimOpFun fun;
std::optional<ExperimentalFeature> experimentalFeature;
};
typedef std::vector<Info> PrimOps;
static PrimOps * primOps;
/**
@@ -18,7 +28,12 @@ struct RegisterPrimOp
* will get called during EvalState initialization, so there
* may be primops not yet added and builtins is not yet sorted.
*/
RegisterPrimOp(PrimOp && primOp);
RegisterPrimOp(
std::string name,
size_t arity,
PrimOpFun fun);
RegisterPrimOp(Info && info);
};
/* These primops are disabled without enableNativeCode, but plugins
+5 -49
View File
@@ -12,11 +12,7 @@ static void prim_unsafeDiscardStringContext(EvalState & state, const PosIdx pos,
v.mkString(*s);
}
static RegisterPrimOp primop_unsafeDiscardStringContext({
.name = "__unsafeDiscardStringContext",
.arity = 1,
.fun = prim_unsafeDiscardStringContext
});
static RegisterPrimOp primop_unsafeDiscardStringContext("__unsafeDiscardStringContext", 1, prim_unsafeDiscardStringContext);
static void prim_hasContext(EvalState & state, const PosIdx pos, Value * * args, Value & v)
@@ -26,16 +22,7 @@ static void prim_hasContext(EvalState & state, const PosIdx pos, Value * * args,
v.mkBool(!context.empty());
}
static RegisterPrimOp primop_hasContext({
.name = "__hasContext",
.args = {"s"},
.doc = R"(
Return `true` if string *s* has a non-empty context. The
context can be obtained with
[`getContext`](#builtins-getContext).
)",
.fun = prim_hasContext
});
static RegisterPrimOp primop_hasContext("__hasContext", 1, prim_hasContext);
/* Sometimes we want to pass a derivation path (i.e. pkg.drvPath) to a
@@ -64,11 +51,7 @@ static void prim_unsafeDiscardOutputDependency(EvalState & state, const PosIdx p
v.mkString(*s, context2);
}
static RegisterPrimOp primop_unsafeDiscardOutputDependency({
.name = "__unsafeDiscardOutputDependency",
.arity = 1,
.fun = prim_unsafeDiscardOutputDependency
});
static RegisterPrimOp primop_unsafeDiscardOutputDependency("__unsafeDiscardOutputDependency", 1, prim_unsafeDiscardOutputDependency);
/* Extract the context of a string as a structured Nix value.
@@ -136,30 +119,7 @@ static void prim_getContext(EvalState & state, const PosIdx pos, Value * * args,
v.mkAttrs(attrs);
}
static RegisterPrimOp primop_getContext({
.name = "__getContext",
.args = {"s"},
.doc = R"(
Return the string context of *s*.
The string context tracks references to derivations within a string.
It is represented as an attribute set of [store derivation](@docroot@/glossary.md#gloss-store-derivation) paths mapping to output names.
Using [string interpolation](@docroot@/language/string-interpolation.md) on a derivation will add that derivation to the string context.
For example,
```nix
builtins.getContext "${derivation { name = "a"; builder = "b"; system = "c"; }}"
```
evaluates to
```
{ "/nix/store/arhvjaf6zmlyn8vh8fgn55rpwnxq0n7l-a.drv" = { outputs = [ "out" ]; }; }
```
)",
.fun = prim_getContext
});
static RegisterPrimOp primop_getContext("__getContext", 1, prim_getContext);
/* Append the given context to a given string.
@@ -232,10 +192,6 @@ static void prim_appendContext(EvalState & state, const PosIdx pos, Value * * ar
v.mkString(orig, context);
}
static RegisterPrimOp primop_appendContext({
.name = "__appendContext",
.arity = 2,
.fun = prim_appendContext
});
static RegisterPrimOp primop_appendContext("__appendContext", 2, prim_appendContext);
}
+65 -186
View File
@@ -5,150 +5,37 @@
namespace nix {
/**
* Handler for the content addressed case.
*
* @param state Evaluator state and store to write to.
* @param fromStore Store containing the path to rewrite.
* @param fromPath Source path to be rewritten.
* @param toPathMaybe Path to write the rewritten path to. If empty, the error shows the actual path.
* @param v Return `Value`
*/
static void runFetchClosureWithRewrite(EvalState & state, const PosIdx pos, Store & fromStore, const StorePath & fromPath, const std::optional<StorePath> & toPathMaybe, Value &v) {
// establish toPath or throw
if (!toPathMaybe || !state.store->isValidPath(*toPathMaybe)) {
auto rewrittenPath = makeContentAddressed(fromStore, *state.store, fromPath);
if (toPathMaybe && *toPathMaybe != rewrittenPath)
throw Error({
.msg = hintfmt("rewriting '%s' to content-addressed form yielded '%s', while '%s' was expected",
state.store->printStorePath(fromPath),
state.store->printStorePath(rewrittenPath),
state.store->printStorePath(*toPathMaybe)),
.errPos = state.positions[pos]
});
if (!toPathMaybe)
throw Error({
.msg = hintfmt(
"rewriting '%s' to content-addressed form yielded '%s'\n"
"Use this value for the 'toPath' attribute passed to 'fetchClosure'",
state.store->printStorePath(fromPath),
state.store->printStorePath(rewrittenPath)),
.errPos = state.positions[pos]
});
}
auto toPath = *toPathMaybe;
// check and return
auto resultInfo = state.store->queryPathInfo(toPath);
if (!resultInfo->isContentAddressed(*state.store)) {
// We don't perform the rewriting when outPath already exists, as an optimisation.
// However, we can quickly detect a mistake if the toPath is input addressed.
throw Error({
.msg = hintfmt(
"The 'toPath' value '%s' is input-addressed, so it can't possibly be the result of rewriting to a content-addressed path.\n\n"
"Set 'toPath' to an empty string to make Nix report the correct content-addressed path.",
state.store->printStorePath(toPath)),
.errPos = state.positions[pos]
});
}
state.mkStorePathString(toPath, v);
}
/**
* Fetch the closure and make sure it's content addressed.
*/
static void runFetchClosureWithContentAddressedPath(EvalState & state, const PosIdx pos, Store & fromStore, const StorePath & fromPath, Value & v) {
if (!state.store->isValidPath(fromPath))
copyClosure(fromStore, *state.store, RealisedPath::Set { fromPath });
auto info = state.store->queryPathInfo(fromPath);
if (!info->isContentAddressed(*state.store)) {
throw Error({
.msg = hintfmt(
"The 'fromPath' value '%s' is input-addressed, but 'inputAddressed' is set to 'false' (default).\n\n"
"If you do intend to fetch an input-addressed store path, add\n\n"
" inputAddressed = true;\n\n"
"to the 'fetchClosure' arguments.\n\n"
"Note that to ensure authenticity input-addressed store paths, users must configure a trusted binary cache public key on their systems. This is not needed for content-addressed paths.",
state.store->printStorePath(fromPath)),
.errPos = state.positions[pos]
});
}
state.mkStorePathString(fromPath, v);
}
/**
* Fetch the closure and make sure it's input addressed.
*/
static void runFetchClosureWithInputAddressedPath(EvalState & state, const PosIdx pos, Store & fromStore, const StorePath & fromPath, Value & v) {
if (!state.store->isValidPath(fromPath))
copyClosure(fromStore, *state.store, RealisedPath::Set { fromPath });
auto info = state.store->queryPathInfo(fromPath);
if (info->isContentAddressed(*state.store)) {
throw Error({
.msg = hintfmt(
"The store object referred to by 'fromPath' at '%s' is not input-addressed, but 'inputAddressed' is set to 'true'.\n\n"
"Remove the 'inputAddressed' attribute (it defaults to 'false') to expect 'fromPath' to be content-addressed",
state.store->printStorePath(fromPath)),
.errPos = state.positions[pos]
});
}
state.mkStorePathString(fromPath, v);
}
typedef std::optional<StorePath> StorePathOrGap;
static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value * * args, Value & v)
{
state.forceAttrs(*args[0], pos, "while evaluating the argument passed to builtins.fetchClosure");
std::optional<std::string> fromStoreUrl;
std::optional<StorePath> fromPath;
std::optional<StorePathOrGap> toPath;
std::optional<bool> inputAddressedMaybe;
bool toCA = false;
std::optional<StorePath> toPath;
for (auto & attr : *args[0]->attrs) {
const auto & attrName = state.symbols[attr.name];
auto attrHint = [&]() -> std::string {
return "while evaluating the '" + attrName + "' attribute passed to builtins.fetchClosure";
};
if (attrName == "fromPath") {
NixStringContext context;
fromPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
fromPath = state.coerceToStorePath(attr.pos, *attr.value, context,
"while evaluating the 'fromPath' attribute passed to builtins.fetchClosure");
}
else if (attrName == "toPath") {
state.forceValue(*attr.value, attr.pos);
bool isEmptyString = attr.value->type() == nString && attr.value->string.s == std::string("");
if (isEmptyString) {
toPath = StorePathOrGap {};
}
else {
toCA = true;
if (attr.value->type() != nString || attr.value->string.s != std::string("")) {
NixStringContext context;
toPath = state.coerceToStorePath(attr.pos, *attr.value, context, attrHint());
toPath = state.coerceToStorePath(attr.pos, *attr.value, context,
"while evaluating the 'toPath' attribute passed to builtins.fetchClosure");
}
}
else if (attrName == "fromStore")
fromStoreUrl = state.forceStringNoCtx(*attr.value, attr.pos,
attrHint());
else if (attrName == "inputAddressed")
inputAddressedMaybe = state.forceBool(*attr.value, attr.pos, attrHint());
"while evaluating the 'fromStore' attribute passed to builtins.fetchClosure");
else
throw Error({
@@ -163,18 +50,6 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value * * arg
.errPos = state.positions[pos]
});
bool inputAddressed = inputAddressedMaybe.value_or(false);
if (inputAddressed) {
if (toPath)
throw Error({
.msg = hintfmt("attribute '%s' is set to true, but '%s' is also set. Please remove one of them",
"inputAddressed",
"toPath"),
.errPos = state.positions[pos]
});
}
if (!fromStoreUrl)
throw Error({
.msg = hintfmt("attribute '%s' is missing in call to 'fetchClosure'", "fromStore"),
@@ -199,40 +74,55 @@ static void prim_fetchClosure(EvalState & state, const PosIdx pos, Value * * arg
auto fromStore = openStore(parsedURL.to_string());
if (toPath)
runFetchClosureWithRewrite(state, pos, *fromStore, *fromPath, *toPath, v);
else if (inputAddressed)
runFetchClosureWithInputAddressedPath(state, pos, *fromStore, *fromPath, v);
else
runFetchClosureWithContentAddressedPath(state, pos, *fromStore, *fromPath, v);
if (toCA) {
if (!toPath || !state.store->isValidPath(*toPath)) {
auto remappings = makeContentAddressed(*fromStore, *state.store, { *fromPath });
auto i = remappings.find(*fromPath);
assert(i != remappings.end());
if (toPath && *toPath != i->second)
throw Error({
.msg = hintfmt("rewriting '%s' to content-addressed form yielded '%s', while '%s' was expected",
state.store->printStorePath(*fromPath),
state.store->printStorePath(i->second),
state.store->printStorePath(*toPath)),
.errPos = state.positions[pos]
});
if (!toPath)
throw Error({
.msg = hintfmt(
"rewriting '%s' to content-addressed form yielded '%s'; "
"please set this in the 'toPath' attribute passed to 'fetchClosure'",
state.store->printStorePath(*fromPath),
state.store->printStorePath(i->second)),
.errPos = state.positions[pos]
});
}
} else {
if (!state.store->isValidPath(*fromPath))
copyClosure(*fromStore, *state.store, RealisedPath::Set { *fromPath });
toPath = fromPath;
}
/* In pure mode, require a CA path. */
if (evalSettings.pureEval) {
auto info = state.store->queryPathInfo(*toPath);
if (!info->isContentAddressed(*state.store))
throw Error({
.msg = hintfmt("in pure mode, 'fetchClosure' requires a content-addressed path, which '%s' isn't",
state.store->printStorePath(*toPath)),
.errPos = state.positions[pos]
});
}
state.mkStorePathString(*toPath, v);
}
static RegisterPrimOp primop_fetchClosure({
.name = "__fetchClosure",
.args = {"args"},
.doc = R"(
Fetch a store path [closure](@docroot@/glossary.md#gloss-closure) from a binary cache, and return the store path as a string with context.
This function can be invoked in three ways, that we will discuss in order of preference.
**Fetch a content-addressed store path**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1;
}
```
This is the simplest invocation, and it does not require the user of the expression to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
If your store path is [input addressed](@docroot@/glossary.md#gloss-input-addressed-store-object) instead of content addressed, consider the other two invocations.
**Fetch any store path and rewrite it to a fully content-addressed store path**
Example:
Fetch a Nix store closure from a binary cache, rewriting it into
content-addressed form. For example,
```nix
builtins.fetchClosure {
@@ -242,42 +132,31 @@ static RegisterPrimOp primop_fetchClosure({
}
```
This example fetches `/nix/store/r2jd...` from the specified binary cache,
fetches `/nix/store/r2jd...` from the specified binary cache,
and rewrites it into the content-addressed store path
`/nix/store/ldbh...`.
Like the previous example, no extra configuration or privileges are required.
If `fromPath` is already content-addressed, or if you are
allowing impure evaluation (`--impure`), then `toPath` may be
omitted.
To find out the correct value for `toPath` given a `fromPath`,
use [`nix store make-content-addressed`](@docroot@/command-ref/new-cli/nix3-store-make-content-addressed.md):
you can use `nix store make-content-addressed`:
```console
# nix store make-content-addressed --from https://cache.nixos.org /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1
rewrote '/nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1' to '/nix/store/ldbhlwhh39wha58rm61bkiiwm6j7211j-git-2.33.1'
```
Alternatively, set `toPath = ""` and find the correct `toPath` in the error message.
This function is similar to `builtins.storePath` in that it
allows you to use a previously built store path in a Nix
expression. However, it is more reproducible because it requires
specifying a binary cache from which the path can be fetched.
Also, requiring a content-addressed final store path avoids the
need for users to configure binary cache public keys.
**Fetch an input-addressed store path as is**
Example:
```nix
builtins.fetchClosure {
fromStore = "https://cache.nixos.org";
fromPath = /nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1;
inputAddressed = true;
}
```
It is possible to fetch an [input-addressed store path](@docroot@/glossary.md#gloss-input-addressed-store-object) and return it as is.
However, this is the least preferred way of invoking `fetchClosure`, because it requires that the input-addressed paths are trusted by the Nix configuration.
**`builtins.storePath`**
`fetchClosure` is similar to [`builtins.storePath`](#builtins-storePath) in that it allows you to use a previously built store path in a Nix expression.
However, `fetchClosure` is more reproducible because it specifies a binary cache from which the path can be fetched.
Also, using content-addressed store paths does not require users to configure [`trusted-public-keys`](@docroot@/command-ref/conf-file.md#conf-trusted-public-keys) to ensure their authenticity.
This function is only available if you enable the experimental
feature `fetch-closure`.
)",
.fun = prim_fetchClosure,
.experimentalFeature = Xp::FetchClosure,
+1 -5
View File
@@ -88,10 +88,6 @@ static void prim_fetchMercurial(EvalState & state, const PosIdx pos, Value * * a
state.allowPath(tree.storePath);
}
static RegisterPrimOp r_fetchMercurial({
.name = "fetchMercurial",
.arity = 1,
.fun = prim_fetchMercurial
});
static RegisterPrimOp r_fetchMercurial("fetchMercurial", 1, prim_fetchMercurial);
}
+6 -13
View File
@@ -22,7 +22,7 @@ void emitTreeAttrs(
{
assert(input.isLocked());
auto attrs = state.buildBindings(10);
auto attrs = state.buildBindings(8);
state.mkStorePathString(tree.storePath, attrs.alloc(state.sOutPath));
@@ -56,11 +56,6 @@ void emitTreeAttrs(
}
if (auto dirtyRev = fetchers::maybeGetStrAttr(input.attrs, "dirtyRev")) {
attrs.alloc("dirtyRev").mkString(*dirtyRev);
attrs.alloc("dirtyShortRev").mkString(*fetchers::maybeGetStrAttr(input.attrs, "dirtyShortRev"));
}
if (auto lastModified = input.getLastModified()) {
attrs.alloc("lastModified").mkInt(*lastModified);
attrs.alloc("lastModifiedDate").mkString(
@@ -199,11 +194,7 @@ static void prim_fetchTree(EvalState & state, const PosIdx pos, Value * * args,
}
// FIXME: document
static RegisterPrimOp primop_fetchTree({
.name = "fetchTree",
.arity = 1,
.fun = prim_fetchTree
});
static RegisterPrimOp primop_fetchTree("fetchTree", 1, prim_fetchTree);
static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v,
const std::string & who, bool unpack, std::string name)
@@ -254,8 +245,10 @@ static void fetch(EvalState & state, const PosIdx pos, Value * * args, Value & v
auto expectedPath = state.store->makeFixedOutputPath(
name,
FixedOutputInfo {
.method = unpack ? FileIngestionMethod::Recursive : FileIngestionMethod::Flat,
.hash = *expectedHash,
.hash = {
.method = unpack ? FileIngestionMethod::Recursive : FileIngestionMethod::Flat,
.hash = *expectedHash,
},
.references = {}
});
+3 -33
View File
@@ -3,8 +3,6 @@
#include "../../toml11/toml.hpp"
#include <sstream>
namespace nix {
static void prim_fromTOML(EvalState & state, const PosIdx pos, Value * * args, Value & val)
@@ -60,18 +58,8 @@ static void prim_fromTOML(EvalState & state, const PosIdx pos, Value * * args, V
case toml::value_t::offset_datetime:
case toml::value_t::local_date:
case toml::value_t::local_time:
{
if (experimentalFeatureSettings.isEnabled(Xp::ParseTomlTimestamps)) {
auto attrs = state.buildBindings(2);
attrs.alloc("_type").mkString("timestamp");
std::ostringstream s;
s << t;
attrs.alloc("value").mkString(s.str());
v.mkAttrs(attrs);
} else {
throw std::runtime_error("Dates and times are not supported");
}
}
// We fail since Nix doesn't have date and time types
throw std::runtime_error("Dates and times are not supported");
break;;
case toml::value_t::empty:
v.mkNull();
@@ -90,24 +78,6 @@ static void prim_fromTOML(EvalState & state, const PosIdx pos, Value * * args, V
}
}
static RegisterPrimOp primop_fromTOML({
.name = "fromTOML",
.args = {"e"},
.doc = R"(
Convert a TOML string to a Nix value. For example,
```nix
builtins.fromTOML ''
x=1
s="a"
[table]
y=2
''
```
returns the value `{ s = "a"; table = { y = 2; }; x = 1; }`.
)",
.fun = prim_fromTOML
});
static RegisterPrimOp primop_fromTOML("fromTOML", 1, prim_fromTOML);
}
-56
View File
@@ -1,56 +0,0 @@
#include "search-path.hh"
#include "util.hh"
namespace nix {
std::optional<std::string_view> SearchPath::Prefix::suffixIfPotentialMatch(
std::string_view path) const
{
auto n = s.size();
/* Non-empty prefix and suffix must be separated by a /, or the
prefix is not a valid path prefix. */
bool needSeparator = n > 0 && n < path.size();
if (needSeparator && path[n] != '/') {
return std::nullopt;
}
/* Prefix must be prefix of this path. */
if (path.compare(0, n, s) != 0) {
return std::nullopt;
}
/* Skip next path separator. */
return {
path.substr(needSeparator ? n + 1 : n)
};
}
SearchPath::Elem SearchPath::Elem::parse(std::string_view rawElem)
{
size_t pos = rawElem.find('=');
return SearchPath::Elem {
.prefix = Prefix {
.s = pos == std::string::npos
? std::string { "" }
: std::string { rawElem.substr(0, pos) },
},
.path = Path {
.s = std::string { rawElem.substr(pos + 1) },
},
};
}
SearchPath parseSearchPath(const Strings & rawElems)
{
SearchPath res;
for (auto & rawElem : rawElems)
res.elements.emplace_back(SearchPath::Elem::parse(rawElem));
return res;
}
}
-108
View File
@@ -1,108 +0,0 @@
#pragma once
///@file
#include <optional>
#include "types.hh"
#include "comparator.hh"
namespace nix {
/**
* A "search path" is a list of ways look for something, used with
* `builtins.findFile` and `< >` lookup expressions.
*/
struct SearchPath
{
/**
* A single element of a `SearchPath`.
*
* Each element is tried in succession when looking up a path. The first
* element to completely match wins.
*/
struct Elem;
/**
* The first part of a `SearchPath::Elem` pair.
*
* Called a "prefix" because it takes the form of a prefix of a file
* path (first `n` path components). When looking up a path, to use
* a `SearchPath::Elem`, its `Prefix` must match the path.
*/
struct Prefix;
/**
* The second part of a `SearchPath::Elem` pair.
*
* It is either a path or a URL (with certain restrictions / extra
* structure).
*
* If the prefix of the path we are looking up matches, we then
* check if the rest of the path points to something that exists
* within the directory denoted by this. If so, the
* `SearchPath::Elem` as a whole matches, and that *something* being
* pointed to by the rest of the path we are looking up is the
* result.
*/
struct Path;
/**
* The list of search path elements. Each one is checked for a path
* when looking up. (The actual lookup entry point is in `EvalState`
* not in this class.)
*/
std::list<SearchPath::Elem> elements;
/**
* Parse a string into a `SearchPath`
*/
static SearchPath parse(const Strings & rawElems);
};
struct SearchPath::Prefix
{
/**
* Underlying string
*
* @todo Should we normalize this when constructing a `SearchPath::Prefix`?
*/
std::string s;
GENERATE_CMP(SearchPath::Prefix, me->s);
/**
* If the path possibly matches this search path element, return the
* suffix that we should look for inside the resolved value of the
* element
* Note the double optionality in the name. While we might have a matching prefix, the suffix may not exist.
*/
std::optional<std::string_view> suffixIfPotentialMatch(std::string_view path) const;
};
struct SearchPath::Path
{
/**
* The location of a search path item, as a path or URL.
*
* @todo Maybe change this to `std::variant<SourcePath, URL>`.
*/
std::string s;
GENERATE_CMP(SearchPath::Path, me->s);
};
struct SearchPath::Elem
{
Prefix prefix;
Path path;
GENERATE_CMP(SearchPath::Elem, me->prefix, me->path);
/**
* Parse a string into a `SearchPath::Elem`
*/
static SearchPath::Elem parse(std::string_view rawElem);
};
}
-90
View File
@@ -1,90 +0,0 @@
#include <gtest/gtest.h>
#include <gmock/gmock.h>
#include "search-path.hh"
namespace nix {
TEST(SearchPathElem, parse_justPath) {
ASSERT_EQ(
SearchPath::Elem::parse("foo"),
(SearchPath::Elem {
.prefix = SearchPath::Prefix { .s = "" },
.path = SearchPath::Path { .s = "foo" },
}));
}
TEST(SearchPathElem, parse_emptyPrefix) {
ASSERT_EQ(
SearchPath::Elem::parse("=foo"),
(SearchPath::Elem {
.prefix = SearchPath::Prefix { .s = "" },
.path = SearchPath::Path { .s = "foo" },
}));
}
TEST(SearchPathElem, parse_oneEq) {
ASSERT_EQ(
SearchPath::Elem::parse("foo=bar"),
(SearchPath::Elem {
.prefix = SearchPath::Prefix { .s = "foo" },
.path = SearchPath::Path { .s = "bar" },
}));
}
TEST(SearchPathElem, parse_twoEqs) {
ASSERT_EQ(
SearchPath::Elem::parse("foo=bar=baz"),
(SearchPath::Elem {
.prefix = SearchPath::Prefix { .s = "foo" },
.path = SearchPath::Path { .s = "bar=baz" },
}));
}
TEST(SearchPathElem, suffixIfPotentialMatch_justPath) {
SearchPath::Prefix prefix { .s = "" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("any/thing"), std::optional { "any/thing" });
}
TEST(SearchPathElem, suffixIfPotentialMatch_misleadingPrefix1) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("fooX"), std::nullopt);
}
TEST(SearchPathElem, suffixIfPotentialMatch_misleadingPrefix2) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("fooX/bar"), std::nullopt);
}
TEST(SearchPathElem, suffixIfPotentialMatch_partialPrefix) {
SearchPath::Prefix prefix { .s = "fooX" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo"), std::nullopt);
}
TEST(SearchPathElem, suffixIfPotentialMatch_exactPrefix) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo"), std::optional { "" });
}
TEST(SearchPathElem, suffixIfPotentialMatch_multiKey) {
SearchPath::Prefix prefix { .s = "foo/bar" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo/bar/baz"), std::optional { "baz" });
}
TEST(SearchPathElem, suffixIfPotentialMatch_trailingSlash) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo/"), std::optional { "" });
}
TEST(SearchPathElem, suffixIfPotentialMatch_trailingDoubleSlash) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo//"), std::optional { "/" });
}
TEST(SearchPathElem, suffixIfPotentialMatch_trailingPath) {
SearchPath::Prefix prefix { .s = "foo" };
ASSERT_EQ(prefix.suffixIfPotentialMatch("foo/bar/baz"), std::optional { "bar/baz" });
}
}
-236
View File
@@ -1,236 +0,0 @@
#include "tests/libexpr.hh"
#include "value.hh"
namespace nix {
using namespace testing;
struct ValuePrintingTests : LibExprTest
{
template<class... A>
void test(Value v, std::string_view expected, A... args)
{
std::stringstream out;
v.print(state.symbols, out, args...);
ASSERT_EQ(out.str(), expected);
}
};
TEST_F(ValuePrintingTests, tInt)
{
Value vInt;
vInt.mkInt(10);
test(vInt, "10");
}
TEST_F(ValuePrintingTests, tBool)
{
Value vBool;
vBool.mkBool(true);
test(vBool, "true");
}
TEST_F(ValuePrintingTests, tString)
{
Value vString;
vString.mkString("some-string");
test(vString, "\"some-string\"");
}
TEST_F(ValuePrintingTests, tPath)
{
Value vPath;
vPath.mkString("/foo");
test(vPath, "\"/foo\"");
}
TEST_F(ValuePrintingTests, tNull)
{
Value vNull;
vNull.mkNull();
test(vNull, "null");
}
TEST_F(ValuePrintingTests, tAttrs)
{
Value vOne;
vOne.mkInt(1);
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
test(vAttrs, "{ one = 1; two = 2; }");
}
TEST_F(ValuePrintingTests, tList)
{
Value vOne;
vOne.mkInt(1);
Value vTwo;
vTwo.mkInt(2);
Value vList;
state.mkList(vList, 5);
vList.bigList.elems[0] = &vOne;
vList.bigList.elems[1] = &vTwo;
vList.bigList.size = 3;
test(vList, "[ 1 2 (nullptr) ]");
}
TEST_F(ValuePrintingTests, vThunk)
{
Value vThunk;
vThunk.mkThunk(nullptr, nullptr);
test(vThunk, "<CODE>");
}
TEST_F(ValuePrintingTests, vApp)
{
Value vApp;
vApp.mkApp(nullptr, nullptr);
test(vApp, "<CODE>");
}
TEST_F(ValuePrintingTests, vLambda)
{
Value vLambda;
vLambda.mkLambda(nullptr, nullptr);
test(vLambda, "<LAMBDA>");
}
TEST_F(ValuePrintingTests, vPrimOp)
{
Value vPrimOp;
vPrimOp.mkPrimOp(nullptr);
test(vPrimOp, "<PRIMOP>");
}
TEST_F(ValuePrintingTests, vPrimOpApp)
{
Value vPrimOpApp;
vPrimOpApp.mkPrimOpApp(nullptr, nullptr);
test(vPrimOpApp, "<PRIMOP-APP>");
}
TEST_F(ValuePrintingTests, vExternal)
{
struct MyExternal : ExternalValueBase
{
public:
std::string showType() const override
{
return "";
}
std::string typeOf() const override
{
return "";
}
virtual std::ostream & print(std::ostream & str) const override
{
str << "testing-external!";
return str;
}
} myExternal;
Value vExternal;
vExternal.mkExternal(&myExternal);
test(vExternal, "testing-external!");
}
TEST_F(ValuePrintingTests, vFloat)
{
Value vFloat;
vFloat.mkFloat(2.0);
test(vFloat, "2");
}
TEST_F(ValuePrintingTests, vBlackhole)
{
Value vBlackhole;
vBlackhole.mkBlackhole();
test(vBlackhole, "«potential infinite recursion»");
}
TEST_F(ValuePrintingTests, depthAttrs)
{
Value vOne;
vOne.mkInt(1);
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
BindingsBuilder builder2(state, state.allocBindings(10));
builder2.insert(state.symbols.create("one"), &vOne);
builder2.insert(state.symbols.create("two"), &vTwo);
builder2.insert(state.symbols.create("nested"), &vAttrs);
Value vNested;
vNested.mkAttrs(builder2.finish());
test(vNested, "{ nested = «too deep»; one = «too deep»; two = «too deep»; }", false, 1);
test(vNested, "{ nested = { one = «too deep»; two = «too deep»; }; one = 1; two = 2; }", false, 2);
test(vNested, "{ nested = { one = 1; two = 2; }; one = 1; two = 2; }", false, 3);
test(vNested, "{ nested = { one = 1; two = 2; }; one = 1; two = 2; }", false, 4);
}
TEST_F(ValuePrintingTests, depthList)
{
Value vOne;
vOne.mkInt(1);
Value vTwo;
vTwo.mkInt(2);
BindingsBuilder builder(state, state.allocBindings(10));
builder.insert(state.symbols.create("one"), &vOne);
builder.insert(state.symbols.create("two"), &vTwo);
Value vAttrs;
vAttrs.mkAttrs(builder.finish());
BindingsBuilder builder2(state, state.allocBindings(10));
builder2.insert(state.symbols.create("one"), &vOne);
builder2.insert(state.symbols.create("two"), &vTwo);
builder2.insert(state.symbols.create("nested"), &vAttrs);
Value vNested;
vNested.mkAttrs(builder2.finish());
Value vList;
state.mkList(vList, 5);
vList.bigList.elems[0] = &vOne;
vList.bigList.elems[1] = &vTwo;
vList.bigList.elems[2] = &vNested;
vList.bigList.size = 3;
test(vList, "[ «too deep» «too deep» «too deep» ]", false, 1);
test(vList, "[ 1 2 { nested = «too deep»; one = «too deep»; two = «too deep»; } ]", false, 2);
test(vList, "[ 1 2 { nested = { one = «too deep»; two = «too deep»; }; one = 1; two = 2; } ]", false, 3);
test(vList, "[ 1 2 { nested = { one = 1; two = 2; }; one = 1; two = 2; } ]", false, 4);
test(vList, "[ 1 2 { nested = { one = 1; two = 2; }; one = 1; two = 2; } ]", false, 5);
}
} // namespace nix
+4 -11
View File
@@ -2,7 +2,6 @@
///@file
#include <cassert>
#include <climits>
#include "symbol-table.hh"
#include "value/context.hh"
@@ -138,11 +137,11 @@ private:
friend std::string showType(const Value & v);
void print(const SymbolTable &symbols, std::ostream &str, std::set<const void *> *seen, int depth) const;
void print(const SymbolTable & symbols, std::ostream & str, std::set<const void *> * seen) const;
public:
void print(const SymbolTable &symbols, std::ostream &str, bool showRepeated = false, int depth = INT_MAX) const;
void print(const SymbolTable & symbols, std::ostream & str, bool showRepeated = false) const;
// Functions needed to distinguish the type
// These should be removed eventually, by putting the functionality that's
@@ -219,11 +218,8 @@ public:
/**
* Returns the normal type of a Value. This only returns nThunk if
* the Value hasn't been forceValue'd
*
* @param invalidIsThunk Instead of aborting an an invalid (probably
* 0, so uninitialized) internal type, return `nThunk`.
*/
inline ValueType type(bool invalidIsThunk = false) const
inline ValueType type() const
{
switch (internalType) {
case tInt: return nInt;
@@ -238,10 +234,7 @@ public:
case tFloat: return nFloat;
case tThunk: case tApp: case tBlackhole: return nThunk;
}
if (invalidIsThunk)
return nThunk;
else
abort();
abort();
}
/**
+4 -2
View File
@@ -217,8 +217,10 @@ StorePath Input::computeStorePath(Store & store) const
if (!narHash)
throw Error("cannot compute store path for unlocked input '%s'", to_string());
return store.makeFixedOutputPath(getName(), FixedOutputInfo {
.method = FileIngestionMethod::Recursive,
.hash = *narHash,
.hash = {
.method = FileIngestionMethod::Recursive,
.hash = *narHash,
},
.references = {},
});
}
+1 -8
View File
@@ -243,13 +243,6 @@ std::pair<StorePath, Input> fetchFromWorkdir(ref<Store> store, Input & input, co
"lastModified",
workdirInfo.hasHead ? std::stoull(runProgram("git", true, { "-C", actualPath, "--git-dir", gitDir, "log", "-1", "--format=%ct", "--no-show-signature", "HEAD" })) : 0);
if (workdirInfo.hasHead) {
input.attrs.insert_or_assign("dirtyRev", chomp(
runProgram("git", true, { "-C", actualPath, "--git-dir", gitDir, "rev-parse", "--verify", "HEAD" })) + "-dirty");
input.attrs.insert_or_assign("dirtyShortRev", chomp(
runProgram("git", true, { "-C", actualPath, "--git-dir", gitDir, "rev-parse", "--verify", "--short", "HEAD" })) + "-dirty");
}
return {std::move(storePath), input};
}
} // end namespace
@@ -290,7 +283,7 @@ struct GitInputScheme : InputScheme
if (maybeGetStrAttr(attrs, "type") != "git") return {};
for (auto & [name, value] : attrs)
if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "shallow" && name != "submodules" && name != "lastModified" && name != "revCount" && name != "narHash" && name != "allRefs" && name != "name" && name != "dirtyRev" && name != "dirtyShortRev")
if (name != "type" && name != "url" && name != "ref" && name != "rev" && name != "shallow" && name != "submodules" && name != "lastModified" && name != "revCount" && name != "narHash" && name != "allRefs" && name != "name")
throw Error("unsupported Git input attribute '%s'", name);
parseURL(getStrAttr(attrs, "url"));
+5 -3
View File
@@ -77,8 +77,10 @@ DownloadFileResult downloadFile(
*store,
name,
FixedOutputInfo {
.method = FileIngestionMethod::Flat,
.hash = hash,
.hash = {
.method = FileIngestionMethod::Flat,
.hash = hash,
},
.references = {},
},
hashString(htSHA256, sink.s),
@@ -310,7 +312,7 @@ struct TarballInputScheme : CurlInputScheme
input = immutableInput;
}
if (result.lastModified && !input.attrs.contains("lastModified"))
if (result.lastModified && !input.attrs.count("lastModified"))
input.attrs.insert_or_assign("lastModified", uint64_t(result.lastModified));
return {result.tree.storePath, std::move(input)};
+10 -6
View File
@@ -309,8 +309,10 @@ StorePath BinaryCacheStore::addToStoreFromDump(Source & dump, std::string_view n
*this,
name,
FixedOutputInfo {
.method = method,
.hash = nar.first,
.hash = {
.method = method,
.hash = nar.first,
},
.references = {
.others = references,
// caller is not capable of creating a self-reference, because this is content-addressed without modulus
@@ -426,8 +428,10 @@ StorePath BinaryCacheStore::addToStore(
*this,
name,
FixedOutputInfo {
.method = method,
.hash = h,
.hash = {
.method = method,
.hash = h,
},
.references = {
.others = references,
// caller is not capable of creating a self-reference, because this is content-addressed without modulus
@@ -461,8 +465,8 @@ StorePath BinaryCacheStore::addTextToStore(
*this,
std::string { name },
TextInfo {
.hash = textHash,
.references = references,
{ .hash = textHash },
references,
},
nar.first,
};
+2 -5
View File
@@ -9,7 +9,6 @@
#include "archive.hh"
#include "compression.hh"
#include "worker-protocol.hh"
#include "worker-protocol-impl.hh"
#include "topo-sort.hh"
#include "callback.hh"
#include "local-store.hh" // TODO remove, along with remaining downcasts
@@ -1151,11 +1150,9 @@ HookReply DerivationGoal::tryBuildHook()
throw;
}
WorkerProto::WriteConn conn { hook->sink };
/* Tell the hook all the inputs that have to be copied to the
remote system. */
WorkerProto::write(worker.store, conn, inputPaths);
workerProtoWrite(worker.store, hook->sink, inputPaths);
/* Tell the hooks the missing outputs that have to be copied back
from the remote system. */
@@ -1166,7 +1163,7 @@ HookReply DerivationGoal::tryBuildHook()
if (buildMode != bmCheck && status.known && status.known->isValid()) continue;
missingOutputs.insert(outputName);
}
WorkerProto::write(worker.store, conn, missingOutputs);
workerProtoWrite(worker.store, hook->sink, missingOutputs);
}
hook->sink = FdSink();
+5 -5
View File
@@ -31,11 +31,11 @@ void Store::buildPaths(const std::vector<DerivedPath> & reqs, BuildMode buildMod
}
if (failed.size() == 1 && ex) {
ex->status = worker.failingExitStatus();
ex->status = worker.exitStatus();
throw std::move(*ex);
} else if (!failed.empty()) {
if (ex) logError(ex->info());
throw Error(worker.failingExitStatus(), "build of %s failed", showPaths(failed));
throw Error(worker.exitStatus(), "build of %s failed", showPaths(failed));
}
}
@@ -102,10 +102,10 @@ void Store::ensurePath(const StorePath & path)
if (goal->exitCode != Goal::ecSuccess) {
if (goal->ex) {
goal->ex->status = worker.failingExitStatus();
goal->ex->status = worker.exitStatus();
throw std::move(*goal->ex);
} else
throw Error(worker.failingExitStatus(), "path '%s' does not exist and cannot be created", printStorePath(path));
throw Error(worker.exitStatus(), "path '%s' does not exist and cannot be created", printStorePath(path));
}
}
@@ -128,7 +128,7 @@ void Store::repairPath(const StorePath & path)
goals.insert(worker.makeDerivationGoal(*info->deriver, OutputsSpec::All { }, bmRepair));
worker.run(goals);
} else
throw Error(worker.failingExitStatus(), "cannot repair path '%s'", printStorePath(path));
throw Error(worker.exitStatus(), "cannot repair path '%s'", printStorePath(path));
}
}
+3 -3
View File
@@ -5,14 +5,14 @@ namespace nix {
HookInstance::HookInstance()
{
debug("starting build hook '%s'", concatStringsSep(" ", settings.buildHook.get()));
debug("starting build hook '%s'", settings.buildHook);
auto buildHookArgs = settings.buildHook.get();
auto buildHookArgs = tokenizeString<std::list<std::string>>(settings.buildHook.get());
if (buildHookArgs.empty())
throw Error("'build-hook' setting is empty");
auto buildHook = canonPath(buildHookArgs.front());
auto buildHook = buildHookArgs.front();
buildHookArgs.pop_front();
Strings args;
+46 -68
View File
@@ -1,15 +1,16 @@
#include "local-derivation-goal.hh"
#include "indirect-root-store.hh"
#include "gc-store.hh"
#include "hook-instance.hh"
#include "worker.hh"
#include "builtins.hh"
#include "builtins/buildenv.hh"
#include "path-references.hh"
#include "references.hh"
#include "finally.hh"
#include "util.hh"
#include "archive.hh"
#include "compression.hh"
#include "daemon.hh"
#include "worker-protocol.hh"
#include "topo-sort.hh"
#include "callback.hh"
#include "json-utils.hh"
@@ -64,9 +65,8 @@ void handleDiffHook(
const Path & tryA, const Path & tryB,
const Path & drvPath, const Path & tmpDir)
{
auto & diffHookOpt = settings.diffHook.get();
if (diffHookOpt && settings.runDiffHook) {
auto & diffHook = *diffHookOpt;
auto diffHook = settings.diffHook;
if (diffHook != "" && settings.runDiffHook) {
try {
auto diffRes = runProgram(RunOptions {
.program = diffHook,
@@ -395,9 +395,8 @@ static void linkOrCopy(const Path & from, const Path & to)
bind-mount in this case?
It can also fail with EPERM in BeegFS v7 and earlier versions
or fail with EXDEV in OpenAFS
which don't allow hard-links to other directories */
if (errno != EMLINK && errno != EPERM && errno != EXDEV)
if (errno != EMLINK && errno != EPERM)
throw SysError("linking '%s' to '%s'", to, from);
copyPath(from, to);
}
@@ -594,10 +593,6 @@ void LocalDerivationGoal::startBuilder()
else
dirsInChroot[i.substr(0, p)] = {i.substr(p + 1), optional};
}
if (hasPrefix(worker.store.storeDir, tmpDirInSandbox))
{
throw Error("`sandbox-build-dir` must not contain the storeDir");
}
dirsInChroot[tmpDirInSandbox] = tmpDir;
/* Add the closure of store paths to the chroot. */
@@ -912,13 +907,15 @@ void LocalDerivationGoal::startBuilder()
openSlave();
/* Drop additional groups here because we can't do it
after we've created the new user namespace. */
if (setgroups(0, 0) == -1) {
if (errno != EPERM)
throw SysError("setgroups failed");
if (settings.requireDropSupplementaryGroups)
throw Error("setgroups failed. Set the require-drop-supplementary-groups option to false to skip this step.");
}
after we've created the new user namespace. FIXME:
this means that if we're not root in the parent
namespace, we can't drop additional groups; they will
be mapped to nogroup in the child namespace. There does
not seem to be a workaround for this. (But who can tell
from reading user_namespaces(7)?)
See also https://lwn.net/Articles/621612/. */
if (getuid() == 0 && setgroups(0, 0) == -1)
throw SysError("setgroups failed");
ProcessOptions options;
options.cloneFlags = CLONE_NEWPID | CLONE_NEWNS | CLONE_NEWIPC | CLONE_NEWUTS | CLONE_PARENT | SIGCHLD;
@@ -1200,7 +1197,7 @@ struct RestrictedStoreConfig : virtual LocalFSStoreConfig
/* A wrapper around LocalStore that only allows building/querying of
paths that are in the input closures of the build or were added via
recursive Nix calls. */
struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual IndirectRootStore, public virtual GcStore
struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual LocalFSStore, public virtual GcStore
{
ref<LocalStore> next;
@@ -1251,13 +1248,11 @@ struct RestrictedStore : public virtual RestrictedStoreConfig, public virtual In
void queryReferrers(const StorePath & path, StorePathSet & referrers) override
{ }
std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(
const StorePath & path,
Store * evalStore = nullptr) override
std::map<std::string, std::optional<StorePath>> queryPartialDerivationOutputMap(const StorePath & path) override
{
if (!goal.isAllowed(path))
throw InvalidPath("cannot query output map for unknown path '%s' in recursive Nix", printStorePath(path));
return next->queryPartialDerivationOutputMap(path, evalStore);
return next->queryPartialDerivationOutputMap(path);
}
std::optional<StorePath> queryPathFromHashPart(const std::string & hashPart) override
@@ -1433,8 +1428,7 @@ void LocalDerivationGoal::startDaemon()
Store::Params params;
params["path-info-cache-size"] = "0";
params["store"] = worker.store.storeDir;
if (auto & optRoot = getLocalStore().rootDir.get())
params["root"] = *optRoot;
params["root"] = getLocalStore().rootDir;
params["state"] = "/no-such-path";
params["log"] = "/no-such-path";
auto store = make_ref<RestrictedStore>(params,
@@ -1463,7 +1457,7 @@ void LocalDerivationGoal::startDaemon()
(struct sockaddr *) &remoteAddr, &remoteAddrLen);
if (!remote) {
if (errno == EINTR || errno == EAGAIN) continue;
if (errno == EINVAL || errno == ECONNABORTED) break;
if (errno == EINVAL) break;
throw SysError("accepting connection");
}
@@ -1493,22 +1487,8 @@ void LocalDerivationGoal::startDaemon()
void LocalDerivationGoal::stopDaemon()
{
if (daemonSocket && shutdown(daemonSocket.get(), SHUT_RDWR) == -1) {
// According to the POSIX standard, the 'shutdown' function should
// return an ENOTCONN error when attempting to shut down a socket that
// hasn't been connected yet. This situation occurs when the 'accept'
// function is called on a socket without any accepted connections,
// leaving the socket unconnected. While Linux doesn't seem to produce
// an error for sockets that have only been accepted, more
// POSIX-compliant operating systems like OpenBSD, macOS, and others do
// return the ENOTCONN error. Therefore, we handle this error here to
// avoid raising an exception for compliant behaviour.
if (errno == ENOTCONN) {
daemonSocket.close();
} else {
throw SysError("shutting down daemon socket");
}
}
if (daemonSocket && shutdown(daemonSocket.get(), SHUT_RDWR) == -1)
throw SysError("shutting down daemon socket");
if (daemonThread.joinable())
daemonThread.join();
@@ -1519,8 +1499,7 @@ void LocalDerivationGoal::stopDaemon()
thread.join();
daemonWorkerThreads.clear();
// release the socket.
daemonSocket.close();
daemonSocket = -1;
}
@@ -2400,21 +2379,18 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
continue;
auto references = *referencesOpt;
auto rewriteOutput = [&](const StringMap & rewrites) {
auto rewriteOutput = [&]() {
/* Apply hash rewriting if necessary. */
if (!rewrites.empty()) {
if (!outputRewrites.empty()) {
debug("rewriting hashes in '%1%'; cross fingers", actualPath);
/* FIXME: Is this actually streaming? */
auto source = sinkToSource([&](Sink & nextSink) {
RewritingSink rsink(rewrites, nextSink);
dumpPath(actualPath, rsink);
rsink.flush();
});
Path tmpPath = actualPath + ".tmp";
restorePath(tmpPath, *source);
/* FIXME: this is in-memory. */
StringSink sink;
dumpPath(actualPath, sink);
deletePath(actualPath);
movePath(tmpPath, actualPath);
sink.s = rewriteStrings(sink.s, outputRewrites);
StringSource source(sink.s);
restorePath(actualPath, source);
/* FIXME: set proper permissions in restorePath() so
we don't have to do another traversal. */
@@ -2463,7 +2439,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
"since recursive hashing is not enabled (one of outputHashMode={flat,text} is true)",
actualPath);
}
rewriteOutput(outputRewrites);
rewriteOutput();
/* FIXME optimize and deduplicate with addToStore */
std::string oldHashPart { scratchPath->hashPart() };
HashModuloSink caSink { outputHash.hashType, oldHashPart };
@@ -2501,14 +2477,16 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
Hash::dummy,
};
if (*scratchPath != newInfo0.path) {
// If the path has some self-references, we need to rewrite
// them.
// (note that this doesn't invalidate the ca hash we calculated
// above because it's computed *modulo the self-references*, so
// it already takes this rewrite into account).
rewriteOutput(
StringMap{{oldHashPart,
std::string(newInfo0.path.hashPart())}});
// Also rewrite the output path
auto source = sinkToSource([&](Sink & nextSink) {
RewritingSink rsink2(oldHashPart, std::string(newInfo0.path.hashPart()), nextSink);
dumpPath(actualPath, rsink2);
rsink2.flush();
});
Path tmpPath = actualPath + ".tmp";
restorePath(tmpPath, *source);
deletePath(actualPath);
movePath(tmpPath, actualPath);
}
HashResult narHashAndSize = hashPath(htSHA256, actualPath);
@@ -2530,7 +2508,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
outputRewrites.insert_or_assign(
std::string { scratchPath->hashPart() },
std::string { requiredFinalPath.hashPart() });
rewriteOutput(outputRewrites);
rewriteOutput();
auto narHashAndSize = hashPath(htSHA256, actualPath);
ValidPathInfo newInfo0 { requiredFinalPath, narHashAndSize.first };
newInfo0.narSize = narHashAndSize.second;
@@ -2542,7 +2520,7 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
},
[&](const DerivationOutput::CAFixed & dof) {
auto & wanted = dof.ca.hash;
auto wanted = dof.ca.getHash();
// Replace the output by a fresh copy of itself to make sure
// that there's no stale file descriptor pointing to it
@@ -2551,13 +2529,13 @@ SingleDrvOutputs LocalDerivationGoal::registerOutputs()
renameFile(tmpOutput, actualPath);
auto newInfo0 = newInfoFromCA(DerivationOutput::CAFloating {
.method = dof.ca.method,
.method = dof.ca.getMethod(),
.hashType = wanted.type,
});
/* Check wanted hash */
assert(newInfo0.ca);
auto & got = newInfo0.ca->hash;
auto got = newInfo0.ca->getHash();
if (wanted != got) {
/* Throw an error after registering the path as
valid. */
+1 -2
View File
@@ -21,8 +21,7 @@ void setPersonality(std::string_view system)
&& (std::string_view(SYSTEM) == "x86_64-linux"
|| (!strcmp(utsbuf.sysname, "Linux") && !strcmp(utsbuf.machine, "x86_64"))))
|| system == "armv7l-linux"
|| system == "armv6l-linux"
|| system == "armv5tel-linux")
|| system == "armv6l-linux")
{
if (personality(PER_LINUX32) == -1)
throw SysError("cannot set 32-bit personality");
+9 -2
View File
@@ -468,9 +468,16 @@ void Worker::waitForInput()
}
unsigned int Worker::failingExitStatus()
unsigned int Worker::exitStatus()
{
// See API docs in header for explanation
/*
* 1100100
* ^^^^
* |||`- timeout
* ||`-- output hash mismatch
* |`--- build failure
* `---- not deterministic
*/
unsigned int mask = 0;
bool buildFailure = permanentFailure || timedOut || hashMismatch;
if (buildFailure)
+1 -22
View File
@@ -280,28 +280,7 @@ public:
*/
void waitForInput();
/***
* The exit status in case of failure.
*
* In the case of a build failure, returned value follows this
* bitmask:
*
* ```
* 0b1100100
* ^^^^
* |||`- timeout
* ||`-- output hash mismatch
* |`--- build failure
* `---- not deterministic
* ```
*
* In other words, the failure code is at least 100 (0b1100100), but
* might also be greater.
*
* Otherwise (no build failure, but some other sort of failure by
* assumption), this returned value is 1.
*/
unsigned int failingExitStatus();
unsigned int exitStatus();
/**
* Check whether the given valid path exists and has the right
+92 -33
View File
@@ -4,6 +4,11 @@
namespace nix {
std::string FixedOutputHash::printMethodAlgo() const
{
return makeFileIngestionPrefix(method) + printHashType(hash.type);
}
std::string makeFileIngestionPrefix(FileIngestionMethod m)
{
switch (m) {
@@ -37,6 +42,21 @@ ContentAddressMethod ContentAddressMethod::parsePrefix(std::string_view & m)
return method;
}
std::string ContentAddress::render() const
{
return std::visit(overloaded {
[](const TextHash & th) {
return "text:"
+ th.hash.to_string(Base32, true);
},
[](const FixedOutputHash & fsh) {
return "fixed:"
+ makeFileIngestionPrefix(fsh.method)
+ fsh.hash.to_string(Base32, true);
}
}, raw);
}
std::string ContentAddressMethod::render(HashType ht) const
{
return std::visit(overloaded {
@@ -49,20 +69,6 @@ std::string ContentAddressMethod::render(HashType ht) const
}, raw);
}
std::string ContentAddress::render() const
{
return std::visit(overloaded {
[](const TextIngestionMethod &) -> std::string {
return "text:";
},
[](const FileIngestionMethod & method) {
return "fixed:"
+ makeFileIngestionPrefix(method);
},
}, method.raw)
+ this->hash.to_string(Base32, true);
}
/**
* Parses content address strings up to the hash.
*/
@@ -112,12 +118,22 @@ ContentAddress ContentAddress::parse(std::string_view rawCa)
{
auto rest = rawCa;
auto [caMethod, hashType] = parseContentAddressMethodPrefix(rest);
auto [caMethod, hashType_] = parseContentAddressMethodPrefix(rest);
auto hashType = hashType_; // work around clang bug
return ContentAddress {
.method = std::move(caMethod).raw,
.hash = Hash::parseNonSRIUnprefixed(rest, hashType),
};
return std::visit(overloaded {
[&](TextIngestionMethod &) {
return ContentAddress(TextHash {
.hash = Hash::parseNonSRIUnprefixed(rest, hashType)
});
},
[&](FileIngestionMethod & fim) {
return ContentAddress(FixedOutputHash {
.method = fim,
.hash = Hash::parseNonSRIUnprefixed(rest, hashType),
});
},
}, caMethod.raw);
}
std::pair<ContentAddressMethod, HashType> ContentAddressMethod::parse(std::string_view caMethod)
@@ -140,10 +156,52 @@ std::string renderContentAddress(std::optional<ContentAddress> ca)
return ca ? ca->render() : "";
}
ContentAddress ContentAddress::fromParts(
ContentAddressMethod method, Hash hash) noexcept
{
return std::visit(overloaded {
[&](TextIngestionMethod _) -> ContentAddress {
return TextHash {
.hash = std::move(hash),
};
},
[&](FileIngestionMethod m2) -> ContentAddress {
return FixedOutputHash {
.method = std::move(m2),
.hash = std::move(hash),
};
},
}, method.raw);
}
ContentAddressMethod ContentAddress::getMethod() const
{
return std::visit(overloaded {
[](const TextHash & th) -> ContentAddressMethod {
return TextIngestionMethod {};
},
[](const FixedOutputHash & fsh) -> ContentAddressMethod {
return fsh.method;
},
}, raw);
}
const Hash & ContentAddress::getHash() const
{
return std::visit(overloaded {
[](const TextHash & th) -> auto & {
return th.hash;
},
[](const FixedOutputHash & fsh) -> auto & {
return fsh.hash;
},
}, raw);
}
std::string ContentAddress::printMethodAlgo() const
{
return method.renderPrefix()
+ printHashType(hash.type);
return getMethod().renderPrefix()
+ printHashType(getHash().type);
}
bool StoreReferences::empty() const
@@ -159,20 +217,19 @@ size_t StoreReferences::size() const
ContentAddressWithReferences ContentAddressWithReferences::withoutRefs(const ContentAddress & ca) noexcept
{
return std::visit(overloaded {
[&](const TextIngestionMethod &) -> ContentAddressWithReferences {
[&](const TextHash & h) -> ContentAddressWithReferences {
return TextInfo {
.hash = ca.hash,
.hash = h,
.references = {},
};
},
[&](const FileIngestionMethod & method) -> ContentAddressWithReferences {
[&](const FixedOutputHash & h) -> ContentAddressWithReferences {
return FixedOutputInfo {
.method = method,
.hash = ca.hash,
.hash = h,
.references = {},
};
},
}, ca.method.raw);
}, ca.raw);
}
std::optional<ContentAddressWithReferences> ContentAddressWithReferences::fromPartsOpt(
@@ -184,7 +241,7 @@ std::optional<ContentAddressWithReferences> ContentAddressWithReferences::fromPa
return std::nullopt;
return ContentAddressWithReferences {
TextInfo {
.hash = std::move(hash),
.hash = { .hash = std::move(hash) },
.references = std::move(refs.others),
}
};
@@ -192,8 +249,10 @@ std::optional<ContentAddressWithReferences> ContentAddressWithReferences::fromPa
[&](FileIngestionMethod m2) -> std::optional<ContentAddressWithReferences> {
return ContentAddressWithReferences {
FixedOutputInfo {
.method = m2,
.hash = std::move(hash),
.hash = {
.method = m2,
.hash = std::move(hash),
},
.references = std::move(refs),
}
};
@@ -208,7 +267,7 @@ ContentAddressMethod ContentAddressWithReferences::getMethod() const
return TextIngestionMethod {};
},
[](const FixedOutputInfo & fsh) -> ContentAddressMethod {
return fsh.method;
return fsh.hash.method;
},
}, raw);
}
@@ -217,10 +276,10 @@ Hash ContentAddressWithReferences::getHash() const
{
return std::visit(overloaded {
[](const TextInfo & th) {
return th.hash;
return th.hash.hash;
},
[](const FixedOutputInfo & fsh) {
return fsh.hash;
return fsh.hash.hash;
},
}, raw);
}
+61 -30
View File
@@ -113,6 +113,37 @@ struct ContentAddressMethod
* Mini content address
*/
/**
* Somewhat obscure, used by \ref Derivation derivations and
* `builtins.toFile` currently.
*/
struct TextHash {
/**
* Hash of the contents of the text/file.
*/
Hash hash;
GENERATE_CMP(TextHash, me->hash);
};
/**
* Used by most store objects that are content-addressed.
*/
struct FixedOutputHash {
/**
* How the file system objects are serialized
*/
FileIngestionMethod method;
/**
* Hash of that serialization
*/
Hash hash;
std::string printMethodAlgo() const;
GENERATE_CMP(FixedOutputHash, me->method, me->hash);
};
/**
* We've accumulated several types of content-addressed paths over the
* years; fixed-output derivations support multiple hash algorithms and
@@ -127,17 +158,19 @@ struct ContentAddressMethod
*/
struct ContentAddress
{
/**
* How the file system objects are serialized
*/
ContentAddressMethod method;
typedef std::variant<
TextHash,
FixedOutputHash
> Raw;
/**
* Hash of that serialization
*/
Hash hash;
Raw raw;
GENERATE_CMP(ContentAddress, me->method, me->hash);
GENERATE_CMP(ContentAddress, me->raw);
/* The moral equivalent of `using Raw::Raw;` */
ContentAddress(auto &&... arg)
: raw(std::forward<decltype(arg)>(arg)...)
{ }
/**
* Compute the content-addressability assertion
@@ -150,6 +183,20 @@ struct ContentAddress
static std::optional<ContentAddress> parseOpt(std::string_view rawCaOpt);
/**
* Create a `ContentAddress` from 2 parts:
*
* @param method Way ingesting the file system data.
*
* @param hash Hash of ingested file system data.
*/
static ContentAddress fromParts(
ContentAddressMethod method, Hash hash) noexcept;
ContentAddressMethod getMethod() const;
const Hash & getHash() const;
std::string printMethodAlgo() const;
};
@@ -172,8 +219,7 @@ std::string renderContentAddress(std::optional<ContentAddress> ca);
* References to other store objects are tracked with store paths, self
* references however are tracked with a boolean.
*/
struct StoreReferences
{
struct StoreReferences {
/**
* References to other store objects
*/
@@ -200,13 +246,8 @@ struct StoreReferences
};
// This matches the additional info that we need for makeTextPath
struct TextInfo
{
/**
* Hash of the contents of the text/file.
*/
Hash hash;
struct TextInfo {
TextHash hash;
/**
* References to other store objects only; self references
* disallowed
@@ -216,18 +257,8 @@ struct TextInfo
GENERATE_CMP(TextInfo, me->hash, me->references);
};
struct FixedOutputInfo
{
/**
* How the file system objects are serialized
*/
FileIngestionMethod method;
/**
* Hash of that serialization
*/
Hash hash;
struct FixedOutputInfo {
FixedOutputHash hash;
/**
* References to other store objects or this one.
*/
+81 -89
View File
@@ -1,13 +1,11 @@
#include "daemon.hh"
#include "monitor-fd.hh"
#include "worker-protocol.hh"
#include "worker-protocol-impl.hh"
#include "build-result.hh"
#include "store-api.hh"
#include "store-cast.hh"
#include "gc-store.hh"
#include "log-store.hh"
#include "indirect-root-store.hh"
#include "path-with-outputs.hh"
#include "finally.hh"
#include "archive.hh"
@@ -261,13 +259,13 @@ struct ClientSettings
}
};
static std::vector<DerivedPath> readDerivedPaths(Store & store, unsigned int clientVersion, WorkerProto::ReadConn conn)
static std::vector<DerivedPath> readDerivedPaths(Store & store, unsigned int clientVersion, Source & from)
{
std::vector<DerivedPath> reqs;
if (GET_PROTOCOL_MINOR(clientVersion) >= 30) {
reqs = WorkerProto::Serialise<std::vector<DerivedPath>>::read(store, conn);
reqs = WorkerProto<std::vector<DerivedPath>>::read(store, from);
} else {
for (auto & s : readStrings<Strings>(conn.from))
for (auto & s : readStrings<Strings>(from))
reqs.push_back(parsePathWithOutputs(store, s).toDerivedPath());
}
return reqs;
@@ -275,14 +273,11 @@ static std::vector<DerivedPath> readDerivedPaths(Store & store, unsigned int cli
static void performOp(TunnelLogger * logger, ref<Store> store,
TrustedFlag trusted, RecursiveFlag recursive, unsigned int clientVersion,
Source & from, BufferedSink & to, WorkerProto::Op op)
Source & from, BufferedSink & to, unsigned int op)
{
WorkerProto::ReadConn rconn { .from = from };
WorkerProto::WriteConn wconn { .to = to };
switch (op) {
case WorkerProto::Op::IsValidPath: {
case wopIsValidPath: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
bool result = store->isValidPath(path);
@@ -291,8 +286,8 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryValidPaths: {
auto paths = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
case wopQueryValidPaths: {
auto paths = WorkerProto<StorePathSet>::read(*store, from);
SubstituteFlag substitute = NoSubstitute;
if (GET_PROTOCOL_MINOR(clientVersion) >= 27) {
@@ -305,11 +300,11 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
}
auto res = store->queryValidPaths(paths, substitute);
logger->stopWork();
WorkerProto::write(*store, wconn, res);
workerProtoWrite(*store, to, res);
break;
}
case WorkerProto::Op::HasSubstitutes: {
case wopHasSubstitutes: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
StorePathSet paths; // FIXME
@@ -320,16 +315,16 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QuerySubstitutablePaths: {
auto paths = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
case wopQuerySubstitutablePaths: {
auto paths = WorkerProto<StorePathSet>::read(*store, from);
logger->startWork();
auto res = store->querySubstitutablePaths(paths);
logger->stopWork();
WorkerProto::write(*store, wconn, res);
workerProtoWrite(*store, to, res);
break;
}
case WorkerProto::Op::QueryPathHash: {
case wopQueryPathHash: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
auto hash = store->queryPathInfo(path)->narHash;
@@ -338,27 +333,27 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryReferences:
case WorkerProto::Op::QueryReferrers:
case WorkerProto::Op::QueryValidDerivers:
case WorkerProto::Op::QueryDerivationOutputs: {
case wopQueryReferences:
case wopQueryReferrers:
case wopQueryValidDerivers:
case wopQueryDerivationOutputs: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
StorePathSet paths;
if (op == WorkerProto::Op::QueryReferences)
if (op == wopQueryReferences)
for (auto & i : store->queryPathInfo(path)->references)
paths.insert(i);
else if (op == WorkerProto::Op::QueryReferrers)
else if (op == wopQueryReferrers)
store->queryReferrers(path, paths);
else if (op == WorkerProto::Op::QueryValidDerivers)
else if (op == wopQueryValidDerivers)
paths = store->queryValidDerivers(path);
else paths = store->queryDerivationOutputs(path);
logger->stopWork();
WorkerProto::write(*store, wconn, paths);
workerProtoWrite(*store, to, paths);
break;
}
case WorkerProto::Op::QueryDerivationOutputNames: {
case wopQueryDerivationOutputNames: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
auto names = store->readDerivation(path).outputNames();
@@ -367,16 +362,16 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryDerivationOutputMap: {
case wopQueryDerivationOutputMap: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
auto outputs = store->queryPartialDerivationOutputMap(path);
logger->stopWork();
WorkerProto::write(*store, wconn, outputs);
workerProtoWrite(*store, to, outputs);
break;
}
case WorkerProto::Op::QueryDeriver: {
case wopQueryDeriver: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
auto info = store->queryPathInfo(path);
@@ -385,7 +380,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryPathFromHashPart: {
case wopQueryPathFromHashPart: {
auto hashPart = readString(from);
logger->startWork();
auto path = store->queryPathFromHashPart(hashPart);
@@ -394,11 +389,11 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddToStore: {
case wopAddToStore: {
if (GET_PROTOCOL_MINOR(clientVersion) >= 25) {
auto name = readString(from);
auto camStr = readString(from);
auto refs = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
auto refs = WorkerProto<StorePathSet>::read(*store, from);
bool repairBool;
from >> repairBool;
auto repair = RepairFlag{repairBool};
@@ -480,7 +475,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddMultipleToStore: {
case wopAddMultipleToStore: {
bool repair, dontCheckSigs;
from >> repair >> dontCheckSigs;
if (!trusted && dontCheckSigs)
@@ -497,10 +492,10 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddTextToStore: {
case wopAddTextToStore: {
std::string suffix = readString(from);
std::string s = readString(from);
auto refs = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
auto refs = WorkerProto<StorePathSet>::read(*store, from);
logger->startWork();
auto path = store->addTextToStore(suffix, s, refs, NoRepair);
logger->stopWork();
@@ -508,7 +503,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::ExportPath: {
case wopExportPath: {
auto path = store->parseStorePath(readString(from));
readInt(from); // obsolete
logger->startWork();
@@ -519,7 +514,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::ImportPaths: {
case wopImportPaths: {
logger->startWork();
TunnelSource source(from, to);
auto paths = store->importPaths(source,
@@ -531,8 +526,8 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::BuildPaths: {
auto drvs = readDerivedPaths(*store, clientVersion, rconn);
case wopBuildPaths: {
auto drvs = readDerivedPaths(*store, clientVersion, from);
BuildMode mode = bmNormal;
if (GET_PROTOCOL_MINOR(clientVersion) >= 15) {
mode = (BuildMode) readInt(from);
@@ -556,8 +551,8 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::BuildPathsWithResults: {
auto drvs = readDerivedPaths(*store, clientVersion, rconn);
case wopBuildPathsWithResults: {
auto drvs = readDerivedPaths(*store, clientVersion, from);
BuildMode mode = bmNormal;
mode = (BuildMode) readInt(from);
@@ -572,12 +567,12 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
auto results = store->buildPathsWithResults(drvs, mode);
logger->stopWork();
WorkerProto::write(*store, wconn, results);
workerProtoWrite(*store, to, results);
break;
}
case WorkerProto::Op::BuildDerivation: {
case wopBuildDerivation: {
auto drvPath = store->parseStorePath(readString(from));
BasicDerivation drv;
readDerivation(from, *store, drv, Derivation::nameFromPath(drvPath));
@@ -649,12 +644,12 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
DrvOutputs builtOutputs;
for (auto & [output, realisation] : res.builtOutputs)
builtOutputs.insert_or_assign(realisation.id, realisation);
WorkerProto::write(*store, wconn, builtOutputs);
workerProtoWrite(*store, to, builtOutputs);
}
break;
}
case WorkerProto::Op::EnsurePath: {
case wopEnsurePath: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
store->ensurePath(path);
@@ -663,7 +658,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddTempRoot: {
case wopAddTempRoot: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
store->addTempRoot(path);
@@ -672,12 +667,12 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddIndirectRoot: {
case wopAddIndirectRoot: {
Path path = absPath(readString(from));
logger->startWork();
auto & indirectRootStore = require<IndirectRootStore>(*store);
indirectRootStore.addIndirectRoot(path);
auto & gcStore = require<GcStore>(*store);
gcStore.addIndirectRoot(path);
logger->stopWork();
to << 1;
@@ -685,14 +680,14 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
}
// Obsolete.
case WorkerProto::Op::SyncWithGC: {
case wopSyncWithGC: {
logger->startWork();
logger->stopWork();
to << 1;
break;
}
case WorkerProto::Op::FindRoots: {
case wopFindRoots: {
logger->startWork();
auto & gcStore = require<GcStore>(*store);
Roots roots = gcStore.findRoots(!trusted);
@@ -711,10 +706,10 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::CollectGarbage: {
case wopCollectGarbage: {
GCOptions options;
options.action = (GCOptions::GCAction) readInt(from);
options.pathsToDelete = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
options.pathsToDelete = WorkerProto<StorePathSet>::read(*store, from);
from >> options.ignoreLiveness >> options.maxFreed;
// obsolete fields
readInt(from);
@@ -735,7 +730,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::SetOptions: {
case wopSetOptions: {
ClientSettings clientSettings;
@@ -772,7 +767,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QuerySubstitutablePathInfo: {
case wopQuerySubstitutablePathInfo: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
SubstitutablePathInfos infos;
@@ -784,22 +779,22 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
else {
to << 1
<< (i->second.deriver ? store->printStorePath(*i->second.deriver) : "");
WorkerProto::write(*store, wconn, i->second.references);
workerProtoWrite(*store, to, i->second.references);
to << i->second.downloadSize
<< i->second.narSize;
}
break;
}
case WorkerProto::Op::QuerySubstitutablePathInfos: {
case wopQuerySubstitutablePathInfos: {
SubstitutablePathInfos infos;
StorePathCAMap pathsMap = {};
if (GET_PROTOCOL_MINOR(clientVersion) < 22) {
auto paths = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
auto paths = WorkerProto<StorePathSet>::read(*store, from);
for (auto & path : paths)
pathsMap.emplace(path, std::nullopt);
} else
pathsMap = WorkerProto::Serialise<StorePathCAMap>::read(*store, rconn);
pathsMap = WorkerProto<StorePathCAMap>::read(*store, from);
logger->startWork();
store->querySubstitutablePathInfos(pathsMap, infos);
logger->stopWork();
@@ -807,21 +802,21 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
for (auto & i : infos) {
to << store->printStorePath(i.first)
<< (i.second.deriver ? store->printStorePath(*i.second.deriver) : "");
WorkerProto::write(*store, wconn, i.second.references);
workerProtoWrite(*store, to, i.second.references);
to << i.second.downloadSize << i.second.narSize;
}
break;
}
case WorkerProto::Op::QueryAllValidPaths: {
case wopQueryAllValidPaths: {
logger->startWork();
auto paths = store->queryAllValidPaths();
logger->stopWork();
WorkerProto::write(*store, wconn, paths);
workerProtoWrite(*store, to, paths);
break;
}
case WorkerProto::Op::QueryPathInfo: {
case wopQueryPathInfo: {
auto path = store->parseStorePath(readString(from));
std::shared_ptr<const ValidPathInfo> info;
logger->startWork();
@@ -842,14 +837,14 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::OptimiseStore:
case wopOptimiseStore:
logger->startWork();
store->optimiseStore();
logger->stopWork();
to << 1;
break;
case WorkerProto::Op::VerifyStore: {
case wopVerifyStore: {
bool checkContents, repair;
from >> checkContents >> repair;
logger->startWork();
@@ -861,17 +856,19 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddSignatures: {
case wopAddSignatures: {
auto path = store->parseStorePath(readString(from));
StringSet sigs = readStrings<StringSet>(from);
logger->startWork();
if (!trusted)
throw Error("you are not privileged to add signatures");
store->addSignatures(path, sigs);
logger->stopWork();
to << 1;
break;
}
case WorkerProto::Op::NarFromPath: {
case wopNarFromPath: {
auto path = store->parseStorePath(readString(from));
logger->startWork();
logger->stopWork();
@@ -879,7 +876,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::AddToStoreNar: {
case wopAddToStoreNar: {
bool repair, dontCheckSigs;
auto path = store->parseStorePath(readString(from));
auto deriver = readString(from);
@@ -887,7 +884,7 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
ValidPathInfo info { path, narHash };
if (deriver != "")
info.deriver = store->parseStorePath(deriver);
info.references = WorkerProto::Serialise<StorePathSet>::read(*store, rconn);
info.references = WorkerProto<StorePathSet>::read(*store, from);
from >> info.registrationTime >> info.narSize >> info.ultimate;
info.sigs = readStrings<StringSet>(from);
info.ca = ContentAddress::parseOpt(readString(from));
@@ -931,21 +928,21 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryMissing: {
auto targets = readDerivedPaths(*store, clientVersion, rconn);
case wopQueryMissing: {
auto targets = readDerivedPaths(*store, clientVersion, from);
logger->startWork();
StorePathSet willBuild, willSubstitute, unknown;
uint64_t downloadSize, narSize;
store->queryMissing(targets, willBuild, willSubstitute, unknown, downloadSize, narSize);
logger->stopWork();
WorkerProto::write(*store, wconn, willBuild);
WorkerProto::write(*store, wconn, willSubstitute);
WorkerProto::write(*store, wconn, unknown);
workerProtoWrite(*store, to, willBuild);
workerProtoWrite(*store, to, willSubstitute);
workerProtoWrite(*store, to, unknown);
to << downloadSize << narSize;
break;
}
case WorkerProto::Op::RegisterDrvOutput: {
case wopRegisterDrvOutput: {
logger->startWork();
if (GET_PROTOCOL_MINOR(clientVersion) < 31) {
auto outputId = DrvOutput::parse(readString(from));
@@ -953,14 +950,14 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
store->registerDrvOutput(Realisation{
.id = outputId, .outPath = outputPath});
} else {
auto realisation = WorkerProto::Serialise<Realisation>::read(*store, rconn);
auto realisation = WorkerProto<Realisation>::read(*store, from);
store->registerDrvOutput(realisation);
}
logger->stopWork();
break;
}
case WorkerProto::Op::QueryRealisation: {
case wopQueryRealisation: {
logger->startWork();
auto outputId = DrvOutput::parse(readString(from));
auto info = store->queryRealisation(outputId);
@@ -968,16 +965,16 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
if (GET_PROTOCOL_MINOR(clientVersion) < 31) {
std::set<StorePath> outPaths;
if (info) outPaths.insert(info->outPath);
WorkerProto::write(*store, wconn, outPaths);
workerProtoWrite(*store, to, outPaths);
} else {
std::set<Realisation> realisations;
if (info) realisations.insert(*info);
WorkerProto::write(*store, wconn, realisations);
workerProtoWrite(*store, to, realisations);
}
break;
}
case WorkerProto::Op::AddBuildLog: {
case wopAddBuildLog: {
StorePath path{readString(from)};
logger->startWork();
if (!trusted)
@@ -994,10 +991,6 @@ static void performOp(TunnelLogger * logger, ref<Store> store,
break;
}
case WorkerProto::Op::QueryFailedPaths:
case WorkerProto::Op::ClearFailedPaths:
throw Error("Removed operation %1%", op);
default:
throw Error("invalid operation %1%", op);
}
@@ -1052,8 +1045,7 @@ void processConnection(
auto temp = trusted
? store->isTrustedClient()
: std::optional { NotTrusted };
WorkerProto::WriteConn wconn { .to = to };
WorkerProto::write(*store, wconn, temp);
workerProtoWrite(*store, to, temp);
}
/* Send startup error messages to the client. */
@@ -1066,9 +1058,9 @@ void processConnection(
/* Process client requests. */
while (true) {
WorkerProto::Op op;
WorkerOp op;
try {
op = (enum WorkerProto::Op) readInt(from);
op = (WorkerOp) readInt(from);
} catch (Interrupted & e) {
break;
} catch (EndOfFile & e) {
+12 -18
View File
@@ -5,7 +5,6 @@
#include "util.hh"
#include "split.hh"
#include "worker-protocol.hh"
#include "worker-protocol-impl.hh"
#include "fs-accessor.hh"
#include <boost/container/small_vector.hpp>
#include <nlohmann/json.hpp>
@@ -232,10 +231,9 @@ static DerivationOutput parseDerivationOutput(const Store & store,
validatePath(pathS);
auto hash = Hash::parseNonSRIUnprefixed(hashS, hashType);
return DerivationOutput::CAFixed {
.ca = ContentAddress {
.method = std::move(method),
.hash = std::move(hash),
},
.ca = ContentAddress::fromParts(
std::move(method),
std::move(hash)),
};
} else {
experimentalFeatureSettings.require(Xp::CaDerivations);
@@ -396,7 +394,7 @@ std::string Derivation::unparse(const Store & store, bool maskOutputs,
[&](const DerivationOutput::CAFixed & dof) {
s += ','; printUnquotedString(s, maskOutputs ? "" : store.printStorePath(dof.path(store, name, i.first)));
s += ','; printUnquotedString(s, dof.ca.printMethodAlgo());
s += ','; printUnquotedString(s, dof.ca.hash.to_string(Base16, false));
s += ','; printUnquotedString(s, dof.ca.getHash().to_string(Base16, false));
},
[&](const DerivationOutput::CAFloating & dof) {
s += ','; printUnquotedString(s, "");
@@ -629,7 +627,7 @@ DrvHash hashDerivationModulo(Store & store, const Derivation & drv, bool maskOut
auto & dof = std::get<DerivationOutput::CAFixed>(i.second.raw());
auto hash = hashString(htSHA256, "fixed:out:"
+ dof.ca.printMethodAlgo() + ":"
+ dof.ca.hash.to_string(Base16, false) + ":"
+ dof.ca.getHash().to_string(Base16, false) + ":"
+ store.printStorePath(dof.path(store, drv.name, i.first)));
outputHashes.insert_or_assign(i.first, std::move(hash));
}
@@ -751,8 +749,7 @@ Source & readDerivation(Source & in, const Store & store, BasicDerivation & drv,
drv.outputs.emplace(std::move(name), std::move(output));
}
drv.inputSrcs = WorkerProto::Serialise<StorePathSet>::read(store,
WorkerProto::ReadConn { .from = in });
drv.inputSrcs = WorkerProto<StorePathSet>::read(store, in);
in >> drv.platform >> drv.builder;
drv.args = readStrings<Strings>(in);
@@ -781,7 +778,7 @@ void writeDerivation(Sink & out, const Store & store, const BasicDerivation & dr
[&](const DerivationOutput::CAFixed & dof) {
out << store.printStorePath(dof.path(store, drv.name, i.first))
<< dof.ca.printMethodAlgo()
<< dof.ca.hash.to_string(Base16, false);
<< dof.ca.getHash().to_string(Base16, false);
},
[&](const DerivationOutput::CAFloating & dof) {
out << ""
@@ -800,9 +797,7 @@ void writeDerivation(Sink & out, const Store & store, const BasicDerivation & dr
},
}, i.second.raw());
}
WorkerProto::write(store,
WorkerProto::WriteConn { .to = out },
drv.inputSrcs);
workerProtoWrite(store, out, drv.inputSrcs);
out << drv.platform << drv.builder << drv.args;
out << drv.env.size();
for (auto & i : drv.env)
@@ -971,7 +966,7 @@ nlohmann::json DerivationOutput::toJSON(
[&](const DerivationOutput::CAFixed & dof) {
res["path"] = store.printStorePath(dof.path(store, drvName, outputName));
res["hashAlgo"] = dof.ca.printMethodAlgo();
res["hash"] = dof.ca.hash.to_string(Base16, false);
res["hash"] = dof.ca.getHash().to_string(Base16, false);
// FIXME print refs?
},
[&](const DerivationOutput::CAFloating & dof) {
@@ -1018,10 +1013,9 @@ DerivationOutput DerivationOutput::fromJSON(
else if (keys == (std::set<std::string_view> { "path", "hashAlgo", "hash" })) {
auto [method, hashType] = methodAlgo();
auto dof = DerivationOutput::CAFixed {
.ca = ContentAddress {
.method = std::move(method),
.hash = Hash::parseNonSRIUnprefixed((std::string) json["hash"], hashType),
},
.ca = ContentAddress::fromParts(
std::move(method),
Hash::parseNonSRIUnprefixed((std::string) json["hash"], hashType)),
};
if (dof.path(store, drvName, outputName) != store.parseStorePath((std::string) json["path"]))
throw Error("Path doesn't match derivation output");
+56
View File
@@ -1,4 +1,5 @@
#include "derived-path.hh"
#include "derivations.hh"
#include "store-api.hh"
#include <nlohmann/json.hpp>
@@ -29,6 +30,30 @@ nlohmann::json DerivedPath::Built::toJSON(ref<Store> store) const {
return res;
}
nlohmann::json BuiltPath::Built::toJSON(ref<Store> store) const {
nlohmann::json res;
res["drvPath"] = store->printStorePath(drvPath);
for (const auto& [output, path] : outputs) {
res["outputs"][output] = store->printStorePath(path);
}
return res;
}
StorePathSet BuiltPath::outPaths() const
{
return std::visit(
overloaded{
[](const BuiltPath::Opaque & p) { return StorePathSet{p.path}; },
[](const BuiltPath::Built & b) {
StorePathSet res;
for (auto & [_, path] : b.outputs)
res.insert(path);
return res;
},
}, raw()
);
}
std::string DerivedPath::Opaque::to_string(const Store & store) const
{
return store.printStorePath(path);
@@ -96,4 +121,35 @@ DerivedPath DerivedPath::parseLegacy(const Store & store, std::string_view s)
return parseWith(store, s, "!");
}
RealisedPath::Set BuiltPath::toRealisedPaths(Store & store) const
{
RealisedPath::Set res;
std::visit(
overloaded{
[&](const BuiltPath::Opaque & p) { res.insert(p.path); },
[&](const BuiltPath::Built & p) {
auto drvHashes =
staticOutputHashes(store, store.readDerivation(p.drvPath));
for (auto& [outputName, outputPath] : p.outputs) {
if (experimentalFeatureSettings.isEnabled(
Xp::CaDerivations)) {
auto drvOutput = get(drvHashes, outputName);
if (!drvOutput)
throw Error(
"the derivation '%s' has unrealised output '%s' (derived-path.cc/toRealisedPaths)",
store.printStorePath(p.drvPath), outputName);
auto thisRealisation = store.queryRealisation(
DrvOutput{*drvOutput, outputName});
assert(thisRealisation); // We’ve built it, so we must
// have the realisation
res.insert(*thisRealisation);
} else {
res.insert(outputPath);
}
}
},
},
raw());
return res;
}
}
+42
View File
@@ -3,6 +3,7 @@
#include "util.hh"
#include "path.hh"
#include "realisation.hh"
#include "outputs-spec.hh"
#include "comparator.hh"
@@ -108,6 +109,47 @@ struct DerivedPath : _DerivedPathRaw {
static DerivedPath parseLegacy(const Store & store, std::string_view);
};
/**
* A built derived path with hints in the form of optional concrete output paths.
*
* See 'BuiltPath' for more an explanation.
*/
struct BuiltPathBuilt {
StorePath drvPath;
std::map<std::string, StorePath> outputs;
nlohmann::json toJSON(ref<Store> store) const;
static BuiltPathBuilt parse(const Store & store, std::string_view);
GENERATE_CMP(BuiltPathBuilt, me->drvPath, me->outputs);
};
using _BuiltPathRaw = std::variant<
DerivedPath::Opaque,
BuiltPathBuilt
>;
/**
* A built path. Similar to a DerivedPath, but enriched with the corresponding
* output path(s).
*/
struct BuiltPath : _BuiltPathRaw {
using Raw = _BuiltPathRaw;
using Raw::Raw;
using Opaque = DerivedPathOpaque;
using Built = BuiltPathBuilt;
inline const Raw & raw() const {
return static_cast<const Raw &>(*this);
}
StorePathSet outPaths() const;
RealisedPath::Set toRealisedPaths(Store & store) const;
};
typedef std::vector<DerivedPath> DerivedPaths;
typedef std::vector<BuiltPath> BuiltPaths;
}
+2 -6
View File
@@ -2,7 +2,6 @@
#include "store-api.hh"
#include "archive.hh"
#include "worker-protocol.hh"
#include "worker-protocol-impl.hh"
#include <algorithm>
@@ -46,9 +45,7 @@ void Store::exportPath(const StorePath & path, Sink & sink)
teeSink
<< exportMagic
<< printStorePath(path);
WorkerProto::write(*this,
WorkerProto::WriteConn { .to = teeSink },
info->references);
workerProtoWrite(*this, teeSink, info->references);
teeSink
<< (info->deriver ? printStorePath(*info->deriver) : "")
<< 0;
@@ -76,8 +73,7 @@ StorePaths Store::importPaths(Source & source, CheckSigsFlag checkSigs)
//Activity act(*logger, lvlInfo, "importing path '%s'", info.path);
auto references = WorkerProto::Serialise<StorePathSet>::read(*this,
WorkerProto::ReadConn { .from = source });
auto references = WorkerProto<StorePathSet>::read(*this, source);
auto deriver = readString(source);
auto narHash = hashString(htSHA256, saved.s);
+9 -26
View File
@@ -71,36 +71,19 @@ struct GCResults
};
/**
* Mix-in class for \ref Store "stores" which expose a notion of garbage
* collection.
*
* Garbage collection will allow deleting paths which are not
* transitively "rooted".
*
* The notion of GC roots actually not part of this class.
*
* - The base `Store` class has `Store::addTempRoot()` because for a store
* that doesn't support garbage collection at all, a temporary GC root is
* safely implementable as no-op.
*
* @todo actually this is not so good because stores are *views*.
* Some views have only a no-op temp roots even though others to the
* same store allow triggering GC. For instance one can't add a root
* over ssh, but that doesn't prevent someone from gc-ing that store
* accesed via SSH locally).
*
* - The derived `LocalFSStore` class has `LocalFSStore::addPermRoot`,
* which is not part of this class because it relies on the notion of
* an ambient file system. There are stores (`ssh-ng://`, for one),
* that *do* support garbage collection but *don't* expose any file
* system, and `LocalFSStore::addPermRoot` thus does not make sense
* for them.
*/
struct GcStore : public virtual Store
{
inline static std::string operationName = "Garbage collection";
/**
* Add an indirect root, which is merely a symlink to `path` from
* `/nix/var/nix/gcroots/auto/<hash of path>`. `path` is supposed
* to be a symlink to a store path. The garbage collector will
* automatically remove the indirect root when it finds that
* `path` has disappeared.
*/
virtual void addIndirectRoot(const Path & path) = 0;
/**
* Find the roots of the garbage collector. Each root is a pair
* `(link, storepath)` where `link` is the path of the symlink
Loaded 100 of 277 files, more files were not shown because too many files have changed in this diff. Show more